This page intentionally left blank
Probability Theory An Analytic View, Second Edition This second edition of Daniel W. Stroock’s text is suitable for first-year graduate students with a good grasp of introductory undergraduate probability. It provides a reasonably thorough introduction to modern probability theory with an emphasis on the mutually beneficial relationship between probability theory and analysis. It includes more than 750 exercises and offers new material on Levy processes, large deviations theory, Gaussian measures on a Banach space, and the relationship between a Wiener measure and partial differential equations. The first part of the book deals with independent random variables, Central Limit phenomena, the general theory of weak convergence and several of its applications, as well as elements of both the Gaussian and Markovian theories of measures on function space. The introduction of conditional expectation values is postponed until the second part of the book, where it is applied to the study of martingales. This part also explores the connection between martingales and various aspects of classical analysis and the connections between a Wiener measure and classical potential theory. Dr. Daniel W. Stroock is the Simons Professor of Mathematics Emeritus at the Massachusetts Institute of Technology. He has published many articles and is the author of six books, most recently Partial Differential Equations for Probabilists (2008).
Probability Theory An Analytic View Second Edition
Daniel W. Stroock Massachusetts Institute of Technology
cambridge university press Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, S˜ a o Paulo, Delhi, Dubai, Tokyo, Mexico City Cambridge University Press 32 Avenue of the Americas, New York, NY 10013-2473, USA www.cambridge.org Information on this title: www.cambridge.org/9780521132503 c Daniel W. Stroock 1994, 2011 This publication is in copyright. Subject to statutory exception and to the provisions of relevant collective licensing agreements, no reproduction of any part may take place without the written permission of Cambridge University Press. First edition published 1994 First paperback edition 2000 Second edition published 2011 Printed in the United States of America A catalog record for this publication is available from the British Library. Library of Congress Cataloging in Publication data Stroock, Daniel W. Probability theory : an analytic view/ Daniel W. Stroock. – 2nd ed. p. cm. Includes bibliographical references and index. ISBN 978-0-521-76158-1 (hardback) – ISBN 978-0-521-13250-3 (pbk.) 1. Probabilities. I. Title. QA 273.S763 2010 519.2–dc22 2010027652 ISBN 978-0-521-76158-1 Hardback ISBN 978-0-521-13250-3 Paperback Cambridge University Press has no responsibility for the persistence or accuracy of URLs for external or third-party Internet Web sites referred to in this publication and does not guarantee that any content on such Web sites is, or will remain, accurate or appropriate.
This book is dedicated to my teachers: M. Kac, H.P. McKean, Jr., and S.R.S. Varadhan
Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii Table of Dependence . . . . . . . . . . . . . . . . . . . . . . xxi Chapter 1 Sums of Independent Random 1.1 Independence . . . . . . . . . . . . 1.1.1. Independent σ-Algebras . . . . . . 1.1.2. Independent Functions . . . . . . . 1.1.3. The Rademacher Functions . . . . . Exercises for § 1.1 . . . . . . . . . . . . 1.2 The Weak Law of Large Numbers . . . 1.2.1. Orthogonal Random Variables . . . 1.2.2. Independent Random Variables . . . 1.2.3. Approximate Identities . . . . . . . Exercises for § 1.2 . . . . . . . . . . . . 1.3 Cram´er’s Theory of Large Deviations . . Exercises for § 1.3 . . . . . . . . . . . . 1.4 The Strong Law of Large Numbers . . . Exercises for § 1.4 . . . . . . . . . . . . 1.5 Law of the Iterated Logarithm . . . . Exercises for § 1.5 . . . . . . . . . . . .
Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Chapter 2 The Central Limit Theorem . . . . 2.1 The Basic Central Limit Theorem . . . . . . 2.1.1. Lindeberg’s Theorem . . . . . . . . . . 2.1.2. The Central Limit Theorem . . . . . . . Exercises for § 2.1 . . . . . . . . . . . . . . . 2.2 The Berry–Esseen Theorem via Stein’s Method 2.2.1. L1 -Berry–Esseen . . . . . . . . . . . . 2.2.2. The Classical Berry–Esseen Theorem . . . Exercises for § 2.2 . . . . . . . . . . . . . . . 2.3 Some Extensions of The Central Limit Theorem 2.3.1. The Fourier Transform . . . . . . . . . . 2.3.2. Multidimensional Central Limit Theorem . . 2.3.3. Higher Moments . . . . . . . . . . . . Exercises for § 2.3 . . . . . . . . . . . . . . . vii
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . .
1 1 1 4 5 7 14 14 15 16 20 22 31 35 42 49 56 59 60 60 62 65 71 72 75 81 82 82 84 87 90
viii
Contents
2.4 An Application to Hermite Multipliers 2.4.1. Hermite Multipliers . . . . . . . 2.4.2. Beckner’s Theorem . . . . . . . 2.4.3. Applications of Beckner’s Theorem Exercises for § 2.4 . . . . . . . . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. 96 . 96 . 101 . 105 . 110
Chapter 3 Infinitely Divisible Laws . . . . . . . . . . . . . . . 115 3.1 Convergence of Measures on RN . . . 3.1.1. Sequential Compactness in M1 (RN ) 3.1.2. L´evy’s Continuity Theorem . . . . Exercises for § 3.1 . . . . . . . . . . . 3.2 The L´evy–Khinchine Formula . . . . 3.2.1. I(RN ) Is the Closure of P(RN ) . . 3.2.2. The Formula . . . . . . . . . . Exercises for § 3.2 . . . . . . . . . . . 3.3 Stable Laws . . . . . . . . . . . . 3.3.1. General Results . . . . . . . . . 3.3.2. α-Stable Laws . . . . . . . . . Exercises for § 3.3 . . . . . . . . . . . Chapter 4 L´ evy Processes
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
115 116 117 119 122 123 126 137 139 139 141 147
. . . . . . . . . . . . . . . . . . . 151
4.1 Stochastic Processes, Some Generalities . . . 4.1.1. The Space D(RN ) . . . . . . . . . . . 4.1.2. Jump Functions . . . . . . . . . . . . Exercises for § 4.1 . . . . . . . . . . . . . . 4.2 Discontinuous L´evy Processes . . . . . . . 4.2.1. The Simple Poisson Process . . . . . . . 4.2.2. Compound Poisson Processes . . . . . . 4.2.3. Poisson Jump Processes . . . . . . . . 4.2.4. L´evy Processes with Bounded Variation . 4.2.5. General, Non-Gaussian L´evy Processes . . Exercises for § 4.2 . . . . . . . . . . . . . . 4.3 Brownian Motion, the Gaussian L´evy Process 4.3.1. Deconstructing Brownian Motion . . . . 4.3.2. L´evy’s Construction of Brownian Motion . 4.3.3. L´evy’s Construction in Context . . . . . 4.3.4. Brownian Paths Are Non-Differentiable . 4.3.5. General L´evy Processes . . . . . . . . Exercises for § 4.3 . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . .
152 153 156 159 160 161 163 168 170 171 174 177 178 180 182 183 185 187
Contents
ix
Chapter 5 Conditioning and Martingales . . . . . . . . . . . . 193 5.1 Conditioning . . . . . . . . . . . . . . . . 5.1.1. Kolmogorov’s Definition . . . . . . . . . . 5.1.2. Some Extensions . . . . . . . . . . . . . Exercises for § 5.1 . . . . . . . . . . . . . . . . 5.2 Discrete Parameter Martingales . . . . . . . . 5.2.1. Doob’s Inequality and Marcinkewitz’s Theorem 5.2.2. Doob’s Stopping Time Theorem . . . . . . . 5.2.3. Martingale Convergence Theorem . . . . . . 5.2.4. Reversed Martingales and De Finetti’s Theory . 5.2.5. An Application to a Tracking Algorithm . . . Exercises for § 5.2 . . . . . . . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
193 194 198 202 205 206 212 214 217 221 226
Chapter 6 Some Extensions and Applications of Martingale Theory . . . . . . . . . . . . . . . . . . . . . . 233 6.1 Some Extensions . . . . . . . . . . . . . . . 6.1.1. Martingale Theory for a σ-Finite Measure Space 6.1.2. Banach Space–Valued Martingales . . . . . . Exercises for § 6.1 . . . . . . . . . . . . . . . . 6.2 Elements of Ergodic Theory . . . . . . . . . . 6.2.1. The Maximal Ergodic Lemma . . . . . . . . 6.2.2. Birkhoff’s Ergodic Theorem . . . . . . . . 6.2.3. Stationary Sequences . . . . . . . . . . . 6.2.4. Continuous Parameter Ergodic Theory . . . . Exercises for § 6.2 . . . . . . . . . . . . . . . . 6.3 Burkholder’s Inequality . . . . . . . . . . . . 6.3.1. Burkholder’s Comparison Theorem . . . . . 6.3.2. Burkholder’s Inequality . . . . . . . . . . Exercises for § 6.3 . . . . . . . . . . . . . . . . Chapter 7 Continuous Parameter Martingales 7.1 Continuous Parameter Martingales . . . . . . 7.1.1. Progressively Measurable Functions . . . . 7.1.2. Martingales: Definition and Examples . . . 7.1.3. Basic Results . . . . . . . . . . . . . . 7.1.4. Stopping Times and Stopping Theorems . . 7.1.5. An Integration by Parts Formula . . . . . Exercises for § 7.1 . . . . . . . . . . . . . . . 7.2 Brownian Motion and Martingales . . . . . . 7.2.1. L´evy’s Characterization of Brownian Motion 7.2.2. Doob–Meyer Decomposition, an Easy Case . 7.2.3. Burkholder’s Inequality Again . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
233 233 239 240 244 245 248 251 253 256 257 257 262 263
. . . . . . . . . 266 . . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
266 266 267 270 272 276 280 282 282 284 289
x
Contents Exercises for § 7.2 . . . . . . . . . . . . 7.3 The Reflection Principle Revisited . . . 7.3.1. Reflecting Symmetric L´evy Processes 7.3.2. Reflected Brownian Motion . . . . . Exercises for § 7.3 . . . . . . . . . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
290 292 292 294 298
Chapter 8 Gaussian Measures on a Banach Space . 8.1 The Classical Wiener Space . . . . . . . . . . . 8.1.1. Classical Wiener Measure . . . . . . . . . . 8.1.2. The Classical Cameron–Martin Space . . . . . Exercises for § 8.1 . . . . . . . . . . . . . . . . . 8.2 A Structure Theorem for Gaussian Measures . . . 8.2.1. Fernique’s Theorem . . . . . . . . . . . . . 8.2.2. The Basic Structure Theorem . . . . . . . . . 8.2.3. The Cameron–Marin Space . . . . . . . . . . Exercises for § 8.2 . . . . . . . . . . . . . . . . . 8.3 From Hilbert to Abstract Wiener Space . . . . . 8.3.1. An Isomorphism Theorem . . . . . . . . . . 8.3.2. Wiener Series . . . . . . . . . . . . . . . . 8.3.3. Orthogonal Projections . . . . . . . . . . . . 8.3.4. Pinned Brownian Motion . . . . . . . . . . . 8.3.5. Orthogonal Invariance . . . . . . . . . . . . Exercises for § 8.3 . . . . . . . . . . . . . . . . . 8.4 A Large Deviations Result and Strassen’s Theorem 8.4.1. Large Deviations for Abstract Wiener Space . . 8.4.2. Strassen’s Law of the Iterated Logarithm . . . . Exercises for § 8.4 . . . . . . . . . . . . . . . . . 8.5 Euclidean Free Fields . . . . . . . . . . . . . 8.5.1. The Ornstein–Uhlenbeck Process . . . . . . . 8.5.2. Ornstein–Uhlenbeck as an Abstract Wiener Space 8.5.3. Higher Dimensional Free Fields . . . . . . . . Exercises for § 8.5 . . . . . . . . . . . . . . . . . 8.6 Brownian Motion on a Banach Space . . . . . . . 8.6.1. Abstract Wiener Formulation . . . . . . . . . 8.6.2. Brownian Formulation . . . . . . . . . . . . 8.6.3. Strassen’s Theorem Revisited . . . . . . . . . Exercises for § 8.6 . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
299 299 299 303 306 306 306 307 310 313 317 317 318 322 326 328 330 337 337 340 342 343 344 346 349 355 358 358 361 363 365
Chapter 9 Convergence of Measures on a Polish Space 9.1 Prohorov–Varadarajan Theory . . . . . . . . . . . 9.1.1. Some Background . . . . . . . . . . . . . . . . 9.1.2. The Weak Topology . . . . . . . . . . . . . . . 9.1.3. The L´evy Metric and Completeness of M1 (E) . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
367 367 367 370 377
Contents Exercises for § 9.1 . . . . . . . . . . . . . . . 9.2 Regular Conditional Probability Distributions . 9.2.1. Fibering a Measure . . . . . . . . . . . 9.2.2. Representing L´evy Measures via the Itˆo Map Exercises for § 9.2 . . . . . . . . . . . . . . . 9.3 Donsker’s Invariance Principle . . . . . . . . 9.3.1. Donsker’s Theorem . . . . . . . . . . . 9.3.2. Rayleigh’s Random Flights Model . . . . . Exercise for § 9.3 . . . . . . . . . . . . . . .
xi . . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
381 386 388 390 392 392 393 396 399
Chapter 10 Wiener Measure and Partial Differential Equations 10.1 Martingales and Partial Differential Equations . . . . . . . . 10.1.1. Localizing and Extending Martingale Representations . . . . 10.1.2. Minimum Principles . . . . . . . . . . . . . . . . . . 10.1.3. The Hermite Heat Equation . . . . . . . . . . . . . . . 10.1.4. The Arcsine Law . . . . . . . . . . . . . . . . . . . . 10.1.5. Recurrence and Transience of Brownian Motion . . . . . . Exercises for § 10.1 . . . . . . . . . . . . . . . . . . . . . . 10.2 The Markov Property and Potential Theory . . . . . . . . . 10.2.1. The Markov Property for Wiener Measure . . . . . . . . . 10.2.2. Recurrence in One and Two Dimensions . . . . . . . . . . 10.2.3. The Dirichlet Problem . . . . . . . . . . . . . . . . . Exercises for § 10.2 . . . . . . . . . . . . . . . . . . . . . . 10.3 Other Heat Kernels . . . . . . . . . . . . . . . . . . . . 10.3.1. A General Construction . . . . . . . . . . . . . . . . . 10.3.2. The Dirichlet Heat Kernel . . . . . . . . . . . . . . . . 10.3.3. Feynman–Kac Heat Kernels . . . . . . . . . . . . . . . 10.3.4. Ground States and Associated Measures on Pathspace . . . 10.3.5. Producing Ground States . . . . . . . . . . . . . . . . Exercises for § 10.3 . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
400 400 401 404 405 407 411 415 416 416 417 418 426 429 429 431 436 439 445 449
Chapter 11 Some Classical Potential Theory . . . . . . 11.1 Uniqueness Refined . . . . . . . . . . . . . . . . . 11.1.1. The Dirichlet Heat Kernel Again . . . . . . . . . . 11.1.2. Exiting Through ∂reg G . . . . . . . . . . . . . . 11.1.3. Applications to Questions of Uniqueness . . . . . . . 11.1.4. Harmonic Measure . . . . . . . . . . . . . . . . Exercises for § 11.1 . . . . . . . . . . . . . . . . . . . 11.2 The Poisson Problem and Green Functions . . . . . . . 11.2.1. Green Functions when N ≥ 3 . . . . . . . . . . . 11.2.2. Green Functions when N ∈ {1, 2} . . . . . . . . . . Exercises for § 11.2 . . . . . . . . . . . . . . . . . . . 11.3 Excessive Functions, Potentials, and Riesz Decompositions
. . . . . . . . . . . .
456 456 456 459 463 468 472 475 476 477 486 487
. . . . . . . . . . . .
. . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
xii
Contents
11.3.1. Excessive Functions . . . . . . . . . 11.3.2. Potentials and Riesz Decomposition . Exercises for § 11.3 . . . . . . . . . . . . 11.4 Capacity . . . . . . . . . . . . . . 11.4.1. The Capacitory Potential . . . . . . 11.4.2. The Capacitory Distribution . . . . . 11.4.3. Wiener’s Test . . . . . . . . . . . 11.4.4. Some Asymptotic Expressions Involving Exercises for § 11.4 . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . Capacity . . . .
. . . . . . .
. . . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
488 489 496 497 497 500 504 507 514
Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 521
Preface
From the Preface to the First Edition When writing a graduate level mathematics book during the last decade of the twentieth century, one probably ought not inquire too closely into one’s motivation. In fact, if ones own pleasure from the exercise is not sufficient to justify the effort, then one should seriously consider dropping the project. Thus, to those who (either before or shortly after opening it) ask for whom was this book written, my pale answer is me; and, for this reason, I thought that I should preface this preface with an explanation of who I am and what were the peculiar educational circumstances that eventually gave rise to this somewhat peculiar book. My own introduction to probability theory began with a private lecture from H.P. McKean, Jr. At the time, I was a (more accurately, the) graduate student of mathematics at what was then called The Rockefeller Institute for Biological Sciences. My official mentor there was M. Kac, whom I had cajoled into becoming my adviser after a year during which I had failed to insert even one micro-electrode into the optic nerves of innumerable limuli. However, as I soon came to realize, Kac had accepted his role on the condition that it would not become a burden. In particular, he had no intention of wasting much of his own time on a reject from the neurophysiology department. On the other hand, he was most generous with the time of his younger associates, and that is how I wound up in McKean’s office. Never one to bore his listeners with a lot of dull preliminaries, McKean launched right into a wonderfully lucid explanation of P. L´evy’s interpretation of the infinitely divisible laws. I have to admit that my appreciation of the lucidity of his lecture arrived nearly a decade after its delivery, and I can only hope that my reader will reserve judgment of my own presentation for an equal length of time. In spite of my perplexed state at the end of McKean’s lecture, I was sufficiently intrigued to delve into the readings that he suggested at its conclusion. Knowing that the only formal mathematics courses that I would be taking during my graduate studies would be given at N.Y.U. and guessing that those courses would be oriented toward partial differential equations, McKean directed me to material which would help me understand the connections between partial differential equations and probability theory. In particular, he suggested that I start with the, then recently translated, two articles by E.B. Dynkin which had appeared originally in the famous 1956 volume of Teoriya Veroyatnostei i ee Primeneniya. Dynkin’s articles turned out to be a godsend. They were beautifully crafted to xiii
xiv
Preface
tell the reader enough so that he could understand the ideas and not so much that he would become bored by them. In addition, they gave me an introduction to a host of ideas and techniques (e.g., stopping times and the strong Markov property), all of which Kac himself consigned to the category of overelaborated measure theory. In fact, it would be reasonable to say that my thesis was simply the application of techniques which I picked up from Dynkin to a problem that I picked up by reading some notes by Kac. Of course, along the way I profited immeasurably from continued contact with McKean, a large number of courses at N.Y.U. (particularly ones taught by M. Donsker, F. John, and L. Nirenberg), and my increasingly animated conversations with S.R.S. Varadhan. As I trust the preceding description makes clear, my graduate education was anything but deprived; I had ready access to some of the very best analysts of the day. On the other hand, I never had a proper introduction to my field, probability theory. The first time that I ever summed independent random variables was when I was summing them in front of a class at N.Y.U. Thus, although I now admire the magnificent body of mathematics created by A.N. Kolmogorov, P. L´evy, and the other twentieth-century heroes of the field, I am not a dyed-in-the-wool probabilist (i.e., what Donsker would have called a true coin-tosser). In particular, I have never been able to develop sufficient sensitivity to the distinction between a proof and a probabilistic proof. To me, a proof is clearly probabilistic only if its punch-line comes down to an argument like P (A) ≤ P (B) because A ⊆ B; and there are breathtaking examples of such arguments. However, to base an entire book on these examples would require a level of genius that I do not possess. In fact, I myself enjoy probability theory best when it is inextricably interwoven with other branches of mathematics and not when it is presented as an entity unto itself. For this reason, the reader should not be surprised to discover that he finds some of the material presented in this book does not belong here; but I hope that he will make an effort to figure out why I disagree with him. Preface to the Second Edition My favorite “preface to a second edition” is the one that G.N. Watson wrote for the second edition of his famous treatise on Bessel functions. The first edition appeared in 1922, the second came out in 1941, and Watson had originally intended to stay abreast of developments and report on them in the second edition. However, in his preface to the second edition Watson admits that his interest in the topic had “waned” during the intervening years and apologizes that, as a consequence, the new edition contains less new material than he had thought it would. My excuse for not incorporating more new material into this second edition is related to but somewhat different from Watson’s. In my case, what has waned is not my interest in probability theory but instead my ability to assimilate the transformations that the subject has undergone. When I was a student,
Preface
xv
probabilists were still working out the ramifications of Kolmogorov’s profound insights into the connections between probability and analysis, and I have spent my career investigating and exploiting those connections. However, about the time when the first edition of this book was published, probability theory began a return to its origins in combinatorics, a topic in which my abilities are woefully deficient. Thus, although I suspect that, for at least a decade, the most exciting developments in the field will have a strong combinatorial component, I have not attempted to prepare my readers for those developments. I repeat that my decision not to incorporate more combinatorics into this new edition in no way reflects my assessment of the direction in which probability is likely to go but instead reflects my assessment of my own inability to do justice to the beautiful combinatorial ideas that have been introduced in the recent past. In spite of the preceding admission, I believe that the material in this book remains valuable and that, no matter how probability theory evolves, the ideas and techniques presented here will play an important role. Furthermore, I have made some substantive changes. In particular, I have given more space to infinitely divisible laws and their associated L´evy processes, both of which are now developed in RN rather than just in R. In addition, I have added an entire chapter devoted to Gaussian measures in infinite dimensions from the perspective of the Segal–Gross school. Not only have recent developments in Malliavin calculus and conformal field theory sparked renewed interest in this topic, but it seems to me that most modern texts pay either no or too little attention to this beautiful material. Missing from the new edition is the treatment of singular integrals. I included it in the first edition in the hope that it would elucidate the similarity between cancellations that underlie martingale theory, especially Burkholder’s Inequality, and Calderon–Zygmund theory. I still believe that these similarities are worth thinking about, but I have decided that my explanation of them led me too far astray and was more of a distraction than a pedagogically valuable addition. Besides those mentioned above, minor changes have been made throughout. For one thing, I have spent a lot of time correcting old errors and, undoubtedly, inserting new ones. Secondly, I have made several organizational changes as well as others that are remedial. A summary of the contents follows. Summary 1: Chapter 1 contains a sampling of the standard, pointwise convergence theorems dealing with partial sums of independent random variables. These include the Weak and Strong Laws of Large Numbers as well as Hartman–Wintner’s Law of the Iterated Logarithm. In preparation for the Law of the Iterated Logarithm, Cram´er’s theory of large deviations from the Law of Large Numbers is developed in § 1.4. Everything here is very standard, although I feel that my passage from the bounded to the general case of the Law of the Iterated Logarithm has been
xvi
Preface
considerably smoothed by the ideas that I learned during a conversation with M. Ledoux. 2: The whole of Chapter 2 is devoted to the classical Central Limit Theorem. After an initial (and slightly flawed) derivation of the basic result via moment considerations, Lindeberg’s general version is derived in § 2.1. Although Lindeberg’s result has become a sine qua non in the writing of probability texts, the Berry–Esseen estimate has not. Indeed, until recently, the Berry–Esseen estimate required a good many somewhat tedious calculations with characteristic functions (i.e., Fourier transforms), and most recent authors seem to have decided that the rewards did not justify the effort. I was inclined to agree with them until P. Diaconis brought to my attention E. Bolthausen’s adaptation of C. Stein’s techniques (the so-called Stein’s method) to give a proof that is not only brief but also, to me, aesthetically pleasing. In any case, no use of Fourier methods is made in the derivation given in § 2.2. On the other hand, Fourier techniques are introduced in § 2.3, where it is shown that even elementary Fourier analytic tools lead to important extensions of the basic Central Limit Theorem to more than one dimension. Finally, in § 2.4, the Central Limit Theorem is applied to the study of Hermite multipliers and (following Wm. Beckner) is used to derive both E. Nelson’s hypercontraction estimate for the Mehler kernel as well as Beckner’s own estimate for the Fourier transform. I am afraid that, with this flagrant example of the sort of thing that does not belong here, I may be trying the patience of my purist colleagues. However, I hope that their indignation will be somewhat assuaged by the fact that the rest of the book is essentially independent of the material in § 2.4. 3: This chapter is devoted to the study of infinitely divisible laws. It begins in § 3.1 with a few refinements (especially The L´evy Continuity Theorem) of the Fourier techniques introduced in § 2.3. These play a role in § 3.2, where the L´evy–Khinchine formula is first derived and then applied to the analysis of stable laws. 4: In Chapter 4 I construct the L´evy processes (a.k.a. independent increment processes) corresponding to infinitely divisible laws. Secton 4.1 provides the requisite information about the pathspace D(RN ) of right-continuous paths with left limits, and § 4.2 gives the construction of L´evy processes with discontinuous paths, the ones corresponding to infinitely divisible laws having no Gaussian part. Finally, in § 4.3 I construct Brownian motion, the L´evy process with continuous paths, following the prescription given by L´evy. 5: Because they are not needed earlier, conditional expectations do not appear until Chapter 5. The advantage gained by this postponement is that, by the time I introduce them, I have an ample supply of examples to which conditioning can be applied; the disadvantage is that, with considerable justice, many probabilists feel that one is not doing probability theory until one is conditioning. Be that as it may, Kolmogorov’s definition is given in § 5.1 and is shown
Preface
xvii
to extend naturally both to σ-finite measure spaces as well as to random variables with values in a Banach space. Section 5.2 presents Doob’s basic theory of real-valued, discrete parameter martingales: Doob’s Inequality, his Stopping Time Theorem, and his Martingale Convergence Theorem. In the last part of § 5.2, I introduce reversed martingales and apply them to DeFinetti’s theory of exchangeable random variables. 6: Chapter 6 opens with extensions of martingale theory in two directions: to σ-finite measures and to random variables with values in a Banach space. The results in § 6.1 are used in § 6.2 to derive Birkhoff’s Individual Ergodic Theorem and a couple of its applications. Finally, in § 6.3 I prove Burkholder’s Inequality for martingales with values in a Hilbert space. The derivation that I give is essentially the same as Burkholder’s second proof, the one that gives optimal constants. 7: Section 7.1 provides a brief introduction to the theory of martingales with a continuous parameter. As anyone at all familiar with the topic knows, anything approaching a full account of this theory requires much more space than a book like this can give it. Thus, I deal with only its most rudimentary aspects, which, fortunately, are sufficient for the applications to Brownian motion that I have in mind. Namely, in § 7.2 I first discuss the intimate relationship between continuous martingales and Brownian motion (L´evy’s martingale characterization of Brownian motion), then derive the simplest (and perhaps most widely applied) case of the Doob–Meyer Decomposition Theory, and finally show what Burkholder’s Inequality looks like for continuous martingales. In the concluding section, § 7.3, the results in §§ 7.1–7.2 are applied to derive the Reflection Principle for Brownian motion. 8: In § 8.1 I formulate the description of Brownian motion in terms of its Gaussian, as opposed to its independent increment, properties. More precisely, following Segal and Gross, I attempt to convince the reader that Wiener measure (i.e., the distribution of Brownian motion) would like to be the standard Gauss measure on the Hilbert space H 1 (RN ) of absolutely continuous paths with a square integrable derivative, but, for technical reasons, cannot live there and has to settle for a Banach space in which H 1 (RN ) is densely embedded. Using Wiener measure as the model, in § 8.2 I show that, at an abstract level, any non-degenerate, centered Gaussian measure on an infinite dimensional, separable Banach space shares the same structure as Wiener measure in the sense that there is always a densely embedded Hilbert space, known as the Cameron– Martin space, for which it would like to be the standard Gaussian measure but on which it does not fit. In order to carry out this program, I need and prove Fernique’s Theorem for Gaussian measures on a Banach space. In § 8.3 I begin by going in the opposite direction, showing how to pass from a Hilbert space H to a Gaussian measure on a Banach space E for which H is the Cameron–Martin space. The rest of § 8.3 gives two applications: one to “pinned Brownian” motion
xviii
Preface
and the second to a very general statement of orthogonal invariance for Gaussian measures. The main goal of § 8.4 is to prove a large deviations result, known as Schilder’s Theorem, for abstract Wiener spaces; and once I have Schilder’s Theorem, I apply it to derive a version of Strassen’s Law of the Iterated Logarithm. Starting with the Ornstein–Uhlenbeck process, I construct in § 8.5 a family of Gaussian measures known in the mathematical physics literature as Euclidean free fields. In the final section, § 8.6, I first show how to construct Banach space– valued Brownian motion and then derive the original form of Strassen’s Law of the Iterated Logarithm in that context. 9: The central topic here is the abstract theory of weak convergence of probability measures on a Polish space. The basic theory is developed in § 9.1. In § 9.2 I apply the theory to prove the existence of regular conditional probability distributions, and in § 9.3 I use it to derive Donsker’s Invariance Principle (i.e., the pathspace statement of the Central Limit Theorem). 10: Chapter 10 is an introduction to the connections between probability theory and partial differential equations. At the beginning of § 10.1 I show that martingale theory provides a link between probability theory and partial differential equations. More precisely, I show how to represent in terms of Wiener integrals solutions to parabolic and elliptic partial differential equations in which the Laplacian is the principal part. In the second part of § 10.1, I use this link to calculate various Wiener integrals. In § 10.2 I introduce the Markov property of Wiener measure and show how it not only allows one to evaluate other Wiener integrals in terms of solutions to elliptic partial differential equations but also enables one to prove interesting facts about solutions to such equations as a consequence of their representation in terms of Wiener integrals. Continuing in the same spirit, I show in § 10.2 how to represent solutions to the Dirichlet problem in terms of Wiener integrals, and in § 10.3 I use Wiener measure to construct and discuss heat kernels related to the Laplacian. 11: The final chapter is an extended example of the way in which probability theory meshes with other branches of analysis, and the example that I have chosen is the marriage between Brownian motion and classical potential theory. Like an ideal marriage, this one is simultaneously intimate and mutually beneficial to both partners. Indeed, the more one knows about it, the more convinced one becomes that the properties of Brownian paths are a perfect reflection of properties of harmonic functions, and vice versa. In any case, in § 11.1 I sharpen the results in § 10.2.3 and show that, in complete generality, the solution to the Dirichlet problem is given by the Wiener integral of the boundary data evaluated at the place where Brownian paths exit from the region. Next, in § 11.2, I discuss the Green function for a region and explain how its existence reflects the recurrence and transience properties of Brownian paths. In preparation for § 11.4, § 11.3 is devoted to the Riesz Decomposition Theorem for excessive functions. Finally, in § 11.4, I discuss the capacity of regions, derive Chung’s representation of the
Preface
xix
capacitory measure in terms of the last place where a Brownian path visits a region, apply the probabilistic interpretation of capacity to give a derivation of Wiener’s test for regularity, and conclude with two asymptotic calculations in which capacity plays a crucial role. Suggestions about the Use of This Book In spite of the realistic assessment contained in the first paragraph of its preface, when I wrote the first edition of this book I harbored the na¨ıve hope that it might become the standard graduate text in probability theory. By the time that I started preparing the second edition, I was significantly older and far less na¨ıve about its prospects. Although the first edition has its admirers, it has done little to dent the sales record of its competitors. In particular, the first edition has seldom been adopted as the text for courses in probability, and I doubt that the second will be either. Nonetheless, I close this preface with a few suggestions for anyone who does choose to base a course on it. I am well aware that, except for those who find their way into the poorly stocked library of some prison camp, few copies of this book will be read from cover to cover. For this reason, I have attempted to organize it in such a way that, with the help of the table of dependence that follows, a reader can select a path which does not require his reading all the sections preceding the information he is seeking. For example, the contents of §§ 1.1–1.2, § 1.4, § 2.1, § 2.3, and § 5.1– 5.2 constitute the backbone of a one semester, graduate level introduction to probability theory. What one attaches to this backbone depends on the speed with which these sections are covered and the content of the courses for which the course is the introduction. If the goal is to prepare the students for a career as a “quant” in what is left of the financial industry, an obvious choice is § 4.3 and as much of Chapter 7 as time permits, thereby giving one’s students a reasonably solid introduction to Brownian motion. On the other hand, if one wants the students to appreciate that white noise is not the only noise that they may encounter in life, one might defer the discussion of Brownian motion and replace it with the material in Chapter 3 and §§ 4.1–4.2. Alternatively, one might use this book in a more advanced course. An introduction to stochastic processes with an emphasis on their relationship to partial differential equations can be constructed out of Chapters 6, 7, 10, and 11, and § 4.3 combined with Chapter 8 could be used to provide background for a course on Gaussian processes. Whatever route one takes through this book, it will be a great help to your students for you to suggest that they consult other texts. Indeed, it is a familiar fact that the third book one reads on a subject is always the most lucid, and so one should suggest at least two other books. Among the many excellent choices available, I mention Wm. Feller’s An Introduction to Probability Theory and Its Applications, Vol. II, and M. Lo´eve’s classic Probability Theory. In addition, for
xx
Preface
background, precision (including accuracy of attribution), and supplementary material, R. Dudley’s Real Analysis and Probability is superb.
Table of Dependence
§§11.1–11.4
§§10.1–10.3
§§7.4–7.5
§§8.1–8.5
§9.3
§§7.1–7.3
§§3.2–3.3
§§4.1–4.3
§§6.1 & 6.3
§6.2
§2.3
§§3.1 & 3.4
§§5.1 & 5.3
§1.5
§§2.1 & 2.2
§§1.1–1.4
§§9.1–9.2
xxi
Chapter 1 Sums of Independent Random Variables
In one way or another, most probabilistic analysis entails the study of large families of random variables. The key to such analysis is an understanding of the relations among the family members; and of all the possible ways in which members of a family can be related, by far the simplest is when there is no relationship at all! For this reason, I will begin by looking at families of independent random variables. § 1.1 Independence In this section I will introduce Kolmogorov’s way of describing independence and prove a few of its consequences. § 1.1.1. Independent σ-Algebras. Let (Ω, F, P) be a probability space (i.e., Ω is a nonempty set, F is a σ-algebra over Ω, and P is a non-negative measure on the measurable space (Ω, F) having total mass 1), and, for each i from the (non-empty) index set I, let Fi be a sub-σ-algebra of F. I will say that the σ-algebras Fi , i ∈ I, are mutually P-independent, or, less precisely, P-independent, if, for every finite subset {i1 , . . . , in } of distinct elements of I and every choice of Aim ∈ Fim , 1 ≤ m ≤ n, (1.1.1)
P(Ai1 ∩ · · · ∩ Ain ) = P(Ai1 ) · · · P(Ain ).
In particular, if {Ai : i ∈ I} is a family of sets from F, I will say that Ai , i ∈ I, are P-independent if the associated σ-algebras Fi = {∅, Ai , Ai {, Ω}, i ∈ I, are. To gain an appreciation for the intuition on which this definition is based, it is important to notice that independence of the pair A1 and A2 in the present sense is equivalent to P(A1 ∩ A2 ) = P(A1 )P(A2 ), the classical definition that one encounters in elementary treatments. Thus, the notion of independence just introduced is no more than a simple generalization of the classical notion of independent pairs of sets encountered in non-measure theoretic presentations, and therefore the intuition that underlies the elementary notion applies equally well to the definition given here. (See Exercise 1.1.8 for more information about the connection between the present definition and the classical one.) As will become increasing evident as we proceed, infinite families of independent objects possess surprising and beautiful properties. In particular, mutually 1
2
1 Sums of Independent Random Variables
independent σ-algebras tend to fill up space in a sense made precise by the following beautiful thought experiment designed by A.N. Kolmogorov. Let I be any index set, take F∅ = {∅, Ω}, and, for each non-empty subset Λ ⊆ I, let ! FΛ =
_
Fi ≡ σ
i∈Λ
[
Fi
i∈I
S be the σ-algebra generated by i∈Λ Fi (i.e., FΛ is the smallest σ-algebra conS taining i∈Λ Fi ). Next, define the tail σ-algebra T to be the intersection over all finite Λ ⊆ I of the σ-algebras FΛ{ . When I itself is finite, T = {∅, Ω} and is therefore P-trivial in the sense that P(A) ∈ {0, 1} for every A ∈ T . The interesting remark made by Kolmogorov is that even when I is infinite, T is P-trivial whenever the original Fi ’s are P-independent. To see this, for a given non-empty Λ ⊆ I, let CΛ denote the collection of sets of the form Ai1 ∩ · · · Ain where {i1 , . . . , in } are distinct elements of Λ and Aim ∈ Fim for each 1 ≤ m ≤ n. Clearly CΛ is closed under intersection and FΛ = σ(CΛ ). In addition, by assumption, P(A ∩ B) = P(A)P(B) for all A ∈ CΛ and B ∈ CΛ{ . Hence, by Exercise 1.1.12, FΛ is independent of FΛ{ . But this means that T is independent of FF for every finite F ⊆ I, and therefore, again by Exercise 1.1.12, T is independent of [ FI = σ {FF : F a finite subset of Λ} . Since T ⊆ FI , this implies that T is independent of itself ; that is, P(A ∩ B) = P(A)P(B) for all A, B ∈ T . Hence, for every A ∈ T , P(A) = P(A)2 , or, equivalently, P(A) ∈ {0, 1}, and so I have now proved the following famous result. Theorem 1.1.2 (Kolmogorov’s 0–1 Law). Let {Fi : i ∈ I} be a family of P-independent sub-σ-algebras of (Ω, F, P), and define the tail σ-algebra T accordingly, as above. Then, for every A ∈ T , P(A) is either 0 or 1. To develop a feeling for the kind of conclusions that can be drawn from Kolmogorov’s 0–1 Law (cf. Exercises 1.1.18 and 1.1.19 as well), let {An : n ≥ 1} be a sequence of subsets of Ω, and recall the notation
lim An ≡
n→∞
∞ [ \
An = ω : ω ∈ An for infinitely many n ∈ Z+ .
m=1 n≥m
Obviously, limn→∞ An is measurable with respect to the tail field determined by the sequence of σ-algebras {∅, An , An {, Ω}, n ∈ Z+ ; and therefore, if the An ’s are P-independent elements of F, then P lim An ∈ {0, 1}. n→∞
§ 1.1 Independence
3
In words, this conclusion can be summarized as follows: for any sequence of P-independent events An , n ∈ Z+ , either P-almost every ω ∈ Ω is in infinitely many An ’s or P-almost every ω ∈ Ω is in at most finitely many An ’s. A more quantitative statement of this same fact is contained in the second part of the following useful result. Let {An : n ∈ Z+ } ⊆ F be given.
Lemma 1.1.3 (Borel–Cantelli Lemma). Then ∞ X
(1.1.4)
P(An ) < ∞ =⇒ P
n=1
lim An = 0.
n→∞
In fact, if the An ’s are P-independent sets, then ∞ X
(1.1.5)
P(An ) = ∞ ⇐⇒ P
n=1
lim An = 1.
n→∞
(See part (iii) of Exercise 5.2.40 and Lemma 11.4.14 for generalizations.) Proof: The first assertion, which is due to E. Borel, is an easy application of countable additivity. Namely, by countable additivity, [ X An ≤ lim P(An ) = 0 P lim An = lim P n→∞
m→∞
m→∞
n≥m
n≥m
P∞
n=1 P(An ) < ∞. To complete the proof of (1.1.5) when the An ’s are independent, note that, by countable additivity, P limn→∞ An = 1 if and only if
if
\ ∞ \ [ lim An { = 0. lim P An { = P An { = P
m→∞
n→∞
m=1 n≥m
n≥m
But, by independence and another application of countable additivity, for any given m ≥ 1 we have that
P
∞ \ n=m
! An {
= lim
N →∞
N Y n=m
"
1 − P(An ) ≤ lim exp − N →∞
N X
# P An
=0
n=m
P∞ if n=1 P(An ) = ∞. (In the preceding, I have used the trivial inequality 1 − t ≤ e−t , t ∈ [0, ∞).) A second, and perhaps more transparent, way of dealing with the contents of the preceding is to introduce the non-negative random variable N (ω) ∈ Z+ ∪
4
1 Sums of Independent Random Variables
{∞}, that counts theP number of n ∈ Z+ such that ω ∈ An . Then, by Tonelli’s ∞ 1 P Theorem, E [N ] = n=1 P(An ), and so Borel’s contribution is equivalent to the EP [N ] < ∞ =⇒ P(N < ∞) = 1, which is obvious, whereas Cantelli’s contribution is that, for mutually independent An ’s, P(N < ∞) =⇒ EP [N ] < ∞, which is not obvious. § 1.1.2. Independent Functions. Having described what it means for the σalgebras to be P-independent, I will now transfer the notion to random variables on (Ω, F, P). Namely, for each i ∈ I, let Xi be a random variable (i.e., a measurable function on (Ω, F)) with values in the measurable space (Ei , Bi )). I will say that the random variables Xi , i ∈ I, are (mutually) P-independent if the σ-algebras σ(Xi ) = Xi−1 Bi ≡ Xi−1 (Bi ) : Bi ∈ Bi , i ∈ I, are P-independent. If B(E; R) = B (E, B); R denotes the space of bounded measurable R-valued functions on the measurable space (E, B), then it should be clear that P-independence of {Xi : i ∈ I} is equivalent to the statement that EP fi1 ◦ Xi1 · · · fin ◦ Xin = EP fi1 ◦ Xi1 · · · EP fin ◦ Xin for all finite subsets {i1 , . . . , in } of distinct elements of I and all choices of fi1 ∈ B Ei1 ; R , . . . , fin ∈ B Ein ; R . Finally, if 1A given by 1A (ω) ≡
1
if ω ∈ A
0
if ω ∈ /A
denotes the indicator function of the set A ⊆ Ω, notice that the family of sets {Ai : i ∈ I} ⊆ F is P-independent if and only if the random variables 1Ai , i ∈ I, are P-independent. Thus far I have discussed only the abstract notion of independence and have yet to show that the concept is not vacuous. In the modern literature, the standard way to construct lots of independent quantities is to take products of probability spaces. Namely, if E , B , µ is a probability space for each i ∈ I, i i i Q one sets Ω = i∈I Ei ; defines πi : Ω −→ Ei to be the natural projection map W for each i ∈ I; takes Fi = πi−1 (Bi ), i ∈ I, and F = i∈I Fi ; and shows that there is a unique probability measure P on (Ω, F) with the properties that P πi−1 Γi = µi Γi ) 1
for all
i ∈ I and Γi ∈ Bi
Throughout this book, I use EP [X, A] to denote the expected value under P of X over the set R A. That is, EP [X, A] = X dP. Finally, when A = Ω, I will write EP [X]. Tonelli’s Theorem A is the version of Fubini’s Theorem for non-negative functions. Its virtue is that it applies whether or not the integrand is integrable.
§ 1.1 Independence
5
and the σ-algebras Fi , i ∈ I, are P-independent. Although this procedure is extremely powerful, it is rather mechanical. For this reason, I have chosen to defer the details of the product construction to Exercises 1.1.14 and 1.1.16 and to, instead, spend the rest of this section developing a more hands-on approach to constructing independent sequences of real-valued random variables. Indeed, although the product method is more ubiquitous and has become the construction of choice, the one that I am about to present has the advantage that it shows independent random variables can arise “naturally” and even in a familiar places. § 1.1.3. The Rademacher Functions. Until further notice, take (Ω, F) = [0, 1), B[0,1) (when E is a metric space, I use BE to denote the Borel field over E) and P to be the restriction λ[0,1) of Lebesgue measure λR to [0, 1). Next define the Rademacher functions Rn , n ∈ Z+ , on Ω as follows. Take the integer part btc of t ∈ R to be the largest integer dominated by t, and consider the function R : R −→ {−1, 1} given by −1 if t − btc ∈ 0, 12 R(t) = 1 if t − btc ∈ 12 , 1 .
The function Rn is then defined on [0, 1) by Rn (ω) = R 2n−1 ω ,
n ∈ Z+ and ω ∈ [0, 1).
I will now show that the Rademacher functions are P-independent. To this end, first note that every real-valued function f on {−1, 1} is of the form α + βx, x ∈ {−1, 1}, for some pair of real numbers α and β. Thus, all that I have to show is that EP (α1 + β1 R1 ) · · · (αn + βn Rn ) = α1 · · · αn for any n ∈ Z+ and (α1 , β1 ), . . . , (αn , βn ) ∈ R2 . Since this is obvious when n = 1, I will assume that it holds for n and need only check that it must also hold for n + 1, and clearly this comes down to checking that EP F (R1 , . . . , Rn ) Rn+1 = 0 for any F : {−1, 1}n −→ R. But (R1 , . . . , Rn ) is constant on each interval m m+1 , 0 ≤ m < 2n , Im,n ≡ n , 2n 2
whereas Rn+1 integrates to 0 on each Im,n . Hence, by writing the integral over Ω as the sum of integrals over the Im,n ’s, we get the desired result. At this point I have produced a countably infinite sequence of independent Bernoulli random variables (i.e., two-valued random variables whose range is usually either {−1, 1} or {0, 1}) with mean value 0. In order to get more general
6
1 Sums of Independent Random Variables
random variables, I will combine our Bernoulli random variables together in a clever way. Recall that a random variable U is said to be uniformly distributed on the finite interval [a, b] if P(U ≤ t) =
t−a b−a
for t ∈ [a, b].
Lemma 1.1.6. Let {X` : ` ∈ Z+ } be a sequence of P-independent {0, 1}valued Bernoulli random variables with mean value 12 on some probability space (Ω, F, P), and set ∞ X X` . U= 2` `=1
Then U is uniformly distributed on [0, 1]. Proof: Because the assertion only involves properties of distributions, it will be proved in general as soon as I prove it for a particular realization of independent, mean value 12 , {0, 1}-valued Bernoulli random variables. In particular, by the preceding discussion, I need only consider the random variables
n (ω) ≡
1 + Rn (ω) , 2
n ∈ Z+ and ω ∈ [0, 1),
on [0, 1), B[0,1) , λ[0,1) . But, as is easily checked (cf. part (i) of Exercise 1.1.11), P∞ for each ω ∈ [0, 1], ω = n=1 2−n n (ω). Hence, the desired conclusion is trivial in this case. Now let (k, `) ∈ Z+ × Z+ 7−→ n(k, `) ∈ Z+ be any one-to-one mapping of + Z × Z+ onto Z+ , and set Yk,` =
1 + Rn(k,`) , 2
2 (k, `) ∈ Z+ .
Clearly, each Yk,` is a {0, 1}-valued, Bernoulli random variable with mean value 1 + 2 is P-independent. Hence, by Lemma 2 , and the family Yk,` : (k, `) ∈ Z 1.1.6, each of the random variables
Uk ≡
∞ X Yk,` `=1
2`
,
k ∈ Z+ ,
is uniformly distributed on [0, 1). In addition, the Uk ’s are obviously mutually independent. Hence, I have now produced a sequence of mutually independent random variables, each of which is uniformly distributed on [0, 1). To complete our program, I use the time-honored transformation that takes a uniform random
Exercises for § 1.1
7
variable into an arbitrary one. Namely, given a distribution function F on R (i.e., F is a right-continuous, non-decreasing function that tends to 0 at −∞ and 1 at +∞), define F −1 on [0, 1] to be the left-continuous inverse of F . That is, F −1 (t) = inf{s ∈ R : F (s) ≥ t}, t ∈ [0, 1]. (Throughout, the infemum over the empty set is taken to be +∞.) It is then an easy matter to check that when U is uniformly distributed on [0, 1) the random variable X = F −1 ◦ U has distribution function F : P(X ≤ t) = F (t),
t ∈ R.
Hence, after combining this with what we already know, I have now completed the proof of the following theorem. Theorem 1.1.7. Let Ω = [0, 1), F = B[0,1) , and P = λ[0,1) . Then, for any sequence {Fk : k ∈ Z+ } of distribution functions on R, there exists a sequence {Xk : k ∈ Z+ } ofP-independent random variables on (Ω, F, P) with the property that P Xk ≤ t = Fk (t), t ∈ R, for each k ∈ Z+ . Exercises for § 1.1 Exercise 1.1.8. As I pointed out, P A1 ∩ A2 = P A1 )P A2 if and only if the σ-algebra generated by A1 is P-independent of the one generated by A2 . Construct an example to show that the analogous statement is false when dealing with three, instead of two, sets. That is, just because P A1 ∩ A2 ∩ A3 = P A1 P A2 P A3 , show that it is not necessarily true that the three σ-algebras generated by A1 , A2 , and A3 are P-independent. Exercise 1.1.9. This exercise deals with three elementary, but important, properties of independent random variables. Throughout, (Ω, F, P) is a given probability space. (i) Let X1 and X2 be a pair of P-independent random variables with values in the measurable spaces (E1 , B1 ) and (E2 , B2 ), respectively. Given a B1 × B2 measurable function F : E1 × E2 −→ R that is bounded below, use Tonelli’s or Fubini’s Theorem to show that x2 ∈ E2 7−→ f (x2 ) ≡ EP F X1 , x2 ∈ R is B2 -measurable and that EP F X1 , X2 = EP f X2 . (ii) Suppose that X1 , . . . , Xn are P-independent, real-valued random variables. If each of the Xm ’s is P-integrable, show that X1 · · · Xn is also P-integrable and that EP X1 · · · Xn = EP X1 · · · EP Xn . (iii) Let {Xn : n ∈ Z+ } be a sequence of independent random variables taking values in some separable metric space E. If P(X n = x) = 0 for all x ∈ E and n ∈ Z+ , show that P Xm = Xn for some m 6= n = 0.
8
1 Sums of Independent Random Variables
Exercise 1.1.10. As an application of Lemma 1.1.6 and part (ii) of Exercise 1.1.9, prove the identity sin z = z
∞ Y
cos 2−n z
for all z ∈ C.
n=1
Exercise 1.1.11. Define {n (ω) : n ≥ 1} for ω ∈ [0, 1) as in the proof of Lemma 1.1.6. +
Z (i) Show that {P n (ω) : n ≥ 1} is the unique sequence {αn : n ≥ 1} ⊆ {0, 1} n such that ω − m=1 2−m αm < 2−n , and conclude that 1 (ω) = b2ωc and n+1 (ω) = b2n+1 ωc − 2b2n ωc for n ≥ 1.
(ii) Define F : [0, 1) −→ [0, 1)2 by F (ω) =
∞ X
2
−n
2n−1 (ω),
n=1
∞ X
! −n
2
2n (ω) ,
n=1
and show that λ[0,1)2 = F∗ λ[0,1) . That is, λ[0,1) {ω : F (ω) ∈ Γ} = λ2[0,1) (Γ) for all Γ ∈ B[0,1)2 . (iii) Define G : [0, ∞)2 −→ [0, 1) by ∞ X 2n (ω1 ) + n (ω2 ) , G (ω1 , ω2 ) = 4n n=1
and show that λ[0,1) = G∗ λ[0,1)2 . Parts (ii) and (iii) are special cases of a general principle that says, under very general circumstances, measures can be transformed into one another. Exercise 1.1.12. Given a non-empty set Ω, recall2 that a collection C of subsets of Ω is called a π-system if C is closed under finite intersections. At the same time, recall that a collection L is called a λ-system if Ω ∈ L, A ∪ B ∈ L whenever A and B are disjoint members S∞ of L, B \ A ∈ L whenever A and B are members of L with A ⊆ B, and 1 An ∈ L whenever {An : n ≥ 1} is a non-decreasing sequence of members of L. Finally, recall (cf. Lemma 3.1.3 in my Concise Introduction to the Theory of Integration) that if C is a π-system, then the σ-algebra σ(C) generated by C is the smallest L-system L ⊇ C. Show that if C is a π-system and F = σ(C), then two probability measures P and Q are equal on F if they are equal on C. Next use this to see that if {Ci : i ∈ I} is a family of π-systems contained in F and if (1.1.1) holds when the Ai ’s are from the Ci ’s, then the family of σ-algebras {σ(Ci ) : i ∈ I} is independent. 2
See, for example, § 3.1 in the author’s A Concise Introduction to the Theory of Integration, Third Edition, Birkh¨ auser (1998).
Exercises for § 1.1
9
Exercise 1.1.13. In this exercise I discuss two criteria for determining when random variables on the probability space (Ω, F, P) are independent. (i) Let X1 , . . . , Xn be bounded, real-valued random variables. Using Weierstrass’s Approximation Theorem, show that the Xm ’s are P-independent if and only if EP X1m1 · · · Xnmn = EP X1m1 · · · EP Xnmn for all m1 , . . . , mn ∈ N. (ii) Let X : Ω −→ Rm and Y : Ω −→ Rn be random variables. Show that X and Y are P-independent if and only if h√ i P E exp −1 α, X Rm + β, Y Rn
h√ h√ i i P = E exp −1 α, X Rm E exp −1 β, Y Rn P
for all α ∈ Rm and β ∈ Rn . Hint: The only if assertion is obvious. To prove the if assertion, first check that X and Y are independent if EP f (X) g(Y) = EP f (X) EP g(Y) for all f ∈ Cc∞ Rm ; C and g ∈ Cc∞ Rn ; C . Second, given such f and g, apply elementary Fourier analysis to write Z Z √ √ −1 (α,x)Rm ϕ(α) dα and g(y) = e −1 (β,y)Rn ψ(β) dβ, f (x) = e Rm
Rn
where ϕ and ψ are smooth functions with rapidly decreasing (i.e., tending to 0 as |x| → ∞ faster than any power of (1 + |x|)−1 ) derivatives of all orders. Finally, apply Fubini’s Theorem. Exercise 1.1.14. Given a pair of measurable spaces (E1 , B1 ) and (E 2 , B2 ), recall that their product is the measurable space E1 × E2 , B1 × B2 , where B1 × B2 is the σ-algebra over the Cartesian product space E1 × E2 generated by the sets Γ1 × Γ2 , Γi ∈ Bi . Further, recall that, for any probability measures µi on (Ei , Bi ), there is a unique probability measure µ1 × µ2 on E1 × E2 , B1 × B2 such that (µ1 × µ2 ) Γ1 × Γ2 = µ1 (Γ1 )µ2 (Γ2 ) for Γi ∈ Bi . More Q generally, for any n ≥ 2 and measurable spaces {(Ei , Bi ) : 1Q≤ i ≤ n}, one Qn n n takes 1 Bi to be the σ-algebra over 1 Ei generated by the sets 1 Γi , Γi ∈ Bi . Qn+1 Qn+1 Qn In particular, since 1 Ei and 1 Bi can be identified with ( 1 Ei ) ×
10
1 Sums of Independent Random Variables
Qn En+1 and ( 1 Bi ) × Bn+1 , respectively, one can use induction to show that, for every choice measures µi on (Ei , Bi ), there is a unique probability Qn of probability Qn Qn measure 1 µi on ( 1 Ei , 1 Bi ) such that ! n ! n n Y Y Y µi Γi = µi (Γi ), Γi ∈ Bi . 1
1
1
The purpose of this exercise is to generalize the preceding construction to infinite collections. Thus, let I be an infinite index set, and, for each i ∈ I, let (Ei , Bi ) be a measurable 6 Λ ⊆ I, use EΛ to denote the Q space. Given ∅ = Cartesian product space i∈Λ Ei and πQ Λ to denote the natural projection map taking EI onto EΛ . Further, let BI = i∈I Bi stand for the σ-algebra over EI generated by the collection C of subsets ! Y −1 πF Γi , Γi ∈ Bi , i∈F
as F varies over non-empty, finite subsets of I (abbreviated by ∅ = 6 F ⊂⊂ I). In the following steps, I outline a proof that, for every choice of Q probability measures µ on the (E , B )’s, there is a unique probability measure i i i i∈I µi on EI , BI with the property that ! !! Y Y Y −1 (1.1.15) µi πF Γi = µi Γi , Γi ∈ Bi , i∈I
i∈F
i∈F
for every ∅ = 6 F ⊂⊂ I. Not surprisingly, the probability space ! Y Y Y Ei , Bi , µi i∈I
i∈I
i∈I
is called the product over I of the spaces Ei , Bi , µi ; and when all the factors are the same space E, B, µ , it is customary to denote it by E I , B I , µI , and if, in addition, I = {1, . . . , N }, one uses E N , B N , µN . (i) After noting (cf. Exercise 1.1.12) that two probability measures that agree on a π-system agree on the σ-algebra generated bythat π-system, show that there is at most one probability measure on EI , BI that satisfies the condition in (1.1.15). Hence, the problem is purely one of existence. (ii) Let A be the algebra over EI generated by C, and show that there is a finitely additive µ : A −→ [0, 1] with the property that ! Y µ πF−1 ΓF = µi ΓF , ΓF ∈ BF , i∈F
Exercises for § 1.1
11
for all ∅ = 6 F ⊂⊂ I. Hence, all that one has to do is check that µ admits a σ-additive extension to BI , and, by a standard extension theorem, this comes down to checking that µ(An ) & 0 whenever {An : n ≥ 1} ⊆ A and An & ∅. Thus, let {An : n ≥ 1} be a non-increasing sequence from A, and T∞assume that µ(An ) ≥ for some > 0 and all n ∈ Z+ . One must show that 1 An 6= ∅. (iii) Referring to the last part of (ii), show that there is no loss in generality to assume that An = πF−1 ΓFn , where, for each n ∈ Z+ , ∅ = 6 Fn ⊂⊂ I and n ΓFn ∈ BFn . In addition, show that one may assume that F1 = {i1 } and that Fn = Fn−1 ∪ {in }, n ≥ 2, where {in : n ≥ 1} is a sequence of distinct elements of I. Now, make these assumptions, and show that it suffices to find a` ∈ Ei` , ` ∈ Z+ , with the property that, for each m ∈ Z+ , (a1 , . . . , am ) ∈ ΓFm . ( iv) Continuing (iii), for each m, n ∈ Z+ , define gm,n : EFm −→ [0, 1] so that gm,n xFm = 1ΓFn xi1 , . . . , xin
if n ≤ m
and
Z
gm,n xFm = EFn \Fm
1ΓFn xFm , yFn \Fm
n Y
! µi`
dyFn \Fm
if n > m.
`=m+1
After noting that, for each m and n, gm,n+1 ≤ gm,n and Z gm,n xFm = gm+1,n xFm , yim+1 µim+1 dyim+1 , Eim+1
set gm = limn→∞ gm,n and conclude that Z gm xFm = gm+1 xFm , yim+1 µim+1 dyim+1 . Eim+1
In addition, note that Z Z g1 xi1 µi1 dxi1 = lim Ei1
n→∞
g1,n xi1 µi1 dxi1
Ei1
= lim µ(An ) ≥ , n→∞
and proceed by induction to produce a` ∈ Ei` , ` ∈ Z+ , so that gm (a1 , . . . , am ) ≥ for all m ∈ Z+ . Finally, check that {am : m ≥ 1} is a sequence of the sort for which we were looking at the end of part (iii).
12
1 Sums of Independent Random Variables
Exercise 1.1.16. Recall that if Φ is a measurable map from one measurable space (E, B) into a second one (E 0 , B 0 ), then the distribution of Φ under a measure µ on (E, B) is the pushforward measure Φ∗ µ (also denoted by µ◦Φ−1 ) defined on (E 0 , B 0 ) by Φ∗ µ(Γ) = µ Φ−1 (Γ) for Γ ∈ B 0 . Given a non-empty index set I and, for each i ∈ I, a measurable space (Ei , Bi ) and an Ei -valued random variable Xi on the probability space (Ω, F, P), define Q X : Ω −→ i∈I Ei so that X(ω)i = Xi (ω) for each i ∈ I and ω ∈ Ω. Show that XQ i : i ∈ I is a family of P-independent random variables if and only if X∗ P = i∈I (Xi )∗ P. In particular, given probability measures µi on (Ei , Bi ), set Y Y Y Ω= Ei , F = Bi , P = µi , i∈I
i∈I
i∈I
let Xi : Ω −→ Ei be the natural projection map from Ω onto Ei , and show that {Xi : i ∈ I} is a family of mutually P-independent random variables such that, for each i ∈ I, Xi has distribution µi . Exercise 1.1.17. Although it does not entail infinite product spaces, an interesting example of the way in which the preceding type of construction can be effectively applied is provided by the following elementary version of a coupling argument. (i) Let (Ω, B, P) be a probability space and X and Y a pair of P-square integrable R-valued random variables with the property that X(ω) − X(ω 0 ) Y (ω) − Y (ω 0 ) ≥ 0 for all (ω, ω 0 ) ∈ Ω2 . Show that EP X Y ≥ EP [X] EP [Y ]. Hint: Define Xi and Yi on Ω2 for i ∈ {1, 2} so that Xi (ω) = X(ωi ) and Yi (ω) = Y (ωi ) when ω = (ω1 , ω2 ), and integrate the inequality 0 ≤ X(ω1 ) − X(ω2 ) Y (ω1 ) − Y (ω2 ) = X1 (ω) − X2 (ω) Y1 (ω) − Y2 (ω) with respect to P2 . (ii) Suppose that n ∈ Z+ and that f and g are R-valued, Borel measurable functions on Rn that are non-decreasing with respect to each coordinate (separately). Show that if X = X1 , . . . , Xn is an Rn -valued random variable on a probability space (Ω, B, P) whose coordinates are mutually P-independent, then EP f (X) g(X) ≥ EP f (X) EP g(X) so long as f (X) and g(X) are both P-square integrable.
Exercises for § 1.1
13
Hint: First check that the case when n = 1 reduces to an application of (i). Next, describe the general case in terms of a multiple integral, apply Fubini’s Theorem, and make repeated use of the case when n = 1. Exercise 1.1.18. A σ-algebra is said to be countably generated if it contains a countable collection of sets that generate it. The purpose of this exercise is to show that just because a σ-algebra is itself countably generated does not mean that all its sub-σ-algebras are. Let (Ω, F, P) be a probability space and {An : n ∈ Z+ ⊆ F a sequence of P-independent sub-subsets of F with the property that α ≤ P(An ) ≤ 1 − α for some α ∈ (0, 1). Let Fn be the sub-σ-algebra generated by An . Show that the tail σ-algebra T determined by Fn : n ∈ Z+ cannot be countably generated. Hint: Show that C ∈ T is an atom in T (i.e., B = C whenever B ∈ T \ {∅} is contained in C) only if one can write C = lim Cn ≡ n→∞
∞ \ [
Cn ,
m=1 n≥m
where, for each n ∈ Z+ , Cn equals either An or An {. Conclude that every atom in T must have P-measure 0. Now suppose that T were generated by B` : ` ∈ N . By Kolmogorov’s 0–1 Law, P B` ∈ {0, 1} for every ` ∈ N. Take ˆ` = B
B` B` {
if P B` = 1 if P B` = 0
and set
C=
\
ˆ` . B
`∈N
Note that, on the one hand, P(C) = 1, while, on the other hand, C is an atom in T and therefore has probability 0. Exercise 1.1.19. Here is an interesting application of Kolmogorov’s 0–1 Law to a property of the real numbers. (i) Referring to the discussion preceding Lemma 1.1.6 and part (i) of Exercise 1.1.11, define the transformations Tn : [0, 1) −→ [0, 1) for n ∈ Z+ so that Tn (ω) = ω −
Rn (ω) , 2n
ω ∈ [0, 1),
and notice (cf. the proof of Lemma 1.1.6) that Tn (ω) simply flips the nth coefficient in the binary expansion ω. Next, let Γ ∈ B[0,1) , and show that Γ is measurable with respect to the σ-algebra σ {Rn : n > m} generated by {Rn : n > m} if and only if Tn (Γ) = Γ for each 1 ≤ n ≤ m. In particular, conclude that λ[0,1) (Γ) ∈ {0, 1} if Tn Γ = Γ for every n ∈ Z+ .
14
1 Sums of Independent Random Variables
(ii) Let F denote the set of all finite subsets of Z+ , and for each F ∈ F, define T F : [0, 1) −→ [0, 1) so that T ∅ is the identity mapping and T F ∪{m} = T F ◦ Tm
for each F ∈ F and m ∈ Z+ \ F.
As an application of (i), show that for every Γ ∈ B[0,1) with λ[0,1) (Γ) > 0, ! [ λ[0,1) T F (Γ) = 1. F ∈F
In particular, this means that if Γ has positive measure, then almost every ω ∈ [0, 1) can be moved to Γ by flipping a finite number of the coefficients in the binary expansion of ω. § 1.2 The Weak Law of Large Numbers Starting with this section, and for the rest of this chapter, I will be studying what happens when one averages independent, real-valued random variables. The remarkable fact, which will be confirmed repeatedly, is that the limiting behavior of such averages depends hardly at all on the variables involved. Intuitively, one can explain this phenomenon by pretending that the random variables are building blocks that, in the averaging process, first get homothetically shrunk and then reassembled according to a regular pattern. Hence, by the time that one passes to the limit, the peculiarities of the original blocks get lost. Throughout the discussion, (Ω, F, P) will be a probability space on which there is a sequence {Xn : n ≥ 1} of real-valued random variables. Given n ∈ Z+ , use Sn to denote the partial sum X1 + · · · + Xn and S n to denote the average: n 1X Sn X` . = n n `=1
§ 1.2.1. Orthogonal Random Variables. My first result is a very general one; in fact, it even applies to random variables that are not necessarily independent and do not necessarily have mean 0. Lemma 1.2.1. Assume that EP Xn2 < ∞ for n ∈ Z+
and EP Xk X` = 0 if k 6= `.
Then, for each > 0, n 2 1 X P 2 E X` for n ∈ Z+ . (1.2.2) 2 P S n ≥ ≤ EP S n = 2 n `=1
In particular, if M ≡ sup EP Xn2 < ∞, n∈Z+
then
2 M , n ∈ Z+ and > 0; 2 P S n ≥ ≤ EP S n ≤ n and so S n −→ 0 in L2 (P; R) and therefore also in P-probability. (1.2.3)
§ 1.2 The Weak Law of Large Numbers
15
Proof: To prove the equality in (1.2.2), note that, by orthogonality, n X EP Sn2 = EP X`2 . `=1
The rest is just an application of Chebyshev’s inequality, the estimate that results after integrating the inequality 2 1[,∞) |Y | ≤ Y 2 1[,∞) |Y | ≤ Y 2 for any random variable Y . § 1.2.2. Independent Random Variables. Although Lemma 1.2.1 does not use independence, independent random variables provide a ready source of orthogonal functions. To wit, recall that for any P-square integrable random variable X, its variance Var(X) satisfies h 2 i 2 Var(X) ≡ EP X − EP [X] = EP X 2 − EP [X] ≤ EP X 2 . In particular, if the random variables Xn , n ∈ Z+ , are P-square integrable and P-independent, then the random variables ˆ n ≡ Xn − EP Xn , n ∈ Z+ , X are still P-square integrable, have mean value 0, and therefore are orthogonal. Hence, the following statement is an immediate consequence of Lemma 1.2.1. Theorem 1.2.4. Let Xn : n ∈ Z+ be a sequence of P-independent, P-square integrable random variables with mean value m and variance dominated by σ 2 . Then, for every n ∈ Z+ and > 0, h 2 i σ 2 . ≤ (1.2.5) 2 P S n − m ≥ ≤ EP S n − m n In particular, S n −→ m in L2 (P; R) and therefore in P-probability.
As yet I have made only minimal use of independence: all that I have done is subtract off the mean of independent random variables and thereby made them orthogonal. In order to bring the full force of independence into play, one has to exploit the fact that one can compose independent random variables with any (measurable) functions without destroying their independence; in particular, truncating independent random variables does not destroy independence. To see how such a property can be brought to bear, I will now consider the problem of extending the last part of Theorem 1.2.4 to Xn ’s that are less than P-square integrable. In order to understand the statement, recall that a family of random variables Xi : i ∈ I is said to be uniformly P-integrable if h i lim sup EP Xi , Xi ≥ R = 0. R%∞ i∈I
As the proof of the following theorem illustrates, the importance of this condition is that it allows one to simultaneously approximate the random variables Xi , i ∈ I, by bounded random variables.
16
1 Sums of Independent Random Variables
Theorem 1.2.6 (The Weak Law of Large Numbers). Let Xn : n ∈ Z+ be a uniformly P-integrable sequence of P-independent random variables. Then n
1X Xm − EP [Xm ] −→ 0 in L1 (P; R) n 1 and therefore also in P-probability. In particular, if Xn : n ∈ Z+ is a sequence of P-independent, P-integrable random variables that are identically distributed, then S n −→ EP [X1 ] in L1 (P; R) and P-probability. (Cf. Exercise 1.2.11.)
Proof: Without loss in generality, I will assume that EP [Xn ] = 0 for every n ∈ Z+ . For each R ∈ (0, ∞), define fR (t) = t 1[−R,R] (t), t ∈ R, m(R) = EP fR ◦ Xn , n
Xn(R) = fR ◦ Xn − m(R) n ,
and set (R)
Sn
n
=
1 X (R) X` n
and
(R)
Tn
n
=
Since E[Xn ] = 0 =⇒
1 X (R) Y` . n `=1
`=1
(R) mn
and Yn(R) = Xn − Xn(R) ,
= −E Xn , |Xn | > R ,
(R) (R) EP |S n | ≤ EP |S n | + EP |T n | (R) 1 ≤ EP |S n |2 2 + 2 max EP |X` |, |X` | ≥ R 1≤`≤n
R EP |X` |, |X` | ≥ R ; ≤ √ + 2 max n `∈Z+ and therefore, for each R > 0,
lim EP |S n | ≤ 2 sup EP |X` |, |X` | ≥ R .
n→∞
`∈Z+
Hence, because the X` ’s are uniformly P-integrable, we get the desired convergence in L1 (P; R) by letting R % ∞. § 1.2.3. Approximate Identities. The name of Theorem 1.2.6 comes from a somewhat invidious comparison with the result in Theorem 1.4.9. The reason why the appellation weak is not entirely fair is that, although The Weak Law is indeed less refined than the result in Theorem 1.4.9, it is every bit as useful as the one in Theorem 1.4.9 and maybe even more important when it comes to applications. What The Weak Law provides is a ubiquitous technique for constructing an approximate identity (i.e., a sequence of measures that approximate a point mass) and measuring how fast the approximation is taking
§ 1.2 The Weak Law of Large Numbers
17
place. To illustrate how clever selections of the random variables entering The Weak Law can lead to interesting applications, I will spend the rest of this section discussing S. Bernstein’s approach to Weierstrass’s Approximation Theorem. + For a given p ∈ [0, 1], let Xn : n ∈ Z be a sequence of P-independent {0, 1}-valued Bernoulli random variables with mean value p. Then P Sn = ` =
n ` p (1 − p)n−` `
for
0 ≤ ` ≤ n.
Hence, for any f ∈ C [0, 1]; R , the nth Bernstein polynomial n X n ` p` (1 − p)n−` Bn (p; f ) ≡ f n `
(1.2.7)
`=0
of f at p is equal to
EP f ◦ S n . In particular,
f (p) − Bn (p; f ) = EP f (p) − f ◦ S n ≤ EP f (p) − f ◦ S n ≤ 2kf ku P S n − p ≥ + ρ(; f ),
where kf ku is the uniform norm of f (i.e., the supremum of |f | over the domain of f ) and ρ(; f ) ≡ sup |f (t) − f (s)| : 0 ≤ s < t ≤ 1 with t − s ≤ is the modulus of continuity of f . Noting that Var Xn = p(1 − p) ≤ applying (1.2.5), we conclude that, for every > 0,
1 4
and
f (p) − Bn (p; f ) ≤ kf ku + ρ(; f ). u 2n2
In other words, for all n ∈ Z+ , (1.2.8)
f − Bn (· ; f ) u ≤ β(n; f ) ≡ inf
kf ku + ρ(; f ) : > 0 . 2n2
Obviously, (1.2.8) not only shows that, as n → ∞, Bn (· ; f ) −→ f uniformly on [0, 1], it even provides a rate of convergence in terms of the modulus of continuity of f . Thus, we have done more than simply prove Weierstrass’s theorem; we have produced a rather explicit and tractable sequence of approximating polynomials, the sequence Bn (· ; f ) : n ∈ Z+ . Although this sequence is, by no means, the
18
1 Sums of Independent Random Variables
most efficient one,1 as we are about to see, the Bernstein polynomials have a lot to recommend them. In particular, they have the feature that they provide non-negative polynomial approximates to non-negative functions. In fact, the following discussion reveals much deeper non-negativity preservation properties possessed by the Bernstein approximation scheme. In order to bring out the virtues of the Bernstein polynomials, it is important to replace (1.2.7) with an expression in which the coefficients of Bn ( · ; f ) (as polynomials) are clearly displayed. To this end, introduce the difference operator ∆h for h > 0 given by f (t + h) − f (t) . ∆h f (t) = h
A straightforward inductive argument (using Pascal’s Identity for the binomial coefficients) shows that m X m ` m (−h) ∆h f (t) = (−1) f (t + `h) ` m
for m ∈ Z+ ,
`=0
(m)
where ∆h see that
denotes the mth iterate of the operator ∆h . Taking h =
Bn (p; f ) =
n n−` X X nn − `
`
`=0 k=0 n X
k
1 n,
we now
(−1)k f (`h)p`+k
r X n n−` = p (−1)r−` f (`h) ` r − ` r=0 r
`=0
n X
X r n r = (−p) (−1)` f (`h) r ` r=0 r
`=0
=
n X r=0
n (ph)r ∆rh f (0), r
where ∆0h f ≡ f . Hence, we have proved that (1.2.9)
Bn (p; f ) =
n X `=0
n−`
n ` ∆ 1 f (0)p` n `
for
p ∈ [0, 1].
The marked resemblance between the expression on the right-hand side of (1.2.9) and a Taylor polynomial is more than coincidental. To demonstrate how 1
See G.G. Lorentz’s Bernstein Polynomials, Chelsea Publ. Co. (1986) for a lot more information.
§ 1.2 The Weak Law of Large Numbers
19
one can exploit the relationship between the Bernstein and Taylor polynomials, say that a function ϕ ∈ C ∞ (a, b); R is absolutely monotone if its mth derivative Dm ϕ is non-negative for every m ∈ N. Also, say that ϕ ∈ C ∞ [0, 1]; [0, 1]) is a probability generating function if there exists a un : n ∈ N ⊆ [0, 1] such that ∞ ∞ X X un = 1 and ϕ(t) = un tn for t ∈ [0, 1]. n=0
n=0
Obviously, every probability generating function is absolutely monotone on (0, 1). The somewhat surprising (remember that most infinitely differentiable functions do not admit power series expansions) fact which I am about to prove is that, apart from a multiplicative constant, the converse is also true. In fact, one does not need to know, a priori, that the function is smooth so long as it satisfies a discrete version of absolute monotonicity. Theorem 1.2.10. Let ϕ ∈ C [0, 1]; R with ϕ(1) = 1 be given. Then the following are equivalent: (i) ϕ is a probability generating function, (ii) the of ϕ to (0, 1) is absolutely monotone; restriction ϕ (0) ≥ 0 for every n ∈ N and 0 ≤ m ≤ n. (iii) ∆m 1 n
Proof: The implication (i) =⇒ (ii) is trivial. To see that (ii) implies (iii), first observe that if ψ is absolutely monotone on (a, b) and h ∈ (0, b − a), then ∆h ψ is absolutely monotone on (a, b − h). Indeed, because D ◦ ∆h ψ = ∆h ◦ Dψ on (a, b − h), we have that h Dm ◦ ∆h ψ (t) =
Z
t+h
Dm+1 ψ(s) ds ≥ 0,
t ∈ (a, b − h),
t
for any m ∈ N. Returning to the function ϕ, we now know that ∆m h ϕ is absolutely monotone on (0, 1 − mh) for all m ∈ N and h > 0 with mh < 1. In particular, m [∆m h ϕ](0) = lim [∆h ϕ](t) ≥ 0 t&0
and so ∆m h ϕ (0) ≥ 0 when h =
1 n
if
mh < 1,
and 0 ≤ m < n. Moreover, since
[∆n1 ϕ](0) = lim1 [∆nh ϕ](0), n
h% n
we also know that ∆nh ϕ (0) ≥ 0 when h = n1 , and this completes the proof that (ii) implies (iii). Finally, assume that (iii) holds and set ϕn = Bn ( · ; ϕ). Then, from (1.2.9) and the equality ϕn (1) = ϕ(1) = 1, we see that each ϕn is a probability generating function. Thus, in order to complete the proof that (iii) implies (i), all that
20
1 Sums of Independent Random Variables
one has to do is check that a uniform limit of probability generating functions is itself a probability generating function. To this end, write ϕn (t) =
∞ X
un,` t` ,
t ∈ [0, 1] for each n ∈ Z+ .
`=0
Because the un,` ’s are all elements of [0, 1], one can use a diagonalization procedure to choose {nk : k ∈ Z+ } so that lim unk ,` = u` ∈ [0, 1]
k→∞
exists for each ` ∈ N. But, by Lebesgue’s Dominated Convergence Theorem, this means that ϕ(t) = lim ϕnk (t) = k→∞
∞ X
u` t`
for every t ∈ [0, 1).
`=0
Finally, by the Monotone Convergence Theorem, the preceding extends immediately to t = 1, and so ϕ is a probability generating function. (Notice that the argument just given does not even use the assumed uniform convergence and shows that the pointwise limit of probability generating functions is again a probability generating function.) The preceding is only one of many examples in which The Weak Law leads to useful ways of forming an approximate identity. A second example is given in Exercises 1.2.12 and 1.2.13. My treatment of these is based on that of Wm. Feller.2 Exercises for § 1.2 Exercise 1.2.11. Although, for historical reasons, The Weak Law is usually thought of as a theorem about convergence in P-probability, the forms in which I have presented it are clearly results about convergence in either P-mean or even P-square mean. Thus, it is interesting to discover that one can replace the uniform integrability assumption made in Theorem 1.2.6 with a weak uniform integrability assumption if one is willing to settle for convergence in P-probability. Namely, let X1 , . . . , Xn , . . . be mutually P-independent random variables, assume that F (R) ≡ sup RP |Xn | ≥ R −→ 0 as R % ∞, n∈Z+ 2
Wm. Feller, An Introduction to Probability Theory and Its Applications, Vol. II, Wiley, Series in Probability and Math. Stat. (1968). Feller provides several other similar applications of The Weak Law, including the ones in the following exercises.
Exercises for § 1.2 and set
21
n
i 1 X Ph E X` , |X` | ≤ n , mn = n
n ∈ Z+ .
`=1
Show that, for each > 0, n i 1 X P h 2 X` > n E X , X ≤ n + P max P S n − mn ≥ ≤ ` ` 1≤`≤n (n)2 `=1 Z n 2 F (t) dt + F (n), ≤ 2 n 0 and conclude that S n − mn −→ 0 in P-probability. (See part (ii) of Exercises 1.4.26 and 1.4.27 for a partial converse to this statement.)
Hint: Use the formula Var(Y ) ≤ EP Y 2 = 2
Z
t P |Y | > t dt.
[0,∞)
Exercise 1.2.12. Show that, for each T ∈ [0, ∞) and t ∈ (0, ∞), X (nt)k 1 if T > t = lim e−nt n→∞ k! 0 if T < t. 0≤k≤nT
Hint: Let X1 , . . . , Xn , . . . be P-independent, N-valued Poisson random variables with mean value t. That is, the Xn ’s are P-independent and tk for k ∈ N. P Xn = k = e−t k! Show that Sn is an N-valued Poisson random variable with mean value nt, and conclude that, for each T ∈ [0, ∞) and t ∈ (0, ∞), X (nt)k = P Sn ≤ T . e−nt k! 0≤k≤nT
Exercise 1.2.13. Given a right-continuous function F : [0, ∞) −→ R of bounded variation with F (0) = 0, define its Laplace transform ϕ(λ), λ ∈ [0, ∞), by the Riemann–Stieltjes integral: Z ϕ(λ) = e−λt dF (t). [0,∞)
Using Exercise 1.2.12, show that X (−n)k Dk ϕ (n) −→ F (T ) k!
as n → ∞
k≤nT
for each T ∈ [0, ∞) at which F is continuous. Conclude, in particular, that F can be recovered from its Laplace transform. Although this is not the most practical recovery method, it is distinguished by the fact that it does not involve complex analysis.
22
1 Sums of Independent Random Variables
§ 1.3 Cram´ er’s Theory of Large Deviations From Theorem 1.2.4, we know that if Xn : n ∈ Z+ is a sequence of Pindependent, P-square integrable random variables with mean value 0, and if the averages S n , n ∈ Z+ , are defined accordingly, then, for every > 0, max 1≤m≤n Var(Xm ) , n ∈ Z+ . P S n ≥ ≤ n2 Thus, so long as Var(Xn ) −→ 0 as n → ∞, n the S n ’s are becoming more and more concentrated near 0, and the rate at which this concentration is occurring can be estimated in terms of the variances Var(Xn ). In this section, we will see that, by placing more stringent integrability requirements on the Xn ’s, one can gain more information about the rate at which the S n ’s are concentrating at 0. In all of this analysis, the trick is to see how independence can be combined with 0 mean value to produce unexpected cancellations; and, as a preliminary warm-up exercise, I begin with the following.
Theorem 1.3.1. Let {Xn : n ∈ Z+ } be a sequence of P-independent, Pintegrable random variables with mean value 0, and assume that M4 ≡ sup EP Xn4 < ∞. n∈Z+
Then, for each > 0,
4 3M4 4 P |S n | ≥ ≤ EP S n ≤ 2 , n In particular, S n −→ 0 P-almost surely. (1.3.2)
n ∈ Z+ .
Proof: Obviously, in order to prove (1.3.2), it suffices to check the second inequality, which is equivalent to EP Sn4 ≤ 3M4 n2 . But n X
E Sn4 = P
EP Xm1 · · · Xm4 ,
m1 ,...,m4 =1
and, by Schwarz’s Inequality, each of these terms is dominated by M4 . In addition, of these terms, the only ones that do not vanish have either all their factors the same or two pairs of equal factors. Thus, the number of non-vanishing terms is n + 3n(n − 1) = 3n2 − 2n. Given (1.3.2), the proof of the last part becomes an easy application of the Borel–Cantelli Lemma. Indeed, for any > 0, we know from (1.3.2) that ∞ X P S n ≥ < ∞, n=1
and therefore, by (1.1.4), that P limn→∞ S n ≥ = 0.
§ 1.3 Cram´er’s Theory of Large Deviations
23
Remark 1.3.3. The final assertion in Theorem 1.3.1 is a primitive version of The Strong Law of Large Numbers. Although The Strong Law will be taken up again, and considerably refined, in Section 1.4, the principle on which its proof here was based is an important one: namely, control more moments and you will get better estimates; get better estimates and you will reach more refined conclusions. With the preceding adage in mind, I will devote the rest of this section to examining what one can say when one has all moments at one’s disposal. In fact, from now on, I will be assuming that X1 , . . . , Xn , . . . are independent random variables with common distribution µ having the property that the moment generating function Z (1.3.4) Mµ (ξ) ≡ eξ x µ(dx) < ∞ for all ξ ∈ R. R
Obviously, (1.3.4) is more than sufficient to guarantee that the Xn ’s have moments of all orders. In fact, as an application of Lebesgue’s Dominated Convergence Theorem, one sees that ξ ∈ R 7−→ M (ξ) ∈ (0, ∞) is infinitely differentiable and that Z dn M (0) for all n ∈ N. EP X1n = xn µ(dx) = dξ n R
In the discussion that follows, I will use m and σ 2 to denote, respectively, the common mean value and variance of the Xn ’s. In order to develop some intuition for the considerations that follow, I will first consider an example, which, for many purposes, is the canonical example in probability theory. Namely, let g : R −→ (0, ∞) be the Gauss kernel |y|2 1 , y ∈ R, (1.3.5) g(y) ≡ √ exp − 2 2π
and recall that a random variable X is standard normal if Z P X∈Γ = g(y) dy, Γ ∈ BR . Γ
In spite of their somewhat insultingly bland moniker, standard normal random variables are the building blocks for the most honored family in all of probability theory. Indeed, given m ∈ R and σ ∈ [0, ∞), the random variable Y is said to be normal (or Gaussian) with mean valuem and variance σ 2 (often this is abbreviated by saying that X is an N m, σ 2 -random variable) if and only if the distribution of Y is γm,σ2 , where γm,σ2 is the distribution of the variable σX + m when X is standard normal. That is, Y is an N (m, σ 2 ) random variable if, when σ = 0, P(Y = m) = 1 and, when σ > 0, Z y−m 1 dy for Γ ∈ BR . g P Y ∈Γ = σ Γ σ
24
1 Sums of Independent Random Variables
There are two obvious reasons for the honored position held by Gaussian random variables. In the first place, they certainly have finite moment generating functions. In fact, since 2 Z ξ ξy , ξ ∈ R, e g(y) dy = exp 2 R
it is clear that
σ2 ξ2 . Mγm,σ2 (ξ) = exp ξm + 2
(1.3.6)
Secondly, they add nicely. To be precise, it is a familiar fact from elemenˆ is tary probability theory that if X is an N (m, σ 2 )-random variable and X 2 ˆ an N (m, ˆ σ ˆ )-random variable that is independent of X, then X + X is an N m + m, ˆ σ2 + σ ˆ 2 -random variable. In particular, if X1 , . . . , Xn are mutually independent, standard normal random variables, then S n is an N 0, n1 -random variable. That is, r Z n|y|2 n dy. exp − P Sn ∈ Γ = 2 2π Γ
Thus (cf. Exercise 1.3.16), for any Γ we see that (1.3.7)
h i 1 log P S n ∈ Γ = −ess inf lim n→∞ n
|y|2 : y∈Γ , 2
where the “ess” in (1.3.7) stands for essential and means that what follows is taken modulo a set of measure 0. (Hence, apart from a minus sign, the right2 hand side of (1.3.7) is the greatest number dominated by |y|2 for Lebesgue-almost every y ∈ Γ.) In fact, because
Z
∞
g(y) dy ≤ x−1 g(x) for all x ∈ (0, ∞),
x
we have the rather precise upper bound r n2 2 exp − P |S n | ≥ ≤ 2 nπ2
for > 0.
At the same time, it is clear that, for 0 < < |a|,
r P |S n − a| < ≥
n(|a| + )2 22 n . exp − 2 π
§ 1.3 Cram´er’s Theory of Large Deviations
25
More generally, if the Xn ’s are mutually independent N (m, σ 2 )-random variables, then one finds that
r P |S n − m| ≥ σ ≤
n2 2 exp − 2 nπ2
for > 0;
and, for 0 < < |a| and sufficiently large n’s,
r P |S n − (m + a)| < σ ≥
n(|a| + )2 22 n . exp − 2 π
Of course, in general one cannot hope to know such explicit expressions for the distribution of S n . Nonetheless, on the basis of the preceding, one can start to see what is going on. Namely, when the distribution µ falls off rapidly outside of compacts, averaging n independent random variables with distribution µ has the effect of building an exponentially deep well in which the mean value m lies at the bottom. More precisely, if one believes that the Gaussian random variables are normal in the sense that they are typical, then one should conjecture that, even when the random variables are not normal, the behavior of P S n − m ≥ for large n’s should resemble that of Gaussians with the same variance; and it is in the verification of this conjecture that the moment generating function Mµ plays a central role. Namely, although an expression in terms of µ for the distribution of Sn is seldom readily available, the moment generating function for Sn is easily expressed in terms of Mµ . To wit, as a trivial application of independence, we have EP eξSn = Mµ (ξ)n , ξ ∈ R.
Hence, by Markov’s Inequality applied to eξSn , we see that, for any a ∈ R,
P S n ≥ a ≤ e−nξa Mµ (ξ)n = exp −n ξa − Λµ (ξ) ,
ξ ∈ [0, ∞),
where (1.3.8)
Λµ (ξ) ≡ log Mµ (ξ)
is the logarithmic moment generating function of µ. The preceding relation is one of those lovely situations in which a single quantity is dominated by a whole family of quantities, which means that one should optimize by minimizing over the dominating quantities. Thus, we now have (1.3.9)
" P S n ≥ a ≤ exp −n
sup ξ∈[0,∞)
# ξa − Λµ (ξ) .
26
1 Sums of Independent Random Variables
Notice that (1.3.9) is really very good. For instance, when the Xn ’s are N (m, σ 2 )random variables and σ > 0, then (cf. (1.3.6)) the preceding leads quickly to the estimate n2 P S n − m ≥ ≤ exp − 2 , 2σ
which is essentially the upper bound at which we arrived before. Taking a hint from the preceding, I now introduce the Legendre transform (1.3.10) Iµ (x) ≡ sup ξx − Λµ (ξ) : ξ ∈ R , x ∈ R, of Λµ and, before proceeding further, make some elementary observations about the structure of the functions Λµ and Iµ . Lemma 1.3.11. The function Λµ is infinitely differentiable. In addition, for each ξ ∈ R, the probability measure νξ on R given by Z 1 eξx µ(dx) for Γ ∈ BR νξ (Γ) = Mµ (ξ) Γ
has moments of all orders, Z x νξ (dx) = R
Λ0µ (ξ),
Z
2
Z
x νξ (dx) −
and R
2 x νξ (dx) = Λ00µ (ξ).
R
Next, the function Iµ is a [0, ∞]-valued, lower semicontinuous, convex function that vanishes at m. Moreover, Iµ (x) = sup ξx − Λµ (ξ) : ξ ≥ 0 for x ∈ [m, ∞) and Iµ (x) = sup ξx − Λµ (ξ) : ξ ≤ 0
for x ∈ (−∞, m].
Finally, if α = inf x ∈ R : µ (−∞, x] > 0 and β = sup x ∈ R : µ [x, ∞) > 0 , then Iµ is smooth on (α, β) and identically +∞ off of [α, β]. In fact, either µ({m}) = 1 and α = m = β or m ∈ (α, β), in which case Λ0µ is a smooth, strictly increasing mapping from R onto (α, β), Iµ (x) = Ξµ (x) x − Λµ Ξµ (x) , x ∈ (α, β),
where
Ξµ = Λ0µ
−1
is the inverse of Λ0µ , µ({α}) = e−Iµ (α) if α > −∞, and µ({β}) = e−Iµ (β) if β < ∞.
§ 1.3 Cram´er’s Theory of Large Deviations
27
Proof: For notational convenience, I will drop the subscript “µ” during the proof. Further, note that the smoothness of Λ follows immediately from the positivity and smoothness of M , and the identification of Λ0 (ξ) and Λ00 (ξ) with the mean and variance of νξ is elementary calculus combined with the remark following (1.3.4). Thus, I will concentrate on the properties of the function I. As the pointwise supremum of functions that are linear, I is certainly lower semicontinuous and convex. Also, because Λ(0) = 0, it is obvious that I ≥ 0. Next, by Jensen’s Inequality, Z Λ(ξ) ≥ ξ x µ(dx) = ξ m, R
and, therefore, ξx − Λ(ξ) ≤ 0 if x ≤ m and ξ ≥ 0 or if x ≥ m and ξ ≤ 0. Hence, because I is non-negative, this proves the one-sided extremal characterizations of Iµ (x) depending on whether x ≥ m or x ≤ m. Turning to the final part, note first that there is nothing more to do in the case when µ({m}) = 1. Thus, assume that µ({m}) < 1, in which case it is clear that m ∈ (α, β) and that none of the measures νξ is degenerate (i.e., concentrate at one point). In particular, because Λ00 (ξ) is the variance of the νξ , we know that Λ00 > 0 everywhere. Hence, Λ0 is strictly increasing and therefore admits a smooth inverse Ξ on its image. Furthermore, because Λ0 (ξ) is the mean of νξ , it is clear that the image of Λ0 is contained in (α, β). At the same time, given an x ∈ (α, β), note that Z e−ξx eξy µ(dy) −→ ∞ as |ξ| → ∞, R
and therefore ξ ξx − Λ(ξ) achieves a maximum at some point ξx ∈ R. In addition, by the first derivative test, Λ0 (ξx ) = x, and so ξx = Ξ−1 (x). Finally, suppose that β < ∞. Then Z Z −ξβ ξy e e µ(dy) = e−ξ(β−y) µ(dy) & µ({β}) as ξ → ∞, R
(−∞,β]
and therefore e−I(β) = inf ξ≥0 e−ξβ M (ξ) = µ({β}). Since the same reasoning applies when α > −∞, we are done. Theorem 1.3.12 (Cram´ er’s Theorem). Let {Xn : n ≥ 1} be a sequence of P-independent random variables with common distribution µ, assume R that the associated moment generating function Mµ satisfies (1.3.4), set m = R x µ(dx), and define Iµ accordingly, as in (1.3.10). Then,
P S n ≥ a ≤ e−nIµ (a) P S n ≤ a ≤ e−nIµ (a)
for all a ∈ [m, ∞),
for all a ∈ (−∞, m].
28
1 Sums of Independent Random Variables
Moreover, for a ∈ (α, β) (cf. Lemma 1.3.11), > 0, and n ∈ Z+ , ! h i Λ00µ Ξµ (a) exp −n I (a) + |Ξ (a)| , P S n − a < ≥ 1 − µ µ n2 −1 where Λµ is the function given in (1.3.8) and Ξµ ≡ Λµ 0 .
Proof: To prove the first part, suppose that a ∈ [m, ∞), and apply the second part of Lemma 1.3.11 to see that the exponent in (1.3.9) equals −nIµ (a), and, after replacing {Xn : n ≥ 1} by {−Xn : n ≥ 1}, one also gets the desired estimate when a ≤ m. To prove the lower bound, let a ∈ [m, β) be given, and set ξ = Ξµ (a) ∈ [0, ∞). Next, recall the probability measure νξ described in Lemma 1.3.11, and remember that νξ has mean value a = Λ0µ (ξ) and variance Λ00µ (ξ). Further, if Yn : n ∈ Z+ is a sequence of independent, identically distributed random variables with common distribution νξ , then it is an easy matter to check that, for any n ∈ Z+ and every BRn -measurable F : Rn −→ [0, ∞), h h i i 1 P ξSn E e F X , . . . , X EP F Y1 , . . . , Yn = 1 n . Mµ (ξ)n In particular, if n X Tn , Tn = Y` and T n = n `=1
then, because Iµ (a) = ξa − Λµ (ξ), i h P S n − a < = M (ξ)n EP e−ξTn , T n − a < ≥ e−nξ(a+) M (ξ)n P T n − a < h i = exp −n Iµ (a) + ξ P T n − a < .
But, because the mean value and variance of the Yn ’s are, respectively, a and Λ00µ (ξ), (1.2.5) leads to Λ00 (ξ) µ . P T n − a ≥ ≤ n2
The case when a ∈ (α, m] is handled in the same way.
Results like the ones obtained in Theorem 1.3.12 are examples of a class of results known as large deviations estimates. They are large deviations because the probability of their occurrence is exponentially small. Although large deviation estimates are available in a variety of circumstances,1 in general one has to settle for the cruder sort of information contained in the following. 1
In fact, some people have written entire books on the subject. See, for example, J.-D. Deuschel and D. Stroock, Large Deviations, now available from the A.M.S. in the Chelsea Series.
§ 1.3 Cram´er’s Theory of Large Deviations
29
Corollary 1.3.13. For any Γ ∈ BR ,
h i 1 log P S n ∈ Γ n→∞ n h i 1 log P S n ∈ Γ ≤ − inf Iµ (x). ≤ lim n→∞ n x∈Γ
− inf◦ Iµ (x) ≤ lim x∈Γ
(I use Γ◦ and Γ to denote the interior and closure of a set Γ. Also, recall that I take the infemum over the empty set to be +∞.)
Proof: To prove the upper bound, let Γ be a closed set, and define Γ+ = Γ ∩ [m, ∞) and Γ− = Γ ∩ (−∞, m]. Clearly, P S n ∈ Γ ≤ 2P S n ∈ Γ+ ∨ P S n ∈ Γ− .
Moreover, if Γ+ 6= ∅ and a+ = min{x : x ∈ Γ+ }, then, by Lemma 1.3.11 and Theorem 1.3.12, Iµ (a+ ) = inf Iµ (x) : x ∈ Γ+ and P S n ∈ Γ+ ≤ e−nIµ (a+ ) .
Similarly, if Γ− 6= ∅ and a− = max{x : x ∈ Γ− }, then Iµ (a− ) = inf Iµ (x) : x ∈ Γ− and P S n ∈ Γ− ≤ e−nIµ (a− ) .
Hence, either Γ = ∅, and there is nothing to do anyhow, or P S n ∈ Γ ≤ 2 exp −n inf Iµ (x) : x ∈ Γ , n ∈ Z+ ,
which certainly implies the asserted upper bound. To prove the lower bound, assume that Γ is a non-empty open set. What I have to show is that h i 1 log P S n ∈ Γ ≥ −Iµ (a) lim n→∞ n
for every a ∈ Γ. If a ∈ Γ ∩ (α, β), choose δ > 0 so that (a − δ, a + δ) ⊆ Γ and use the second part of Theorem 1.3.12 to see that
h i 1 log P S n ∈ Γ ≥ −Iµ (a) − Ξµ (a) n→∞ n lim
for every ∈ (0, δ). If a ∈ / [α, β], then Iµ (a) = ∞, and so there is nothing to do. Finally, if a ∈ {α, β}, then µ({a}) = e−Iµ (a) and therefore P S n ∈ Γ ≥ P S n = a ≥ e−nIµ (a) .
30
1 Sums of Independent Random Variables
Remark 1.3.14. The upper bound in Theorem 1.3.12 is often called Chernoff ’s Inequality. The idea underlying its derivation is rather mundane by comparison to the subtle idea underlying the proof of the lower bound. Indeed, it may not be immediately obvious what that idea was! Thus, consider once again the second part of the proof of Theorem 1.3.12. What I had to do is estimate the probability that S n lies in a neighborhood of a. When a is the mean value m, such an estimate is provided by the Weak Law. On the other hand, when a 6= m, the Weak Law for the Xn ’s has very little to contribute. Thus, what I did is replace the original Xn ’s by random variables Yn , n ∈ Z+ , whose mean value is a. Furthermore, the transformation from the Xn ’s to the Yn ’s was sufficiently simple that it was easy to estimate Xn -probabilities in terms of Yn probabilities. Finally, the Weak Law Pn applied to the Yn ’s gave strong information about the rate of approach of n1 `=1 Y` to a.
I close this section by verifying the conjecture (cf. the discussion preceding Lemma 1.3.11) that the Gaussian case is normal. In particular, I want to check that the well around m in which the distribution of S n becomes concentrated looks Gaussian, and, in view of Theorem 1.3.12, this comes down to the following.
Theorem 1.3.15. Let everything be as in Lemma 1.3.11, and assume that the variance σ 2 > 0. There exists a δ ∈ (0, 1] and a K ∈ (0, ∞) such that [m − δ, m + δ] ⊆ (α, β) (cf. Lemma 1.3.11), Λ00µ Ξ(x) ≤ K, Ξµ (x) ≤ K|x − m|,
and
2 Iµ (x) − (x − m) ≤ K|x − m|3 2σ 2
for all x ∈ [m − δ, m + δ]. In particular, if 0 < < δ, then 2 3 − K , P |S n − m| ≥ ≤ 2 exp −n 2σ 2
and if |a − m| < δ and > 0, then |a − m|2 K 2 + K|a − m| + |a − m| . P |S n − a| < ≥ 1 − 2 exp −n 2σ 2 n
Proof: Without loss in generality (cf. Exercise 1.3.17), I will assume that m = 0 and σ 2 = 1. Since, in this case, Λµ (0) = Λ0µ (0) = 0 and Λ00µ (0) = 1, it follows that Ξµ (0) = 0 and Ξ0µ (0) = 1. Hence, we can find an M ∈ (0, ∞) and a δ ∈ (0, 1] with α < −δ < δ < β for which Ξµ (x) − x ≤ M |x|2 and Λµ (ξ) − ξ2 ≤ M |ξ|3 whenever |x| ≤ δ and |ξ| ≤ (M + 1)δ, respectively. In 2 particular, this leads immediately to Ξµ (x) ≤ (M + 1)|x| for |x| ≤ δ, and the estimate for Iµ comes easily from the preceding combined with equation Iµ (x) = Ξ(x)x − Λµ Ξµ (x) .
Exercises for § 1.3
31
Exercises for § 1.3 Exercise 1.3.16. Let E, F, µ be a measure space and f a non-negative, F-measurable function. If either µ(E) < ∞ or f is µ-integrable, show that kf kLp (µ;R) −→ kf kL∞ (µ;R)
as p → ∞.
Hint: Handle the case µ(E) < ∞ first, and treat the case when f ∈ L1 (µ; R) by considering the measure ν(dx) = f (x) µ(dx). Exercise 1.3.17. Referring to the notation used in this section, assume that µ is a non-degenerate (i.e., it is not concentrated at a single point) probability measure on R for which (1.3.4) holds. Next, let m and σ 2 be the mean and variance of µ, use ν to denote the distribution of x ∈ R 7−→
x−m ∈R σ
under µ,
and define Λν , Iν , and Ξν accordingly. Show that Λµ (ξ) = ξm + Λν (σξ), ξ ∈ R, x−m , x ∈ R, Iµ (x) = Iν σ Image Λ0µ = m + σ Image Λ0ν , 1 x−m , x ∈ Image Λ0µ . Ξµ (x) = Ξν σ σ
Exercise 1.3.18. Continue with the same notation as in the preceding. (i) Show that Iν ≤ Iµ if Mµ ≤ Mν . (ii) Show that Iµ (x) =
(x − m)2 , 2σ 2
x ∈ R,
when µ is the N m, σ 2 distribution with σ > 0, and show that Iµ (x) =
b−x b−x x−a x−a , log + log p(b − a) (1 − p)(b − a) b − a b−a
x ∈ (a, b),
when a < b, p ∈ (0, 1), and µ({a}) = 1 − µ({b}) = p. (iii) When µ is the hcentered Bernoulli distribution given by µ {±1} = 12 , show i 2 2 that Mµ (ξ) ≤ exp ξ2 , ξ ∈ R, and conclude that Iµ (x) ≥ x2 , x ∈ R. More
generally, given n ∈ Z+ , {σk : 1 ≤ k ≤ n} ⊆ R, and independent random variables X1 , . . . , Xn with this µ as their common distribution, let ν denote the
32
1 Sums of Independent Random Variables
Pn Pn x2 2 ≡ 1 σk2 . distribution of S ≡ 1 σk Xk and show that Iν (x) ≥ 2Σ 2 , where Σ In particular, conclude that a2 P |S| ≥ a ≤ 2 exp − 2 , a ∈ [0, ∞). 2Σ
Exercise 1.3.19. Although it is not exactly the direction in which I have been going, it seems appropriate to include here a derivation of Stirling’s formula. Namely, recall Euler’s Gamma function: Z (1.3.20) Γ(t) ≡ xt−1 e−x dx, t ∈ (−1, ∞). [0,∞)
The goal of this exercise is to prove that t √ t (1.3.21) Γ(t + 1) ∼ 2πt e
as
t % ∞,
where the tilde “∼” means that the two sides are asymptotic to one another in the sense that their ratio tends to 1. (See Exercise 2.1.16 for another approach.) The first step is to make the problem look like one to which Exercise 1.3.16 is applicable. Thus, make the substitution x = ty, and apply Exercise 1.3.16 to see that ! 1t 1 Z Γ(t + 1) t −→ e−1 . = y t e−ty dy tt+1 [0,∞)
This is, of course, far less than we want to know. Nonetheless, it does show that all the action is going to take place near y = 1 and that the principal factor in −t the asymptotics of Γ(t+1) tt+1 is e . In order to highlight these observations, make the substitution y = z + 1 and obtain Z Γ(t + 1) = (1 + z)t e−tz dz. tt+1 e−t (−1,∞) 2
Before taking the next step, introduce the function R(z) = log(1 + z) − z + z2
for z ∈ (−1, 1), and check that R(z) ≤ 0 if z ∈ (−1, 0] and that |R(z)| ≤ everywhere in (−1, 1). Now let δ ∈ (0, 1) be given, and show that Z −δ t tz tδ 2 −δ t 1 + z e dz ≤ (1 − δ) (1 − δ)e ≤ exp − 2 −1
and Z
∞
h it−1 Z ∞ e−tz dz ≤ 1 + δ e−δ (1 + z)e−z dz δ δ3 tδ 2 . + ≤ 2 exp 1 − 3(1 − δ) 2 1+z
δ
t
|z|3 3(1−|z|)
Exercises for § 1.3
33
tz 2
Next, write (1 + z)t e−tz = e− 2 etR(z) . Then Z Z t −tz tz 2 1+z e dz = e− 2 dz + E(t, δ), |z|≤δ
|z|≤δ
where
Z E(t, δ) =
e−
tz 2 2
etR(z) − 1 dz.
|z|≤δ
Check that Z r Z 2 tδ 2 z2 1 2 2π − tz2 e− 2 dz ≤ 1 e− 2 . dz − e = t− 2 1 |z|≤δ t t2 δ |z|≥t 2 δ
At the same time, show that Z Z 2 − tz2 +|R(z)| dz ≤ t |E(t, δ)| ≤ t |R(z)|e
|z|3 e−
|z|≤δ
|z|≤δ
tz 2 3−5δ 2 3(1−δ)
dz ≤
12(1 − δ) (3 − 5δ)2 t
p as long as δ < 35 . Finally, take δ = 2t−1 log t, and combine these to conclude that there is a C < ∞ such that C Γ(t + 1) − 1 ≤ , t ∈ [1, ∞). √ 2πt t t t e
Exercise 1.3.22. Inspired by T.H. Carne,2 here is a rather different sort of application of large deviation estimates. Namely, the goal is to show that for each n ≥ 2 and 1 ≤ m < n there exists an (m − 1)st order polynomial pm,n with the property that 2 n x − pm,n (x) ≤ 2 exp − m for x ∈ [−1, 1]. 2n
(i) Given a C-valued f on Z, define Af : Z −→ C by f (n + 1) + f (n − 1) , n ∈ Z, 2 and show that, for any n ≥ 1, An f = EP f (Sn ) , where Sn is the sum of n P-independent, {−1, 1}-valued Bernoulli random variables with mean value 0. Af (n) =
2
T.H. Carne, “A transformation formula for Markov chains,” Bull. Sc. Math., 109, pp. 399– 405 (1985). As Carne points out, what he is doing is the discrete analog of Hadamard’s representation, via the Weierstrass transform, of solutions to heat equations in terms of solutions to the wave equations.
34
1 Sums of Independent Random Variables
(ii) Show that, for each z ∈ C, there is a unique sequence {Q(m, z) : m ∈ Z} ⊆ C satisfying Q(0, z) = 1, Q(−m, z) = Q(m, z), and AQ( · , z) (m) = zQ(m, z) for all m ∈ Z. In fact, show that, for each m ∈ Z+ : Q(m, · ) is a polynomial of degree m and Q(m, cos θ) = cos(mθ),
θ ∈ C.
In particular, this means that |Q(n, x)| ≤ 1 for all x ∈ [−1, 1]. (It also means that Q(n, · ) is the nth Chebychev polynomial.) (iii) Using induction on n ∈ Z+ , show that n A Q( · , z) (m) = z n Q(m, z),
m ∈ Z and z ∈ C,
and conclude that h i z n = E Q Sn , z ,
n ∈ Z+
and z ∈ C.
In particular, if h i pm,n (z) ≡ E Q Sn , z), Sn < m = 2−n
X |2`−n|<m
n Q(2` − n, z), `
conclude that (cf. Exercise 1.3.18)
m2 sup |x − pm,n (x)| ≤ P |Sn | ≥ m ≤ 2 exp − 2n x∈[−1,1] n
for all 1 ≤ m ≤ n.
(iv) Suppose that A is a self-adjoint contraction on the real or complex Hilbert space H (i.e., (f, Ag)H = (g, Af )H and kAf kH ≤ kf kH for all f, g ∈ H). Next, assume that f, A` g H = 0 for some f, g ∈ H and each 0 ≤ ` < m. Show that
2 f, An g ≤ 2kf kH kgkH exp − m H 2n
for n ≥ m.
(See Exercise 2.3.30 for an application.) Hint: Note that f, pm,n (A)g H = 0, and use the Spectral Theorem to see that, for any polynomial p, kp(A)f kH ≤
sup |p(x)| kf kH , x∈[−1,1]
f ∈ H.
§ 1.4 The Strong Law of Large Numbers
35
§ 1.4 The Strong Law of Large Numbers In this section I will discuss a few almost sure convergence properties of averages of independent random variables. Thus, once again, {Xn : n ≥ 1} will be a sequence of independent random variables on a probability space Ω, F, P , and Sn and S n will be, respectively, the sum and average of X1 , . . . , Xn . Throughout this section, the reader should notice how much more immediately important a role independence (as opposed to orthogonality) plays than it did 1.2. in Section To get started, I point out that, for both {Sn : n ≥ 1} and S n : n ≥ 1 , the set on which convergence occurs has P-measure either 0 or 1. In fact, we have the following simple application of Kolmogorov’s 0–1 Law (Theorem 1.1.2). + Lemma 1.4.1. For any sequence a : n ∈ Z ⊆ R and any sequence n bn : n ∈ Z+ ⊆ (0, ∞) that converges to an element of (0, ∞], the set on which
lim
n→∞
Sn − an bn
exists in R
has P-measure either 0 or 1. In fact, if bn −→ ∞ as n → ∞, then both
lim
n→∞
Sn − an bn
and
Sn − an bn n→∞ lim
are P-almost surely constant. Proof: Simply observe that all of the events and functions involved can be expressed in terms of {Sm+n − Sm : n ≥ 1} for each m ∈ Z+ and are therefore tail-measurable. The following beautiful statement, which was proved originally by Kolmogorov, is the driving force behind many of the almost sure convergence results about both {Sn : n ≥ 1} and S n : n ≥ 1 .
Theorem 1.4.2. variables, and if
If the Xn ’s are independent, P-square integrable random ∞ X
(1.4.3)
Var Xn ) < ∞,
n=1
then
∞ X
Xn − EP Xn converges P-almost surely.
n=1
Note that, since (1.4.4)
! ∞ n X 1 X Var X` , sup P X` − EP X` ≥ ≤ 2 n≥N `=N
`=N
36
1 Sums of Independent Random Variables
P∞ (1.4.3) certainly implies that the series n=1 Xn − EP Xn converges in Pmeasure. Thus, all that I am attempting to do here is replace a convergence in measure statement with an almost sure one. Obviously, this replacement would be trivial if the “supn≥N ” in (1.4.3) appeared on the other side of P. The remarkable fact which we are about to prove is that, in the present situation, the “supn≥N ” can be brought inside! Theorem 1.4.5 (Kolmogorov’s Inequality). and P-square integrable, then (1.4.6)
If the Xn ’s are independent
n ! ∞ X 1 X Var Xn P sup X` − EP X` ≥ ≤ 2 n=1 n≥1 `=1
for each > 0. (See Exercise 1.4.21 for more information.) Proof: Without loss in generality, assume that each Xn has mean value 0. Given 1 ≤ n < N , note that 2 SN − Sn2 = SN − Sn
2
+ 2 SN − Sn Sn ≥ 2 SN − Sn Sn ;
and therefore, since SN −Sn has mean value 0 and is independent of the σ-algebra σ {X1 , . . . , Xn } , (*)
2 EP SN , An ≥ EP Sn2 , An for any An ∈ σ {X1 , . . . , Xn } .
In particular, if A1 = |S1 | > and n o An+1 = Sn+1 > and max S` ≤ , 1≤`≤n
n ∈ Z+ ,
then, the An ’s are mutually disjoint, BN ≡
N [ max Sn > = An ,
1≤n≤N
n=1
and so (*) implies that N N 2 X 2 X P E SN , BN = E SN , An ≥ EP Sn2 , An P
n=1
≥ 2
N X n=1
n=1
P An = 2 P B N .
§ 1.4 The Strong Law of Large Numbers
37
Thus, ∞ 2 X P sup Sn > = lim 2 P BN ≤ lim EP SN ≤ EP Xn2 , 2
n≥1
N →∞
N →∞
n=1
and so the result follows after one takes left limits with respect to > 0. Proof of Theorem 1.4.2: Again assume that the Xn ’s have mean value 0. By (1.4.6) applied to XN +n : n ∈ Z+ , we see that (1.4.3) implies ∞ 1 X EP Xn2 −→ 0 as N → ∞ P sup Sn − SN ≥ ≤ 2 n>N n=N +1
for every > 0, and this is equivalent to the P-almost sure Cauchy convergence of {Sn : n ≥ 1}. In order to convert the conclusion in Theorem 1.4.2 into a statement about S n : n ≥ 1 , I will need the following elementary summability fact about sequences of real numbers. Lemma 1.4.7 (Kronecker). Let bn : n ∈ Z+ be a non-decreasing sequence of positive numbers that tend to ∞, and set βn = bn − bn−1 , where b0 ≡ 0. If {sn : n ≥ 1} ⊆ R is a sequence that converges to s ∈ R, then n 1 X β` s` −→ s. bn `=1
In particular, if {xn : n ≥ 1} ⊆ R, then n ∞ X 1 X xn x` −→ 0 as n → ∞. converges in R =⇒ bn b n=1 n `=1
Proof: To prove the first part, assume that s = 0, and for given > 0 choose N ∈ Z+ so that |s` | < for ` ≥ N . Then, with M = supn≥1 |sn |, n Mb 1 X N + −→ as n → ∞. β` s` ≤ bn bn `=1 Pn Turning to the second part, set y` = xb`` , s0 = 0, and sn = `=1 y` . After summation by parts, n n 1 X 1 X β` s`−1 ; x` = sn − bn bn `=1
`=1
and so, since sn −→ s ∈ R as n → ∞, the first part gives the desired conclusion. After combining Theorem 1.4.2 with Lemma 1.4.7, we arrive at the following interesting statement.
38
1 Sums of Independent Random Variables
Corollary 1.4.8. Assume that {bn : n ≥ 1} ⊆ (0, ∞) increases to infinity as n → ∞, and suppose that {Xn : n ≥ 1} is a sequence of independent, P-square integrable random variables. If ∞ X Var Xn < ∞, b2n n=1
then
n 1 X X` − EP X` −→ 0 bn
P-almost surely.
`=1
As an immediate consequence of the preceding, we see that S n −→ m P-almost surely if the Xn ’s are identically distributed and P-square integrable. In fact, without very much additional effort, we can also prove the following much more significant refinement of the last part of Theorem 1.3.1. Theorem 1.4.9 (Kolmogorov’s Strong Law). Let Xn : n ∈ Z+ be a sequence of P-independent, identically distributed random variables. If X1 is P-integrable and has mean value m, then, as n → ∞, S n −→ m P-almost surely and in L1 (P; R). Conversely, if S n converges (in R) on a set of positive P-measure, then X1 is P-integrable. P Proof: Assume that X is P-integrable and that E X1 = 0. Next, set Yn = 1 Xn 1[0,n] |Xn | , and note that ∞ X
P Yn 6= Xn =
n=1
≤
∞ X
P |Xn | > n
n=1 ∞ Z n X n=1
P |X1 | > t dt = EP |X1 | < ∞.
n−1
Thus, by the first part of the Borel–Cantelli Lemma, P ∃n ∈ Z+ ∀N ≥ n YN = XN = 1. Pn In particular, if T n = n1 `=1 Y` for n ∈ Z+ , then, for P-almost every ω ∈ Ω, T n (ω) −→ 0 if and only if S n (ω) −→ 0. Finally, to see that T n −→ 0 P-almost surely, first observe that, because EP [X1 ] = 0, by the first part of Lemma 1.4.7, n
1X P E [Y` ] = lim EP X1 , |X1 | ≤ n = 0, n→∞ n→∞ n lim
`=1
and therefore, by Corollary 1.4.8, it suffices for us to check that ∞ X EP [Yn2 ] < ∞. n2 n=1
§ 1.4 The Strong Law of Large Numbers To this end, set C = sup ` `∈Z+
and note that ∞ X EP [Y 2 ] n
n=1
n2
39
∞ X 1 , n2 n=`
n ∞ X 1 X P 2 E X , ` − 1 < |X | ≤ ` = 1 1 n2 n=1 `=1
=
∞ X `=1
≤C
∞ X 1 EP X12 , ` − 1 < |X1 | ≤ ` n2 n=`
∞ X 1 `=1
`
EP X12 , ` − 1 < |X1 | ≤ ` ≤ C EP |X1 | < ∞.
Thus, the P-almost sure convergence is now established, and the L1 (P; R)-convergence result was proved already in Theorem 1.2.6. Turning to the converse assertion, first note that (by Lemma 1.4.1) if S n converges in R on a set of positive P-measure, then it converges P-almost surely to some m ∈ R. In particular,
|Xn | = lim S n − S n−1 = 0 P-almost surely; n→∞ n→∞ n and so, if An ≡ |Xn | > n , then P limn→∞ An = 0. But the An ’s are mutually independent, and by the second part of the Borel–Cantelli Lemma, we Ptherefore, ∞ now know that n=1 P An < ∞. Hence, Z ∞ ∞ X P E |X1 | = P |X1 | > t dt ≤ 1 + P |Xn | > n < ∞. lim
0
n=1
Remark 1.4.10. A reason for being interested in the converse part of Theorem 1.4.9 is that it provides a reconciliation between the measure theory vs. frequency schools of probability theory. Although Theorem 1.4.9 is the centerpiece of this section, I want to give another approach to the study of the almost sure convergence properties of {Sn : n ≥ 1}. In fact, following P. L´evy, I am going to show that {Sn : n ≥ 1} converges P-almost surely if it converges in P-measure. Hence, for example, Theorem 1.4.2 can be proved as a direct consequence of (1.4.4), without appeal to Kolmogorov’s Inequality. The key to L´evy’s analysis lies in a version of the reflection principle, whose statement requires the introduction of a new concept. Given an R-valued random variable Y , say that α ∈ R is a median of Y and write α ∈ med(Y ), if (1.4.11) P Y ≤ α ∧ P Y ≥ α ≥ 12 .
40
1 Sums of Independent Random Variables
Notice that (as distinguished from a mean value) every Y admits a median; for example, it is easy to check that
α ≡ inf t ∈ R : P Y ≤ t ≥ 12 is a median of Y . In addition, it is clear that med(−Y ) = −med(Y )
and
med (β + Y ) = β + med (Y ) for all β ∈ R.
On the other hand, the notion of median is flawed by the fact that, in general, a random variable will admit an entire non-degenerate interval of medians. In addition, it is neither easy to compute the medians of a sum in terms of the medians of the summands nor to relate the medians of an integrable random variable to its mean value. Nonetheless, at least if Y ∈ Lp (P; R) for some p ∈ [1, ∞), the following estimate provides some information. Namely, since, for α ∈ med(Y ) and β ∈ R,
|α − β|p ≤ |α − β|p P Y ≥ α ∧ P Y ≤ α ≤ EP |Y − β|p , 2 we see that, for any p ∈ [1, ∞) and Y ∈ Lp (P; R), p1 for all β ∈ R and α ∈ med (Y ). |α − β| ≤ 2EP |Y − β|p
In particular, if Y ∈ L2 (P ) and m is the mean value of Y , then (1.4.12)
|α − m| ≤
p
2Var(Y )
for all α ∈ med(Y ).
Theorem 1.4.13 (L´ evy’s Reflection Principle). Let Xn : n ∈ Z+ be a sequence of P-independent random variables, and, for k ≤ `, choose α`,k ∈ med S` − Sk . Then, for any N ∈ Z+ and > 0, (1.4.14)
max Sn + αN,n ≥
P
1≤n≤N
≤ 2P SN ≥ ,
and therefore (1.4.15)
P
max Sn + αN,n ≥
1≤n≤N
≤ 2P |SN | ≥ .
Proof: Clearly (1.4.15) follows by applying (1.4.14) to both the sequences Xn : n ≥ 1} and {−Xn : n ≥ 1} and then adding the two results.
§ 1.4 The Strong Law of Large Numbers
41
To prove (1.4.14), set A1 = S1 + αN,1 ≥ and An+1 = max S` + αN,` < and Sn+1 + αN,n+1 ≥ 1≤`≤n
for 1 ≤ n < N . Obviously, the An ’s are mutually disjoint and N [
An =
n=1
max Sn + αN,n ≥ .
1≤n≤N
In addition, {SN ≥ ⊇ An ∩ SN − Sn ≥ αN,n
for each 1 ≤ n ≤ N.
Hence, N X P SN ≥ ≥ P An ∩ SN − Sn ≥ αN,n n=1
N 1 1X P An = P max Sn + αN,n ≥ , ≥ 1≤n≤N 2 2 n=1 where, in the passage to the last line, I have used the independence of the sets An and SN − Sn ≥ αN,n . + Corollary 1.4.16. Let X : n ∈ Z be n a sequence of independent random + variables, and assume that Sn : n ∈ Z converges in P-measure to an Rvalued random variable S. Then Sn −→ S P-almost surely. (Cf. Exercise 1.4.25 as well.) Proof: What I must show is that, for each > 0, there is an M ∈ Z+ such that sup P max Sn+M − SM ≥ < . N ≥1
1≤n≤N
To this end, let 0 < < 1 be given, and choose M ∈ Z+ so that for all 1 ≤ k < n. < P Sn+M − Sk+M ≥ 2 2 Next, for a given N ∈ Z+ , choose αN,n ∈ med SM +N − SM +n for 0 ≤ n ≤ N . Then |αN,n | ≤ 2 , and so, by (1.4.15) applied to {XM +n : n ≥ 1}, P max SM +n − SM ≥ ≤ P max SM +n − SM + αN,n ≥ 1≤n≤N 1≤n≤N 2 < . ≤ 2P SM +N − SM ≥ 2
42
1 Sums of Independent Random Variables
Remark 1.4.17. The most beautiful and startling feature of L´evy’s line of reasoning is that it requires no integrability assumptions. Of course, in many applications of Corollary 1.4.16, integrability considerations enter into the proof that {Sn : n ≥ 1} converges in P-measure. Finally, a word of caution may be in order. Namely, the result in Corollary 1.4.16 applies to the quantities Sn themselves; it does not apply to associated quantities like S n . Indeed, suppose that {Xn : n ≥ 1} is a sequence of independent, identically distributed random variables that satisfy
− 12 P Xn ≤ −t = P Xn ≥ t = 1 + t2 log e4 + t2
for all t ≥ 0.
On the one hand, by Exercise 1.2.11, we know that the associated averages S n tend to 0 in probability. On the other hand, by the second part of Theorem 1.4.9, we know that the sequence S n : n ≥ 1 diverges almost surely.
Exercises for § 1.4 Exercise 1.4.18. Let X and Y be non-negative random variables, and suppose that
i 1 h P X ≥ t ≤ EP Y, X ≥ t , t
(1.4.19)
t ∈ (0, ∞).
Show that
(1.4.20)
p P p p1 , E Y p−1
p1 ≤ EP X p
p ∈ (1, ∞).
Hint: First, reduce to the case when X is bounded. Next, recall that, for any measure space E, F, µ , any non-negative, measurable f on E, F , and any α ∈ (0, ∞), Z
Z
α
α−1
f (x) µ(dx) = α E
t
Z
µ f > t dt = α
(0,∞)
tα−1 µ f ≥ t dt.
(0,∞)
Use this together with (1.4.19) to justify the relation E Xp ≤ p
Z
P
tp−2 EP Y, X ≥ t dt
(0,∞)
" Z = pE Y P
0
#
X
t
p−2
dt =
p EP X p−1 Y , p−1
and arrive at (1.4.20) after an application of H¨ older’s Inequality.
Exercises for § 1.4
43
Exercise 1.4.21. Let {Xn : n ≥ 1} be a sequence of mutually independent, integrable random variables with mean value 0, and assume that 2 P∞ P-square E X < ∞. Let S denote the random variable (guaranteed by Theorem n 1 1.4.2) to which {Sn : n ≥ 1} converges P-almost surely, and, using elementary orthogonality considerations, check that Sn −→ S in L2 (P; R) as well. Next, after examining the proof of Kolmogorov’s Inequality (cf. (1.4.6)), show that 2 2 1 P 2 P sup Sn ≥ t ≤ E S , sup Sn ≥ t , t > 0. t n∈Z+ n∈Z+
Finally, by applying (1.4.20), show that p h i 2p 2p p P EP S , (1.4.22) E sup Sn ≤ p−1
p ∈ (1, ∞),
n∈Z+
and conclude from this that, for each p ∈ (2, ∞), {Sn : n ≥ 1} converges to S in Lp (P ) if and only if S ∈ Lp (P ). Exercise 1.4.23. If X ∈ L2 (P; R), then it is easy to characterize its mean m as the c ∈ R that minimizes EP (X − c)2 . Assuming that X ∈ L1 (P; R), show that α ∈ med(X) if and only if EP |X − α| = min EP |X − c| . c∈R
Hint: Show that, for any a, b ∈ R, E |X − b| − EP |X − a| =
Z
P
b
P(X ≤ t) − P(X ≥ t) dt.
a
Exercise 1.4.24. Let {Xn : n ≥ 1} be a sequence of P-square integrable random variables that converges in probability to a random variable X, and assume that supn≥1 Var(Xn ) < ∞. Show that X is square integrable and that EP |Xn − X| −→ 0. In particular, if, in addition, Var(Xn ) −→ Var(X), show that EP |Xn − X|2 −→ 0. Hint: Let αn ∈ med(Xn ), and show that α+ = limn→∞ αn and α− = limn→∞ αn are both of med(X). Combine this with (1.4.12) to conclude that elements supn≥1 EP [Xn ] < ∞ and therefore that supn≥1 EP [X 2 ] < ∞.
Exercise 1.4.25. The following variant of Theorem 1.4.13 is sometimes useful and has the advantage that it avoids the introduction of medians. Namely, show that, for any t ∈ (0, ∞) and n ∈ Z+ , P |Sn | > t . P max |Sn | ≥ 2t ≤ 1≤m≤n 1 − max P |Sn − Sm | > t 1≤m≤n
Note that this can be used in place of (1.4.15) when proving results like the one in Corollary 1.4.16.
44
1 Sums of Independent Random Variables
Exercise 1.4.26. A random variable X is said to be symmetric if −X has the same distribution as X itself. Obviously, the most natural choice of median for a symmetric random variable is 0; and thus, because sums of independent, symmetric random variables are again symmetric, (1.4.14) and (1.4.15) are particularly useful when the Xn ’s are symmetric, since the α`,k ’s can then be taken to be 0. In this connection, consider the following interesting variation on the theme of Theorem 1.4.13. (i) Let X1 , . . . , Xn , . . . be independent, symmetric random variables, set Mn (ω) = |X` (ω)|, let τn (ω) be the smallest 1 ≤ ` ≤ n with the property that max1≤`≤n X` (ω) = Mn (ω), and define Yn (ω) = Xτn (ω) (ω)
and Sˆn = Sn − Yn .
Show that ω ∈ Ω 7−→ Sˆn (ω), Yn (ω) ∈ R2
and ω ∈ Ω 7−→ −Sˆn (ω), Yn (ω) ∈ R2
have the same distribution, and conclude first that P Yn ≥ t ≤ P Yn ≥ t & Sˆn ≥ 0 + P Yn ≥ t & Sˆn ≤ 0 = 2P Yn ≥ t & Sˆn ≥ 0 ≤ 2P Sn ≥ t , for all t ∈ R, and then that P max X` ≥ t ≤ 2P Sn ≥ t , 1≤`≤n
t ∈ [0, ∞).
(ii) Continuing in the same setting, add the assumption that the Xn ’s are identically distributed, and use part (i) to show that
lim P |S n | ≤ C = 1
n→∞
for some C ∈ (0, ∞) =⇒ lim nP |X1 | ≥ n = 0. n→∞
Hint: Note that P and that
1−(1−x)n x
max |X` | > t = 1 − P(|X1 | ≤ t)n
1≤`≤n
−→ n as x & 0.
In conjunction with Exercise 1.2.11, this proves that if {Xn : n ≥ 1} is a sequence of independent, identically distributed symmetric random variables, then S n −→ 0 in P-probability if and only if limn→∞ nP |X1 | ≥ n = 0.
Exercises for § 1.4
45
Exercise 1.4.27. Let X and X 0 be a pair of independent random variables that have the same distribution, let α be a median of X, and set Y = X − X 0 . (i) Show that Y is symmetric and that P |X − α| ≥ t ≤ 2P |Y | ≥ t
for all
t ∈ [0, ∞),
and conclude that, for any p ∈ (0, ∞),
1 1 1 2− p ∨1 EP |Y |p p ≤ 2EP |Y |p p + |α| . In particular, |X|p is integrable if and only if |Y |p is. (ii) The result in (i) leads to my final refinement of The Weak Law of Large Numbers. Namely, let {Xn : n ≥ 1} be a sequence of independent, identically distributed random variables. By combining Exercise 1.2.11, part (ii) in Exercise 1.4.26, and part (i) above, show that1
lim P S n ≤ C = 1
for some C ∈ (0, ∞) =⇒ lim nP |X1 | ≥ n = 0 n→∞ =⇒ S n − EP X1 , |X1 | ≤ n −→ 0 in P-probability. n→∞
Exercise 1.4.28. Let {Xn : n ≥ 1} be a sequence of mutually independent, identically distributed, P-integrable random variables with mean value m. As we already know, when m > 0, the partial sums Sn tend, P-almost surely, to +∞ at an asymptotic linear rate m; and, of course, when m < 0, the situation is similar at −∞. On the other hand, when m = 0, we know that, if |Sn | tends to ∞ at all, then, P-almost surely, it does so at a strictly sublinear rate. In this exercise, you are to sharpen this statement by proving that m = 0 =⇒ lim |Sn | < ∞ P-almost surely. n→∞
The beautiful argument given below is due to Y. Guivarc’h, but its full power cannot be appreciated in the present context (cf. Exercise 6.2.19). Furthermore, a classic result (cf. Exercise 5.2.43) due to K.L. Chung and W.H. Fuchs gives a much better result for the independent random variables. Their result says that limn→∞ |Sn | = 0 P-almost surely. In order to prove the assertion here, assume that limn→∞ |Sn | = ∞ with positive P-probability, use Kolmogorov’s 0–1 Law to see that |Sn | −→ ∞ Palmost surely, and proceed as follows. 1
These ideas are taken from the book by Wm. Feller cited at the end of § 1.2. They become even more elegant when combined with a theorem due to E.J.G. Pitman, which is given in Feller’s book.
46
1 Sums of Independent Random Variables
(i) Show that there must exist an > 0 with the property that P ∀` > k S` − Sk ≥ ≥ for some k ∈ Z+ and therefore that P(A) ≥ ,
where A ≡ ω : ∀` ∈ Z+ S` (ω) ≥ .
(ii) For each ω ∈ Ω and n ∈ Z+ , set o n Γn (ω) = t ∈ R : ∃1 ≤ ` ≤ n t − S` (ω) < 2
and
o n Γn0 (ω) = t ∈ R : ∃1 ≤ ` ≤ n t − S`0 (ω) < 2 ,
Pn where Sn0 ≡ `=1 X`+1 . Next, let Rn (ω) and Rn0 (ω) denote the Lebesgue measure of Γn (ω) and Γn0 (ω), respectively; and, using the translation invariance of Lebesgue measure, show that Rn+1 (ω) − Rn0 (ω) ≥ 1A0 (ω), where A0 ≡ ω : ∀` ≥ 2 S` (ω) − S1 (ω) ≥ . On the other hand, show that EP Rn0 = EP Rn
and P(A0 ) = P(A),
and conclude first that P(A) ≤ EP Rn+1 − Rn , and then that
n ∈ Z+ ,
1 P E Rn . n→∞ n
P(A) ≤ lim
(iii) In view of parts (i) and (ii), what remains to be done is show that m = 0 =⇒ lim
n→∞
1 P E Rn = 0. n
But, clearly, 0 ≤ Rn (ω) ≤ n. Thus, it is enough to show that, when m = 0, Rn n −→ 0 P-almost surely; and, to this end, first check that
Rn (ω) Sn (ω) −→ 0, −→ 0 =⇒ n n
and, finally, apply The Strong Law of Large Numbers.
Exercises for § 1.4
47
Exercise 1.4.29. As I have already said, for many applications The Weak Law of Large Numbers is just as good as and even preferable to the Strong Law. Nonetheless, here is an application in which the full strength of the Strong Law plays an essential role. Namely, I want to use the Strong Law to produce examples of continuous, strictly increasing functions F on [0, 1] with the property that their derivative F (y) − F (x) =0 y→x y−x
F 0 (x) ≡ lim
at Lebesgue-almost every x ∈ (0, 1).
By familiar facts about functions of a real variable, one knows that such functions F are in one-to-one correspondence with non-atomic, Borel probability measures µ on [0, 1] which charge every non-empty open subset but are singular to Lebesgue’s measure. Namely, F is the distribution function determined by µ: F (x) = µ (−∞, x] . +
+
(i) Set Ω = {0, 1}Z , and, for each p ∈ (0, 1), take Mp = (βp )Z , where βp on {0, 1} is the Bernoulli measure with βp ({1}) = p = 1 − βp ({0}). Next, define ω ∈ Ω 7−→ Y (ω) ≡
∞ X
2−n ωn ∈ [0, 1],
n=1
and let µp denote the Mp -distribution of Y . Given n ∈ Z+ and 0 ≤ m < 2n , show that µp m2−n , (m + 1)2−n = p`m,n (1 − p)n−`m,n , Pn n −n where ` = = m,n k=1 ωk and (ω1 , . . . , ωn ) ∈ {0, 1} is determined by m2 Pn −k ωk . Conclude, in particular, that µp is non-atomic and charges every k=1 2 non-empty open subset of [0, 1]. (iii) Given x ∈ [0, 1) and n ∈ Z+ , define n (x) =
1
if 2n−1 x − b2n−1 xc ≥
0
if 2n−1 x − b2n−1 xc <
1 2 1 2,
where denotes the integer part of s. If {n : n ≥ 1} ⊆ {0, 1} satisfies Pbsc ∞ x = 1 2−m m , show that m = m (x) for all m ≥ 1 if and only if m = 0 for infinitely many m ≥ 1. In particular, conclude first that ωn = n Y (ω) , n ∈ Z+ , for Mp -almost every ω ∈ Ω and, second, by the Strong Law, that n 1 X n (x) −→ p n m=1
Thus, µp1 ⊥ µp2 whenever p1 6= p2 .
for µp -almost every x ∈ [0, 1].
48
1 Sums of Independent Random Variables
(iv) By Lemma 1.1.6, we know that µ 12 is Lebesgue measure λ[0,1] on [0, 1]. Hence, we now know that µp ⊥ λ[0,1] when p 6= 12 . In view of the introductory remarks, this completes the proof that, for each p ∈ (0, 1) \ { 12 }, the function Fp (x) = µp (−∞, x] is a strictly increasing, continuous function on [0, 1] whose derivative vanishes at Lebesgue-almost every point. Here, one can do better. Namely, referring to part (iii), let ∆p denote the set of x ∈ [0, 1) such that
1 Σn (x) = p, n→∞ n lim
where Σn (x) ≡
n X
m (x).
m=1
We know that ∆ 12 has Lebesgue measure 1. Show that, for each x ∈ ∆ 12 and p ∈ (0, 1) \ { 12 }, Fp is differentiable with derivative 0 at x.
Hint: Given x ∈ [0, 1), define Ln (x) =
n X
2−m m (x)
and Rn (x) = Ln (x) + 2−n .
m=1
Show that
Fp Rn (x) − Fp Ln (x) = Mp
n X
! 2−m ωm = Ln (x)
= pΣn (x) (1 − p)n−Σn (x) .
m=1
When p ∈ that
(0, 1) \ { 12 }
and x ∈ ∆ 12 , use this together with 4p(1 − p) < 1 to show ! Fp Rn (x) − Fp Ln (x) < 0. lim n log n→∞ Rn (x) − Ln (x)
To complete the proof, for given x ∈ ∆ 12 and n ≥ 2 such that Σn (x) ≥ 2, let
mn (x) denote the largest m < n such that m (x) = 1, and show that mnn(x) −→ 1 as n → ∞. Hence, since 2−n−1 < h ≤ 2−n implies that Fp (x) − Fp (x − h) n−mn (x)+1 Fp Rn (x) − Fp Ln (x) , ≤2 Rn (x) − Ln (x) h
one concludes that Fp is left-differentiable at x and has left derivative equal to 0 there. To get the same conclusion about right derivatives, simply note that Fp (x) = 1 − F1−p (1 − x). (v) Again let p ∈ (0, 1) \ { 12 } be given, but this time choose x ∈ ∆p . Show that
lim
h&0
Fp (x + h) − Fp (x) = +∞. h
The argument is similar to the one used to handle part (iv). However, this time the role played by the inequality 4pq < 1 is played here by (2p)p (2q)q > 1 when q = 1 − p.
§ 1.5 Law of the Iterated Logarithm
49
§ 1.5 Law of the Iterated Logarithm Let X1 , . . . , Xn , . . . be a sequence of independent, identically distributed random variables with mean value 0 and variance 1. In this section, I will investigate exactly how large {Sn : n ∈ Z+ } can become as n → ∞. To get a feeling for what one should be expecting, first note that, by Corollary 1.4.8, for any non-decreasing {bn : n ≥ 1} ⊆ (0, ∞), Sn −→ 0 bn
P-almost surely if
∞ X 1 < ∞. 2 b n=1 n 1
Thus, for example, Sn grows more slowly than n 2 log n. On the other hand, if Sn ; the Xn ’s are N (0, 1)-random variables, then so are the random variables √ n and therefore, for every R ∈ (0, ∞),
P
Sn lim √ ≥ R n
n→∞
[ Sn S √ ≥ R ≥ lim P √N ≥ R > 0. = lim P N →∞ N →∞ n N
n≥N
Hence, at least for normal random variables, one can use Lemma 1.4.1 to see that Sn lim √ = ∞ P-almost surely; n→∞ n 1
and so Sn grows faster than n 2 . If, as we did in Section 1.3, we proceed on the assumption that Gaussian random variables are typical, we should expect the growth rate of the Sn ’s to be 1 1 something between n 2 and n 2 log n. What, in fact, turns out to be the precise growth rate is
(1.5.1)
Λn ≡
q 2n log(2) (n ∨ 3),
where log(2) x ≡ log log x (not the logarithm with base 2) for x ∈ [e, ∞). That is, one has The Law of the Iterated Logarithm: (1.5.2)
Sn = 1 P-almost surely. n→∞ Λn lim
This remarkable fact was discovered first for Bernoulli random variables by Khinchine, was extended by Kolmogorov to random variables possessing 2 + moments, and eventually achieved its final form in the work of Hartman and Wintner. The approach that I will adopt here is based on ideas (taught to me by M. Ledoux) introduced originally to handle generalizations of (1.5.2) to random
50
1 Sums of Independent Random Variables
variables with values in a Banach space.1 This approach consists of two steps. The first establishes a preliminary version of (1.5.2) that, although it is far cruder than (1.5.2) itself, will allow me to justify a reduction of the general case to the case of bounded random variables. In the second step, I deal with bounded random variables and more or less follow Khinchine’s strategy for deriving (1.5.2) once one has estimates like the ones provided by Theorem 1.3.12. In what follows, I will use the notation Λβ = Λ[β]
S[β] and S˜β = Λβ
for β ∈ [3, ∞),
where [β] is the integer part of β. Lemma 1.5.3. Let {Xn : n ≥ 1} be a sequence of independent, identically distributed random variables with mean value 0 and variance 1. Then, for any a ∈ (0, ∞) and β ∈ (1, ∞),2
lim S˜n ≤ a
n→∞
(a.s., P) if
∞ X 1 P S˜β m ≥ a β − 2 < ∞. m=1
Proof: Let β ∈ (1, ∞) be given and, for each m ∈ N and 1 ≤ n ≤ β m , let √αm,n be a median (cf. (1.4.11)) of S[β m ] −Sn . Noting that, by (1.4.12), αm,n ≤ 2β m , we know that Sn 1 ˜ ˜ 2 lim Sn = lim m−1max m Sn ≤ β lim m−1max m m→∞ β m→∞ β n→∞ ≤n≤β Λβ m ≤n≤β Sn + αm,n 1 , ≤ β 2 lim maxm m→∞ n≤β Λβ m
and therefore
P lim S˜n ≥ a ≤ P n→∞
lim max
m→∞ n≤β m
Sn + αm,n
Λβ m
! ≥ aβ
− 12
.
But, by Theorem 1.4.13,
! Sn + αm,n 1 1 ≥ aβ − 2 ≤ 2P S˜β m ≥ aβ − 2 , P maxm n≤β Λβ m
and so the desired result follows from the Borel–Cantelli Lemma. 1
See §§ 8.4.2 and 8.6.3 and, for much more information, M. Ledoux and M. Talagrand, Probability in Banach Spaces, Springer-Verlag, Ergebnisse Series 3.Folge·Band 23 (1991). 2 Here and elsewhere, I use (a.s.,P) to abbreviate “P-almost surely.”
§ 1.5 Law of the Iterated Logarithm
51
Lemma 1.5.4. For any sequence {Xn : n ≥ 1} of independent, identically distributed random variables with mean value 0 and variance σ 2 ,
lim S˜n ≤ 8σ
(1.5.5)
n→∞
(a.s., P).
Proof: Without loss in generality, I assume throughout that σ = 1; and, for the moment, I will also assume that the Xn ’s are symmetric (cf. Exercise 1.4.26). By Lemma 1.5.3, we will know that (1.5.5) holds with 8 replaced by 4 once I show that ∞ X 3 P S˜2m ≥ 2 2 < ∞.
(*)
m=0
In order to take maximal advantage of symmetry, let (Ω, F, P) be the probability space on which the Xn ’s are defined, use {Rn : n ≥ 1} to denote the sequence of Rademacher functions on [0, 1) introduced in Section 1.1, and set Q = λ[0,1) × P on [0, 1) × Ω, B[0,1) × F . It is then an easy matter to check that symmetry of the Xn ’s is equivalent to the statement that + ω ∈ Ω −→ X1 (ω), . . . , Xn (ω), . . . ∈ RZ has the same distribution under P as + (t, ω) ∈ [0, 1) × Ω 7−→ R1 (t)X1 (ω), . . . , Rn (t)Xn (ω), . . . ∈ RZ has under Q. Next, using the last part of (iii) in Exercise 1.3.18 with σk = Xk (ω), note that λ[0,1)
2m ! X t ∈ [0, 1) : Rn (t)Xn (ω) ≥ a n=1 # " a2 , a ∈ [0, ∞) and ω ∈ Ω. ≤ 2 exp − P2m 2 n=1 Xn (ω)2
Hence, if
) 2m 1 X 2 Xm (ω) ≥ 2 ω∈Ω: m 2 n=1
( Am ≡
and
2m )! X 3 , t ∈ [0, 1) : Rn (t)Xn (ω) ≥ 2 2 Λ2m
( Fm (ω) ≡ λ[0,1)
n=1
52
1 Sums of Independent Random Variables
then, by Tonelli’s Theorem,
o Z n 3 2 = Fm ω) P(dω) P ω ∈ Ω : S2m (ω) ≥ 2 Λ2m Ω
"
8Λ2m exp − P2m 2 2 n=1 Xn (ω)2 Ω
Z ≤2
#
h i P(dω) ≤ 2 exp −4 log(2) 2m + 2P Am .
P∞ Thus, (*) comes down to proving that m=0 P Am < ∞; and, in order to check this, I argue in much the same way as I did when I proved the converse statement in Kolmogorov’s Strong Law. Namely, set m
Tm =
2 X
Xn2 ,
Bm =
n=1
Tm+1 − Tm ≥2 , 2m
and T m =
Tm 2m
for m ∈ N. Clearly, P Am = P Bm . Moreover, the sets Bm , m ∈ N, are mutually independent; and therefore, by the Borel–Cantelli Lemma, I need only check that Tm+1 − Tm ≥ 2 = 0. P lim Bm = P lim m→∞ m→∞ 2m
But, by the Strong Law, we know that T m −→ 1 (a.s., P), and therefore it is clear that Tm+1 − Tm −→ 1 (a.s., P). 2m
I have now proved (1.5.5) with 4 replacing 8 for symmetric random variables. To eliminate the symmetry assumption, again let (Ω, F, P) be the probability 0 0 0 space on which the Xn ’s are defined, let Ω , F , P be a second copy of the same space, and consider the random variables (ω, ω 0 ) ∈ Ω × Ω0 7−→ Yn ω, ω 0 ≡ Xn (ω) − Xn (ω 0 ) under the measure Q ≡ P × P0 . Since the Yn ’s are obviously (cf. part (i) of Exercise 1.4.21) symmetric, the result which I have already proved says that
lim
n→∞
Sn (ω) − Sn (ω 0 )
Λn
5
≤ 22 ≤ 8
for Q-almost every (ω, ω 0 ) ∈ Ω × Ω0 .
n| Now suppose that limn→∞ |S Λn > 8 on a set of positive P-measure. Then, by Kolmogorov’s 0–1 Law, there would exist an > 0 such that
|Sn (ω)| ≥ 8 + for P-almost every ω ∈ Ω; n→∞ Λn lim
§ 1.5 Law of the Iterated Logarithm
53
and so, by Fubini’sTheorem,3 we would that, for Q-almost every (ω, ω 0 ) ∈ have 0 + + Ω × Ω , there is a nm (ω) : m ∈ Z ⊆ Z such that nm (ω) % ∞ and Sn (ω) (ω) − Sn (ω) (ω 0 ) Sn (ω) (ω) Sn (ω) (ω 0 ) m m m m ≥ . − lim ≥ lim lim m→∞ m→∞ Λnm (ω) Λnm (ω) Λnm (ω) m→∞
But, again by Fubini’s Theorem, this would mean that there exists a {nm : m ∈ Sn (ω0 ) ≥ for P0 -almost every Z+ } ⊆ Z+ such that nm % ∞ and limm→∞ Λmn m ω 0 ∈ Ω0 , and obviously this contradicts " # 2 0 1 Sn P −→ 0. = E 2 log(2) n Λn
We have now got the crude statement alluded to above. In order to get the more precise statement contained in (1.5.2), I will need the following application of the results in § 1.3. Lemma 1.5.6. Let {Xn : n ≥ 1} be a sequence of independent random variables with mean value 0, variance 1, and common distribution µ. Further, assume that (1.3.4) holds. Then, for each R ∈ (0, ∞) there is an N (R) ∈ Z+ such that ! # " r 8R log(2) n R2 log(2) n (1.5.7) P S˜n ≥ R ≤ 2 exp − 1 − K n
for n ≥ N (R). In addition, for each ∈ (0, 1], there is an N () ∈ Z+ such that, for all n ≥ N () and |a| ≤ 1 ,
(1.5.8)
h i 1 P S˜n − a < ≥ exp − a2 + 4K|a| log(2) n . 2
In both (1.5.7) and (1.5.8), the constant K ∈ (0, ∞) is the one in Theorem 1.3.15. Proof: Set Λn = λn = n
2 log(2) (n ∨ 3) n
12
.
To prove (1.5.7), simply apply the upper bound in the last part of Theorem 1.3.15 to see that, for sufficiently large n ∈ Z+ , 3 (Rλn )2 ˜ − K Rλn . P Sn ≥ R = P S n ≥ Rλn ≤ 2 exp −n 2 3
This is Fubini at his best and subtlest. Namely, I am using Fubini to switch between horizontal and vertical sets of measure 0.
54
1 Sums of Independent Random Variables
To prove (1.5.8), first note that
P S˜n − a < = P S n − an < n , where an = aλn and n = λn . Thus, by the lower bound in the last part of Theorem 1.3.15,
2 an K 2 ˜ + K|an | n + an P Sn − a < ≥ 1 − 2 exp −n 2 nn ! h i K exp − a2 + 2K|a| + a2 λn log(2) n ≥ 1− 2 2 log(2) n
for sufficiently large n’s.
Theorem 1.5.9 (Law of Iterated Logarithm). The equation (1.5.2) holds for any sequence {Xn : n ≥ 1} of independent, identically distributed random variables with mean value 0oand variance 1. In fact, P-almost surely, the set of n Sn limit points of Λn : n ≥ 1 coincides with the entire interval [−1, 1]. Equiva lently, for any f ∈ C R; R ,
lim f
(1.5.10)
n→∞
Sn Λn
=
sup f (t)
(a.s., P).
t∈[−1,1]
(Cf. Exercise 1.5.12 for a converse statement and §§ 8.4.2 and 8.6.3 for related results.) Proof: I begin with the observation that, because of (1.5.5), I may restrict my attention to the case when the Xn ’s are bounded random variables. Indeed, for any Xn ’s and any > 0, an easy truncation procedure allows us to find an ψ ∈ Cb (R; R) such that Yn ≡ ψ ◦ Xn again has mean value 0 and variance 1 while Zn ≡ Xn − Yn has variance less than 2 . Hence, if the result is known when the random variables are bounded, then, by (1.5.5) applied to the Zn ’s,
Pn m=1 Zm (ω) ˜ ≤ 1 + 8, lim Sn (ω) ≤ 1 + lim n→∞ n→∞ Λn and, for a ∈ [−1, 1],
Pn Zm (ω) lim S˜n (ω) − a ≤ lim m=1 ≤ 8 n→∞ Λn
n→∞
for P-almost every ω ∈ Ω.
§ 1.5 Law of the Iterated Logarithm
55
In view of the preceding, from now on I may and will assume that the Xn ’s are bounded. To prove that limn→∞ S˜n ≤ 1 (a.s., P), let β ∈ (1, ∞) be given, and use (1.5.7) to see that h i 1 1 P S˜β m ≥ β 2 ≤ 2 exp −β 2 log(2) β m + for all sufficiently large m ∈ Z . Hence, by Lemma 1.5.3 with a = β, we see that limn→∞ S˜n ≤ β (a.s., P) for every β ∈ (1, ∞). To complete the proof, I must still show that, for every a ∈ (−1, 1) and > 0, P lim S˜n − a < = 1.
n→∞
Because I want to get this conclusion as an application of the second part of the Borel–Cantelli Lemma, it is important that we be dealing with independent events, and for this purpose I use the result just proved to see that, for every integer k ≥ 2, lim S˜n − a ≤ inf lim S˜km − a k→∞ m→∞
n→∞
= inf
Skm − Skm−1 − a lim Λk m
P-almost surely.
k→∞ m→∞
Thus, because the events Skm − Skm−1 − a < , Ak,m ≡ Λkm
m ∈ Z+ ,
are independent for each k ≥ 2, all that I need to do is check that ∞ X P Ak,m = ∞ for sufficiently large k ≥ 2. m=1
But P Ak,m
Λkm Λkm a ˜ , < = P Skm −km−1 − Λkm −km−1 Λkm −km−1
and, because
Λkm − 1 = 0, lim max+ k→∞ m∈Z Λ m m−1 k −k
everything reduces to showing that ∞ X (*) P S˜km −km−1 − a < = ∞ m=1
for each k ≥ 2, a ∈ (−1, 1), and > 0. Finally, referring to (1.5.8), choose 0 > 0 so small that ρ ≡ a2 + 4K0 |a| < 1, and conclude that, when 0 < < 0 , h i 1 P S˜n − < ≥ exp −ρ log(2) n 2 for all sufficiently large n’s, from which (*) is easy.
56
1 Sums of Independent Random Variables
Remark 1.5.11. The reader should notice that the Law of the Iterated Logarithm provides a naturally occurring sequence of functions that converge in measure but not almost everywhere. Indeed, it is obvious that S˜n −→ 0 in 2 ˜ L (P; R), but the Law of the Iterated Logarithm says that Sn : n ≥ 1 is wildly divergent when looked at in terms of P-almost sure convergence. Exercises for § 1.5 Exercise 1.5.12. Let {Xn : n ≥ 1} be a sequence of mutually independent, identically distributed random variables for which
|Sn | < ∞ > 0. n→∞ Λn
(1.5.13)
P
lim
In this exercise I4 will outline a proof that X1 is P-square integrable, EP X1 = 0, and (1.5.14)
1 Sn Sn = EP X12 2 = − lim n→∞ Λn n→∞ Λn lim
(a.s., P).
(i) Using Lemma 1.4.1, show that there is a σ ∈ [0, ∞) such that (1.5.15)
lim
n→∞
Sn
Λn
=σ
(a.s., P).
Next, assuming that X1 is P-square integrable, use The Law of Large Strong Numbers together with Theorem 1.5.9 to show that EP X1 = 0 and 1 Sn Sn = − lim σ = EP X12 2 = lim n→∞ Λn n→∞ Λn
(a.s., P).
In other words, everything comes down to proving that (1.5.13) implies that X1 is P-square integrable. (ii) Assume that the Xn ’s are symmetric. For t ∈ (0, ∞), set ˇ nt = Xn 1[0,t] |Xn | − Xn 1(t,∞) |Xn | , X and show that ˇ t, . . . , X ˇt , . . . X 1 n 4
and
X1 , . . . , X n , . . .
I follow Wm. Feller “An extension of the law of the iterated logarithm to variables without variance,” J. Math. Mech., 18 #4, pp. 345–355 (1968), although V. Strassen was the first to prove the result.
Exercises for § 1.5
57
have the same distribution. Conclude first that, for all t ∈ [0, 1), Pn m=1 Xn 1[0,t] |Xn | ≤ σ (a.s., P), lim n→∞ Λn
where σ is the number in (1.5.15), and second that h i EP X12 = lim EP X12 , X1 ≤ t ≤ σ 2 . t%∞
Hint: Use the equation ˇ nt Xn + X , Xn 1[0,t] |Xn | = 2
and apply part (i). (iii) For general {Xn : n ≥ 1}, produce an independent copy {Xn0 : n ≥ 1} (as in the proof of Lemma 1.5.4), and set Yn = Xn − Xn0 . After checking that Pn | m=1 Ym | ≤ 2σ (a.s., P), lim n→∞ Λn conclude first that EP Y12 ≤ 4σ 2 and then (cf. part (i) of Exercise1.4.27) that EP X12 < ∞. Finally, apply (i) to arrive at EP X1 = 0 and (1.5.14).
Exercise 1.5.16. Let {˜ sn : n ≥ 1} be a sequence of real numbers which possess the properties that lim s˜n+1 − s˜n = 0. lim s˜n = 1, lim s˜n = −1, and n→∞
n→∞
n→∞
Show that the set of subsequential limit points of {˜ sn : n ≥ 1} coincides with [−1, 1]. Apply this observation to show that, in order to get the final statement in Theorem 1.5.9, I need only have proved (1.5.10) for the function f (x) = x, x ∈ R. Hint: In proving the last part, use the square integrability of X1 to see that 2 ∞ X Xn ≥ 1 < ∞, P n n=1
and apply the Borel–Cantelli Lemma to conclude that S˜n − S˜n−1 −→ 0 (a.s., P). Exercise 1.5.17. Let {Xn : n ≥ 1} be a sequence of RN -valued, identically distributed random variables on (Ω, F, P) with the property that, for each e ∈ SN −1 = {x ∈ RN : |x| = 1}, e, X1 RN has mean value 0 and variance 1. Set Pn ˜ n | = 1 P-almost surely. ˜ n = Sn , and show that limn→∞ |S Sn = m=1 Xm and S Λn Here are some steps that you might want to follow.
58
1 Sums of Independent Random Variables
(i) Let {ek : k ≥ 1} be a countable, dense subset of SN −1 for which {e1 , . . . , eN } N is orthonormal, and suppose that the sequence {˜sn : n ≥ 1} ⊆ R has the property that limn→∞ ek , ˜sn RN = 1 for each k ≥ 1. Note that |˜sn | ≤ 1 N 2 max1≤k≤N (ek , s˜n RN , and conclude that C ≡ supn≥1 |˜sn | ∈ [1, ∞). S` (ii) Continuing (i), for a given > 0, choose ` ≥ 1 so that SN −1 ⊆ k=1 B ek , C . Show that |˜sn | ≤ max e,˜sn RN + , 1≤k≤`
and conclude first that limn→∞ |˜sn | ≤1 + and then that limn→∞ |˜sn | ≤ 1. At the same time, since |˜sn | ≥ e1 , ˜sn RN , show that limn→∞ |˜sn | ≥ 1. Thus limn→∞ |˜sn | = 1.
(iii) Let {ek : k ≥ 1} be as in (i), and apply Theorem 1.5.9 to show that, for ˜ n (ω) : n ≥ 1} satisfies the condition in (i). P-almost all ω ∈ Ω, the sequence {S ˜ Thus, by (ii), limn→∞ |Sn (ω)| = 1 for P-almost every ω ∈ Ω.
Chapter 2 The Central Limit Theorem
In the preceding chapter I dealt with averages of random variables and showed that, in great generality, those averages converge almost surely or in probability to a constant. At least when all the random variables have the same distribution and moments of all orders, one way of rationalizing this phenomenon is to recognize that the mean value is conserved whereas all higher moments are driven to 0 when one averages. Of course, the reason why it is easy to conserve the first moment is that the mean of the sum is the sum of the means. Thus, if one is going to attempt to find a simple normalization procedure that conserves a quantity involving more than the mean value, one should seek a quantity that shares this additivity property. With this in mind, one is led to ask what happens if one normalizes in a way that conserves the variance. For this purpose, suppose that {Xn : n ∈ Z+ } is a sequence of mutually independent, identicallyP distributed random variables with 1 n mean value 0 and variance 1, and set Sn = 1 Xk . Then S˘n ≡ n− 2 Sn again has mean value 0 and variance 1. On the other hand, because of Theorem 1.5.9, we know that, with probability 1, limn→∞ S˘n = ∞ = − limn→∞ S˘n . Hence, from the point of view of either almost sure convergence or even convergence in probability, there is no hope that the S˘n ’s will converge. Nonetheless, the random variables {S˘n : n ≥ 1} possess remarkable stability when viewed from a distributional perspective. Indeed, if the Xn ’s are Gaussian, then so are the S˘n ’s, and therefore S˘n ∈ N (0, 1) for all n ≥ 1. More generally, even if the Xn ’s are not Gaussian, fixing their mean value and variance in this way forces all their moments to stabilize. To be precise, assume that X1 has finite moments of all orders, that its mean is 0, and that its variance is 1. Trivially, L1 ≡ limn→∞ EP [S˘n ] = 0 and L2 ≡ limn→∞ EP [S˘n2 ] = 1. Next, assume that L` ≡ limn→∞ EP [S˘n` ] exists for 1 ≤ ` ≤ m, where m ≥ 2. I will show now that Lm+1 ≡ limn→∞ EP [S˘nm+1 ] exists and is equal to mLm−1 . To this end, first note that, since EP [Xn ] = 0 and the Xn ’s are independent and identically distributed, m X m+1 m m P j+1 P m−j P E Sn = nE Xn Xn + Sn−1 =n E Xn E Sn−1 j j=0 P
59
60
2 The Central Limit Theorem m X m−1 m P j+1 P m−j = nmE Sn−1 + n E Xn E Sn−1 . j j=2 P
m+1
Thus, after dividing through by n 2 , one gets the desired conclusion when n → ∞. Starting from L1 = 0 and L2 = 1, one now can use induction to check Qm + that L2m−1 = 0 and L2m = `=1 (2` − 1) = 2(2m)! m m! for all m ∈ Z . That is,
lim EP S˘n2m−1 = 0
n→∞
and
m Y (2m)! lim EP S˘n2m = (2` − 1) = m , n→∞ 2 m! `=1
for all m ∈ Z+ . In other words, at least when the Xn ’s have moments of all orders, limn→∞ EP S˘nm exists and is independent of the particular choice of random variables. In particular, since for the Gaussian case, EP [S˘nm ] = EP [X1m ], we conclude that all moments of the S˘n ’s converge to the corresponding moments of a standard normal random variable. In this chapter we will see that the preceding stabilization result is just one manifestation of a general principle known as the Central Limit phenomenon. § 2.1 The Basic Central Limit Theorem In this section I will derive the basic Central Limit Theorem using a beautiful argument which was introduced by J. Lindeberg. Throughout, hϕ, µi denotes the integral of a function ϕ against a measure µ. § 2.1.1. Lindeberg’s Theorem. Let {Xn : n ≥ 1} be a sequence of independent,Psquare integrable random variables with mean value 0, and set 1 n S˘n = n− 2 m=1 Xm . At least when the Xn ’s are identically distributed and have moments of all orders and variance 1, we just saw that (recall that γm,σ2 is the distribution of an N (m, σ 2 )-random variable) (2.1.1) lim EP ϕ(S˘n ) = hϕ, γ0,1 i n→∞
for any polynomial ϕ : R −→ C. In this subsection, I will prove a result that shows that, under much more general conditions, (2.1.1) holds for all ϕ ∈ C 3 R; C) with bounded second and third order derivatives. In the following statement, sX p p Sn 2 , . σm and S˘n ≡ (2.1.2) σm = Var(Xm ) > 0, Σn = Var(Sn ) = Σn m=1
Notice that when the Xk ’s are identically distributed and have variance 1, the S˘n in (2.1.2) is consistent with the notation used above. Finally, set (2.1.3)
σm 1≤m≤n Σn
rn = max
and gn () =
n i 1 X P h 2 E X , X ≥ Σ m n m Σ2n m=1
§ 2.1 The Basic Central Limit Theorem
61 1
for > 0. Clearly, in the identically distributed case, rn = n− 2 and i h 1 gn () = σ1−2 EP X12 , |X1 | ≥ n 2 σ1 −→ 0 as n → ∞ for each > 0.
Theorem 2.1.4 (Lindeberg). Refer to the preceding, and let ϕ be an element of C 3 (R; R) with bounded second and third order derivatives. Then, for each > 0,
rn P
ϕ000 + gn () ϕ00 . + (2.1.5) E ϕ S˘n − hϕ, γ0,1 i ≤ u u 2 6
In particular, because rn2 ≤ 2 + gn (),
(2.1.6)
> 0,
(2.1.1) holds if gn () −→ 0 as n → ∞ for each > 0. Proof: Choose N (0, 1)-random variables Y1 , . . . , Yn which are both mutually independent and independent of the Xm ’s. (After augmenting the probability space, if necessary, this can be done as an application of either Theorem 1.1.7 or Exercise 1.1.14.) Next, set σk Yk Y˘k = Σn
and T˘n =
n X
Y˘k ,
1
and observe that T˘n is again an N (0, 1)-random variable and therefore that ∆ ≡ EP ϕ(S˘n ) − hϕ, γ0,1 i = EP ϕ(S˘n ) − EP ϕ(T˘n ) . ˘k = Further, set X
Xk Σn ,
Um =
and define X
1≤k≤m−1
Y˘k +
X
˘k X
for 1 ≤ m ≤ n,
m+1≤k≤n
where a sum over the empty set is taken to be 0. It is then clear that ∆≤
n X
∆m
˘ m − EP ϕ Um + Y˘m . where ∆m ≡ EP ϕ Um + X
1
Moreover, if Rm (ξ) ≡ ϕ Um + ξ − ϕ(Um ) − ξϕ0 (Um ) −
ξ 2 00 2 ϕ (Um ),
ξ ∈ R,
62
2 The Central Limit Theorem
˘ m and Y˘m are independent of Um and have the same first then (because both X two moments) ˘ m ) − EP Rm (Y˘m )] ≤ EP Rm (X ˘ m ) + EP Rm (Y˘m ) . ∆m = EP Rm (X In order to complete the derivation of (2.1.5), note that, by Taylor’s Theorem,
3 Rm (ξ) ≤ ϕ000 |ξ| ∧ ϕ00 |ξ|2 ; u u 6
and therefore, for each > 0, n X ˘ m )| EP |Rm (X 1
≤
n n X 2 kϕ000 ku X P ˘ 3 ˘ , |Xm | ≥ Σn E |Xm | , |Xm | ≤ Σn + kϕ00 ku EP X m 6 1 1
n 2 kϕ000 ku kϕ000 ku X σm 00 + kϕ00 ku gn (), + kϕ k g () = ≤ u n 2 6 Σ 6 n 1
while n X 1
3 n 3 X kϕ000 ku P 3 4 rn kϕ000 ku σm . ≤ E |Y1 |3 EP |Rm (Y˘n )| ≤ 6 Σ3n 6 1
Hence, (2.1.5) is now proved. Given (2.1.5), all that remains is to prove (2.1.6). However, for any 1 ≤ m ≤ n and > 0, 2 2 2 σm = EP Xm , |Xm | < Σn + EP Xm , |Xm | ≥ Σn ≤ Σ2n 2 + gn () . The condition that gn () −→ 0 for each > 0 is often called Lindeberg’s condition because it introduced by J. Lindeberg and it was he who proved that it is a sufficient condition for (2.1.1) to hold for all (cf. Theorem 2.1.8) ϕ ∈ Cb (RN ; C). Later, Feller proved that (2.1.1) for all ϕ ∈ Cb (RN ; R) plus rn → 0 imply that Lindeberg’s condition holds. Together, these two results are known as the Lindeberg–Feller Theorem. See Exercise 2.3.20 for a proof of Feller’s part. § 2.1.2. The Central Limit Theorem. If one is not concerned about rates of convergence, then the differentiability requirement on ϕ can be dropped from the last part of Theorem 2.1.4. In order to understand the reason for this, it is helpful to couch the statement of Theorem 2.1.4 entirely in terms of measures. Thus, let µn denote the distribution of S˘n . Then, under Lindeberg’s condition, Theorem 2.1.4 allows one to say that hϕ, µn i −→ hϕ, γ0,1 i for all ϕ ∈ C 3 (RN ; C) with bounded second and third order derivatives. Because we are dealing with statements about integration and integration is a very forgiving operation, this sort of result self-improves. To be precise, I prove the following lemma.
§ 2.1 The Basic Central Limit Theorem
63
Lemma 2.1.7. Suppose that {µn : n ≥ 1} is a sequence of (non-negative) locally finite1 Borel measures on RN and that µ is a locally finite Borel mea∞ N sure on RN with the property that hϕ, µn i −→ hϕ, µi for all ϕ ∈ Cc (R ; R). N Then, for any ψ ∈ C R ; [0, ∞) , hψ, µi ≤ limn→∞ hψ, µn i. Moreover, if ψ ∈ C RN ; [0, ∞) is µn -integrable for each n ∈ Z+ and if hψ, µn i −→ hψ, µi ∈ [0, ∞), then for any sequence {ϕn : n ≥ 1} ⊆ C(RN ; C) that converges uniformly on compacts to a ϕ ∈ C(RN ; C) and satisfies |ϕn | ≤ Cψ for some C < ∞ and all n ≥ 1, hϕn , µn i −→ hϕ, µi. Proof: Choose ρ ∈ Cc∞ B(0, 1); [0, ∞) with total integral 1, and set ρ (x) = −N ρ(−1 x) for > 0. Also, choose η ∈ Cc∞ B(0, 2); [0, 1] so that η = 1 on B(0, 1), and set ηR (x) = η(R−1 x) for R > 0. Begin by noting that hϕ, µn i −→ hϕ, µi for all ϕ ∈ Cc∞ (RN ; C). Next, suppose that ϕ ∈ Cc (RN ; C), and, for > 0, set ϕ = ρ ? ϕ, the convolution Z ρ (x − y)ϕ(y) dy RN
of ρ with ϕ. Then, for each > 0, ϕ ∈ Cc∞ (RN ; C) and therefore hϕ , µn i −→ hϕ , µi. In addition, there is an R > 0 such that supp(ϕ ) ⊆ B(0, R) for all ∈ (0, 1]. Hence, lim hϕ, µn i − hϕ, µi ≤ 2hηR , µikϕ − ϕku . n→∞
Since lim&0 kϕ − ϕku = 0, we have now shown that hϕ, µn i −→ hϕ, µi for all ϕ ∈ Cc (RN ; C). Now suppose that ψ ∈ C RN ; [0, ∞) , and set ψR = ηR ψ, where ηR is as above. Then, for each R > 0, hψR , µi = limn→∞ hψR , µn i ≤ limn→∞ hψ, µn i. Hence, by Fatou’s Lemma, hψ, µi ≤ limR→∞ hψR , µi ≤ limn→∞ hψ, µn i. Finally, suppose that ψ ∈ C RN ; [0, ∞) is µn -integrable for each n ∈ Z+ and that hψ, µn i −→ hψ, µi ∈ [0, ∞). Given {ϕn : n ≥ 1} ⊆ C(RN ; C) satisfying |ϕn | ≤ Cψ and converging uniformly on compacts to ϕ, one has hϕn , µn i − hϕ, µi ≤ hϕn − ϕ, µn i + hϕ, µn i − hϕ, µi .
Moreover, for each R > 0, lim hϕn − ϕ, µn i n→∞
≤ lim
sup
n→∞ x∈B(0,2R)
|ϕn (x) − ϕ(x)|hηR , µn i + lim h(1 − ηR )(ϕn − ϕ), µn i n→∞
≤ 2C lim h(1 − ηR )ψ, µn i = lim 2C hψ, µn i − hηR ψ, µn i = 2Ch(1 − ηR )ψ, µi, n→∞
1
n→∞
A Borel measure on a topological space is locally finite if it gives finite measure to compacts.
64
2 The Central Limit Theorem
and similarly lim hϕ, µn i − hϕ, µi n→∞ ≤ lim hηR ϕ, µn i − hηR ϕ, µi + C lim h(1 − ηR )ψ, µn i + Ch(1 − ηR )ψ, µi n→∞
n→∞
= 2Ch(1 − ηR )ψ, µi. Finally, because ψ is µ-integrable, h(1 − ηR )ψ, µi −→ 0 as R → ∞ by Lebesgue’s Dominated Convergence Theorem, and so we are done. By combining Theorem 2.1.4 with the preceding, we have the following version of the famous Central Limit Theorem. Theorem 2.1.8 (Central Limit Theorem). With the setting the same as it was in Theorem 2.1.4, assume that gn () −→ 0 as n → ∞ for each > 0. Then lim EP ϕn (S˘n ) = hϕ, γ0,1 i n→∞
whenever {ϕn : n ≥ 1} ⊆ C(R; C) satisfies ϕn (y) <∞ sup sup 2 n≥1 y∈R 1 + |y|
and tends to ϕ uniformly on compacts. Moreover, if −∞ ≤ a < b ≤ ∞, then 2 Z b y 1 ˘ dy. exp − (2.1.9) lim P a ≤ Sn ≤ b = γ0,1 (a, b] = √ n→∞ 2 2π a
(See Exercise 2.1.10 for more information about the identically distributed case.) Proof: Take µn to be the distribution of S˘n . By Theorem 2.1.4, we know that hϕ, µn i −→ hϕ, γ0,1 i for all ϕ ∈ Cc∞ (RN ; R). In addition, we know that hψ, µn i = 2 = hψ, γ0,1 i when ψ(y) = 1 + y 2 . Hence, the first assertion is an application of Lemma 2.1.7. Turning to the second assertion, let a < b be given. To prove (2.1.9), choose {ϕk : k ≥ 1} ⊆ Cb (R; R) and {ψk : k ≥ 1} ⊆ Cb (R; R) so that 0 ≤ ϕk % 1(a,b) and 1 ≥ ψk & 1[a,b] as k → ∞. Then, Z i P ˘ ˘ ϕk (y) γ0,1 (dy) −→ γ0,1 (a, b) lim P a < Sn < b ≥ lim E ϕk Sn = n→∞
n→∞
R
as k → ∞, and, similarly,
lim P a ≤ S˘n ≤ b ≤ lim EP ψk S˘n =
n→∞
n→∞
Z R
Finally, note that γ0,1 (a, b) = γ0,1 [a, b] .
ψk (y) γ0,1 (dy) −→ γ0,1 [a, b] .
Exercises for § 2.1
65
Exercises for § 2.1 Exercise 2.1.10. Let {Xn : n ≥ 1} be a sequence of independent, identically 1 Pn distributed random variables, set S˘n = n− 2 m=1 Xm , and assume that
lim EP S˘n2 ∧ R2 ≤ 1
n→∞
for every R ∈ [0, ∞).
In particular, by Lemma 2.1.7, this will certainly be the case whenever (2.1.1) holds for every ϕ ∈ Cc (R; R). The purpose of this exercise is to show that the Xn ’s are P-square integrable, have mean value 0, and variance no more than 1; and the method which I will use is based on the same line of reasoning as was given in Exercise 1.5.12. (i) Assuming that X1 ∈ L2 (P; R), show that EP X1 = 0 and EP X12 ≤ 1. In particular, use this together with the result in part (i) of Exercise 1.4.27 to see that it suffices to handle the case when the Xn ’s are symmetric. (ii) In this and the succeeding parts of this exercise, we will be assuming that the Xn ’s are symmetric. Following the same route as was suggested in (ii) of Exercise 1.5.12, set ˇ t = Xn 1[0,t] |Xn | − Xn 1(t,∞) |Xn | , X n
n ∈ Z+ ,
ˇ t, . . . , X ˇ nt , . . . and X1 , . . . , Xn , . . . have the same distribuand recall that X 1 tion for each t ∈ (0, ∞). Use this together with our basic assumption to see that limR→∞ sup n∈Z+ P An (t, R) = 0, where t∈(0,∞)
) ( n n X X 1 ˇt ≥ n2 R . An (t, R) ≡ Xk ∨ X k 1
1
(iii) Continuing in the setting of part (ii), set n 1 X Xk 1[0,t] |Xk | . S˘nt = 1 n2 1
After noting that the Xn 1[0,t] |Xn | ’s are symmetric, check (cf. the proof of Theorem 1.3.1) that EP |S˘nt |4 ≤ 3t4 . In particular, conclude that, for each t ∈ (0, ∞), there is an R(t) ∈ (0, ∞) such that
1 1 EP |S˘nt |2 , An t, R(t) ≤ 3 2 t2 P An t, R(t) 2 ≤ 1
for all n ∈ Z+ .
66
2 The Central Limit Theorem
(iv) Given t ∈ (0, ∞), choose R(t) ∈ (0, ∞) as in the preceding. Taking into account the identity Pn Pn ˇ t X + k t 1 1 Xk , S˘n = 1 2n 2 show that EP X12 , |X1 | ≤ t = EP |S˘nt |2 ≤ EP |S˘nt |2 , An t, R(t) { + 1 h i ≤ EP S˘n2 ∧ R(t)2 + 1
for all n ∈ Z+ and t ∈ (0, use this and our basic hypothesis ∞). In particular, to conclude first that EP X12 , |X1 | ≤ t ≤ 2 for all t ∈ (0, ∞) and then that X1 is square P-integrable. (v) Show that (2.1.1) holds if and only if X1 has mean value 0 and variance 1. Exercise 2.1.11. An interesting way in which to interpret The Central Limit Theorem is as the solution to a certain fixed point problem. Namely, let P denote the set of probability measures µ on R; BR with the properties that Z Z x2 µ(dx) = 1 and x µ(dx) = 0. R
R
Next, define T2 µ for µ ∈ P to be the probability measure on R, BR given by ZZ x+y √ µ(dx)µ(dy) for Γ ∈ BR . T2 µ(Γ) = 1Γ 2 R2
After checking that T2 maps P into itself, use The Central Limit Theorem to show that, for every µ ∈ P, Z Z n lim ϕ d T2 µ = ϕ dγ0,1 , ϕ ∈ Cb (R; C). n→∞
R
R
Conclude, in particular, that γ0,1 is the one and only element µ of P with the property that T2 µ = µ and that this fixed point is attracting. (See Exercise 2.3.21 for more information.) Exercise 2.1.12. Here is another indication of the remarkable stability of normal random variables. Namely, I will outline here a derivation2 of the L´ evy– Cram´ er Theorem which says that if X and Y are independent random variables whose sum is normal (with some mean and variance), then both X and Y are normal. 2
This derivation is based on a note by Z. Sasv´ ari, who himself borrowed some of the ideas from A. R´ enyi. I know of no derivation that does not rely on complex analysis and would be very interested in learning one.
Exercises for § 2.1
67
(i) Assume that X + Y ∈ N (a, σ 2 ), and, by subtracting a from X, reduce to the case in which X + Y ∈ N (0, σ 2 ). Next, show that there is nothing more to do when σ = 0 and that one can always reduce to the case σ = 1 when σ > 0. Thus, from now on, assume that X + Y ∈ N (0, 1). (ii) Choose r ∈ (0, ∞) so that P |X| ∨ |Y | ≥ r ≤ 12 , and conclude that
R2 , P |X| ≥ r + R ∨ P |Y | ≥ r + R ≤ 4 exp − 2
R ∈ (0, ∞).
In particular, show that the moment generating functions z ∈ C 7−→ M (z) = EP ezX ∈ C and z ∈ C 7−→ N (z) = EhP eizY ∈ C exist and are entire functions. 2
Further, note that M (z)N (z) = exp z2 , and conclude that M and N never vanish. Finally, from the fact that X + Y has mean 0, show that one can reduce to the case in which both X and Y have mean 0. Thus, from now on, we assume that M 0 (0) = 0 = N 0 (0).
(iii) Because M never vanishes and M (0) = 1, elementary complex analysis (cf. Lemma 3.2.3) guarantees that there is a unique entire function θ : C −→ C such that θ(0) = 0 and M (z) = eθ(z) for all z ∈ C. Further, from M 0 (0) = 0, note that θ0 (0) = 0. Thus, θ(z) =
∞ X
cn z n
where n!cn =
n=2
Finally, note that N (z) = exp
h
z2 2
xX dn P log E e ∈ R. dxn x=0
i − θ(z) .
(iv) As an application of H¨older’s Inequality, observe that x ∈ R 7−→ θ(x) ∈ R 2 and x ∈ R 7−→ x2 − θ(x) ∈ R are both convex. Thus, since θ0 (0) = 0, both these functions are non-increasing on (−∞, 0] and non-decreasing on [0, ∞). Use this observation to check that
θ(x) ≥ 0 ≤
x2 − θ(x) 2
for all x ∈ R.
Next, use the preceding in conjunction with the trivial remarks exp Re θ(z) = EP ezX ≤ eθ(x) and
h exp Re
z2 2
− θ(z)
i
i h 2 = EP ezY ≤ exp x2 − θ(x)
to arrive at −y 2 ≤ 2Re θ(z) ≤ x2
for z = x +
√
−1 y ∈ C.
68
2 The Central Limit Theorem
In particular, this means that |z|2 , Re θ(z) ≤ 2
z ∈ C.
(v) To complete the program, use Cauchy’s integral formula to show that, for each n ∈ Z+ and r > 0, on the one hand, Z 2π √ √ 1 θ re −1 θ e− −1 nθ dθ, r > 0, cn r n = 2π 0
while, on the other hand (since θ(z) = θ z¯) and therefore ∂z θ(z) = 0),
Z
2π
√
θ re
0=
−1 θ
e−
√
−1 nθ
dθ.
0
Hence, 1 cn r = π n
Z
2π
√ √ e− −1 nθ dθ, Re θ re −1 θ
n ∈ Z+ and r > 0.
0
Finally, in combination with the estimate obtained in (iv) and the fact that c0 = c1 = 0, this leads to the conclusion that cn = 0 for n 6= 2 and therefore that θ(z) = c2 z 2 with 0 ≤ c2 ≤ 12 .
Exercise 2.1.13. An important result that is closely related to The Central Limit Theorem is the following observation, which occupies a central position in the development of classical statistical mechanics.3 (i) For each n ∈ Z+ , let λn denote the normalized surface measure on the (n − 1)-dimensional sphere √ 1 Sn−1 n = x ∈ Rn : |x| = n 2 , (1)
and denote by λn the distribution of the coordinate x1 under λn . Check that, (1) when n ≥ 2, λn (dt) = fn (t) dt, where fn (t) = 3
ωn−2 1 2
n ωn−1
t2 1− n
n−3 2
1 1(−1,1) n− 2 t ,
Although E. Borel seems to have thought he was the first to discover this result and rhap´ sodizes about it a good deal in “Sur les principes de la cin´ etique des gaz,” Ann. l’Ecole Norm. ¨ sup., 3e t. 23, it appears already in the 1866 article “Uber die Entwicklungen einer Funktion von beliebig vielen Variabeln nach Laplaceshen Funktionen h¨ oherer Ordnung,” J. Reine u. Angewandte Math., by F. Mehler and is only a small part of what Mehler discovered there. Be that as it may, Borel deserves credit for recognizing the significance of this result for statistical mechanics.
Exercises for § 2.1
69
and ωk−1 denotes the surface area of the (k − 1)-dimensional unit sphere in Rk . Using polar coordinates to compute the right-hand side of Z |x|2 k e− 2 dx, (2π) 2 = Rk
first check that
k
ωk−1 =
2π 2 , Γ k2
where Γ(t) is Euler’s Γ-function (cf. (1.3.20)), and then apply Stirling’s formula (cf. (1.3.21)) to see that ωn−2 1 2
n ωn−1
1 −→ √ 2π
as
n → ∞.
Now, using g to denote the density for the standard Gauss distribution (i.e., the Gauss kernel in (1.3.5)), apply these computations to show that sup sup n≥3 t∈R
fn (t) < ∞ and that g(t)
fn (t) −→ 1 uniformly on compacts. g(t)
In particular, conclude that, for any ϕ ∈ L1 (γ0,1 ; R), Z Z (1) (2.1.14) ϕ dλn −→ ϕ dγ0,1 . R
R
(ii) A less computational approach to the same calculation is the following. Let {Xn : n variables, and set p≥ 1} be a sequence of independent N (0, 1) random 2 2 Rn = X1 + · · · + Xn . First note that P Rn = 0 = 0 and then that the distribution of 1 n 2 X1 , . . . , Xn θn ≡ Rn R2
is λn . Next, use The Strong Law of Large Numbers to see that nn −→ 1 (a.s., P) and conclude that, for any N ∈ Z+ , lim EP ϕ θn(N ) = EP ϕ X1 , . . . , XN , ϕ ∈ Cc RN ; R , n→∞
(N )
where, for n ≥ N , θn ∈ RN denotes the projection of θn ∈ Rn onto its first (N ) N coordinates. Conclude that if λn on RN , BRN denotes the distribution of x = (x1 , . . . , xn ) ∈ Rn 7−→ x(N ) ≡ x1 , . . . , xN ∈ RN under λn , then Z Z (N ) N lim ϕ dλn = ϕ dγ0,1 for all ϕ ∈ Cb RN ; C . n→∞
RN
RN
70
2 The Central Limit Theorem
(iii) By considering the case when N = 2, show that, for any ϕ ∈ Cb (R; R), Z (2.1.15)
lim
n→∞ √ Sn−1 ( n)
n
1X ϕ xk − n
!2
Z
k=1
ϕ dγ0,1
λn (dx) = 0.
R
Notice that the non-computational argument has the advantage that it immedi(N ) ately generalizes the earlier result to cover λn for all N ∈ Z+ , not just N = 1 (cf. Exercise 2.3.24). On the other hand, the conclusion is weaker in the sense that convergence of the densities has been replaced by convergence of integrals with bounded continuous integrands and that no estimate on the rate of convergence is provided. More work is required to restore the stronger statements when N ≥ 2. When couched in terms of statistical mechanics, this result can be interpreted as a derivation of the Maxwell distribution of velocities for an ideal gas of free particles of mass 2 and having average energy 1. Exercise 2.1.16. The most frequently encountered applications of Stirling’s formula (cf. (1.3.21)) are to cases when t ∈ Z+ . That is, one is usually interested in the formula n n √ . (2.1.17) n! ∼ 2πn e
Here is a derivation of (2.1.17) as an application of The Central Limit Theorem. Namely, take {X n : n ≥ 1} to be a sequence of independent, random variables with P Xn > x = exp −(x + 1)+ , x ∈ R for all n ∈ Z+ . For n ≥ 1, note that √
−1 Z n+n 1 1 1+4 ˘ xn e−x dx P Sn+1 ∈ 0, 4 = n! 1+n Z − 12 1 √ −1 1 n −√n y nn+ 2 e−n n + 4 1+n − 12 dy. y e 1 + n = 1 n! n− 2
By The Central Limit Theorem,
Z 14 1 x2 1 ˘ e− 2 dx. P Sn ∈ 0, 4 −→ √ 2π 0
At the same time, an elementary computation shows that 1
Z
n− 2 + 14
1 n− 2
√
1+n−1
1+n
− 12
y
n
e
√ − ny
1 4
Z dy −→ 0
e−
x2 2
dx,
§ 2.2 The Berry–Esseen Theorem via Stein’s Method
71
and clearly (2.1.17) follows from these. In fact, if one applies the Berry–Esseen estimate proved in the next section, one finds that √
2πn n!
n n e
1 = 1 + O n− 2 .
However, this last observation is not very interesting since we saw in Exercise 1.3.19 that the true correction term is of order n−1 .4 § 2.2 The Berry–Esseen Theorem via Stein’s Method As we will see in the next section, the principles underlying the passage from Theorem 2.1.4 to Theorem 2.1.8 are very general. In fact, as we will see in Chapter 9, some of these principles can be formulated in such a way that they extend to a very abstract setting. However, rather than delve into such extensions here, I will devote this section to a closer examination of the situation at hand. Specifically, in this section we are going to see how to make the final part of Theorem 2.1.8 quantitative. From (2.1.5), we get a rate of convergence in terms of the second and third derivatives of ϕ. In fact, if we assume that (2.2.1)
1 τk ≡ EP |Xk |3 3 < ∞,
1 ≤ k ≤ n,
then (cf. the proof of Theorem 2.1.4), by using the estimates 000 3 Rm (ξ) ≤ kϕ ku |ξ| 6
and σk ≤ τk ,
one sees that (2.1.5) can be replaced by (2.2.2)
Pn 3 Z 000 P E ϕ S˘n − ϕ dγ0,1 ≤ 2kϕ ku 1 τk Σ3n 3 R
when the Xk ’s have third moments. Although both (2.1.5) and (2.2.2) are interesting, neither one of them can be used to get very much information about the rate at which the distribution functions (2.2.3) 4
x ∈ R 7−→ Fn (x) ≡ P S˘n ≤ x ∈ [0, 1]
As this exercise demonstrates, Stirling’s formula is intimately connected to The Central √ Limit Theorem. In fact, apart from the constant 2π, what we now call Stirling’s formula was discovered first by DeMoivre while he was proving The Central Limit Theorem for Bernoulli random variables. For more information, see, for example, Wm. Feller’s discussion of Stirling’s formula in his Introduction to Probability Theory and Its Applications, Vol. I, Wiley, Series in Probability and Math. Stat. (1968).
72
2 The Central Limit Theorem
are tending to the error function (2.2.4)
1 G(x) ≡ γ0,1 (−∞, x] = √ 2π
Z
x
t2
e− 2 dt.
−∞
To see how (2.1.5) and (2.2.2) must be modified in order to gain such information, first observe that Z ϕ0 (x) Fn (x) − G(x) dx R Z (2.2.5) P ˘ = E ϕ(Sn ) − ϕ(y) γ0,1 (dy), ϕ ∈ Cb1 (R; R . R
(To prove (2.2.5), reduce to the case in which ϕ ∈ Cc1 (R; R) and ϕ(0) = 0; and for this case apply either Fubini’s Theorem or integration by parts over the intervals (−∞, 0] and [0, ∞) separately.) Hence, in order to get information about the distance between Fn and G, we will have to learn how to replace the right-hand sides of (2.1.5) and (2.2.2) with expressions that depend only on the first derivative of ϕ. For example, if the dependence is on kϕ0 ku , then we get information about the L1 (R; R) distance between Fn and G, whereas if the dependence is on kϕ0 kL1 (R;R) , then the information will be about the uniform distance between Fn and G. § 2.2.1. L1 -Berry–Esseen. The basic idea that I will use to get estimates in terms of ϕ0 was introduced by C. Stein and is an example of a procedure known as Stein’s method.1 In the case at hand, his method stems from the trivial observation that if µ is a Borel probability measure on R and g is the Gauss kernel in (1.3.5), then µ = γ0,1 if and only if ∂ µg = 0 in the sense of Schwartz
distribution theory. Equivalently, if A+ is the raising operator (cf. 2.4.1) given D §E µ by A+ ϕ(x) = xϕ(x) − ∂ϕ(x), then, because hA+ ϕ, µi = ϕg, ∂ g , µ = γ0,1 if and only if hA+ ϕ, µi = 0 for sufficiently many test functions ϕ. In fact, as will be shown in what follows, µ will be close to γ0,1 if, in an appropriate sense, hA+ ϕ, µi is small. To make mathematics out of the preceding, I will need the following.
Lemma 2.2.6. Let ϕ ∈ C 1 (R; R), assume that kϕ0 ku < ∞, set ϕ˜ = ϕ−hϕ, γ0,1 i, and define Z x 2 x2 − t2 2 dt ∈ R. ϕ(t)e ˜ (2.2.7) x ∈ R 7−→ f (x) ≡ e −∞
Then f ∈ Cb2 (R; R), (2.2.8) 1
kf ku ≤ 2kϕ0 ku ,
q kf 0 ku ≤ 3 π2 kϕ0 ku ,
kf 00 ku ≤ 6kϕ0 ku ,
Stein provided an introduction, by way of examples, to his own method in Approximate Computation of Expectations, I.M.S., Lec. Notes & Monograph Series # 7 (1986).
§ 2.2 The Berry–Esseen Theorem via Stein’s Method
73
and f 0 (x) − xf (x) = ϕ(x), ˜
(2.2.9)
x ∈ R.
Proof: The facts that f ∈ C 1 (R; R) and that (2.2.9) holds are elementary applications of The Fundamental Theorem of Calculus. Moreover, knowing that f ∈ C 1 (R; R) and using (2.2.9), we see that f ∈ C 2 (R; R) and, in fact, that f 00 (x) − xf 0 (x) = f (x) + ϕ0 (x),
(2.2.10)
x ∈ R.
To prove the estimates in (2.2.8), first note that, because ϕ˜ and therefore f are unchanged when ϕ is replaced by ϕ − ϕ(0), I may and will assume that ϕ(0) = 0 and therefore that |ϕ(t)| ≤ kϕ0 ku |t|. In particular, this means that Z Z q ϕ dγ0,1 ≤ kϕ0 ku |t| γ0,1 (dt) = kϕ0 ku 2 . π R
R
t2
− 2 dt = 0, an alternative expression for f ϕ(t)e ˜ Z ∞ 2 x2 − t2 dt, x ∈ R. ϕ(t)e ˜ f (x) = −e 2
Next, observe that, because is
R
R
x
Thus, by using the original expression for f (x) when x ∈ (−∞, 0) and the alternative one when x ∈ [0, ∞), we see first that Z ∞ 2 x2 ϕ˜ −t sgn(x) e− t2 dt, x ∈ R, 2 |f (x)| ≤ e |x|
and then that |f (x)| ≤ kϕ0 ku e
x2 2
Z
∞
t+
|x|
q 2 π
t2
e− 2 dt.
But, since Z ∞ Z ∞ t2 x2 t2 x2 d − t e− 2 dt − 1 = 0 e 2 dt ≤ e 2 e2 dx x x
we have that, for x ∈ R, Z Z ∞ x2 t2 x2 e− 2 dt ≤ e 2 (2.2.11) |x|e 2 |x|
∞
t2
te− 2 dt = 1 and e
for x ∈ [0, ∞),
x2 2
|x|
Z
∞
|x|
t2
e− 2 dt ≤
q
π 2;
which means that I have now proved the first estimate in (2.2.8). To prove the other two estimates there, derive from (2.2.10)
x2 d − x2 0 e 2 f (x) = e− 2 f (x) + ϕ0 (x) dx
74
2 The Central Limit Theorem
and therefore that Z Z x − t2 x2 x2 0 0 f (t)+ϕ (t) e 2 dt = −e 2 f (x) = e 2 −∞
∞
t2 f (t)+ϕ0 (t) e− 2 dt,
x ∈ R.
x
Thus, reasoning as I did above and using the first estimate in (2.2.8) and the relations in (2.2.9), (2.2.10), and (2.2.11), one arrives at the second and third estimates in (2.2.8). I now have the ingredients needed to apply Stein’s method to the following example of a Berry–Esseen type of estimate. Theorem 2.2.12 (L1 -Berry–Esseen Estimate). Continuing in the setting of Theorem 2.1.4, one has that for all > 0 (cf. (2.1.3), (2.2.3), and (2.2.4)) (2.2.13)
√
Fn − G 1 2π gn (2). ≤ 6(r + ) + 3 n L (R;R)
Moreover, if (cf. (2.2.1)) τm < ∞ for each 1 ≤ m ≤ n, then (2.2.14)
Fn − G 1 ≤ L (R;R)
6rn +
3
Pn
3 m=1 τm Σ3n
Pn 3 9 m=1 τm . ∧ Σ3n
2 In particular, if σm = 1 and τm ≤ τ < ∞ for each 1 ≤ m ≤ n, then
8τ 3 6 + 2τ 3
Fn − G 1 √ . √ ≤ ≤ L (R;R) n n
Proof: Let ϕ ∈ C 1 (R; R) having bounded first derivative be given, and define f accordingly, as in (2.2.7). Everything turns on the equality in (2.2.9). Indeed, because of that equality, we know that the right-hand side of (2.2.5) is equal to n X 2 P ˘ m f (S˘n ) , EP f 0 (S˘n ) − EP S˘n f (S˘n ) = σ ˘m E f 0 (S˘n ) − EP X m=1
where I have set σ ˘m =
σm Σn
˘m = and X
Xm Σn .
Next, define
˘m T˘n,m (t) = S˘n + (t − 1)X
for t ∈ [0, 1],
˘ m , and conclude that note that T˘n,m (0) is independent of X ˘ m f (S˘n ) = EP X
Z
1
2 0 ˘ m f T˘n,m (t) dt EP X
0
2 P =σ ˘m E f 0 T˘n,m (0) +
Z 0
1
2 0 ˘ m f (T˘n,m (t) − f 0 T˘n,m (0) dt EP X
§ 2.2 The Berry–Esseen Theorem via Stein’s Method for each 1 ≤ m ≤ n. Hence, we now see that Z n n Z X X 2 (2.2.15) EP ϕ(S˘n ) − ϕ dγ0,1 = σ ˘m Am − R
where
m=1
m=1
75
1
Bm (t) dt,
0
h i Am ≡ EP f 0 S˘n ) − f 0 T˘n,m (0)
and
h i 2 ˘m Bm (t) ≡ EP X f 0 (T˘n,m (t) − f 0 T˘n,m (0) .
Obviously, by Taylor’s Theorem and H¨ older’s Inequality, for each 1 ≤ m ≤ n, τm 00 kf 00 ku (*) |Am | ≤ σ ˘m kf ku ≤ rn ∧ Σn
while, for each t ∈ [0, 1] and > 0,
i kf 0 ku h 2 2 Bm (t) ≤ 2t˘ , |Xm | ≥ 2Σn . σm kf 00 ku + 2 2 EP Xm Σn Thus, after summing over 1 ≤ m ≤ n, integrating with respect to t ∈ [0, 1], and using (2.2.5), (2.2.15), and (*), we arrive at Z ϕ0 (x) Fn (x) − G(x) dx ≤ rn + kf 00 ku + 2gn (2)kf 0 ku , R
which, in conjunction with the estimates in (2.2.8), leads immediately to the estimate in (2.2.13). In order to get (2.2.14), simply note that Z 1 h 3 i ˘ m |3 f 00 T˘n,m (st) ds ≤ tkf 00 ku τm , Bm (t) ≤ t EP |X Σ3n 0
and again use (2.2.15), (2.2.8), and (*). § 2.2.2. The Classical Berry–Esseen Theorem. The result in Theorem 2.2.12 is already significant. However, it is not the classical Berry–Esseen Theorem, which is the analogous statement about kFn − Gku . In order to prove the classical result via Stein’s method, we must learn how to replace the kϕ000 ku in Lindeberg’s Theorem by kϕ0 kL1 (R;R) . It turns out that this replacement is far more challenging than replacing kϕ000 ku by kϕ0 ku , which was the replacement needed to prove Theorem 2.2.12. The argument that I will use is a clever induction procedure that was introduced into this context by E. Bolthausen.2 But, before I can apply Bolthausen’s argument, I will need the following variation on Lemma 2.2.6. 2
The Berry–Esseen Theorem appears as a warm-up exercise in Bolthausen’s “An estimate of the remainder term in a combinatorial central limit theorem,” Z. Wahr. Gebiete 66, pp. 379–386 (1984).
76
2 The Central Limit Theorem
Lemma 2.2.16. ϕ ∈ C 1 (R; R), and define f accordingly, as in (2.2.7). p π Let 0 Then kf ku ≤ 8 kϕ kL1 (R;R) and kf 0 ku ≤ kϕ0 kL1 (R;R) .
Proof: I will assume, throughout, that kϕ0 kL1 (R;R) = 1. Observe that, by the Fundamental Theorem of Calculus, (cf. the notation in Lemma 2.2.6) Z ϕ(x) ˜ = − ϕ˜y (x) ϕ0 (y) dy, where ϕy = 1(−∞,y] , R
and so (cf. (2.2.4)) Z f (x) = − ψy (x) ϕ0 (y) dy,
where ψy (x) =
√
2πe
x2 2
G(x∧y)−G(x)G(y) ≥ 0.
R
At the same time, these, together with (2.2.9), give Z 0 f (x) = − xψy (x) + ϕ˜y (x) ϕ0 (y) dy. R
Hence, the desired estimates come down to checking that x2 e 2 G(x ∧ y) − G(x)G(y) ≤ 14 ,
and
√ x2 2πxe 2 G(x ∧ y) − G(x)G(y) + 1(−∞,y] (x) − G(y) ≤ 1
for all (x, y) ∈ R × R. But, if x ≤ y, G(x ∧ y) − G(x)G(y) ≤ G(x) − G(x)2 =
2 1 1 − 4 G(x) − 12 4
and G(x) −
1 2 2
|x|
1 = 2π
Z
1 ≥ 8π
ZZ
2 − ξ2
e
!2 dξ
0
e−
ξ2 +η 2 2
dξdη =
x2 1 1 − e− 2 , 4
ξ 2 +η 2 ≤x2
which proves the first inequality. To get the second one, it suffices to consider each of the four cases 0 ≤ x ≤ y, x ≥ 0 & y < x, y < x < 0, and x < 0 & y ≥ x separately and take into account that, from the first part of (2.2.11), √ √ x2 x2 x ≥ 0 =⇒ 2πxe 2 1−G(x) ≤ 1 and x < 0 =⇒ 2π|x|e 2 G(x) ≤ 1.
As distinguished from Lemma 2.2.6, Lemma 2.2.16 contains no estimate on kf 00 ku . Indeed, there is no such estimate in terms of kϕ0 kL1 (R;R) . As a consequence, the proof of the following is much more involved than that of Theorem 2.2.12
§ 2.2 The Berry–Esseen Theorem via Stein’s Method
77
Theorem 2.2.17 (Classical Berry–Esseen Estimate). Let everything be as in Theorem 2.1.4, and assume that (cf. (2.2.1)) τm < ∞ for each 1 ≤ m ≤ n. Then (cf. (2.2.3) and (2.2.4)) Pn kFn − Gku ≤ 10
(2.2.18)
1
3 τm
Σ3n
.
In particular, if σm = 1 for all 1 ≤ m ≤ n, then (2.2.18) can be replaced by Pn kFn − Gku ≤ 10
(2.2.19)
1
3 τm 3
n2
3 max τm √ . ≤ 10 n 1≤m≤n
Proof: For each n ∈ Z+ , let βn denote the smallest number β with the property that Pn 3 τ kFn − Gku ≤ β 1 3 m Σn
for all choices of random variables satisfying the hypotheses under which (2.2.18) is to be proved. My strategy is to give an inductive proof that βn ≤ 10 for all n ∈ Z+ ; and, because Σ1 ≤ τ1 and therefore β1 ≤ 1, I need only be concerned with n ≥ 2. ˘m, σ Given n ≥ 2 and X1 , . . . , Xn , define X ˘m , and T˘n,m (t) for 1 ≤ m ≤ n and t ∈ [0, 1] as in the proof of Theorem 2.2.12. Next, for each 1 ≤ m ≤ n, set Σn,m =
p
2 , Σ2n − σm
τ˘m =
τm , Σn
ρn =
n X
3 τ˘m ,
and ρn,m =
1
X τ` 3 . Σn,m
1≤`≤n `6=m
Finally, set Sn,m =
X
X`
1≤`≤n `6=m
Sn,m , and S˘n,m = Σn,m
and let x ∈ R 7−→ Fn,m (x) ≡ P S˘n,m ≤ x ∈ [0, 1] denote the distribution function for S˘n,m . Notice that, by definition, kFn,m − Gku ≤ βn−1 ρn,m for each 1 ≤ m ≤ n. Furthermore, because (cf. (2.1.3)) Σ2n,m 2 =1−σ ˘m ≥ 1 − rn2 Σ2n
we know first that ρn,m ≤
and ρn,m ≤
ρn 3
(1 − rn2 ) 2
,
Σn Σn,m
1 ≤ m ≤ n,
3 ρn ,
78
2 The Central Limit Theorem
and therefore that max kFn,m − Gku ≤
(2.2.20)
1≤m≤n
ρn βn−1 3
(1 − rn2 ) 2
.
Now let ϕ ∈ Cb2 (R; R) with kϕ00 kL1 (R) < ∞ be given, define f accordingly, as in (2.2.7), and let {Am : 1 ≤ m ≤ n}
and {Bm (t) : 1 ≤ m ≤ n & t ∈ [0, 1]}
be the associated quantities appearing in (2.2.15). By (2.2.9), we have that h i h i ˘ m f (S˘n ) + EP T˘n,m (0) f (S˘n ) − f T˘n,m (0) |Am | ≤ EP X h i + EP ϕ(S˘n ) − ϕ T˘n,m (0) ˘ m | kf ku + EP |X ˘ m T˘n,m (0)| kf 0 ku ≤ EP |X Z 1 P ˘ m ϕ0 T˘n,m (ξ) dξ E X + 0
Σn,m 0 ˘ m ϕ0 T˘n,m (ξ) kf ku + max EP X kf ku + Σn ξ∈[0,1] ˘ m ϕ0 T˘n,m (ξ) . kf ku + kf 0 ku + max EP X
≤σ ˘m
≤σ ˘m
ξ∈[0,1]
˘ m from T˘m,n (0), one sees that Similarly, from (2.2.9)) and the independence of X |Bm (t)| is dominated by h i h 2 i ˘ 3 f T˘n,m (t) + EP X ˘ T˘n,m (0) f T˘n,m (t) − f T˘n,m (0) t EP X m
m
h i ˘ 2 ϕ T˘n,m (t) − ϕ T˘n,m (0) + EP X m ˘ m |3 kf ku + tEP |X ˘ m |3 EP |T˘n,m (0)| kf 0 ku ≤ tEP |X Z 1 P 3 0 ˘ m ϕ T˘n,m (tξ) dξ E X +t 0
≤
3 t˘ τm
3 0 ˘ m ϕ T˘n,m (ξ) . kf ku + kf 0 ku + t max EP X ξ∈[0,1]
In order to handle the second term in the last line of each of these calculations, introduce the function Σn,m 0 ˘ y ∈ R. (ξ, ω, y) ∈ [0, 1] × Ω × R 7−→ ψ(ξ, ω, y) ≡ ϕ ξ Xm (ω) + Σn
§ 2.2 The Berry–Esseen Theorem via Stein’s Method
79
˘ m is independent of T˘n,m (0), Then, because X h Z Z P k 0 i k ˘ m ϕ T˘n,m (ξ) − ˘ E X Xm (ω) ψ(ξ, ω, y) γ0,1 (dy) P(dω) Ω R Z Z Z k ˘ ≤ Xm (ω) ψ(ξ, ω, y) dFn,m (y) − ψ(ξ, ω, y) dG(y) P(dω) Ω
Z = Ω
≤
R
R
Z ˘ m (ω) k ψ 0 (t, ω, y) G(y) − Fn,m (y) dy P(dω) X R
h k 00 i 1 ˘ m k kϕ00 kL1 (R;R) ≤ τ˘m βn−1 kϕ kL 3(R;R) ρn , EP X (1 − rn2 ) 2 (1 − rn2 ) βn−1 ρn
3 2
k ∈ {1, 3},
where I have used ψ 0 (t, ω, y) to denote the first derivative of y ∈ R 7−→ ψ(ξ, ω, y), applied (2.2.5) and (2.2.20), and noted that kψ 0 (ξ, ω, ·)kL1 (R;R) = kϕ00 kL1 (R;R)
for all (ξ, ω) ∈ [0, 1] × Ω.
At the same time, because kψ(ξ, ω, · )kL1 (R;R) =
Σn kϕ0 kL1 (R;R) Σn,m
for all (ξ, ω) ∈ [0, 1] × Ω,
we have that, for each ξ ∈ [0, 1], Z Z k kϕ0 kL1 (R;R) τ˘m ˘ m (ω)k X ψ(ξ, ω, y) γ (dy) P(dω) ≤ 0,1 1 . Ω R 2π(1 − rn2 ) 2
Hence, by combining these estimates, we arrive at 0 1 kϕ kL (R;R) βn−1 ρn 00 |Am | ≤ τ˘m kf ku + kf 0 ku + 3 kϕ kL1 (R;R) 12 + 2 2 (1 − rn ) 2π(1 − rn2 )
and
0 1 kϕ k βn−1 ρn L (R;R) 00 3 |Bm (t)| ≤ t˘ τm kf ku + kf 0 ku + 3 kϕ kL1 (R;R) 12 + 2 2 2 (1 − rn ) 2π(1 − rn )
for all 1 ≤ m ≤ n and t ∈ [0, 1], and, after putting these together with (2.2.5) and (2.2.15), we conclude that Z ϕ0 (y) G(y) − Fn (y) dy R 3 kf ku + kf 0 ku ≤ (2.2.21) 2 βn−1 kϕ00 kL1 (R;R) ρn kϕ0 kL1 (R;R) ρn . + 3 1 + (1 − rn2 ) 2 2π(1 − r2 ) 2 n
80
2 The Central Limit Theorem
I next apply (2.2.21) to a special class of ϕ’s. Namely, set 1 h(x) = 1 − x 0
if x < 0 if x ∈ [0, 1] if x > 1,
and define h (x) =
−1
Z
η −1 y h(x − y) dy
for > 0 and x ∈ R,
R
R where η ∈ Cc∞ R; [0, ∞) satisfies R η(y) dy = 1. Finally, let a ∈ R be given, and set x ∈ R and , L > 0. ϕ,L (x) = h x−a Lρn ,
It is then an easy matter to check that kϕ0,L kL1 (R;R) = 1 while kϕ00,L kL1 (R;R) ≤ 2 Lρn . Hence, by plugging the estimates from Lemma 2.2.16 into (2.2.21) and then letting & 0, we find that, for each L > 0,
1 Z a+Lρn G(y) − Fn (y) dy sup a∈R Lρn a r 2βn−1 1 π 3 ρn . + 1+ ≤ 3 1 + 8 2 (1 − rn2 ) 2 L 2π(1 − r2 ) 2
(2.2.22)
n
But 1 Lρn
Z
a
1 Fn (y) dy ≤ Fn (a) ≤ Lρn a−Lρn
Z
a+Lρn
Fn (y) dy, a
while 0≤
1 Lρn
Z
a+Lρn
G(y) dy − G(a) = a
1 Lρn
and, similarly, 1 0 ≤ G(a) − Lρn
Z
Z a
a+Lρn
Lρn (a + Lρn − y) γ0,1 (dy) ≤ √ , 8π
a
Lρn G(y) dy ≤ √ . 8π a−Lρn
Thus, from (2.2.22), we first obtain, for each L ∈ (0, ∞), 3 kFn − Gku ≤ + 2
r
L 3βn−1 3 9π ρn , + + 1 3 1 + 32 (1 − rn2 ) 2 L (8π) 2 8π(1 − rn2 ) 2
Exercises for § 2.2
81
and then, after minimizing with respect to L ∈ (0, ∞), r r − 1 9 9π 3 1 − rn2 2 + + kFn − Gku ≤ 8π 32 2 (2.2.23) r 1 3 4 18 2 −4 2 ρn . 1 − rn β + π n−1
In order to complete the proof starting from (2.2.23), we have to consider the 1 1 . Because kFn − Gku ≤ 1, or ρn < 10 two cases determined by whether ρn ≥ 10 it is obvious that we can take βn ≤ 10 in the first case. On the other hand, if 1 and we assume that βn−1 ≤ 10, then, because ρn ≤ 10 n n n X 1 X P 2 32 1 X P 3 3 σ ˘m ≥ rn3 , E Xm = E |Xm | ≥ 3 ρn = 3 Σn 1 Σn 1 1
(2.2.23) says that kFn − Gku ≤ 10ρn . Hence, in either case, βn−1 ≤ 10 =⇒ βn ≤ 10. It is clear from the preceding derivation (in particular, the final step) that the constant 10 appearing in (2.2.18) and (2.2.19) can be replaced by the smallest β > 1 that satisfies the equation r r r 1 2 − 3 1 9 9π 3 4 18 − 23 − 2 β 2 1 − β− 3 4 . + 1−β + β= + π 8π 32 2
Numerical experimentation indicates that 10 is quite a good approximation to the actual solution of this minimization problem. However, it should be recognized that, with sufficient diligence and entirely different techniques, one can show that the 10 in (2.2.18) can be replaced by a number that is less than 1. Thus, I do not claim that Stein’s method gives the best result, only that it gives whatever it gives with relatively little pain. Exercises for § 2.2 Exercise 2.2.24. It is important to know that, at least qualitatively, one cannot do better than Berry–Esseen. To see this, consider independent, symmetric, {−1, 1}-valued Bernoulli random variables, and define Fn accordingly. Next, 1 observe that when tn = −(2n + 1)− 2 , Z 0 x2 1 √ e− 2 dx F2n+1 (tn ) − G(tn ) = 2π tn 1
and therefore that limn→∞ n 2 kFn − Gku ≥ √12π . In particular, since τm = 1 for these Bernoulli random variables, we conclude that the constant in the Berry– 1 Esseen estimate cannot be smaller than (2π)− 2 .
82
2 The Central Limit Theorem
Exercise 2.2.25. Because the derivation of Theorem 2.2.12 is so elegant and simple, one wonders whether (2.2.14) can be used as the starting point for a proof of (2.2.19). Unfortunately, the following na¨ıve idea falls considerably short of the mark. Let X1 , . . . , Xn satisfy the hypotheses of Theorem 2.2.17. Starting from (2.2.14) and proceeding as I did in the passage from (2.2.22) to (2.2.23), show that for every L > 0 Pn 3 6 1 τm L +√ , kFn − Gku ≤ 3 LΣn 8π
and conclude that kFn − Gku ≤
72 π
14 Pn 1
3 τm
Σ3n Pn −3
Obviously, this is unacceptably poor when Σn
1
12
.
3 τm is small.
§ 2.3 Some Extensions of The Central Limit Theorem In most modern treatments of The Central Limit Theorem, Fourier analysis plays a central role. Indeed, the Fourier transform makes the argument so simple that it can mask what is really happening. However, now that we know Lindeberg’s argument, it is time to introduce Fourier techniques and begin to see how they facilitate reasoning involving independent random variables. § 2.3.1. The Fourier Transform. The Fourier transform of finite, C-valued, Borel measure µ on RN is the function µ ˆ : RN −→ C given by Z h√ i (2.3.1) µ ˆ(ξ) = exp −1 ξ, x RN µ(dx) for x ∈ RN . RN
When µ is a probability measure which is the distribution of an RN -valued random variable X, probabilists usual call its Fourier transform the characteristic function of X, and when µ admits a density ϕ with respect to Lebesgue measure λRN , one uses Z h√ i (2.3.2) ϕ(ξ) ˆ = exp −1 ξ, x RN ϕ(x) dx for ξ ∈ RN RN
in place of µ ˆ to denote its Fourier transform. Obviously, µ ˆ is a continuous function that is bounded by the total variation kµkvar of µ; and only slightly less obvious1 is the fact that, for ϕ ∈ Cc∞ RN ; C , ϕˆ ∈ C ∞ RN ; C and that ϕˆ as well as all its derivatives are rapidly decreasing −1 (i.e., they tend to 0 at infinity faster than 1 + |ξ|2 to any power). √ α ϕ(ξ) = (− −1ξ)α ϕ(ξ) ˆ and concludes that One uses integration byP parts to check that ∂d α ϕk |ξ|n |ϕ(ξ)| ˆ is bounded by k∂ . 1 N L (R ) kαk=n 1
§ 2.3 Some Extensions of The Central Limit Theorem
83
Lemma 2.3.3. Let µ be a finite, measure on RN . Then, for C-valued Borel N 1 N 1 N every ϕ ∈ Cb R ; C ∩ L R ; C with ϕˆ ∈ L (R ; C), Z Z 1 ˆ(ξ) dξ. ϕ(ξ) ˆ µ (2.3.4) hϕ, µi = ϕ dµ = (2π)N RN RN
Moreover, given a sequence {µn : n ∈ Z+ } of Borel probability measures and a Borel probability measure µ on RN , µ cn −→ µ ˆ uniformly on compacts if hϕ, µn i −→ hϕ, µi for every ϕ ∈ Cc RN , R . Conversely, if µ cn (ξ) −→ µ ˆ(ξ) pointwise, then hϕn , µn i −→ hϕ, µi whenever {ϕn : n ≥ 1} is a uniformly bounded sequence in Cb (RN ; C) that tends to ϕ uniformly on compacts. (Cf. Theorem 3.1.8 for more refined information on this subject.) R Proof: Choose ρ ∈ Cc∞ RN ; [0, ∞) to be an even function that satisfies RN ρ dx = 1, and set ρ (x) = −N ρ(−1 x) for ∈ (0, ∞). Next, define ψ for ∈ (0, ∞) to be the convolution ρ ? µ of ρ with µ. That is, Z ψ (x) = ρ (x − y) µ(dy) for x ∈ RN . RN
It is then easy to check that ψ ∈ Cb RN ; C and kψ kL1 (RN ;R) ≤ kµkvar for every ˆ ∈ (0, ∞). In addition, one sees Fubini’s µ(ξ). (by Theorem) that ψ (ξ) = ρˆ( ξ)ˆ N 1 N Thus, for any ϕ ∈ Cb (R ; C ∩ L R ; C , Fubini’s Theorem followed by the classical Parseval Identity (cf. Exercise 2.3.23) yields Z Z 1 ρˆ( ξ) ϕ(ξ) ˆ µ ˆ(−ξ) dξ, hϕ , µi = ϕ(x) ψ (x) dx = (2π)N RN RN
where ϕ ≡ ρ ? ϕ is the convolution of ρ with ϕ. Since, as & 0, ϕ −→ ϕ while ρˆ( ξ) −→ 1 boundedly and pointwise, (2.3.4) now follows from Lebesgue’s Dominated Convergence Theorem. Turning to the second part of the theorem, first suppose that hϕ, µn i −→ hϕ, µi for every ϕ ∈ Cc (RN ; R), and let ξn −→ ξ in C. Then, by the last part of Lemma √ √ cn (ξn ) −→ µ ˆ(ξ). 2.1.7 applied to ϕn (x) = e −1 (ξn ,x)RN and ϕ(x) = e −1 (ξ,x)RN , µ Hence, µ cn −→ µ ˆ uniformly on compacts. Conversely, suppose that µ cn −→ µ ˆ pointwise. Again by Lemma 2.1.7, we need only check that hϕ, µ i −→ hϕ, µi n when ϕ ∈ Cc∞ RN ; C . But, for such a ϕ, ϕˆ is smooth and rapidly decreasing, and therefore the result follows immediately from the first part of the present lemma together with Lebesgue’s Dominated Convergence Theorem.
Remark 2.3.5. Although it may seem too obvious to mention, an important, and rather amazing, consequence of Lemma 2.3.3 is that a finite Borel measure on RN is completely determined by its 1-dimensional marginals. To understand this remark, recall that for a linear subspace L of RN , the marginal distribution of µ on L is the measure (ΠL )∗ µ, where ΠL denotes orthogonal projection
84
2 The Central Limit Theorem
onto L. In particular, if e ∈ SN −1 and µe is the marginal distribution of µ on the 1-dimensional subspace spanned by e, then µ ˆ(ξe) = µ ce (ξ). Hence, the Fourier transform of µ is determined by the Fourier transforms of {µe : e ∈ SN −1 }, and therefore, by Lemma 2.3.3, µ can be recovered from its 1-dimensional marginals. Of course, one should be careful when applying this observation. For instance, when applied to an RN -valued random variable X = (X1 , . . . , XN ), it says that the distribution of X can be recovered from a knowledge of the distributions of (e, X)RN for all e ∈ SN −1 , but it does not say that the distributions of the coordinates Xi , 1 ≤ i ≤ N , determine the distribution of X. § 2.3.2. Multidimensional Central Limit Theorem. The great virtue of the Fourier transform is that it behaves so well under operations built out of translation. In applications to probability theory, this virtue is of particular importance when adding independent random variables. Specifically, if X and Y are independent, then the characteristic function of X + Y is the product of the characteristic functions of X and Y. This observation, combined with Lemma 2.3.3 leads to the following easy proof of The Central Limit Theorem for independent, identically distributed, R-valued random variables {Xn : n ≥ 1} with mean value 0 and variance 1. Namely, if µn is the distribution of S˘n , then n n ξ2 ξ2 −→ e− 2 = γd + o n1 = 1− µ ˆn (ξ) = µ ˆ √ξn 0,1 (ξ) 2n
for every ξ ∈ R. Actually, as we are about to see, a slight variation on the preceding will allow us to lift the results that we already know for R-valued random variables to random variables with values in RN . However, before I can state this result, I must introduce the analogs of the mean value and variance for vector-valued random variables. Thus, given an RN -valued random variable X on the probability space (Ω, F, P) with |X| ∈ L1 (P; R), the mean value EP [X] of X is the m ∈ RN that is determined by the property that (ξ, m)RN = EP ξ, X RN for all ξ ∈ RN . Similarly, if |X| is P-square integrable, then the covariance cov(X) of X is the symmetric linear transformation C on RN determined by ξ, C η RN = EP ξ, X − EP [X] RN η, X − EP [X] RN for ξ, η ∈ RN . Notice that cov(X) is not only symmetric but is also non-negative definite, since for each ξ ∈ RN , ξ, cov(X) ξ RN is nothing but the variance of (ξ, X)RN . Finally, given m ∈ RN and a symmetric, non-negative C ∈ RN ⊗ RN , I will use γm,C to denote the Borel probability measure on RN determined by the property that Z Z N 1 (dy), ϕ ∈ Cb (RN ; R), (2.3.6) ϕ dγm,C = ϕ m + C 2 y γ0,1 RN
RN
§ 2.3 Some Extensions of The Central Limit Theorem
85
1
where C 2 is the non-negative definite, symmetric square root of C Clearly, an RN -valued random variable Y has distribution γm,C if and only if, for each ξ ∈ RN , (ξ, Y)RN is a normal random variable with mean value (ξ, m)RN and variance (ξ, C ξ)RN . For this reason, γm,C is called the normal or Gaussian distribution with mean value m and covariance C. For the same reason, a random variable with γm,C as its distribution is called a normal or Gaussian random variable with mean value m and covariance C, or, more briefly, an N (m, C)-random variable. Finally, one can use this characterization to see that h√ i 1 ξ, Cξ . −1 ξ, m) − (2.3.7) γ[ (ξ) = exp m,C 2 RN
In the following statements, I will be assuming that {Xn : n ∈ Z+ } is a sequence of mutually independent, square P-integrable, RN -valued random variables on the probability space (Ω, F, P ). Further, I will assume that, for each n ∈ Z+ , Xn has mean value 0 and strictly positive definite covariance cov(Xn ). Finally, for n ∈ Z+ , set Sn =
n X
Xm ,
Cn ≡ cov(Sn ) =
m=1
n X
cov(Xm ),
m=1
1 ˘ n = Sn . Σn = det(Cn ) 2N and S Σn
˘ n is consistent Notice that when N = 1, the above use of the notation Σn and S with that in § 2.1.1. With these preparations, I am ready to prove the following multidimensional generalization of Theorem 2.1.8. Theorem 2.3.8. Referring to the preceding, assume that the limit A ≡ lim
(2.3.9)
n→∞
Cn Σ2n
exists and that (2.3.10)
n 1 X P E |Xm |2 , |Xm | ≥ Σn = 0 for each > 0. 2 n→∞ Σn m=1
lim
Then, for every sequence {ϕn : n ≥ 1} ⊆ C(RN ; C) that satisfies (2.3.11)
sup sup n≥1
y∈RN
|ϕn (y)| <∞ 1 + |y|2
86
2 The Central Limit Theorem
and converges uniformly on compacts to ϕ, ˘ n = hϕ, γ0,A i. lim EP ϕn S
(2.3.12)
n→∞
In particular, when the Xn ’s are uniformly square P-integrable random variables with mean value 0 and common covariance C,
Sn = hϕ, γ0,C i lim EP ϕn √ n→∞ n whenever {ϕn : n ≥ 1} ⊆ C(RN ; C) satisfies (2.3.11) and converges to ϕ uniformly on compacts. Proof: Given e ∈ SN −1 , set Σn (e) =
q
e, Cn e RN
and ρn (e) =
Σn (e) . Σn
p Then, ρ(e) ≡ inf n≥1 ρn (e) ∈ (0, 1] and ρn (e) −→ (e, Ae)RN as n → ∞. In particular, if (e1 , . . . , eN ) is an orthonormal basis in RN , then N N X X ˘ n |2 = ˘ n 2N = EP |S EP ei , S ρn (ei )2 R i=1
−→
N X i=1
i=1
ei , Aei
Z
RN
=
|y|2 γ0,A (dy).
RN
Hence, by Lemmas 2.1.7 and 2.3.3 plus (2.3.7), all that we have to do is check that i h √ 1 ˘ (*) fn (ξ) ≡ EP e −1 (ξ,Sn )RN −→ e− 2 (ξ,Aξ)RN
for each ξ ∈ RN . When ξ = 0, (*) is trivial. Thus, assume that ξ 6= 0, set e = (e,Sn )RN . Because S˘n (e) =
ξ |ξ| ,
and take
Σn (e)
n X 2 1 EP e, Xm RN , e, Xm RN ≥ Σn (e) 2 Σn (e) m=1
≤
n X 2 1 EP e, Xm RN , e, Xm RN ≥ ρ(e)Σn (e) 2 2 ρ(e) Σn m=1
§ 2.3 Some Extensions of The Central Limit Theorem
87
tends to 0 for each > 0, Theorem 2.1.8 combined with Lemma 2.3.3 guarantees that, for any η ∈ R, √ 2 1 ˘ EP e −1 ηn Sn (e) −→ e− 2 |η| p for any {ηn : n ≥ 1} ⊆ R that tends to η. In particular, if η = (ξ, Aξ)RN and ηn = ρn (e)|ξ|, we find that √ 1 ˘ fn (ξ) = EP e −1 ηn Sn (e) −→ e− 2 (ξ,Aξ)RN .
When C is non-degenerate, the final part is a trivial application of the initial part. When C is degenerate but not zero, one can reduce to the non-degenerate case by projecting onto the span of its eigenvectors with strictly positive eigenvalues, and when C = 0, there is nothing to do. § 2.3.3. Higher Moments. In this subsection I will show that when the Xn ’s possess higher moments, then (2.1.1) remains true for ϕ’s that can grow faster than 1 + |y|2 . As an initial step in this direction, I give the following simple example. Lemma 2.3.13. Suppose that {Xn : n ≥ 1} is a sequence of independent, identically distributed random variables with mean value 0 and variance 1. If EP [X12` ] < ∞ for some ` ∈ Z+ , then (2.1.1) holds for any ϕ ∈ C(RN ; C) that satisfies (2.3.14)
sup y∈R
|ϕ(y)| < ∞. 1 + |y|2`
Proof: Refer to the discussion in the introduction to this chapter, and observe that the argument there shows that Z (2`)! P ˘2` y 2` γ0,1 (dy) lim E Sn = ` = n→∞ 2 `! R
whenever the 2`th moment of X1 is finite. Hence the desired conclusion is an application of the last part of Lemma 2.1.7 with ψ(y) = 1 + |y|2` . In most situations one cannot carry out the computations needed to give a direct proof that the last part of Lemma 2.1.7 applies, and for this reason the following lemma is often useful. Lemma 2.3.15. Suppose that {µn : n ≥ 1} is a sequence of finite (nonnegative) Borel measures on RN , and assume µ is a finite Borel measure with the property that hϕ, µn i −→ hϕ, µi for all ϕ ∈ Cb∞ (RN ; R). If, for some ψ ∈ C RN ; [0, ∞) and p ∈ (1, ∞), (2.3.16)
suphψ p , µn i < ∞, n≥1
then hϕn , µn i −→ hϕ, µi whenever {ϕn : n ≥ 1} ⊆ C(RN ; C) is a sequence that satisfies |ϕn | ≤ ψ for all n ∈ Z+ and converges to ϕ uniformly on compacts.
88
2 The Central Limit Theorem
Proof: By Lemma 2.1.7, all that we have to prove is that hψ, µn i −→ hψ, µi. For this purpose, note that, under our present hypotheses, Lemma 2.1.7 shows that hψ, µi ≤ limn→∞ hψ, µn i < ∞ and that hψ ∧ R, µn i −→ hψ ∧ R, µi ≤ hψ, µi for each R > 0. Thus, it suffices to observe that Z suph(ψ − ψ ∧ R), µn i = sup ψ dµn ≤ R1−p suphψ p , µn i −→ 0 n≥1
n≥1
n≥1
{ψ>R}
as R → ∞. Knowing Lemma 2.3.15, one’s problem is to find conditions under which one ˘ n )] < ∞ for an interesting class of non-negative can show that supn≥1 EP [ψ(S ψ’s. One such class is provided by the notion of a sub-Gaussian random variable. Given β ∈ [0, ∞), an RN -valued random variable X is said to be β-subGaussian if β 2 |ξ|2 EP e(ξ,X)RN ≤ e 2 ,
(2.3.17)
ξ ∈ RN .
The origin of this terminology should be clear: if X ∈ N (0, σ 2 ), then equality holds in (2.3.17) with β = σ. Lemma 2.3.18. Let X be an RN -valued random variable. If X is a β-subGaussian, then EP [X] = 0, Cov(X) ≤ β 2 I, 2 − R P |X| ≥ R ≤ 2N e 2N β2 ,
R > 0,
and, for each α ∈ [0, β −1 ),
− N α2 |X|2 ≤ 1 − (αβ)2 2 . EP e 2
α2 |X|2 < ∞ for some α ∈ (0, ∞) and EP [X] = 0, then X Conversely, if A ≡ EP e 2 √ 2(1+A) . In particular, if X is a bounded random is β-sub-Gaussian when β = α variable with mean value 0, then X is β-sub-Gaussian with β ≤ 2kXkL∞ (P;RN ) . Finally, if X1 , . . . , Xn are independent random variables, and, Pnfor each 1 ≤ m ≤ n, Xm is βm -sub-Gaussian, then for any a , . . . , a ∈ R, 1 n m=1 am Xm is pPn 2 β-sub-Gaussian when β = m=1 (am βm ) .
Proof: Since the moment generating function of the sum of independent random variables is the product of the moment generating functions of the summands, the final assertion is essentially trivial. To prove the first assertion, use Lebesgue’s Dominated Convergence Theorem to justify β 2 t2 e 2 −1 −1 P t(e,X)RN =0 ±E (e, X)RN = lim t E e − 1 ≤ lim t&0 t&0 t
P
§ 2.3 Some Extensions of The Central Limit Theorem
89
and β 2 t2 EP et(e,X)RN + EP e−t(e,X)RN − 2 e 2 −1 2 = β2 ≤ 2 lim E (e, X)RN = lim t&0 t&0 t2 t2 P
for any e ∈ SN −1 . Next, from P (e, X)RN ≥ R) ≤ e
−tR
t(e,X) N β 2 t2 R E e ≤ exp −tR + 2 P
for any ≥ 0 and e ∈ SN −1 , one gets P (e, X)RN ≥ R) ≤ e over t ≥ 0. Since
−
R2 2β 2
by minimizing
1 P |X| ≥ R ≤ 2N max P (e, X)| RN ≥ N − 2 R , e∈SN −1
α2 |X|2 , use the estimate for P(|X| ≥ R) follows. To get the estimate on EP e 2 Tonelli’s Theorem to see that Z Z − N α2 |X|2 β 2 |ξ|2 = EP e(ξ,X)RN γ0,α2 I (dξ) ≤ e 2 γ0,α2 I (dξ) = 1−(αβ)2 2 . EP e 2 R
R
α2 |X|2 < ∞ for some α ∈ (0, ∞) and that EP [X] Now assume that A = EP e 2 = 0. Then 1
|ξ|2 P 2 |ξ||X| E |X| e (1 − t)EP (ξ, X)2RN et(ξ,X)RN dt ≤ 1 + 2 0 |ξ|2 A|ξ|2 |ξ|2 |ξ|22 |ξ|2 |ξ|22 P 2 α2 |X|2 4 α2 , α α e ≤ 1+ ≤1+A 2 e e E |X| e ≤1+ α2 α 2
EP e(ξ,X)RN = 1 +
Z
from which it is clear that X is β-sub-Gaussian for the prescribed β. In par α2 |X|2 α2 K 2 ≤ e 2 for all α ≥ 0, ticular, if K = kXkL∞ (P;RN ) ∈ (0, ∞), then EP e 2 and X has mean value 0, then X is β-sub-Gaussian for β = p so, if, in addition, √ t−1 (1 + etK 2 ) for all t > 0. Taking t = K −2 , we see that β = K 1 + e ≤ 2K. When kXkL∞ (P;RN ) = 0, there is nothing to do.
By combining Lemmas 2.3.15 and 2.3.18 with Theorem 2.3.8, we get the following. Theorem 2.3.19. Working in the setting and with the notation in Theorem 2.3.8, assume that, for each n ∈ Z+ , 2 EP e(ξ,Xn )RN ≤ eβn |ξ| ,
ξ ∈ RN ,
90
2 The Central Limit Theorem
where βn ∈ (0, ∞). If pPn
m=1
β ≡ sup
2 βm
Σn
n≥1
< ∞,
then (2.3.12) holds for any ϕ ∈ C(RN ; C) satisfying α2 |y|2
|ϕ(y)| ≤ Ce 2 , y ∈ RN , for some C < ∞ and α ∈ 0, β1 . In particular, if the Xn ’s are identically 2 2 distributed with covariance C and if EP eα |X1 | < ∞ for some α ∈ (0, ∞), then, for any ϕ ∈ C(RN ; C),
1 lim |y|−2 log 1 + |ϕ(y)| = 0 =⇒ lim EP ϕ n− 2 Sn = hϕ, γ0,C i. n→∞
|y|→∞
Exercises for § 2.3 Exercise 2.3.20. Here is a proof of Feller’s part of the Lindeberg–Feller Theorem. Referring to Theorem 2.1.4 and the discussion proceeding it, assume that, as n → ∞, √ ξ2 ˘ rn −→ 0 and EP e −1ξSn −→ e− 2 for all ξ ∈ R.
(i) Show that h √ ξXm i ξ 2 σ 2 m , 1 − EP e −1 Σn ≤ 2 and conclude that, for each R > 0, there is an NR such that h √ ξXm i 1 for n ≥ NR and |ξ| ≤ R. max 1 − EP e −1 Σn ≤ 1≤m≤n 2
P∞ (1−ζ)k for ζ ∈ C (ii) Take the branch of the logarithm k given by log ζ = − k=1 with |1 − ζ| < 1, and check that (1 − ζ) + log ζ ≤ |1 − ζ|2 for |1 − ζ| ≤ 12 . Conclude first that n n X h i X h √ ξXm i R2 r2 √ ξX m n log EP e −1 Σn ≤ EP 1 − e −1 Σn + 2 m=1
m=1
for n ≥ NR and |ξ| ≤ R, and then that
n X ξXm ξ2 P −→ 0 − E 1 − cos ∆n (ξ) ≡ Σn 2 m=1 uniformly for ξ’s in compacts.
Exercises for § 2.3
91
(iii) Given > 0, show that n n X ξ2 X P 2 ξXm P E Xm , |Xm | < Σn , |Xm | < Σn ≤ E 1 − cos 2 2Σ Σ n n m=1 m=1
≤
and that
n X
ξ2 ξ2 − gn () 2 2
ξXm , |Xm | ≥ Σn ≤ −2 . E 1 − cos Σ n m=1 P
Finally, combine these and apply (ii) to get limn→∞ ξ 2 gn () ≤ −2 for all ξ ∈ R.
Exercise 2.3.21. It is of some interest to know that the second moment assumption can be removed from the hypotheses in Exercise 2.1.11 and that the result there extends to Borel probability measures on RN .R To explain what I have in mind, first use that exercise to see that if σ 2 = R x2 µ(dx) < ∞, then µ = T2 µ =⇒ µR∈ N (0, σ 2 ). What I want to do now is remove the a priori assumption that R x2 µ(dx) < ∞. That is, I want to show that, for any probability measure µ on R, µ = T2 µ ⇐⇒ µ ∈ N (0, σ 2 ) for some σ ∈ [0, ∞). Since the “⇐=” direction is obvious, R and, by the discussion above, the “ =⇒ ” direction is already covered when R x2 µ(dx) < ∞, all that remains is to show that Z (2.3.22) µ = T2 µ =⇒ x2 µ(dx) < ∞. R
See Exercise 2.4.33 for an interesting application of this result. (i) Check (2.3.22) first under the condition that µ is symmetric (i.e., µ(−Γ) = µ(Γ) for all Γ ∈ BR ). Indeed, if µ is symmetric, show that Z µ ˆ(ξ) = cos(ξx) µ(dx), ξ ∈ R. R
At the same time, show that 1 1 ˆ(ξ) 2 , µ = T2 µ =⇒ µ ˆ 2− 2 ξ = µ
ξ ∈ R.
Conclude from these two that µ ˆ > 0 everywhere and that Z −n n ˆ(ξ)2 , n ∈ N and ξ ∈ R. cos 2− 2 ξx µ(dx) = µ R
Finally, note that 1 − x ≤ − log x for x ∈ (0, 1], apply this to the preceding to get Z n n ˆ(1) < ∞, n ∈ N, 2 1 − cos 2− 2 x µ(dx) ≤ − log µ R
92 and arrive at
2 The Central Limit Theorem Z
x2 µ(dx) ≤ −2 log µ ˆ(1)
R
after an application of Fatou’s Lemma. (ii) To complete the program, let µ be any solution to µ = T2 µ, and define ν by ZZ ν(Γ) = 1Γ (x − y) µ(dx)µ(dy). R2
R Check that ν is symmetric and that ν = T2 ν. Hence, by (i), R x2 ν(dx) < ∞ (in fact, ν is centered normal). Finally, use this and part (i) of Exercise 1.4.27 R to deduce that R x2 µ(dx) < ∞. (iii) Make the obvious extension of T2 to Borel probability measures µ on RN . That is, ZZ x+y µ(dx)µ(dy) for Γ ∈ BRN . T2 µ(Γ) = 1Γ 1 22 RN ×RN
Using the result just proved when N = 1, show that µ = T2 µ if and only if µ = γ0,C for some non-negative definite, symmetric C. Exercise 2.3.23. In connection with the preceding exercise, define Tα µ for α ∈ (0, ∞) and Borel probability measures µ on RN , so that ZZ 1 Tα µ(Γ) = 1Γ 2− α (x + y) µ(dx)µ(dy), Γ ∈ BRN . RN ×RN
The problem under consideration here is that of determining for which α’s there exist nontrivial (i.e., µ 6= δ0 ) solutions to the fixed point equation µ = Tα µ. Begin by reducing the problem to the case when N = 1. Next, repeat the initial argument given in part (ii) of Exercise 2.3.21 to see that there is some solution if and only if there is one that is symmetric. Assuming that µ is a non-trivial, symmetric solution, use the reasoning in part (i) there to see that Z ∞ if α ∈ (0, 2) 2 x µ(dx) = 0 if α ∈ (2, ∞). R In particular, when α ∈ (2, ∞), there are no non-trivial solutions to µ = Tα µ. (See § 3.2.3 for more on this topic.) Exercise 2.3.24. Return to the setting of Exercise 2.1.13. After noting that, so long as e ∈ Sn−1 , the distribution of √ x ∈ Sn−1 n 7−→ (e, x)Rn ∈ R
is independent of e, use Lemma 2.3.3 to prove that the assertion in (2.1.15) follows as a consequence of the one in (2.1.14).
Exercises for § 2.3
93
Exercise 2.3.25. Begin by checking the identity (cf. (1.3.20)) Z ∞ 2 s−1 s+1 − t ts e 2β2 dt = 2 2 β s+1 Γ 2 0
for all β ∈ (0, ∞) and s ∈ (−1, ∞). Use the preceding to see that, for each p ∈ (0, ∞), r p p+1 2p P σ p if X ∈ N (0, σ 2 ). Γ (2.3.26) E |X| = 2 π
The goal of the exercise is to show that the moments of sub-Gaussian random variable display similar behavior. (i) Suppose that X is β-sub-Gaussian, and show that, for each p ∈ (0, ∞), p p p p +1 . = 2 2 +1 Γ EP |X|p ≤ Kp β p where Kp ≡ p2 2 Γ 2 2
(ii) Again suppose that X is β-sub-Gaussian, and let σ 2 be its variance. Show that 2+|p−2| + σ −(1− p 2) βp EP |X|p ≥ K4 β
for each p ∈ (0, ∞). Hint: When p ≥ 2, the inequality is trivial. To prove it when p < 2, show that, for any q ∈ (1, ∞), 1 2q−p 1 σ 2 ≤ EP |X|p q EP |X| q−1 q0 ,
where q 0 =
q q−1
is the H¨ older conjugate of q.
(iii) Suppose that X1 , . . . , Xn are independent and that, for each 1 ≤ m ≤ n, 2 Xm is βm -sub-Gaussian and has variance σm . Given {a1 , . . . , an } ⊆ R, set v v u n u n n X uX uX (am βm )2 , (am σm )2 , and B = t S= am Xm , Σ = t m=1
m=1
m=1
and show that, for each p ∈ (0, ∞), + −(1− p 2) K4
Σ B
2+|p−2|
B p ≤ EP |S|p ≤ Kp B p .
In particular, if βm = β and σm = σ for all 1 ≤ m ≤ n, then + −(1− p 2)
K4
2+|p−2| σ (βA)p ≤ EP |S|p ≤ Kp (βA)p , β
v u n uX a2m . where A = t m=1
94
2 The Central Limit Theorem
(iv) The most famous case of the situation discussed in (iii) is when the Xm ’s are symmetric Bernoulli (i.e., P(Xm = ±1) = 12 ). First use (iii) in Exercise 1.3.17 or direct computation to check that Xm is 1-sub-Gaussian, and then conclude that + −(1− p 2)
(2.3.27)
K4
n X
! p2 a2m
p # " n X P ≤E am Xm ≤ Kp
n X
m=1
m=1
! p2 a2m
m=1
for all {a1 , . . . , an } ⊆ R. This fact is known as Khinchine’s Inequality. Exercise 2.3.28. Let X1 , . . . ,P Xn be independent, symmetric (Exercise 1.4.26) n random variables, and set S = 1 Xm . Show that, for each p ∈ (0, ∞) (cf. part (ii) in Exercise 2.3.25),
+ −(1− p 2)
K4
EP
n X
! p2 ! p2 n h i X 2 2 . ≤ EP |S|p ≤ Kp EP Xm Xm 1
1
Hint: Refer to the beginning of the proof of Lemma 1.1.6, and let R1 , . . . , Rn be the Rademacher functions on [0, 1), set Q = λ[0,1) × P on [0, 1) × Ω, B[0,1) × F , and observe that n X ω ∈ Ω 7−→ S(ω) ≡ Xm (ω) 1
has the same distribution under P as (t, ω) ∈ [0, 1) × Ω 7−→ T (t, ω) ≡
n X
Rm (t)Xm (ω)
1
does under Q. Next, apply Khinchine’s inequality to see that, for each ω ∈ Ω, + −(1− p 2) K4
n X 1
! p2 Xm (ω)2
Z ≤ [0,1)
T (t, ω) p dt ≤ Kp
n X
! p2 Xm (ω)2
,
1
and complete the proof by taking the P-integral of this with respect to ω. At least when p ∈ (1, ∞), I will show later that this sort of inequality holds in much greater generality. Specifically, see Burkholder’s Inequality in Theorem 6.3.6. Exercise 2.3.29. Suppose that X is an RN -valued Gaussian random variable with mean value 0 and covariance C. (i) Show that if A : RN −→ RN is a linear transformation, then AX is an N (0, ACA> ) random variable, where A> is the adjoint transformation.
Exercises for § 2.3
95
(ii) Given a linear subspace L of RN , let FL be the σ-algebra generated by {(ξ, X)RN : ξ ∈ L}, and take L⊥C to be the subspace of η such that (η, Cξ)RN = 0 for all ξ ∈ L. Show that FL is independent of FL⊥C . Hint: Show that, because of linearity, it suffices to check that √ √ √ √ EP e −1(ξ,X)RN e −1(η,X)RN = EP e −1(ξ,X)RN EP e −1(η,X)RN
for all ξ ∈ L and η ∈ L⊥C . (iii) that N = N1 + N2 , where Ni ∈ Z+ for i ∈ {1, 2}, write RN 3 x = Suppose x(1) ∈ RN1 × RN2 , and take L = {x : x(1) = 0(1) }. Show that if Π is a x(2) linear transformation taking RN onto L that satisfies ξ − Πξ, Cη RN = 0 for all ξ ∈ RN and η ∈ L, then Π> X is independent of (I − Π> )X. (iv) Write C=
C(11) C(21)
C(12) C(22)
,
where the block structure corresponds to RN = RN1 × RN2 , and assume that C(22) is non-degenerate. Show that the one and only transformation Π of the sort in part (iii) is given by Π= and therefore that Π> =
0(11) C−1 (22) C(21)
0(11) 0(21)
0(12) I(22)
,
C(12) C−1 (22) I(22)
.
Hint: Note that Πξ = 0 if ξ(2) = 0(2) , Πξ = ξ if ξ(1) = 0(1) , and that C(I − Π) (21) = 0(21) . (v) Continuing with the assumption that C(22) is non-degenerate, show that X=
C(12) C−1 (22) Y Y
+
Z 0
,
where Y is an RN2 -valued, N (0, C(22) )-random variable, Z is an RN1 -valued N (0, B) random variable with B = C(11) − C(12) C−1 (22) C(21) , and Y is indepenN1 N2 dent of Z. Conclude −→ R that is that, for any measurable F : R × R P bounded below, E F (X(1) , X(2) ) equals Z RN2
Z RN1
F x(1) , x(2) γC(12) C−1
(22)
x(2) ,B (dx(1) ) γ0,C(22) (dx(2) ).
96
2 The Central Limit Theorem
Exercise 2.3.30. Given h ∈ L2 (RN ; C), recall that the (n + 2)-fold convolution h?(n+2) is a bounded continuous function for each n ∈ N. Next, assume that h(−x) = h(x) for almost every x ∈ RN and that h ≡ 0 off of BRN (0, 1). As an application of part (iii) in Exercise 1.3.22, show that
"
?(n+2) h (x) ≤ 2khk2 2 L
(|x| − 2)+ n khk exp − N 1 N (R ;C) L (R ;C) 2n
2 #
.
Hint: Note that h ∈ L1 (RN ; C), assume that M ≡ khkL1 (RN ;C) > 0, and define Af = M −1 h ? f for f ∈ L2 (RN ; C). Show that A is a self-adjoint contraction on L2 (RN ; C), check that h?(n+2) (x) = M n Tx h, An h L2 (RN ;C) , where Tx h ≡ h( · + x), and note that Tx h, A` h L2 (RN ;C) = 0
if ` ≤ |x| − 2.
§ 2.4 An Application to Hermite Multipliers This section does not really belong here and should probably be skipped by those readers who want to restrict their attention to purely probabilistic matters. On the other hand, for those who want to see how probability theory interacts with other branches of mathematical analysis, the present section may come as something of a revelation. § 2.4.1. Hermite Multipliers. The topic of this section will be a class of linear operators called Hermite multipliers, and what will be discussed are certain boundedness properties of these operators. The setting is as follows. For n ∈ N, define (2.4.1)
Hn (x) = (−1)n e
x2 2
dn − x2 e 2 , dxn
x ∈ R.
Clearly, Hn is an nth order, real, monic (i.e., 1 is the coefficient of the highest order term) polynomial. Moreover, if we define the raising operator A+ on C 1 (R; C) by
x2 dϕ d − x2 e 2 ϕ(x) = − (x) + xϕ(x), A+ ϕ (x) = −e 2 dx dx
then (2.4.2)
Hn+1 = A+ Hn
for all n ∈ N.
x ∈ R,
§ 2.4 An Application to Hermite Multipliers
97
At the same time, if ϕ and ψ are continuously differentiable functions whose first derivatives are tempered (i.e., have at most polynomial growth at infinity), then (2.4.3)
ϕ, A+ ψ
L2 (γ
0,1 ;C)
= A− ϕ, ψ
L2 (γ0,1 ;C)
where A− is the lowering operator given by A− ϕ = (2.4.2) with (2.4.3), we see that, for all 0 ≤ m ≤ n,
Hm , Hn
L2 (γ0,1 ;C)
= Hm , An+ H0
L2 (γ0,1 ;C)
= An− Hm , H0
,
dϕ dx .
After combining
L2 (γ0,1 ;C)
= m! δm,n ,
where, at the last step, I have used the fact that Hm is a monic mth order polynomial. Hence, the (normalized) Hermite polynomials
(−1)n x2 dn − x2 Hn (x) e 2 , = √ e2 H n (x) = √ dxn n! n!
x ∈ R,
form an orthonormal set in L2 (γ0,1 ; C). (Indeed, they are one choice of the orthogonal polynomials relative to the Gauss weight.) Lemma 2.4.4. For each λ ∈ C, set
λ2 , H(x; λ) = exp λx − 2
x ∈ R.
Then (2.4.5)
H(x; λ) =
∞ X λn Hn (x), n! n=0
x ∈ R,
where the convergence is both uniform on compact subsets of R× C and, for λ’s in compact subsets of C, uniform in L2 (γ0,1 ; C). In particular, H n : n ∈ N is an orthonormal basis in L2 (γ0,1 ; C). x2
Proof: By (2.4.1) and Taylor’s expansion for the function e− 2 , it is clear that (2.4.5) holds for each (x, λ) and that the convergence is uniform on compact subsets of R × C. Furthermore, because the Hn ’s are orthogonal, the asserted uniform convergence in L2 (γ0,1 ; C) comes down to checking that
lim
m→∞
∞ n 2 X λ Hn k2 2 L (γ0,1 ;C) = 0 |λ|≤R n=m n!
sup
for every R ∈ (0, ∞), and obviously this follows from our earlier calculation that
2
Hn 2 = n!. L (γ ;C) 0,1
98
2 The Central Limit Theorem
To prove the assertion that H n : n ∈ N forms an orthonormal basis in L2 (γ0,1 ; C), it suffices to check that any ϕ ∈ L2 (γ0,1 ; C) that is orthogonal to all of the Hn ’s must be 0. But, because of the L2 (γ0,1 ; C) convergence in (2.4.5), we would have that Z ϕ(x) eλx γ0,1 (dx) = 0, λ ∈ C, R
for such a ϕ. Hence, if ψ(x) =
e−
x2 2
√
ϕ(x) , 2π
x ∈ R,
then kψkL1 (R;C) = kϕkL1 (γ0,1 ;C) ≤ kϕkL2 (γ0,1 ;C) < ∞ and (cf. (2.3.2)) ψˆ ≡ 0, which, by the L1 (R; C) Fourier inversion formula Z √ 1 ˆ dξ α&0 −→ ψ in L1 (R; C), e−α|ξ| e− −1 xξ ψ(ξ) 2π R
means that ψ and therefore ϕ vanish Lebesgue-almost everywhere. Now that we know H n : n ∈ N is an orthonormal basis, I can uniquely determine a normal operator Hθ for each θ ∈ C by specifying that
Hθ Hn = θn Hn
for each n ∈ N.
The operator Hθ is called the Hermite multiplier with parameter θ, and clearly ) ( ∞ X 2 Dom Hθ = ϕ ∈ L2 (γ0,1 ; C) : |θ|2n ϕ, H n L2 (γ0,1 ;C) < ∞ n=1
Hθ ϕ =
∞ X
θn ϕ, H N
L2 (γ0,1 ;C)
H n,
ϕ ∈ Dom Hθ .
n=0
In particular, Hθ is a contraction if and only if θ is an element of the closed unit disk D in C, and it is unitary precisely when θ ∈ S1 ≡ ∂D. Also, the adjoint of Hθ is Hθ , and so it is self-adjoint if and only if θ ∈ R. As we are about to see, there are special choices of θ for which the corresponding Hermite multiplier has interesting alternative interpretations and unexpected additional properties. For example, consider the Mehler kernel1 " 2 2 # θx − 2θxy + θy 1 exp − M (x, y; θ) = √ 2 1 − θ2 1 − θ2 1
This kernel appears in the 1866 article by Mehler referred to in the footnote following (2.1.14). √ ∞ . It arises there as the generating function for spherical harmonics on the sphere S∞
§ 2.4 An Application to Hermite Multipliers
99
for θ ∈ (0, 1) and x, y ∈ R. By a straightforward Gaussian computation (i.e., “complete the square” in the exponential) one can easily check that Z H(y; λ) M (x, y; θ) γ0,1 (dy) = H(x; θλ) R
for all θ ∈ (0, 1) and (x, λ) ∈ R × C. In conjunction with (2.4.5), this means that Z (2.4.6)
Hθ ϕ =
M ( · , y; θ) ϕ(y) γ0,1 (dy),
θ ∈ (0, 1) and ϕ ∈ L2 (γ0,1 ; C),
R
and from here it is not very difficult to prove the following properties of Hθ for θ ∈ (0, 1). Lemma 2.4.7. For each ϕ ∈ L2 (γ0,1 ; C), (θ, x) ∈ (0, 1) × R 7−→ Hθ ϕ(x) ∈ C may be chosen to be a continuous function that is non-negative if ϕ ≥ 0 Lebesgue-almost everywhere. In addition, for each θ ∈ (0, 1) and every p ∈ [1, ∞],
Hθ ϕ p L (γ
(2.4.8)
0,1 ;C)
≤ kϕkLp (γ0,1 ;C) .
Proof: The first assertions are immediate consequences of the representation in (2.4.6). To prove the second assertion, observe that Hθ 1 = 1 and therefore, as a special case of (2.4.6), Z M (x, y; θ) γ0,1 (dy) = 1
for all θ ∈ (0, 1) and x ∈ R.
R
Hence, by (2.4.6) and Jensen’s Inequality, for any p ∈ [1, ∞), Hθ ϕ (x) p ≤
Z
At the same time, by symmetry, R, and therefore
R
M (x, y; θ) |ϕ(y)|p γ0,1 (dy).
R
Z
Hθ ϕ (x) p γ0,1 (dx) ≤
ZZ
R
M (x, y; θ) γ0,1 (dx) = 1 for all (θ, y) ∈ (0, 1)×
p
Z
M (x, y; θ) |ϕ(y)| γ0,1 (dx)γ0,1 (dy) =
R
|ϕ|p dγ0,1 .
R R×R
Hence, (2.4.8) is now proved for p ∈ [1, ∞). The case when p = ∞ is even easier and is left to the reader. The conclusions drawn in Lemma 2.4.7 from the Mehler representation in (2.4.6) are interesting but not very deep (cf. Exercise 2.4.36). A deeper fact is
100
2 The Central Limit Theorem
the relationship between Hermite multipliers and the Fourier transform. For the purposes of this analysis, it is best to define the Fourier operator F by Z √ (2.4.9) Ff (ξ) = e −1 2πξx f (x) dx, ξ ∈ R, R
1
for f ∈ L (R; C). The advantage of this choice is that, without the introduction √ of any further factors of 2π, the Parseval Identity (cf. Exercise 2.4.37) becomes the statement that F determines a unitary operator on L2 (R; C). In order to relate F to Hermite multipliers, observe that, after analytically continuing the result of another simple Gaussian computation, Z ζ2 2 eζx e−πx dx = e 4π for all ζ ∈ C, R
we see from (2.4.5) that Z √ ∞ X p 2 λn 2πp x e−πx dx e −1 2πξx Hn n! R n=0
∞ n p p 2 X λ (p − 1)λ2 √ 2πp0 ξ , θpn Hn + −1 λ 2πp ξ = e−πξ =e exp n! 2 n=0 √ 1 p is the H¨ older conjugate of p and θp ≡ −1 (p − 1) 2 . Thus, where p0 = p−1 we have now proved that, for each p ∈ (1, ∞) and n ∈ N, Z √ p p 2 2 2πp0 x e−πξ . 2πp x e−πx dx = θpn Hn (2.4.10) e −1 2πξx Hn
−πξ 2
R
In particular, when p = 2, (2.4.10) says that √ n −1 hn , (2.4.11) Fhn =
n ∈ N,
where hn is the nth (un-normalized) Hermite function given by 2 1 (2.4.12) hn (x) = Hn (4π) 2 x e−πx , n ∈ N and x ∈ R.
More generally, (2.4.10) leads to the following relationship between F and Hermite multipliers. Namely, for each p ∈ (1, ∞), define Up on Lp (γ0,1 ; C) by 2 1 1 Up ϕ (x) = p 2p ϕ (2πp) 2 x e−πx , x ∈ R.
It is then an easy matter to check that Up is an isometric surjection from Lp (γ0,1 ; C) onto Lp (R; C). In addition, (2.4.10) can now be interpreted as the statement that, for every p ∈ (1, ∞) and every polynomial ϕ, ! 12 1 pp . (2.4.13) F ◦ Up ϕ = Ap Up0 ◦ Hθp ϕ where Ap ≡ 1 (p0 ) p0
See Exercise 2.4.35, where it is shown that Ap < 1 for p ∈ (0, 1).
§ 2.4 An Application to Hermite Multipliers
101
§ 2.4.2. Beckner’s Theorem. Having completed this brief introduction to Hermite multipliers, I will now address a problem to which The Central Limit Theorem has something to contribute. The problem is that of determining the set of (θ, p, q) ∈ D × (1, ∞) × (0, ∞) with p ≤ q for which Hθ determines a contraction from Lp (γ0,1 ; C) into Lq (γ0,1 ; C). In view of the preceding discussion, when θ ∈ (0, 1), a solution to this problem has implications for the Mehler transform; and, when q = p0 , the solution tells us about the Fourier operator. The role that The Central Limit Theorem plays in this analysis is hidden in the following beautiful criterion, which was first discovered by Wm. Beckner.2 Theorem 2.4.14 (Beckner). Let θ ∈ D and 1 ≤ p ≤ q < ∞ be given. Then (2.4.15)
Hθ ϕ q L (γ
0,1 ;C)
≤ kϕkLp (γ0,1 ;C)
for all
ϕ ∈ L2 (γ0,1 ; C)
if (2.4.16)
|1 − θζ|q + |1 + θζ|q 2
q1
≤
|1 − ζ|p + |1 + ζ|p 2
p1
for every ζ ∈ C. That (2.4.16) implies (2.4.15) is trivial is quite remarkable. Indeed, it takes a problem in infinite dimensional analysis and reduces it to a calculus question about functions on the complex plane. Even though, as we will see later, this reduction leads to highly non-trivial problems in calculus, Theorem 2.4.14 has to be considered a major step toward understanding the contraction properties of Hermite multipliers.3 The first step in the proof of Theorem 2.4.14 is to interpret (2.4.16) in operator theoretic language. For this the standard Bernoulli purpose, let β denote probability measure on R, BR . That is, β {±1} = 12 . Next, use χ∅ to denote the function on R that is constantly equal to 1 and χ{1} to stand for the identity function on R (i.e., χ{1} (x) = x, x ∈ R). It is then clear that χ∅ and χ{1} constitute an orthonormal basis in L2 (β; C); in fact, they are the orthogonal polynomials there. Hence, for each θ ∈ C, we can define the Bernoulli multiplier Kθ as the unique normal operator on L2 (β; C) prescribed by
Kθ χF = 2
χ∅
if F = ∅
θχ{1}
if F = {1}.
See Beckner’s “Inequalities in Fourier analysis,” Ann. Math., # 102 #1, pp. 159–182 (1975). Later, in his article “Gaussian kernels have only Gaussian maximizers,” Invent. Math. 12, pp. 179–208 (1990), E. Lieb essentially killed this line of research. His argument, which is entirely different from the one discussed here, handles not only the Hermite multipliers but essentially every operator whose kernel can be represented as the exponential of a second order polynomial. 3
102
2 The Central Limit Theorem
Furthermore, (2.4.16) is equivalent to the statement that
Kθ ϕ q (2.4.17) ≤ kϕkLp (β,C) for all ϕ ∈ L2 (β; C). L (β;C)
Indeed, it is obvious that (2.4.16) is equivalent to (2.4.17) restricted to ϕ’s of the form x ∈ R 7−→ 1 + ζx as ζ runs over C; and from this, together with the observation that every element of L2 (β; C) can be represented in the form aχ∅ + bχ{1} as (a, b) runs over C2 , one quickly concludes that (2.4.16) implies (2.4.17) for general ϕ ∈ L2 (β; C). I next want to show that (2.4.17) can be parlayed into a seemingly more general statement. To this end, define the n-fold tensor product operator Kθ⊗n on L2 (β n ; C) as follows. For F ⊆ {1, . . . , n} set χF ≡ 1 if F = ∅ and define Y χF (x) = χ{1} (xj ) for x = x1 , . . . , xn ∈ Rn j∈F
if F 6= ∅. Note that χF : F ⊆ {1, . . . , n} is an orthonormal basis for L2 (β n ; C), and define Kθ⊗n to be the unique normal operator on L2 (β n ; C) for which Kθ⊗n χF = θ|F | χF ,
(2.4.18)
F ⊆ {1, . . . , n},
where |F | is used here to denote the number of elements in the set F . Alternatively, one can describe Kθ⊗ninductively on n ∈ Z+ by saying that Kθ⊗1 = Kθ and that, for Φ ∈ C Rn+1 ; C and (x, y) ∈ Rn × R, ⊗(n+1) Kθ Φ (x, y) = Kθ Ψ(x, · ) (y) where Ψ(x, y) = Kθ⊗n Φ( · , y) (x). It is this alternative description that makes it easiest to see the extension of (2.4.17) alluded to above. Namely, what I will now show is that, for every n ∈ Z+ ,
(2.4.19) (2.4.17) =⇒ K⊗n Φ q n ≤ kΦkLp (β n ;C) , Φ ∈ L2 (β n ; C). θ
L (β ;C)
Obviously, there is nothing to do when n = 1. Next, assume (2.4.19) for n, let Φ ∈ C Rn+1 ; C be given, and define Ψ as in the second description of ⊗(n+1) Kθ Φ. Then, by (2.4.17) applied to Ψ(x, · ) for each x ∈ Rn and by the induction hypothesis applied to Φ( · , y) for each y ∈ R, we have that Z Z
⊗(n+1) q q
K
q n+1 = Φ K Ψ(x, · ) (y) β(dy) β n (dx) θ θ L (β ;C) Rn
Z
Z ≤
pq
p
|Ψ(x, y)| β(dy) Rn
|Ψ( · , y)|p
≤ Rn
Z ≤ R
Z
β (dx) =
n
Rn
R
Z
R
q
L p (β n ;C)
pq p
Ψ( · , y) β(dy)
q
L p (β n ;C)
Z pq pq
p
= Ψ( · , y) Lq (β n ;C) β(dy) β(dy) R
pq
Φ( · , y) p p n β(dy) = kΦkqLp (β n+1 ;C) , L (β ;C)
§ 2.4 An Application to Hermite Multipliers
103
where, in the passage to the third line, I have used the continuous form of Minkowski’s Inequality (it is at this point that the only essential use of the hypothesis p ≤ q is made). I am now ready to take the main step in the proof of Theorem 2.4.14. Lemma 2.4.20. Define An : L2 (β; C) −→ L2 β n ; C) by An ϕ (x) = ϕ
Pn `=1 x` √ n
for x ∈ Rn .
Then, for every pair of tempered ϕ and ψ from C(R; C),
(2.4.21) kϕkLp (γ0,1 ;C) = lim An ϕ Lp (β n ;C) for every p ∈ [1, ∞) n→∞
and (2.4.22)
Hθ ϕ, ψ
L2 (γ0,1 ;C)
= lim Kθ⊗n ◦ An ϕ, An ψ n→∞
L2 (β n ;C)
for every θ ∈ (0, 1). Moreover, if, in addition, either ϕ or ψ is a polynomial, then (2.4.22) continues to hold for all θ ∈ C. Proof: Let ϕ and ψ be tempered elements of C(R; C), and define fn (θ) = Kθ⊗n ◦ An ϕ, An ψ
L2 (β n ;C)
and f (θ) = Hθ ϕ, ψ
L2 (γ0,1 ;C)
for n ∈ Z+ and θ ∈ C. I begin by showing that (2.4.23)
θ ∈ (0, 1).
lim fn (θ) = f (θ),
n→∞
Notice that (2.4.23) is (2.4.22) for θ ∈ (0, 1) and that In (2.4.21) follows from (2.4.22) with ϕ = 1, ψ = |ϕ|p , and any θ ∈ (0, 1). In order to prove (2.4.23), I will need to introduce other expressions for f (θ) and the fn (θ)’s. To this end, set Cθ =
1 θ
θ 1
,
and, using (2.4.6), observe (cf. (2.3.6)) that Z f (θ) = ϕ(x) ψ(y) γ0,Cθ (dx × dy). R2
Next, let, for each x ∈ R\{0}, define kθ (x, · ) to be the probability measure on R such that kθ x, {±sgnx} = 1±θ 2 , and set kθ (0, · ) = β. Then it is easy to check
104
2 The Central Limit Theorem
R R that R χ{0} (y) kθ (±1, dy) =R χ{0} (±1) and R χ{1} (y) kθ (±1, dy) = θχ{1} (±1) and therefore Kθ ϕ(±1) = R ϕ(y) kθ (±1, dy) for all ϕ. Hence, if βθ be the probability measure on R2 determined by βθ (dx × dy) = kθ (x, dy) β(dx) or, equivalently, and βθ {(±1, ∓1)} = 1−θ βθ {(±1, ±1)} = 1+θ 4 , 4
then Kθ ϕ, ψ
Z
L2 (β;C)
ϕ(x) ψ(y) βθ (dx × dy).
= R2
Proceeding by induction, it follows that Z Z ⊗n Kθ Φ, Ψ 2 = ··· Φ(x) Ψ(y) βθ (dx1 × dy1 ) · · · βθ (dxn × dyn ) L (β;C)
R2
R2
for all Φ, Ψ ∈ C(Rn ; C). Hence, if (cf. Exercise 1.1.14) Ω = R2 Z+ and Pθ = βθ , then fn (θ) = E
Pθ
Z+
, F = BΩ ,
Pn 1 Zm √ , F n
where F (z) ≡ ϕ(x) ψ(y) for z = (x, y) ∈ R2 and Zn (ω) = zn , n ∈ Z+ , when ω = (z1 , . . . , zn , . . . ) ∈ Ω. Further, under Pθ , the Zn ’s are mutually independent, identically distributed R2 -valued random variables with mean value 0 and covariance Cθ . In addition, Z1 is bounded, and therefore the last part of Theorem 2.3.19 applies and guarantees that (2.4.23) holds. To complete the proof, suppose that ϕ is a polynomial of degree k. It is then an easy matter to check that An ϕ, χF L2 (β n ;C) = 0 if |F | > k,
and therefore (cf. (2.4.18)) θ ∈ C 7−→ fn (θ) ∈ C is also a polynomial of degree no more than k. Moreover, because X |F | fn (θ) = θ An ϕ, χF L2 (β n ;C) χF , An ψ L2 (β n ;C) , F
we also know that
fn (θ) ≤ |θ| ∨ 1 k An ϕ 2 n An ψ 2 n , L (β ;C) L (β ;C)
n ∈ Z+ and θ ∈ C.
Hence, because of (2.4.21) with p = 2, {fn : n ∈ Z+ } is a family of entire functions on C that are uniformly bounded on compact subsets. At the same
§ 2.4 An Application to Hermite Multipliers
105
time, because (ϕ, Hm )L2 (γ0,1 ;C) = 0 for m > k, f is also a polynomial of degree at most k, and therefore (2.4.23) already implies that the convergence extends to the whole of C and is uniform on compacts. Finally, in the case when ψ, instead of ϕ, is a polynomial, simply note that
Kθ⊗n ◦ An ϕ, An ψ
and Hθ ϕ, ψ
L2 (γ
0,1 ;C)
L2 (β n ;C)
= Hθ¯ψ, ϕ
= Kθ⊗n ¯ ◦ An ψ, An ϕ
L2 (γ0,1 ;C)
L2 (β n ;C)
, and apply the preceding.
Proof of Theorem 2.4.14: Assume that (2.4.16) holds for a given pair 1 < p ≤ q < ∞ and θ ∈ D. We then know that (2.4.19) holds for every n ∈ Z+ . Hence, by Lemma 2.4.20, if ϕ and ψ are tempered elements of C(R; C) and at least one of them is a polynomial, then Hθ ϕ, ψ L2 (γ0,1 ;C) = lim Kθ⊗n ◦ An ϕ, An ψ 2 n n→∞
L (β ;C)
≤ lim An ϕ Lp (β n ;C) An ψ Lq0 (β n ;C) = kϕkLp (γ0,1 ;C) kψkLq0 (γ0,1 ;C) . n→∞
In other words, we now know that, for all tempered ϕ and ψ from C(R; C), (2.4.24) H ϕ, ψ θ ≤ kϕkLp (γ0,1 ;C) kψkLq0 (γ0,1 ;C) L2 (γ0,1 ;C) so long as one or the other is a polynomial. To complete the proof when p ∈ (1, 2], note that, for any fixed polynomial ϕ, (2.4.24) for every tempered ψ ∈ C(R; C) guarantees that the inequality in (2.4.15) holds for that ϕ. At the same time, because p ∈ (1, 2] and the polynomials are dense in L2 (γ0,1 ; C), (2.4.15) follows immediately from its own restriction to polynomials. Finally, assume that p ∈ [2, ∞) and therefore that q 0 ∈ (1, 2]. Then, again because the polynomials are dense in L2 (γ0,1 ; C), (2.4.24) for a fixed tempered ϕ ∈ C(R; C) and all polynomials ψ implies (2.4.15) first for all tempered continuous ϕ’s and thence for all ϕ ∈ L2 (γ0,1 ; C). § 2.4.3. Applications of Beckner’s Theorem. I will now apply Theorem 2.4.14 to two important examples. The first example involves the case when θ ∈ (0, 1) and shows that the contraction property proved in Lemma 2.4.7 can be improved to say that, for each p ∈ (1, ∞) and θ ∈ (0, 1), there is a q = q(p, θ) ∈ (p, ∞) such that Hθ is a contraction on Lp (γ0,1 ; C) into Lq (γ0,1 ; C). Such an operator is said to be hypercontractive, and the fact that Hθ is hypercontractive was first proved by E. Nelson in connection with his renowned construction of a non-trivial, two-dimensional quantum field.4 The proof that 4
Nelson’s own proof appeared in his “The free Markov field,” J. Fnal. Anal. 12, pp. 12–21 (1974).
106
2 The Central Limit Theorem
I will give is entirely different from Nelson’s and is much closer to the ideas introduced by L. Gross5 as they were developed by Beckner. Theorem 2.4.25 (Nelson). Let θ ∈ (0, 1) and p ∈ (1, ∞) be given, and set q(p, θ) = 1 +
p−1 . θ2
Then
Hθ ϕ q ≤ kϕkLp (γ0,1 ;C) , L (γ0,1 ;C)
(2.4.26)
ϕ ∈ L2 (γ0,1 ; C),
for every 1 ≤ q ≤ q(p, θ). Moreover, if q > q(p, θ), then n
sup Hθ ϕ Lq (γ
(2.4.27)
0,1 ;C)
o : kϕkLp (γ0,1 ;C) = 1 = ∞.
Proof: I will leave the proof of (2.4.27) as an exercise. (Try taking ϕ’s of 2 the form eλx .) Also, because γ0,1 is a probability measure and therefore the left-hand side of (2.4.26) is non-decreasing as a function of q, I will restrict my attention to the proof of (2.4.26) for q = q(p, θ). Hence, by Theorem 2.4.14, what I have to do is prove (2.4.16) for every 1 < p < q < ∞ and θ ∈ (0, 1) that are related by (2.4.28)
θ=
p−1 q−1
12
.
I begin with the case when 1 < p < q ≤ 2, and I will first consider ζ ∈ [0, 1). Introducing the generalized binomial coefficients r r(r − 1) · · · (r − ` + 1) ≡ `! `
one can write
for r ∈ R and ` ∈ N,
∞ X q |1 − θζ|q + |1 + θζ|q =1+ (θζ)2k 2 2k k=1
and
∞ X p |1 − ζ|p + |1 + ζ|p =1+ ζ 2k . 2 2k k=1
5
See Gross’s “Logarithmic Sobolev inequalities,” Amer. J. Math. 97 #4, pp. 1061–1083 (1975). In this paper, Gross introduced the idea of proving estimates on Hθ from the corresponding estimates for Kθ . In this connection, have a look at Exercises 2.4.39 and 2.4.41.
§ 2.4 An Application to Hermite Multipliers
107
q Noting that, because q ≤ 2, 2k ≥ 0 for every k ∈ Z+ , and using the fact that, p because pq ∈ (0, 1), (1 + x) q ≤ 1 + pq x for all x ≥ 0, we see that
|1 − θζ|q + |1 + θζ|q 2
pq
∞ pX q (θζ)2k . ≤1+ q 2k k=1
Hence, I will have completed the case under consideration once I check that
∞ ∞ X p pX q 2k (θζ) ≤ ζ 2k , q 2k 2k k=1
k=1
and clearly this will follow if I show that
p p q 2k θ ≤ q 2k 2k
for each k ∈ Z+ .
But the choice of θ in (2.4.28) makes the preceding an equality when k = 1, and, when k ≥ 2, 2k p q 2k−1 Y j−q q 2k θ ≤ 1, ≤ p j−p 2k j=2
since 1 < p < q ≤ 2. At this point, I have proved (2.4.15) for 1 < p < q ≤ 2 and θ given by (2.4.28) when ζ ∈ (0, 1). Continuing with this choice of p, q, and θ, note that (2.4.15) extends immediately to ζ ∈ [−1, 1] by continuity and symmetry. Finally, for general ζ ∈ C, set a=
|1 − ζ| + |1 + ζ| , 2
b=
|1 − ζ| − |1 + ζ| , 2
and c =
b ∈ [−1, 1]. a
Then |1 ± θζ| = 1+θ 2 (1 ± ζ) +
1−θ 2 (1
∓ ζ) ≤ a ∓ θb,
and, therefore, by the preceding applied to c, we have that
1 1 |1 − θc|q + |1 + θc|q q |1 − θζ|q + |1 + θζ|q q ≤a 2 2 1 1 1 |1 − ζ|p + |1 + ζ|p p |a − b|p + |a + b|p p |1 − c|p + |1 + c|p p . = = ≤a 2 2 2
Hence, I have now completed the case when 1 < p < q ≤ 2 and θ is given by (2.4.28).
108
2 The Central Limit Theorem
To handle the other cases, I will use the equivalence of (2.4.16) and (2.4.17). Thus, what we already know is that (2.4.17) holds for 1 < p < q ≤ 2 and the θ in (2.4.28). Next, suppose that 2 ≤ p < q < ∞. Then, since 1 < q 0 < p0 ≤ 2 and
q0 − 1 p−1 , = 0 p −1 q−1
an application to q 0 and p0 of the result that we already have yields
Kθ ϕ q = sup Kθ ϕ, ψ L2 (β;C) : ψ ∈ L2 (β; C) with kψkLq0 (β) = 1 L (β;C) = sup ϕ, Kθ ψ 2 : ψ ∈ L2 (β; C) with kψkLq0 (β) = 1 L (β;C)
≤ kϕkLp (β;C) , where the θ is the one given in (2.4.28). Thus, the only case that remains is the 1 1 one when 1 < p ≤ 2 ≤ q < ∞. But, in this case, set ξ = (p − 1) 2 , η = (q − 1)− 2 , and observe that, because the associated θ in (2.4.28) is the product of ξ with η, Kθ = Kη ◦ Kξ and therefore
Kθ ϕ q ≤ Kξ ϕ L2 (β;C) ≤ kϕkLp (β;C) . L (β;C) As my second, and final, application of Theorem 2.4.14, I present the theorem of Beckner for which he concocted Theorem 2.4.14 in the first place. The result was originally by H. Weyl, who guessed, on the basis of Fh0 = √ conjectured 0 n ( −1) h0 , that the norm kFkp→p0 of F as an operator on Lp (R; C) to Lp (R; C) should be achieved by h0 . Weyl’s conjecture was partially verified by I. Babenko, who proved it when p0 is an even integer. In particular, when combined with the Riesz–Thorin Interpolation Theorem, Babenko’s result already shows (cf. Exercise 2.4.35) that kFkp→p0 < 1 for p ∈ (0, 1).
Theorem 2.4.29 (Beckner). For each p ∈ [1, 2], (2.4.30)
kFf kLp0 (R;C) ≤ Ap kf kLp (R;C) ,
f ∈ Lp (R; C) ∩ L2 (R; C),
where F is the Fourier operator in (2.4.9), A1 = 1, and Ap is the constant in 2 (2.4.13). Moreover, if f is the Gauss kernel e−πx , then (2.4.30) is an equality. Proof: Because of (2.4.11), the second part is a straightforward computation that I leave to the reader. Also, I will only consider (2.4.30) when p ∈ (1, 2), the other cases being well known (cf. Exercise 2.4.37). Because of (2.4.13), the proof of (2.4.30) comes down to showing that
Hθp ϕ p0 (2.4.31) ≤ kϕkLp (γ0,1 ;C) , ϕ ∈ Lp (γ0,1 ; C), L (γ ;C) 0,1
§ 2.4 An Application to Hermite Multipliers where θp =
√
109
1
−1 (p − 1) 2 . Indeed, by (2.4.13), (2.4.31) implies that
kFUp ϕkLp0 (R;C) ≤ Ap kϕkLp (γ0,1 ;C)
(2.4.32)
for all polynomials ϕ. Next, if ϕ ∈ L2 (γ0,1 ; C) and {ϕn : n ≥ 1} is a sequence of polynomials which tend to ϕ in L2 (γ0,1 ; C), then, because p ∈ (1, 2), it is easy to check that ϕn −→ ϕ in Lp (γ0,1 ; C) and Up ϕn −→ Up ϕ in L2 (R; C); and therefore, since F is a bounded on L2 (R; C), Fatou’s Lemma shows that (2.4.32) continues to hold for all ϕ ∈ L2 (γ0,1 ; C). Now let f ∈ Cc (R; C), and set ϕ = Up−1 f . Then, (2.4.32) implies that (2.4.30) holds for f . Finally, if f ∈ L2 (R; C) ∩ Lp (R; C), choose {fn : n ≥ 1} ⊆ Cc (R; C) so that fn −→ f in both L2 (R; C) and Lp (R; C), and conclude that (2.4.30) continues to hold. By Theorem 2.4.14, (2.4.31) will follow as soon as I prove (2.4.16) for θp . For this purpose, write √ 1 ζ = ξ + −1 (p − 1)− 2 η, where ξ, η ∈ R.
Then, because p0 − 1 = (p − 1)−1 , proving (2.4.16) for θp becomes the problem of checking that 1 h i p20 p0 i p20 h 2 2 2 + 1 + η + (p − 1)ξ 2 1 − η + (p − 1)ξ 2
(*) h
1−ξ
2
0
+ (p − 1)η
2
i p2
≤
+
h
1+ξ
2
0
+ (p − 1)η
2
i p2 p1
2
for all ξ, η ∈ R. To prove (*), consider, α ∈ (0, ∞), the function gα : [0, ∞)2 −→ [0, ∞) 1for each 1 α defined by gα (x, y) = x α +y α . It is an easy matter to check that gα is concave or convex depending on whether α ∈ [1, ∞) or α ∈ (0, 1). In particular, since p0 2
∈ (1, ∞), when we set α =
h
1−η
2
p0 2,
p0
0
+ (p − 1)ξ
2
i p20
+
h
1+η
2
+ (p − 1)ξ
2
i p20
2
gα x− , y + gα x+ , y x− + x+ ,y ≤ gα = 2 2
=
0
0
x± = |1 ± η|p , and y = (p − 1) 2 |ξ|p , we get
|1 − η|p + |1 + η|p 2
0
! 20 p
p20 + (p − 1)ξ 2
;
110
2 The Central Limit Theorem
and similarly, because
h
1−ξ
p 2
2
∈ (0, 1), 0
+ (p − 1)η
2
i p2
+
h
1+ξ
2
0
+ (p − 1)η
2
i p2
2 " ≥
|1 − ξ|p + |1 + ξ|p 2
# p2
p2
+ (p0 − 1)η 2
.
Thus, (*) will be proved once I show that 0
(**)
|1 − η|p + |1 + η|p 2
0
! 20 p
2
+(p−1)ξ ≤
|1 − ξ|p + |1 + ξ|p 2
p2
+(p0 −1)η 2 .
But because (cf. Theorems 2.4.14 and 2.4.25) we know that (2.4.16) holds with 1 p replaced by 2, q = p0 , and θ = p − 1 2 , the left side of (**) is dominated by 1
2
(p − 1)ξ +
1 − (p0 − 1) 2 η
2
1
+ 1 + (p0 − 1) 2 η 2
2
= 1 + (p − 1)ξ 2 + (p0 − 1)η 2 .
At the same time, again by (2.4.16), only this time with p, 2, and the same choice of θ, we see that the right-hand side of (**) dominates 1
0
2
(p − 1)η +
1 − (p − 1) 2 ξ
2
1
+ 1 + (p − 1) 2 ξ 2
2
= 1 + (p − 1)ξ 2 + (p0 − 1)η 2 .
Exercises for § 2.4 2 , let π1 and Exercise 2.4.33. Define S : R2 −→ R so that S(x1 , x2 ) = x1√+x 2 π2 be the natural projection maps given by πi (x1 , x2 ) = xi for i ∈ {1, 2}, and let λR denote Lebesgue measure on R. The goal of this exercise is to prove that if f : R −→ R is a Borel measurable function with the property that
(2.4.34)
f ◦S =
f ◦ π 1 + f ◦ π2 √ 2
λ2R -almost everywhere,
then there is an α ∈ R such that f (x) = αx for λR1 -almost every x ∈ R. Here are steps which one can take to prove this result. (i) After noticing that (2.4.34) holds when λR is replaced by γ0,1 , apply Exercise 2.3.21 to see that the γ0,1 -distribution of x f (x) is γ0,α for some α ∈ [0, ∞). Conclude, in particular, that f ∈ L2 (γ0,1 ; R).
Exercises for § 2.4
111
(ii) For each n ≥ 0, let Z (n) denote span {Hn ◦ π1 Hn−m ◦ π2 : 0 ≤ m ≤ n} . S∞ 2 Show that Z (m) ⊥ Z (n) in L2 (γ0,1 ; R) when m 6= n and the span of n=0 Z (n) 2 2 is dense in L2 (γ0,1 ; R). Conclude from these that if F ∈ L2 (γ0,1 ; R), then F = P∞ (n) and the series n=0 Πn F , where Πn denotes orthogonal projection onto Z convergences in L2 (γ0,1 ; R). (iii) Using the generating (2.4.5), show that n X n −n Hm ◦ π1 Hn−m ◦ π2 , Hn ◦ S = 2 2 m m=0
and use this to conclude that for any ϕ ∈ L2 (γ0,1 ; R), ϕ, Hn L2 (γ0,1 ;R) Hn ◦ S. Πn (ϕ ◦ S) = n!
(iv) Show that if ϕ ∈ L2 (γ0,1 ; R), then Πn
ϕ ◦ π 1 + ϕ ◦ π2 √ 2
ϕ, Hn
=
L2 (γ0,1 ;R) 1 2
2 n!
Hn ◦ π1 + Hn ◦ π2 .
(v) By combining (iii) and (iv), show that f, Hn L2 (γ ;R) f, Hn L2 (γ ;R) 0,1 0,1 Hn ◦ π1 + Hn ◦ π2 . (*) Hn ◦ S = 1 n! 2 2 n! From this, show that f, Hn L2 (γ0,1 ;R) = 0 unless n = 1. When n = 0, this is obvious. When n ≥ 2, one can argue that, if f, Hn L2 (γ0,1 ;R) 6= 0, then (*) implies that Hn0 ◦ π1 = Hn0 ◦ π2 , which is possible only if Hn0 is constant. Finally, P∞ 1 f, Hn L2 (γ0,1 ;R) Hn , it follows that since f = n=0 n!
f (x) = f, H1
Z
L2 (γ0,1 ;R)
H1 (x) =
ξf (ξ) γ0,1 (dξ) x
for γ0,1 -almost every x ∈ R. Exercise 2.4.35. Because the Fourier operator F (cf. (2.4.9)) is a contraction from L1 (R; C) to L∞ (R; C) as well as from L2 (R; C) into L2 (R; C), the Riesz– Thorin Interpolation Theorem guarantees that it is a contraction from Lp (R; C) 0 into Lp (R; C) for each p ∈ (0, 1). However, this is a case in which Riesz–Thorin gives a less than optimal result. Indeed, show that t ∈ 12 , 1 7−→ log A 1t ∈ R
is a strictly convex function that tends to 0 at both end points and is therefore strictly negative. Hence, Ap < 1 for p ∈ (1, 2).
112
2 The Central Limit Theorem
Exercise 2.4.36. The inequality in (2.4.8) is an example of a general principle. Namely, if (E, B) is any measurable space, then a map (x, Γ) ∈ E × B 7−→ Π(x, Γ) ∈ [0, 1] is called a transition probability whenever x ∈ E 7−→ Π(x, Γ) is B-measurable for each Γ ∈ B and Γ ∈ B 7−→ Π(x, Γ) is a probability measure on (E, B) for each x ∈ E. Given a transition probability Π(x, · ), define the linear operator Π on B(E; C) (the space of bounded, B-measurable ϕ : E −→ C) by Z Πϕ (x) = ϕ(y) Π(x, dy), x ∈ E, for ϕ ∈ B(E; C). E
Check that Π takes B(E; C) into itself and that kΠϕku ≤ kϕku . Next, given a σ-finite measure µ on (E, B), say that µ is Π-invariant if Z µ(Γ) = Π(x, Γ) µ(dx) for all Γ ∈ B. E
Using Jensen’s Inequality, first show that, for each p ∈ [1, ∞), p Πϕ (x) ≤ Π|ϕ|p (x), x ∈ E, and then that, for any Π-invariant µ, kΠϕkLp (µ;C) ≤ kϕkLp (µ;C) ,
ϕ ∈ B(E; C).
Finally, show that µ is Π-invariant if it is Π-reversing in the sense that Z Z Π x, Γ2 µ(dx) = Π y, Γ1 µ(dy) for all Γ1 , Γ2 ∈ B. Γ1
Γ2
Exercise 2.4.37. Recall the Hermite functions hn , n ∈ N, in (2.4.12) and define the normalized Hermite functions hn , n ∈ N by 1
hn =
24 1
(n!) 2
hn ,
n ∈ N.
By noting that (cf. the discussion following (2.4.12)) hn = U2 H n , show that hn : n ∈ N constitutes an orthonormal basis in L2 (R; C), and from this together with (2.4.11), arrive at Parseval’s Identity:
kFf kL2 (R;C) = kf kL2 (R;C) ,
f ∈ L1 (R; C) ∩ L2 (R; C),
and conclude that F determines a unique unitary operator F on L2 (R; C) such that Ff = Ff for f ∈ L1 (R; C) ∩ L2 (R; C). Finally, use this to verify the L2 −1 ˜ (x) ≡ Ff (−x), x ∈ R, for ˜ where Ff = F, Fourier inversion formula F f ∈ L1 (R; C) ∩ L2 (R; C).
Exercises for § 2.4
113
Exercise 2.4.38. By the same reasoning as I used to prove Theorem 2.4.29, show√ that, for any pair 1 < p ≤ 2 ≤ q < ∞ and any complex number θ = ξ + −1 η, (2.4.16) and therefore (2.4.15) hold if both (q − 1)η 2 + ξ 2 ≤ 1 and (q − 2)(ξη)2 ≤ 1 − ξ 2 − (q − 1)η 2 (p − 1) − (q − 1)α2 − β 2 .
Exercise 2.4.39. L. Gross had a somewhat different approach to the proof of (2.4.26). As in the proof that I have given, he reduced everything to checking (2.4.17). However, he did this in a different way. Namely, given b ∈ (0, 1), he set f (x) = 1 + bx and introduced the functions −t −t ft (x) ≡ Ke−t f (x) = 1+e2 f (x) + 1−e2 f (−x), (t, x) ∈ [0, ∞) × R,
and q(t) = 1 + (p − 1)e2t , t ∈ [0, ∞), and proved that
d
ft q(t) ≤ 0. (*) L (β;C) dt Following the steps below, see if you can reproduce Gross’s calculation.
(i) Set F (t) = kft kLq(t) (β;C) , and, by somewhat tedious but completely elementary differential calculus, show that Z q(t) F (t)1−q(t) ft q(t) dF dβ − q(t) ˙ f log F (t) dt (t) = q(t)2 R Z q(t)2 q(t)−1 ft (x) ft (−x) − ft (x) β(dx) . + 2 R
Next, check that Z ft (x)q(t)−1 ft (−x) − ft (x) β(dx) R Z 1 ft (x)q(t)−1 − ft (−x)q(t)−1 ft (x) − ft (−x) β(dx), = −2 R
and, after verifying that q
ξ
q−1
−η
q−1
q
4(q − 1) ξ 2 − η 2 (ξ − η) ≥ q2
2
ξ, η ∈ (0, ∞) and q ∈ (1, ∞),
,
conclude that dF dt
(**)
Z q(t) F (t)1−q(t) ft q(t) dβ − q(t) ˙ f log (t) ≤ F (t) q(t)2 R Z q(t) 2 q(t) 2 + q(t) − 1 ft (x) 2 − ft (−x) β(dx) . R
(ii) Prove the Logarithmic Sobolev Inequality Z Z 2 2 ϕ 2 dβ ≤ 2 ϕ(x) − ϕ(−x) β(dx) (2.4.40) ϕ log kϕk 2 R
for strictly positive ϕ’s on R.
L (β;C)
R
114
2 The Central Limit Theorem
Hint: Reduce to the case when ϕ(x) = 1 + bx for some b ∈ (0, 1), and, in this case, check that (2.4.40) is the elementary calculus inequality (1 + b)2 log(1 + b) + (1 − b)2 log(1 − b) − (1 + b2 ) log(1 + b2 ) ≤ 2b2 ,
b ∈ (0, 1).
(iii) By plugging (2.4.40) into (**), arrive at (*), and conclude that (2.4.17) holds for θ ∈ (0, 1) and q = 1 + p−1 θ2 .
Exercise 2.4.41. The major difference between Gross’s and Beckner’s approaches to proving Nelson’s Theorem 2.4.25 is that Gross based his proof on the equivalence of contraction results like (2.4.17) and (2.4.15) to Logarithmic Sobolev Inequalities like (2.4.40). In Exercise 2.4.38, I outlined how one passes from a Logarithmic Sobolev Inequality to a contraction result. The object of this exercise is to go in the opposite direction. Specifically, starting from (2.4.26), show that Z (2.4.42)
2
ϕ log R
2
ϕ
kϕkL2 (γ
0,1 ;C)
Z dγ0,1 ≤ 2
|ϕ0 |2 γ0,1 (dx)
R
for non-negative, continuously differentiable ϕ ∈ L2 (γ0,1 ; C) \ {0} with ϕ0 ∈ L2 (γ0,1 ; C). See Exercise 8.4.8 for another derivation. Exercise 2.4.43. As an application of Theorem 2.4.25, show that kHn kLp (γ0,1 ;C) ≤
p
n!(p − 1) for n ∈ N and p ∈ [2, ∞).
To see that this estimate is quite good, show that kH1 kpLp (γ0,1 ;C) =
p
22 1 π2
Γ
and apply Stirling’s formula (1.3.21) to conclude that kH1 kLp (γ0,1 ;C) ∼ as p → ∞.
p+1 2
p1
, 1 p−1 2 e
Chapter 3 Infinitely Divisible Laws
The results in this chapter are an attempt to answer the following question. GivenPan RN -valued random variable Y with the property that, for each n ∈ Z+ , n Y = m=1 Xm , where X1 , . . . , Xn are independent and identically distributed, what can one say about the distribution of Y? Recall that the convolution ν1 ? ν2 of two finite Borel measures ν1 and ν2 on RN is given by ZZ ν1 ? ν2 (Γ) = 1Γ (x + y) ν1 (dx)ν2 (dy), Γ ∈ BRN , RN ×RN
and that the distribution of the sum of two independent random variables is the convolution of their distributions. Thus, the analytic statement of our problem is that of describing those probability measures µ that, for each n ≥ 1, can be of some probability measure µ n1 . written as the n-fold convolution power µ?n 1 n
I will say that such a µ is infinitely divisible and will use I(RN ) to denote the class of infinitely divisible measures on RN . Since the Fourier transform takes convolution into ordinary multiplication, the Fourier formulation of this problem is that of describing those Borel probability measures on RN whose Fourier transform µ ˆ has, for each n ∈ Z+ , an nth root which is again the Fourier transform of a Borel probability measure on RN . Not surprisingly, the Fourier formulation of the problem is, in many ways, the most amenable to analysis, and it is the formulation in terms of which I will solve it in this chapter. On the other hand, this formulation has the disadvantage that, although it yields a quite satisfactory description of µ ˆ, it leaves the problem of extracting information about µ from properties of µ ˆ. For this reason, the following chapter will be devoted to developing a probabilistic understanding of the analytic answer obtained in this chapter. § 3.1 Convergence of Measures on RN In order to carry out our program, I will need two important facts about the convergence of probability measures on RN . The first of these is a minor modification of the classical Helly–Bray Theorem, and the second is an improvement, due to L´evy, of Lemma 2.3.3. 115
116
3 Infinitely Divisible Laws
Say that the sequence {µn : n ≥ 1} ⊆ M1 (RN ) converges weakly to µ ∈ M1 (RN ) and write µn =⇒ µ when hϕ, µn i −→ hϕ, µi for all ϕ ∈ Cb (RN ; C), and apply Lemma 2.3.3 to check that µn =⇒µ if and only if µ cn (ξ) −→ µ ˆ(ξ) for every N ξ∈R . § 3.1.1. Sequential Compactness in M1 (RN ). Given a subset S of M1 (RN ), I will say that S is sequentially relatively compact if, for every sequence {µn : n ≥ 1} ⊆ S, there a subsequence {µnm : m ≥ 1} and a µ ∈ M1 (RN ) such that µnm =⇒ µ. Theorem 3.1.1. and only if
A subset S of M1 (RN ) is sequentially relatively compact if
lim sup µ B(0, R){ = 0.
(3.1.2)
R→∞ µ∈S
Proof: I begin by pointing out that there is a countable set {ϕk : k ∈ Z+ } ⊆ Cc (RN ; R) of linear independent functions whose span is dense, with respect to uniform convergence, in Cc (RN ; R). To see this, choose η ∈ Cc RN ; [0, 1] so that η = 1 on B(0, 1) and 0 off B(0, 2), and set ηR (y) = η(R−1 y) for R > 0. Next, for each ` ∈ Z+ , apply the Stone–Weierstrass Theorem to choose a countable dense subset {ψj,` : j ∈ Z+ } of C B(0, 2`); R , and set ϕj,` = η` ψj,` . Clearly {ϕj,` : (j, `) ∈ (Z+ )2 } is dense in Cc (RN ; R). Finally, using lexicographic ordering of (Z+ )2 , extract a linearly independent subset {ϕk : k ∈ Z+ } by taking ϕk = ϕjk ,`k , where (j1 , `1 ) = (1, 1) and (jk+1 , `k+1 ) is the first (j, `) such that ϕj,` is linearly independent of {ϕ1 , . . . , ϕk }. Given a sequence {µn : n ≥ 1} ⊆ S, we can use a diagonalization procedure to find a subsequence {µnm : m ≥ 1} such that ak = limm→∞ hϕk , µnm i exists for every k ∈ Z+ . Next, define the linear functional Λ on the span of {ϕk : k ∈ Z+ } PK so that Λ(ϕk ) = ak . Notice that if ϕ = k=1 αk ϕk , then
K X αk hϕk , µnm i = lim hϕ, µnm i ≤ kϕku , m→∞ m→∞
Λ(ϕ) = lim
k=1
and similarly that Λ(ϕ) = limm→∞ hϕ, µnm i ≥ 0 if ϕ ≥ 0. Hence, Λ admits a unique extension as a non-negativity preserving linear functional on Cc (RN ; R) that satisfies |Λ(ϕ)| ≤ kϕku for all ϕ ∈ Cc (RN ; R). Now assume that (3.1.2) holds. For each ` ∈ Z+ , apply the Riesz Representation Theorem to produce a non-negative Borel measure ν` supported on B(0, 2`) so that hϕ, ν` i = Λ(η` ϕ) for ϕ ∈ Cc (RN ; R). Since hϕ, ν`+1 i = Λ(ϕ) = hϕ, ν` i whenever ϕ vanishes off of B(0, `), it is clear that
ν`+1 Γ ∩ B(0, ` + 1) ≥ ν`+1 Γ ∩ B(0, `) = ν` Γ ∩ B(0, `)
for all Γ ∈ BRN .
§ 3.1 Convergence of Measures on RN
117
Hence, if ∞ X µ(Γ) ≡ lim µ` Γ ∩ B(0, `) = µ` Γ ∩ B(0, `) \ B(0, ` − 1) , `→∞
`=1
then µ is a non-negative Borel measure on RN whose restriction to B(0, `) is ν` for each ` ∈ Z+ . In particular, µ(RN ) ≤ 1 and hϕ, µi = limm→∞ hϕ, µnm i for every ϕ ∈ Cc (RN ; R). Thus, by Lemma 2.1.7, all that remains is to check that µ(RN ) = 1. But µ(RN ) ≥ hη` , µi = lim hη` , µnm i ≥ lim µnm B(0, `) m→∞ m→∞ = 1 − lim µnm B(0, `){ ,
m→∞
and, by (3.1.2), the final term tends to 0 as ` → ∞. To prove the converse assertion, suppose that S is sequentially relatively compact. If (3.1.2) failed, then we could find an θ ∈ (0, 1) and, for each n ∈ Z+ , a µn ∈ S such that µn B(0, n) ≤ θ. By sequential relative compactness, this would mean that there is a subsequence {µnm : m ≥ 1} ⊆ S and a µ ∈ M1 (RN ) such that µnm =⇒ µ and µnm B(0, nm ) ≤ θ. On the other hand, for any R > 0, µ B(0, R) ≤ hηR , µi ≤ lim µnm B(0, nm ) ≤ θ, m→∞
and so we would arrive at the contradiction 1 = limR→∞ µ B(0, R) ≤ θ. § 3.1.2. L´ evy’s Continuity Theorem. My next goal is to find a test in terms of the Fourier transform to determine when (3.1.2) holds. Lemma 3.1.3. Define 1−
s(r) = inf
θ≥r
sin θ θ
for r ∈ (0, ∞).
Then s is a strictly positive, non-decreasing, continuous function that tends to 0 as r & 0. Moreover, if µ ∈ M1 (RN ), then, for all (r, R) ∈ (0, ∞)2 , (3.1.4)
1 − µ ˆ(re) ≤ rR + 2µ {y : |(e, y)RN | ≥ R} for all e ∈ SN −1 ,
and
1 µ B(0, N 2 R){ ≤ N sup µ {y : |(e, y)RN | ≥ R} e∈SN −1
(3.1.5)
≤
N max 1 − µ ˆ(ξ) : |ξ| ≤ r . s(rR)
118
3 Infinitely Divisible Laws
In particular, for any S ⊆ M1 (RN ), (3.1.2) holds if and only if lim sup 1 − µ ˆ(ξ) = 0.
(3.1.6)
|ξ|&0 µ∈S
Proof: Given (3.1.4) and (3.1.5), the final assertion is obvious. √ (3.1.4), simply observe that 1 − e −1(re,y)RN ≤ 2 ∧ r|(e, y)RN | . Turning to (3.1.5), note that
1 − µ ˆ(ξ) ≥
Z
RN
To prove
1 − cos(ξ, y)RN µ(dy).
Thus, for each e ∈ SN −1 , 1 r
r
Z
1 − µ ˆ(te) dt ≥
Z RN \{0}
0
sin r(e, y)RN 1− r(e, y)RN
!
µ(dy)
≥ s(rR)µ {y : |(e, y)RN | ≥ R} , and therefore (3.1.7)
ˆ(ξ) ≥ s(rR)µ {y : |(e, y)RN | ≥ R} . sup 1 − µ ξ∈B(0,r)
Since the first inequality in (3.1.5) is obvious, there is nothing more to be done. I am now ready to prove L´evy’s crucial improvement to Lemma 2.3.3. Theorem 3.1.8 (L´ evy’s Continuity Theorem). Let {µn : n ≥ 1} ⊆ M1 (RN ), and assume that f (ξ) = limn→∞ µ ˆn (ξ) exists for each ξ ∈ RN . Then N there is a µ ∈ M 1 (R ) such that ˆ if and only if there is a δ > 0 for which f =µ limn→∞ sup|ξ|≤δ µ ˆn (ξ) − f (ξ) = 0, in which case µn =⇒ µ. (See part (iv) of Exercise 3.1.9 for another version.) Proof: The only assertion not already covered by Lemmas 2.1.7 and 2.3.3 is the “if” part of the equivalence. But, if µ ˆn −→ f uniformly in a neighborhood of 0, then it is easy to check that supn≥1 |1 − µ ˆn (ξ)| must tend to zero as |ξ| → 0. Hence, by the last part of Lemma 3.1.3 and Theorem 3.1.1, we know that there exists a µ and a subsequence {µnm : m ≥ 1} such that µnm =⇒ µ. Since µ ˆ must equal f , Lemma 2.3.3 says that µn =⇒ µ.
Exercises for § 3.1
119
Exercises for § 3.1 Exercise 3.1.9. One might think that to address the sort of problem posed at the beginning of this chapter, it would be helpful to know which functions f : RN −→ C are the Fourier transforms of a probability measure. Such a characterization is the content of Bochner’s Theorem, whose proof will be outlined in this exercise. Unfortunately, his characterization looks more useful than it is in practice. For instance, I will not use it to solve our problem, and it is difficult to see how its use would simplify matters. In order to state Bochner’s Theorem, say that a function f : RN −→ C is N non-negative definite if, for each n ≥ 1 and ξ1 , . . . , ξn ∈ R , the matrix f (ξi − ξj ) 1≤i,j≤n is Hermitian and non-negative definite. Equivalently,1 n X
f (ξi − ξj )ζi ζ¯j ≥ 0
for all ζ1 , . . . , ζn ∈ C.
i,j=1
Then Bochner’s Theorem is the statement that f = µ ˆ for some µ ∈ M1 (RN ) if and only if f (0) = 1 and f is a continuous, non-negative definite function. (i) It is ironic that the necessity assertion is the more useful even though it is nearly trivial. Indeed, if f = µ ˆ, then it is obvious that f (0) = 1 and that f is continuous. To see that it is also non-negative definite, write n X
e
√ −1(ξi −ξj ,x)RN
2 n X √ −1(ξ ,x) i RN ζ , ζi ζ¯j = e i i=1
i,j=1
and integrate in x with respect to µ. (ii) The first step in proving the sufficiency is to use the non-negative definiteness assumption to show that f (−x) = f (x) and f (x) ≤ f (0) for all x ∈ RN . Obviously, this proves that kf ku ≤ 1. Second, using a standard Riemann approximation procedure and the continuity of f , check that, for any rapidly decreasing, continuous ψˆ : RN −→ C, ZZ ˆ ˆ ψ(η) dx dη ≥ 0. f (x − η)ψ(x) RN ×RN
In particular, when f ∈ L1 RN ; C , set −N
Z
m(x) = (2π)
e−
√
−1 (x,ξ)RN
f (ξ) dξ,
RN 1
Recall that a non-negative definite operator on a complex Hilbert space is always Hermitian.
120
3 Infinitely Divisible Laws
and use Parseval’s Identity and Fubini’s Theorem, together with elementary manipulations, to arrive at Z ZZ ˆ ˆ ψ(η) dξ dη ≥ 0 (2π)N m(x) ψ(x)2 dx = f (ξ − η)ψ(ξ) RN
RN ×RN
for all ψ ∈ L1 (RN ; R) ∩ Cb (RN ; R) with ψˆ ∈ L1 (RN ; R). Conclude that m is non negative, and use this to complete the proof in the case when f ∈ L1 RN ; C . (iii) It remains only to pass from the case when f ∈ L1 RN ; C to the general |x|2
case. For each t ∈ (0, ∞), set ft (x) = e−t 2 f (x). Clearly, ft (0) = 1 and ft ∈ Cb (RN ; C) ∩ L1 (RN ; C). In addition, show that Z n n X X ft ξi − ξj ζi ζ¯j = f ξi − ξj ζi (x)ζ¯j (x) γ0,tI (dx) ≥ 0, RN
i,j=1
i,j=1
√
where ζi (x) ≡ ζi e −1 (ξi ,x)RN . Hence, ft is also non-negative definite, and so, by part (ii), we know thatft = µbt for some µt ∈ M1 (RN ). Finally, apply L´evy’s Continuity Theorem to see that µt =⇒µ, where µ ∈ M1 (RN ) satisfies f = µ ˆ.
(iv) Let {µn : n ≥ 1} and f be as in Theorem 3.1.8. Combining Bochner’s Theorem with Lemma 2.1.7, show that there exists a µ ∈ M1 (RN ) such that f =µ ˆ and µn =⇒ µ if and only if f is continuous. Exercise 3.1.10. Suppose that f is a non-negative definite function with f (0) = 1. As we have just seen, if f is continuous, then f = µ ˆ for some µ ∈ M1 (RN ). (i) Assuming that f = µ ˆ, show that (*)
kf ku ≤ 1
and |f (η) − f (ξ)|2 ≤ 2 1 − Re f (η − ξ) ,
ξ, η ∈ RN .
Next, show that (*) follows directly from non-negative definiteness, whether or not f is continuous. Thus, a non-negative definite function is uniformly continuous everywhere if it is continuous at the origin. Hint: Both parts of (*) follow from the fact that f (η) f (ξ) 1 1 f (ξ − η) A = f (ξ)
f (η)
f (ξ − η)
1
is non-negative definite. To get the second part, consider the quadratic form v, Av C3 with v = (v1 , 1, −1).2 2
This choice of v was suggested to me by Linan Chen.
Exercises for § 3.1
121
(ii) To understand how essential a role continuity plays in Bochner’s criterion, show that f = 1{0} is non-negative definite. Even though this f cannot be the Fourier transform of any µ ∈ M1 (RN ), it is nonetheless the “Fourier transform” of a non-negativity preserving linear functional, one for which there is no Riesz representation. To be more precise, consider the linear functional Λ on the space of functions ϕ ∈ Cb (RN ; C) for which Z 1 ϕ(x) dx exists, Λϕ ≡ lim R→∞ |B(0, R)| B(0,R) √
and show that f (ξ) = Λ(eξ ), where eξ (x) = e
−1(ξ,x)RN
.
Exercise 3.1.11. It is important to recognize the extent to which L´evy’s Continuity Theorem and, as a by-product, Bochner’s Theorem, are strictly finite dimensional results. For example, let H be an infinite dimensional, separa2 1 ble, real Hilbert space, and define f (h) = e− 2 khkH . Obviously, f is a continuous and f (0)= 1. Show that it is also non-negative definite in the sense that f (hi − hj ) 1≤i,j≤n is a non-negative definite, Hermitian matrix for each n ∈ Z+ and h1 , . . . , hn ∈ H. Now suppose that there were a Borel probability measure µ on H such that Z √ µ ˆ(h) ≡ e −1(h,x)H µ(dx) = f (h), h ∈ H. H
Show that, for any orthonormal basis {ei : i ∈ Z+ } in H, the functions Xi (h) = (ei , h)H , i ∈ Z+ , would be, under µ, a sequence of independent, N (0, 1)-random variables, and conclude from this that Z Y 2 2 e−khkH µ(dh) = Eµ e−Xi = 0. H
i∈Z+
Hence, no such µ can exist. See Chapter 8 for a much more thorough account of this topic. Hint: The non-negative definiteness of f can be seen as a consequence of the analogous result for Rn . Exercise 3.1.12. The Riemann–Lebesgue Lemma says that fˆ(ξ) −→ 0 as |ξ| → ∞ if f ∈ L1 (RN ; C). Thus µ ˆ(ξ) −→ 0 as |ξ| → ∞ if µ ∈ M1 (R) is absolutely continuous. In this exercise we will examine situations in which µ ∈ M1 (R) but µ ˆ(ξ)−→ 6 0 as |ξ| → ∞. (i) Given a symmetric µ ∈ M1 (R), show that µ ˆ is real valued, and use Bochner’s Theorem to show that µ ˆ(ξ) cannot tend to a strictly negative number as |ξ| → ∞. Hint: Let α > 0, and suppose that µ ˆ(ξ) −→ −2α as |ξ| → ∞. Choose R > 0 so that µ ˆ(ξ) ≤ −α for |ξ| ≥ R and n ∈ Z+ so that (n − 1)α > 1. Set A = µ ˆ(`R − kR) 1≤k,`≤n , and show that A cannot be non-negative definite.
122
3 Infinitely Divisible Laws
(ii) Show that µ ˆ(ξ)−→ 6 0 if µ has an atom (i.e., µ({x}) > 0 for some x ∈ R). Hint: Reduce to the case in which µ is symmetric, and therefore that µ = pδ0 + qν, where p ∈ (0, 1], q = 1 − p, and ν ∈ M1 (R) is symmetric. If p = 1, µ ˆ(ξ) = 1 for all ξ. If p ∈ (0, 1), then µ ˆ(ξ) −→ 0 as |ξ| → ∞ implies νˆ(ξ) −→ − pq < 0.
(iii) To produce an example that is non-atomic, refer to Exercise 1.4.29, take p ∈ (0, 1) \ { 12 }, and let µ = µp , where µp is the measure described in that exercise. Show that µ is a non-atomic element of M1 (R) for which µ ˆ−→ 6 0 as |ξ| → ∞. Hint: Show that µ ˆ never vanishes and that µ ˆ(2m π) is independent of m ∈ Z+ .
§ 3.2 The L´ evy–Khinchine Formula Throughout, I(R ) will be the set of µ ∈ M1 (RN ) that are infinitely divisible. My strategy for characterizing I(RN ) will be to start from an easily understood subset of I(RN ) and to get the rest by taking weak limits. The elements of I(RN ) that first come to mind are the Gaussian measures (cf. (2.3.6)) γm,C . Indeed, if m ∈ RN and C is a symmetric, non-negative definite transformation on RN , then it is clear from (2.3.7) that γm,C = γ ?n m C. n ,n Unfortunately, this is not a good starting place because it is too rigid: limits of Gaussians are again Gaussian. Indeed, suppose that γmn ,Cn =⇒ µ. Then N
√
e
−1 (ξ,mn )RN − 12 (ξ,Cn ξ)RN
−→ µ ˆ(ξ)
for all ξ ∈ RN ,
and so µ = γm,C , where m = limn→∞ mn and C = limn→∞ Cn . In other words, one cannot use weak convergence to escape the class of Gaussian measures. A more fruitful choice is to start with the Poisson measures. Recall that if ν is a probability measure on RN and α ∈ [0, ∞), then the Poisson measure with jump distribution ν and jumping rate α (see § 4.2 for an explanation of this terminology) is the measure πα,ν = e−α
∞ X αn ?n ν . n! n=0
To see that πα,ν is infinitely divisible, note that Z √ −1 (ξ,y)RN − 1 ν(dy) , πd (ξ) = exp α e α,ν
. To see why the Poisson measures provide a and therefore that πα,ν = π ?n α n ,ν more hopeful choice of starting point, let m ∈ RN and a non-negative definite, symmetric C be given, and choose (e1 , . . . , eN ) to be p an orthonormal basis of eigenvectors for C. Next, set mi = (m, ei )RN and σi = (ei , Cei )RN , and take ! N N X 1X 1 . δ σi ei + δ− σ√i ei δ mi ei + νn = n 2 i=1 √n 2N i=1 n
§ 3.2 The L´evy–Khinchine Formula
123
Then the Fourier transform of π2N n,νn is exp
N X
√
n e
−1mi (ξ,ei ) N R n
i=1
! N X σi (ξ, ei )RN −1 , −1 + n cos 1 n2 i=1
which tends to γ[ m,C (ξ) as n → ∞, and so π2N n,νn =⇒ γm,C as n → ∞. Thus, one can use weak convergence to break out to the class of Poisson measures. As I will show in the next subsection, the preceding is a special case of a result (cf. Theorem 3.2.7) that says that every infinitely divisible measure is the weak limit of Poisson measures. However, before proving that result, it will be convenient to alter our description of Poisson measures. For one thing, it should be clear that, without loss in generality, I may always assume that the jump distribution ν assigns no mass to 0. Indeed, if ν({0}) = 1, then πα,ν = δ0 = π0,ν 0 no matter how α and ν 0 are chosen. If β = ν({0}) ∈ (0, 1), then πα,ν = πα0 ,ν 0 , where α0 = α(1 − β) and ν 0 = (1 − β)−1 (ν − βδ0 ). In addition, although the segregation of the rate and jumping distribution provides probabilistic insight, there is no essential reason for doing so. Thus, nothing is lost if one replaces πα,ν by πM , where M is the finite measure αµ, in which case √
Z π d M (ξ) = exp
e
−1(ξ,y)RN
− 1 M (dy) .
With these considerations in mind, let M0 (RN ) be the space of non-negative, finite Borel measures M on RN with M ({0}) = 0, and set P(RN ) = {πM : M ∈ M0 (RN )}, the space of Poisson measures on RN . § 3.2.1. I(RN ) Is the Closure of P(RN ). Let P(RN ) be the closure of P(RN ) under weak convergence. That is, µ ∈ P(RN ) if and only if there exists a sequence {Mn : n ≥ 1} ⊆ M0 (RN ) such that πMn =⇒µ. My goal here is to prove that
I(RN ) = P(RN ).
(3.2.1)
Before turning to the proof of (3.2.1), I need the following simple lemma about non-vanishing, C-valued functions. In its statement, and elsewhere, (3.2.2)
log ζ = −
∞ X (1 − ζ)m m m=1
for ζ ∈ C with |1 − ζ| < 1
is the principle branch of logarithm function on the open unit disk around 1 in the complex plane. Lemma 3.2.3. Let R ∈ (0, ∞) be given. If f ∈ C B(0, R); C \ {0} with f (0) = 1, then there is a unique `f ∈ C B(0; R); C such that `f (0) = 0 and
124
3 Infinitely Divisible Laws
f = e`f . Moreover, if ξ ∈ B(0; R), r ∈ (0, ∞), and 1 −
f (η) f (ξ)
< 1 for all
η ∈ B(ξ, r) ∩ B(0, R), then, for each η ∈ B(ξ, r) ∩ B(0, R),
`f (η) − `f (ξ) = log
f (η) , f (ξ)
and therefore
f (η) ≤ if 1 − f (ξ) if f˜ is a second element of C B(0; R); C \ {0} with Finally, f˜(ξ) 1 − f (ξ) ≤ 12 for all ξ ∈ B(0, R), then f (η) |`f (η) − `f (ξ)| ≤ 2 1 − f (ξ)
˜(ξ) f ` ˜(ξ) − `f (ξ) ≤ 2 1 − f f (ξ)
1 . 2
f˜(0) = 1 and if
for ξ ∈ B(0, R).
In particular, if {fn : n ≥ 1} ⊆ C B(0, R); C \ {0} with fn (0) = 1 for all n ≥ 1, and if fn −→ f ∈ C B(0; R); C \ {0} uniformly on B(0, R), then f (0) = 1 and `fn −→ `f uniformly on B(0; R).
Proof: To prove the existence and uniqueness of `f , begin by observing that there exists an M ∈ Z+ and 0 = r0 < r1 < · · · < rM = R such that 1 f (ξ) 1 − ≤ for 1 ≤ m ≤ M and ξ ∈ B(0, rm ) \ B(0, rm−1 ). ξ 2 f rm−1 |ξ|
Thus, we can define a function `f on B(0, R) so that `f (0) = 0 and
`f (ξ) = `f
rm−1 ξ |ξ|
+ log
f
f (ξ) rm−1 ξ |ξ|
if 1 ≤ m ≤ M and ξ ∈ B(0, rm ) \ B(0, rm−1 ).
Furthermore, working by induction on 1 ≤ m ≤ M , one sees that this `f is continuous and satisfies f = e`f . Finally, for any ` ∈ C B(0, R); C satisfying √ `(0) = 0 and f = e` , ( −12π)−1 (` − `f ) is a continuous, Z-valued function that vanishes at 0, and therefore ` = `f . Next suppose that ξ ∈ B(0, R) and that 1 − f (η) < 1 for all η ∈ B(ξ, r) ∩ B(0, R). f (ξ)
§ 3.2 The L´evy–Khinchine Formula Set `(η) = `f (ξ) + log
f (η) f (ξ)
125
for η ∈ B(ξ, r) ∩ B(0, R),
√ and check that η ( −12π)−1 `(η) − `f (η) is a continuous, Z-valued function that vanishes at ξ. Hence, ` = `f on B(0, R) ∩ B(ξ, r), and therefore on B(0, R) ∩ B(ξ, r). Since | log(1 − ζ)| ≤ 2|ζ| if |ζ| ≤ 12 , this completes the proof of the asserted properties of `f . ˜ 1 Turning to the comparison between `f and `f˜ when 1 − ff (ξ) (ξ) ≤ 2 for all
ξ ∈ B(0, R), set `(ξ) = `f (ξ) + log
f˜(ξ) f (ξ) ,
check that `(0) = 0 and f˜ = e` , and
˜
conclude that `f˜ − `f = log ff . From this, the asserted estimate for |`f˜ − `f | is immediate.
Lemma 3.2.4. Define r s(r) as in Lemma 3.1.3, and let µ ∈ M1 (RN ) and 0 < r < R be given. If |1 − µ ˆ(ξ)| ≤ 12 for all ξ ∈ B(0, r) and there is an ν ∈ M1 (RN ) such that µ = ν ?n for some
n≥
(3.2.5)
16
s
r 4R
,
then |ˆ µ(ξ)| ≥ 2−n for all ξ ∈ B(0, R).
Proof: First apply Lemma 3.2.3 to see that, because µ ˆ(ξ) = νˆ(ξ)n , neither µ ˆ nor νˆ vanishes anywhere on B(0, r) and therefore that there are unique `, `˜ ∈ ˜ ˜ µ ˆ = e` , and νˆ = e` on B(0, r). C B(0, r); C such that `(0) = 0 = `(0), ˜ Further, since µ ˆ = en` , uniqueness requires that `˜ = n1 `. Next, observe that, because ` = log µ ˆ and |1 − µ ˆ| ≤ 12 on B(0, r), |`| ≤ 2 there. Hence, because 1 ` Re` ≤ 0, |1 − νˆ| = 1 − e n ≤ n2 on B(0, r). Using this in (3.1.7), we have, for any ρ > 0 and e ∈ SN −1 , that 2 1 , max 1 − νˆ(ξ) ≤ (3.2.6) ν {y : |(e, y)RN | ≥ ρ} ≤ ns(rρ) s(rρ) ξ∈B(0,r) 4 for ξ ∈ B(0, R). Finally take which, by (3.1.4), leads to 1 − νˆ(ξ) ≤ ρR + ns(rρ) 1 ˆ(ξ) = νˆ(ξ)n to check that this gives the desired ρ = 4R , and use (3.2.5) and µ conclusion. I now have everything that I need to prove the equality (3.2.1).
Theorem 3.2.7. For each µ ∈ I(RN ) there is a unique `µ ∈ C(RN ; C) satis1 fying `µ (0) = 0 and µ ˆ = e`µ . Moreover, for each n ∈ Z+ , e n `µ is the Fourier In addition, if transform of the unique µ n1 ∈ M1 (RN ) such that µ = µ?n 1 . n
Mn ∈ M0 (RN ) is defined by (3.2.8)
Mn (Γ) ≡ nµ n1 Γ ∩ (RN \ {0})
for Γ ∈ BRN ,
126
3 Infinitely Divisible Laws
then πMn =⇒µ. Finally, I(RN ) is closed in the sense that µ ∈ I(RN ) if there exists a sequence {µk : k ≥ 1} ⊆ I(RN ) such that µk =⇒ µ. In particular, µ n1 is uniquely determined and (3.2.1) holds.
Proof: Let µ ∈ I(RN ) be given. Since there is an r > 0 such that |1− µ ˆ(ξ)| ≤ 12 + ?n for all ξ ∈ B(0, r) and, for all n ∈ Z , µ = µ 1 for some µ n1 ∈ M1 (RN ), n Lemma 3.2.4 guarantees that µ ˆ never vanishes. Hence, by Lemma 3.2.3, both the existence and uniqueness of `µ follow. Moreover, if µ = µ?n 1 , then, from n n µ ˆ(ξ) = µcn1 (ξ) , we know first that µcn1 never vanishes and then that `µ = n`, where ` is the unique element of C(RN ; C) satisfying `(0) = 0 and µcn1 = e` . In 1
particular, this proves that µ n1 = e n ` for any µ n1 with µ = µ∗n 1 , and so there is n at most one such µ n1 . Now define Mn as in the statement, and observe that
1 n `µ (ξ) − 1 1 −→ e`µ (ξ) = µ ˆ(ξ) (ξ) − 1 = exp n e πd (ξ) = exp n µ ˆ Mn n
as n → ∞. Hence, πMn =⇒µ. In particular, this proves that I(RN ) ⊆ P(RN ), and therefore, since we already know that P(RN ) ⊆ I(RN ), the final statement will follow once we check that I(RN ) is closed. To prove that I(RN ) is closed, suppose that {µk : k ≥ 1} ⊆ I(RN ) and that µk =⇒ µ. The first step in checking that µ ∈ I(RN ) is to show that µ ˆ never vanishes. To this end, use the fact that µ ˆk −→ µ ˆ uniformly on compacts to see that there must exist an r > 0 such that |1 − µ ˆk (ξ)| ≤ 12 for all k ∈ Z+ and ξ ∈ B(0, r). Hence, because each of the µk ’s is infinitely divisible, one can use Lemma 3.2.4 to see that, for each R ∈ (0, ∞),
inf{|ˆ µk (ξ)| : k ∈ Z+ and ξ ∈ B(0, R)} > 0,
and clearly this is more than enough to show that µ ˆ never vanishes. Thus we can choose a unique ` ∈ C(RN ; C) so that `(0) = 0 and µ ˆ = e` . Moreover, if `k = `µk , then, by Lemma 3.2.3, `k −→ ` uniformly on compacts. Now let n ∈ Z+ be given, . Then we know that and choose {µk, n1 : k ≥ 1} ⊆ M1 (RN ) so that µk = µ?n k, 1 1
1
n
` 1 = e n k , and so, as k → ∞, µ ˆk, n1 −→ e n ` uniformly on compacts. Hence, µ [ k, n 1
ˆ n1 for some µ n1 ∈ M1 (RN ). Since this by L´evy’s Continuity Theorem, e n ` = µ N means that µ = µ?n 1 , we have shown that µ ∈ I(R ). n
§ 3.2.2. The Formula. Theorem 3.2.7 provides interesting information, but it fails to provide a concrete characterization of the infinitely divisible laws. In this subsection I will give an explicit formula for µ ˆ when µ ∈ I(RN ), which, in view of the first part of Theorem 3.2.7, is equivalent to characterizing the functions in {`µ : µ ∈ I(RN )}.
§ 3.2 The L´evy–Khinchine Formula
127
In order to understand what follows, it may be helpful to first guess what the characterization might be. We already know two families of measures which are contained in I(RN ): the Gaussian measures γm,C for m ∈ RN and symmetric, non-negative definite C ∈ Hom(RN ; RN ), and the Poisson measures πM for M ∈ M0 (RN ). Further, it is obvious that µ, ν ∈ I(RN ) =⇒ µ ? ν ∈ I(RN ), and we know that µ ∈ I(RN ) if µn =⇒ ν for some {µn : n ≥ 1} ⊆ I(RN ). Finally, Theorem 3.2.7 tells us that every element of I(RN ) is the limit of Poisson measures. Thus, by L´evy’s Continuity Theorem, we should be asking what sort of functions can arise as the locally uniform limit of functions of the form Z h √ i √ 1 e −1(ξ,y)RN − 1 M (dy), (*) ξ ` = −1 ξ, m RN − 2 ξ, Cξ RN + RN
and, as I already noted, only the Poisson component M offers much flexibility. With this in mind, I introduce for each α ∈ [0, ∞) the class Mα (RN ) of Borel measures M on RN such that Z |y|α M (dy) < ∞. M ({0}) = 0 and α RN 1 + |y|
When M ∈ M0 (RN ), the function ` in (*) equals `µ for µ = γm,C ? πM . More generally, even if M ∈ Mα (RN ) \ M0 (RN ), for each r > 0, Mr given by M (dy) = 1[r,∞) (|y|)M (dy) is an element of M0 (RN ). Furthermore, if M ∈ M1 (RN ), then it is clear that, as r & 0, Z Z i i √−1(ξ,y) N √−1(ξ,y) N R R − 1 M (dy) − 1 Mr (dy) −→ e e RN
RN
uniformly on compacts. Thus, by L´evy’s Continuity Theorem, when M ∈ M1 (RN ), the function ` in (*) is `µ for a µ ∈ I(RN ). In order to handle M ∈ Mα (RN ) for α > 1, we must make the integrand M -integrable at 0 by √ −1(ξ,y)RN . Thus, subtracting off the next term in the Taylor expansion of e choose a Borel measurable function η : RN −→ [0, 1] that equals 1 in a neighborhood of 0, and set `r (ξ) equal to
√
−1 ξ, m
RN
−
1 2
ξ, Cξ
Z
RN
+ RN
h √ √ i e −1(ξ,y)RN − 1 − −1η(y) ξ, y RN Mr (dy).
Because `r (ξ) =
√
−1 ξ, mr
RN
−
1 2
ξ, Cξ
Z
RN
+
h
√
e
−1(ξ,y)RN
RN
Z where mr = m −
η(y)y Mr (dy), RN
i − 1 Mr (dy),
128
3 Infinitely Divisible Laws
we know that `r = `µr for µr = γmr ,C ? πMr . In addition, if M ∈ M2 (RN ) and `(ξ) equals √
1 2 (ξ, Cξ)RN
−1(ξ, m)RN −
Z + RN
h √ √ i e −1(ξ,y)RN − 1 − −1η(y) ξ, y RN M (dy),
then `r −→ ` uniformly on compacts. Hence, again by L´evy’s Continuity Theorem, we know that, for each M ∈ M2 (RN ), the function `(ξ) ≡
ξ
√
(**)
−1 ξ,m RN − 12 ξ, Cξ RN Z h √ √ i + e −1(ξ,y)RN − 1 − −1η(y) ξ, y RN M (dy) RN
equals `µ for some µ ∈ I(RN ). One might think that repeated application of the same procedure would show that one need not stop at M2 (RN ) and that more singular M ’s can occur in the representation of `. More precisely, one might try accommodating M ’s from M3 (RN ) by subtracting off the next term in the Taylor expansion. That is, one would replace Z
h
√
e
−1(ξ,y)RN
−1−
RN
√
−1η(y) ξ, y
i
RN
Mr (dy)
by Z RN
h √ √ 2 i e −1(ξ,y)RN − 1 − −1η(y) ξ, y RN + 12 η(y) ξ, y RN Mr (dy)
in the expression for `r . However, to re-write this `r in the form given in (*), one would have to replace C by Z C−
η(y)y ⊗ y Mr (dy), RN
which would destroy non-negative definiteness as r & 0. The preceding discussion is evidence for the conjecture that the functions ` of the form in (**) coincide with {`µ : µ ∈ I(RN )}, and the rest of this subsection is devoted to the verification of this conjecture. Because of their role here, elements of M2 (RN ) are called L´ evy measures. The strategy that I will adopt derives from the observation that `µ (ξ) = limn→∞ n µ ˆ n1 (ξ) − 1 . Thus, if we can understand the operation
Aµ ϕ = lim n hϕ, µ n1 i − ϕ(0) n→∞
§ 3.2 The L´evy–Khinchine Formula
129
for a sufficiently rich class of functions ϕ, then we can understand `µ (ξ) by √ √ −1(ξ,x)RN −1(ξ,x)RN is not an . Even though x e applying Aµ to x e element, for technical reasons, it turns out that the class of ϕ’s on which it is easiest to understand Aµ is the Schwartz test function space S (RN ; C) (the space of smooth C-valued functions that, together with all of their derivatives, are rapidly decreasing). The basic reason why S (RN ; C) is well suited to our analysis is that the Fourier transform maps S (RN ; C) onto itself. Further, once we understand how Aµ acts on S (RN ; C), it is a relatively simple matter to use that understanding to compute `µ (ξ).
Lemma 3.2.9. Let µ ∈ I(RN ) be given. For each r ∈ (0, ∞) there exists a C(r) < ∞ such that |`µ (ξ)| ≤ C(r)(1 + |ξ|2 ) for all ξ ∈ RN whenever µ ∈ I(RN ) satisfies |1 − µ ˆ(ξ)| ≤ 12 for ξ ∈ B(0, r). Moreover,
Aµ (c1 + ϕ) ≡ lim n hc1 + ϕ, µ n1 i − c + ϕ(0) n→∞ Z 1 `µ (ξ)ϕ(ξ) ˆ dξ = (2π)N RN
(3.2.10)
for each c ∈ C and ϕ ∈ S (RN ; C). Proof: Suppose that µ ∈ I(RN ) satisfies |1 − µ ˆ(ξ)| ≤ 12 for ξ ∈ B(0, r). Applying (3.1.4) and the second inequality in (3.2.6) with ν = µ n1 , we know that, for any (ρ, R) ∈ (0, ∞)2 ,
sup |1 − µcn1 (ξ)| ≤ ρR +
|ξ|≤R
4 . ns(rρ)
1 , we obtain sup|ξ|≤R |1 − µcn1 (ξ)| ≤ 12 Hence, if R ≥ r, then, by taking ρ = 4R and therefore sup|ξ|≤R | n1 `µ (ξ)| ≤ 2 if n satisfies (3.2.5). Finally, observe that 2 there is an > 0 such that s(t) ≥ t for t ∈ (0, 1], and therefore that |`µ (ξ)| ≤
2 1+
64R2 r 2
for |ξ| ≤ R, which completes the proof of the first assertion.
Clearly it suffices to prove (3.2.10) when c = 0. Thus, let ϕ ∈ S (RN ; C) be given. Then, by (2.3.4),
Z 1 ˆ dξ n e n `µ (ξ) − 1 ϕ(ξ) (2π) n hϕ, µ n1 i − ϕ(0) = RN Z Z 1 Z t `µ (ξ)ϕ(ξ) ˆ dξ, ˆ dξ dt −→ = e n `µ (ξ) `µ (ξ)ϕ(ξ) N
0
RN
RN 1
µ n1 (ξ)| ≤ 1, `µ (ξ) has a most quadratic where (keeping in mind that |e n `µ | = |ˆ growth, and ϕ(ξ) ˆ is rapidly decreasing) the passage to the second line is justified
130
3 Infinitely Divisible Laws
by Fubini’s Theorem and the limit is an application of Lebesgue’s Dominated Convergence Theorem. Lemma 3.2.9, especially (3.2.10), provides us with two critical pieces of information about Aµ . Namely, it tells us that Aµ satisfies the minimum principle and that it is quasi-local. To be precise, set D = R⊕S (RN ; R). That is, ϕ ∈ D if and only if there is a ϕ(∞) ∈ R such that ϕ − ϕ(∞)1 ∈ S (RN ; R). I will say that a real-valued linear functional A on D satisfies the minimum principle if (3.2.11)
Aϕ ≥ 0 if ϕ ∈ D and ϕ(0) = min ϕ(x) x∈RN
and that A is quasi-local if (3.2.12)
lim AϕR = 0
R→∞
for all ϕ ∈ D,
x for R > 0. Notice that, by applying the minimum principle where ϕR (x) = ϕ R to both 1 and −1, one knows that A1 = 0. To see that Aµ satisfies both these conditions, first observe that if ϕ(0) = minx∈RN ϕ(x), then hϕ, µ n1 i − ϕ(0) ≥ 0 for all n ∈ Z+ , and therefore that Aµ ϕ ≥ 0. Secondly, to check that Aµ is quasi-local, note that it suffices to treat N ϕ ∈ S (RN ; R) and that for such a ϕ, ϕc ˆ Thus, R (ξ) = R ϕ(Rξ). Z N `µ R−1 ξ ϕ(ξ) ˆ dξ −→ 0, (2π) Aµ ϕR = RN
since `µ (0) = 0 and ξ supR≥1 |`µ (R−1 ξ)ϕ(ξ)| ˆ is rapidly decreasing. As I am about to show, these two properties allow us to say a great deal about Aµ . Before explaining this, first observe that if M ∈ Mα (RN ), then, for every Borel measurable ϕ : RN −→ C, (3.2.13)
|ϕ(y)| < ∞ =⇒ ϕ ∈ L1 (M ; C). α y∈RN \{0} 1 ∧ |y| sup
Using (3.2.13), one can easily check that if ϕ ∈ Cb2 (RN ; C) and η ∈ S (RN ; R) equals 1 in a neighborhood of 0, then y
ϕ(y) − ϕ(0) − η(y) y, ∇ϕ(0) RN
is M -integrable for every M ∈ M2 (RN ). Second, in preparation for the proof of the next lemma, I have to introduce the following partition of unity for RN \ {0}. Choose ψ ∈ C ∞ RN ; [0, 1] so that ψ has compact support in B(0, 2) \ B 0, 14 and ψ(y) = 1 when 12 ≤ |y| ≤ 1, and set ψm (y) = ψ(2m y) for m ∈ Z. Then, if y ∈ RN and 2−m−1 ≤ |y| ≤
§ 3.2 The L´evy–Khinchine Formula
131
2−m , ψmP (y) = 1 and ψn (y) = 0 unless −m − 2 ≤ n ≤ −m + 1. Hence, if Ψ(y) = m∈Z ψm (y) for y ∈ RN \ {0}, then Ψ is a smooth function with values in [1, 4]; and therefore, for each m ∈ Z, the function χm given by χm (0) = 0 m (y) for y ∈ RN \ {0} is a smooth, [0, 1]-valued function that and χm (y) = ψΨ(y)
vanishes off of B(0, 2−m+1 ) \ B(0, 2−m−2 ). In addition, for each y ∈ RN \ {0}, P −m−2 ≤ |y| ≤ 2−m+1 . m∈Z χm (y) = 1 and χm (y) = 0 unless 2 Finally, given n ∈ Z+ and ϕ ∈ C n (RN ; C), define ∇n ϕ(x) to be the multilinear map on (RN )n into C by ! n X n ∂n ϕ x+ tm ξm . ∇ ϕ(x) (ξ1 , . . . , ξn ) = ∂t1 · · · ∂tn t1 =···=tn =0 m=1
Obviously, ∇ϕ and ∇2 ϕ can be identified as the gradient of ϕ and Hessian of ϕ. Lemma 3.2.14. Let D be the space of functions described above. If A : D −→ R is a linear functional on D that satisfiesR (3.2.11) and (3.2.12), then there is a unique M ∈ M2 (RN ) such that Aϕ = RN ϕ(y) M (dy) whenever ϕ is an element of S (RN ; R) for which ϕ(0) = 0, ∇ϕ(0) = 0, and ∇2 ϕ(0) = 0. Next, given η ∈ Cc∞ RN ; [0, 1] satisfying η = 1 in a neighborhood of 0, set ηξ (y) = η(y)(ξ, y)RN for ξ ∈ RN , and define mη ∈ RN and C ∈ Hom(RN ; RN ) by Z (3.2.15) ξ, mη ) = Aηξ and ξ, Cξ 0 RN = A ηξ ηξ0 − (ηξ ηξ0 )(y) M (dy). RN
Then C is symmetric, non-negative definite, and independent of the choice of η. Finally, for any ϕ ∈ D, Aϕ = 12 Trace C∇2 ϕ(0) + mη , ∇ϕ(0) RN Z (3.2.16) + ϕ(y) − ϕ(0) − η(y) y, ∇ϕ(0) RN M (dy). RN
Proof: Referring to the partition of unity described above, define Λm ϕ = A(χm ϕ) for ϕ ∈ C ∞ B(0, 2−m+1 ) \ B(0, 2−m−2 ); R , where
χm ϕ(y) =
χm (y)ϕ(y)
if 2−m−2 ≤ |y| ≤ 2−m+1
0
otherwise.
Clearly Λm is linear. In addition, if ϕ ≥ 0, then χm ϕ ≥ 0 = χm ϕ(0), and so, by ∞ −m+1 ) \ B(0, 2−m−2 ); R , B(0, 2 (3.2.11), Λm ϕ ≥ 0. Similarly, for any ϕ ∈ C kϕku χm ± χm ϕ ≥ 0 = kϕku χm ± χm ϕ (0), and therefore |Λm ϕ| ≤ Km kϕku , where Km = Aχm . Hence, Λm admits a unique extension as a continuous linear functional on C B(0, 2−m+1 ) \ B(0, 2−m−2 ); R that is non-negativity
132
3 Infinitely Divisible Laws
preserving and has norm Km ; and so, by the Riesz Representation Theorem, we now know that there is a unique non-negative Borel measure Mm on RN such that MRm is supported on B(0, 2−m+1 ) \ B(0, 2−m−2 ), Km = Mm (RN ), and A(χm ϕ) = RN ϕ(y) Mm (dy) for all ϕ ∈ S (RN ; R). P Now define the non-negative Borel measure M on RN by M = m∈Z Mm . Clearly, M ({0}) = 0. In addition, if ϕ ∈ Cc∞ RN \ {0}; R , then there is an n ∈ Z+ such that χm ϕ ≡ 0 unless |m| ≤ n. Thus,
Aϕ =
n X
A(χm ϕ) =
m=−n
m=−n
= RN
ϕ(y) Mm (dy)
RN
!
n X
Z
Z n X
χm (y)ϕ(y)
Z M (dy) =
ϕ(y) M (dy), RN
m=−n
and therefore Z (3.2.17)
Aϕ =
ϕ(y) M (dy) RN
for ϕ ∈ Cc∞ RN \ {0}; R . Before taking the next step, observe that, as an application of (3.2.11), if ϕ1 , ϕ2 ∈ D, then ϕ1 ≤ ϕ2 and ϕ1 (0) = ϕ2 (0) =⇒ Aϕ1 ≤ Aϕ2 .
(*)
Indeed, by linearity, this reduces to the observation that, by (3.2.11), if ϕ ∈ D is non-negative and ϕ(0) = 0, then Aϕ ≥ 0. With these preparations, I can show that, for any ϕ ∈ D, Z ϕ ≥ 0 = ϕ(0) =⇒
(**)
ϕ(y) M (dy) ≤ Aϕ. RN
Pn To check this, apply (*) to ϕn = m=−n χm ϕ and ϕ, and use (3.2.17) together with the Monotone Convergence Theorem to conclude that Z
Z ϕn (y) M (dy) = lim Aϕn ≤ Aϕ.
ϕ(y) M (dy) = lim
n→∞
RN
RN
n→∞
Now let η be as in the statement of the lemma, and set ηR (y) = η(R−1 y) for R > 0. By (**) with ϕ(y) = |y|2 η(y) we know that Z RN
|y|2 η(y) M (dy) ≤ Aϕ < ∞.
§ 3.2 The L´evy–Khinchine Formula
133
At the same time, by (3.2.17) and (*), Z 1 − η(y) ηR (y) M (dy) ≤ A(1 − η) RN
for all R > 0, and therefore, by Fatou’s Lemma, Z 1 − η(y) M (dy) ≤ A(1 − η) < ∞. RN
Hence, I have proved that M ∈ M2 (RN ). I am now in a position to show that (3.2.17) continues to hold for any ϕ ∈ S (RN ; R) that vanishes along with its first and second order derivatives at 0. To this end, first suppose that ϕ vanishes in a neighborhood of 0. Then, for each R > 0, (3.2.17) applies to ηR ϕ, and so Z ηR (y)ϕ(y) M (dy) = A(ηR ϕ) = Aϕ + A (1 − ηR )ϕ . RN
By (*) applied to ±(1 − ηR )ϕ and (1 − ηR )kϕku , A (1 − ηR )ϕ ≤ kϕku A(1 − ηR ) = −kϕku AηR −→ 0
as R → ∞,
where I used (3.2.12) to get the limit assertion. Thus, Z Z Aϕ = lim ηR (y)ϕ(y) M (dy) = ϕ(y) M (dy), R→∞
RN
RN
because, since M is finite on the support of ϕ and therefore ϕ is M -integrable, Lebesgue’s Dominated Convergence Theorem applies. I still have to replace the assumption that ϕ vanishes in a neighborhood of 0 by the assumption that it vanishes to second order there. For this purpose, first note that, by (3.2.13), ϕ is certainly M -integrable, and therefore Z ϕ(y) M (dy) = lim A (1 − ηR )ϕ = Aϕ − lim A(ηR ϕ). RN
R&0
R&0
By our assumptions about ϕ at 0, we can find a C < ∞ such that |ηR ϕ(y)| ≤ CR|y|2 η(y) for all R ∈ (0, 1]. Hence, by (*) and the M -integrability of |y|2 η(y), there is a C 0 < ∞ such that |A(ηR ϕ)| ≤ C 0 R for small R > 0, and therefore A(ηR ϕ) −→ 0 as R & 0. To complete the proof from here, let ϕ ∈ S (RN ; R) be given, and set ϕ(x) ˜ = ϕ(x) − ϕ(0) − η(x) x, ∇ϕ(0) RN − 12 η(x)2 x, ∇2 ϕ(0)x RN .
Then, by the preceding, (3.2.17) holds for ϕ˜ and, after one re-arranges terms, says that (3.2.16) holds. Thus, the properties of C are all that remain to be proved. That C is symmetric requires no comment. In addition, from (*), it is clearly non-negative definite. Finally, to see that it is independent of the η chosen, let η 0 be a second choice, note that ηξ0 = ηξ in a neighborhood of 0, and apply (3.2.17).
134
3 Infinitely Divisible Laws
Remark 3.2.18. A careful examination of the proof of Lemma 3.2.14 reveals a lot. Specifically, it shows why the operation performed by the linear functional A cannot be of order greater than 2. The point is, that, because of the minimum principle, A acts as a bounded, non-negative linear functional on the difference between ϕ and its second order Taylor polynomial, and, because of quasi-locality, this action can be represented by integration against a non-negative measure. The reason why the second order Taylor polynomial suffices is that second order polynomials are, apart from constants, the lowest order polynomials that can have a definite sign. In order to complete the program, I need to introduce the notion of a L´ evy system, which is a triple (m, C, M ) consisting of an m ∈ RN , a symmetric, nonnegative definite transformation C on RN , and a L´evy measure M ∈ M2 (RN ). Given a L´evy system (m, C, M ) and a Borel measurable η : RN −→ [0, 1] satisfying ! (3.2.19)
sup
|y|
−1
1 − η(y)
! ∨
sup
η(y)|y|
< ∞,
y∈B(0,1) /
y∈B(0,1)\{0}
we will need to know that Z √ √ 1 −1(ξ,y)RN N M (dy) −1η(y) ξ, y) − 1 − e R 2 1 + |ξ| (3.2.20) RN is bounded and tends to 0 as |ξ| → ∞.
To see this, note that, for each r ∈ (0, 1],
√ √ −1(ξ,y)RN − 1 − −1η(y) ξ, y)RN M (dy) e RN Z √ √ −1(ξ,y)RN − 1 − −1 ξ, y)RN M (dy) ≤ e B(0,r) Z Z + |ξ| 1 − η(y) |y| M (dy) + 2 + |ξ|η(y)|y| M (y)
Z
B(0,r)
≤
|ξ| 2
2
Z
B(0,r){
|y|2 M (dy) + |ξ|
B(0,r)
Z + |ξ|
Z
1 − η(y) M (dy)
B(0,r)
η(y)|y| M (dy) + 2M B(0, r){ .
B(0,r){
Obviously, this proves the boundedness in (3.2.20). In addition, R it shows that, for each r ∈ (0, 1], the limit there as |ξ| → ∞ is dominated by 12 B(0,r) |y|2 M (dy), which tends to 0 as r & 0.
§ 3.2 The L´evy–Khinchine Formula
135
Knowing (3.2.20), we can define
(3.2.21)
√ `η(m,C,M ) (ξ) = −1 m, ξ RN − 12 ξ, Cξ RN Z √ √ + e −1 (ξ,y)RN − 1 − −1η(y) ξ, y RN M (dy) RN
for any L´evy system (m, C, M ) and any Borel measurable η : RN −→ [0, 1] that satisfies (3.2.19). Furthermore, because `η(m,C,Mr ) −→ `η(m,C,M ) uniformly on compacts when Mr (dy) = 1[r,∞) (|y|) M (dy), it is clear that `η(m,C,M ) is continuous. Theorem 3.2.22 (L´ evy–Khinchine). For each µ ∈ I(RN ), there is a unique 1 `µ ∈ C(RN ; C) such that `µ (0) = 0 and µ ˆ = e`µ , and, for each n ∈ Z+ , e n `µ is the Fourier transform of the unique µ n1 ∈ M1 (RN ) satisfying µ = µ?n 1 . Next, n
let η : RN −→ [0, 1] be a Borel measurable function that satisfies (3.2.19). Then, for each µ ∈ I(RN ), there is a unique L´evy system (mηµ , Cµ , Mµ ) such that `µ = `η(mη ,Cµ ,Mµ ) , and, for each L´evy system (m, C, M ), there is a unique µ
µ ∈ I(RN ) such that `µ = `η(m,C,M ) . In fact, if µ ∈ I(RN ), then Z RN
ϕ(y) Mµ (dy) = lim nhϕ, µ n1 i n→∞
for all ϕ ∈ S (R ; C) that satisfy lim |y|−2 |ϕ(y)| = 0, N
|y|&0
Z Cµ = lim n n→∞
Z
2
RN
η0 (y) y ⊗ y µ n1 (dy) −
and mηµ0 = lim n n→∞
η0 (y)2 y ⊗ y Mµ (dy),
RN
Z RN
η0 (y)y µ n1 (dy)
for any if η0 ∈ Cc∞ RN ; [0, 1] satisfying η0 = 1 in a neighborhood of 0. Finally, for any Borel measurable η : RN −→ [0, 1] satisfying (3.2.19), mηµ
=
mηµ0
Z +
η(y) − η0 (y) Mµ (dy).
RN
Proof: The initial assertion is covered by Theorem 3.2.7. To prove the second assertion, let η ∈ Cc∞ RN ; [0, 1] with η = 1 in B(0, 1) be given. For µ ∈ I(RN ), I will show that `µ = `η(mη ,C,M ) , where mη , C, and M are determined from (cf. √ (3.2.10)) Aµ as in Lemma 3.2.14. To this end, define eξ for ξ ∈ RN by eξ (x) = e −1(ξ,x)RN , and set ηR (x) = η(R−1 x) for R > 0. The idea
136
3 Infinitely Divisible Laws
is to show that, as R → ∞, Aµ (ηR eξ ) tends to both `µ (ξ) and to `η(mη ,C,M ) (ξ). To check the first of these, use (3.2.10) to see that Z Z `µ (R−1 ξ 0 − x)ˆ η (ξ 0 ) dξ 0 . `µ (ξ 0 )c ηR (ξ 0 + ξ) dξ 0 = (2π)N Aµ (ηR eξ ) = RN
RN
Hence, since `µ is continuous and, by Lemma 3.2.9, supR≥1 |`µ (R−1 ξ)ˆ η (ξ)| is rapidly decreasing, Lebesgue’s Dominated Convergence Theorem says that Z ηˆR (ξ 0 ) dξ 0 = `µ (ξ). lim Aµ (ηR eξ ) = `µ (−ξ)(2π)−N R→∞
To prove that Aµ (ηR eξ ) also tends to Aµ (ηR eξ ) = `η(mη ,C,M ) (ξ) −
RN η `(mη ,C,M ) (ξ),
√
use (3.2.16) to write
Z
1 − ηR (y) eξ (y) M (dy), RN and observe that the last term is dominated by M B(0, R){ −→ 0. So far we know that, for each µ ∈ I(RN ), there is a L´evy system (mη , C, M ) such that `µ (ξ) = `η(mη ,C,M ) . Moreover, in the preliminary discussion at the beginning of this subsection, it was shown that, for each L´evy system (m, C, M ), there exists a µ ∈ I(RN ) for which `η(m,C,M ) = `µ . Finally, let η0 be as in the statement of this theorem. Given µ ∈ I(RN ), let η0 mµ ∈ RN , Cµ ∈ Hom(RN ; RN ), and Mµ ∈ M2 (RN ) be associated with Aµ as in 0 (3.2.16) of Lemma 3.2.14 when η = η0 . As we have just seen, `µ = `η(m . η0 ,C ,M ) −1
µ
µ
µ
Further, by that lemma and (3.2.10), we know that Z ϕ(y) Mµ (dy) = Aµ ϕ = lim nhϕ, µ n1 i n→∞
RN
for any ϕ ∈ S (RN ; R) that vanishes to second order at 0. In addition, by that same lemma and (3.2.10), we know that Z Z 2 η0 (y)2 y ⊗ y Mµ (dy) Cµ = lim n η0 (y) y ⊗ y µ n1 (dy) − n→∞
RN
RN
and that mηµ0 = lim n n→∞
Z RN
η0 (y) µ n1 (dy).
In particular, mη0 , Cµ , and Mµ are all uniquely determined by µ and η0 . In addition, if η : RN −→ [0, 1] is any other Borel measurable function satisfying (3.2.19), then the preceding combined with Z √ η η0 η0 (y) − η(y) ξ, y RN M (dy) `(m,C,M ) (ξ) = `(m,C,M ) (ξ) + −1 RN R η shows that `(m,C,M ) = `µ if and only if m = mηµ0 + RN η(y) − η0 (y) M (dy), C = Cµ , and M = Mµ . The expression in (3.2.21) for `µ in terms of a L´evy system is known as the L´ evy–Khinchine formula.
Exercises for § 3.2
137
Exercises for § 3.2 Exercise 3.2.23. Referring to (3.2.21), suppose that µ ∈ I(RN ) with `µ = `η(m,C,M ) for some L´evy system (m, C, M ) whose L´evy measure M satisfies R eλ|y| M (dy) < ∞ for all λ ∈ (0, ∞). Show that `µ admits a unique ex|y|≥1 tension as an analytic function on CN and that `µ (ξ) continues to be given by 0 (3.2.21) when the RN -inner product of (ξ1 , . . . , ξN ) ∈ CN with (ξ10 , . . . , ξN )∈ P N N 0 C is i=1 ξi ξi . Further, show that Z
e(ξ,y)RN µ(dy) = e`µ (−
√
−1ξ)
for all ξ ∈ CN .
RN
Hint: The first part is completely elementary complex analysis. To handle the second part, begin by arguing that it is enough to treat the cases when either M = 0 or C = 0. The case M = 0 is trivial, and the case when C = 0 can be further reduced to the one in which µ = πM for an M ∈ M0 (RN ) with compact P∞ m support in RN \ {0}. Finally, use the representation πM = e−α m=0 αm! ν ?m to complete the computation in this case.
Exercise 3.2.24. Given µ ∈ I(RN ) and knowing (3.2.20), show that ξ, Cµ ξ
RN
≡ −2 lim t−2 `µ (tξ) for all µ ∈ I(RN ) and ξ ∈ RN . t→∞
Similarly, when Mµ ∈ M1 (RN ), show that mµ ≡ mηµ −
Z η(y)y Mµ (dy) RN
is independent of the choice of η satisfying (3.2.19) and, for each ξ ∈ RN ,
√ 2 and ξ, mµ = − −1 lim t−1 `µ (tξ) + t2 ξ, Cµ ξ RN t→∞ Z √ √ e −1(ξ,y)RN − 1 Mµ (dy). `µ (ξ) = − 12 ξ, Cµ ξ RN + −1 ξ, mµ RN + RN
Finally, if µ ∈ I(RN ) is symmetric, show that Mµ is also symmetric and that
1 `µ (ξ) = − ξ, Cµ ξ + 2
Z RN
cos ξ, y
RN
− 1 Mµ (dy).
Exercise 3.2.25. Given µ ∈ I(R), show that µ (−∞, 0) = 0 if and only if Cµ = 0, Mµ ∈ M1 (R), Mµ (−∞, 0) = 0, and (cf. the preceding exercise) mµ ≥ 0. The following are steps that you might follow.
138
3 Infinitely Divisible Laws
r (i) To prove the “if” assertion, set M (dy) = 1[r,∞) (y) Mµ (dy) for r > 0, and r show that δmµ ? πM r (−∞, 0] = 0 for all r > 0 and δmµ ? πM =⇒µ as r & 0. Conclude from these that µ (−∞, 0) = 0. (ii) Now assume that µ (−∞, 0) = 0. To see that Cµ = 0, show that if σ > 0, then γ0,σ2 ? ν (−∞, 0) > 0 for any ν ∈ M1 (R). n (iii) Continuing (ii), show that µ (−∞, 0) ≥ µ n1 (−∞, 0) , and conclude first that µ n1 (−∞, 0) = 0 for all n ∈ Z+ and then that Z Mµ (−∞, 0) = 0 and mηµ ≥ η(y)y Mµ (dy).
RN
Finally, deduce from these that Mµ ∈ M1 (R) and that mµ ≥ 0. (iv) Suppose that X ∈ N (0, 1), and show that the distribution of |X| cannot be infinitely divisible. Exercise 3.2.26. The Gamma distributions is an interesting source of infinitely divisible laws. Namely, consider the family {µt : t ∈ (0, ∞)} ⊆ M1 (R) given by xt−1 e−x dx. µt (dx) = 1(0,∞) (x) Γ(t)
(i) Show by direct computation that µs ? µt (dx) =
B(s, t) 1(0,∞) (x)xs+t−1 e−x dx, Γ(s)Γ(t)
where
Z B(s, t) ≡
ξ s−1 (1 − ξ)t−1 dξ
(0,1)
is Euler’s Beta function, and conclude that µs+t = µs ? µt . In particular, one gets, as a dividend, the famous identity B(s, t) = Γ(s)Γ(t) Γ(s+t) .
(ii) As a consequence of (i), we know that the µt ’s are infinitely divisible. Show that their L´evy–Khinchine representation is # " Z √ −1 ξy −y dy . −1 e µbt (ξ) = exp t e y (0,∞)
Exercise 3.2.27. Given a µ ∈ M1 (RN ) for which there exists a strictly increasing sequence {nm : m ≥ 1} ⊆ Z+ and a sequence {µ n1 : m ≥ 1} ⊆ M1 (RN ) m
such that µ = µ?n1 m for all m ≥ 1, show that µ ∈ I(RN ). nm
Hint: First use Lemma 3.2.4 to show that µ ˆ never vanishes and therefore that N there is a unique `µ ∈ C(R ; C) such that `µ (0) = 0 and µ ˆ = e`µ . Next, proceed as in the proof of Theorem 3.2.7 to show that µ ∈ P(RN ), and apply that theorem to conclude that µ ∈ I(RN ).
§ 3.3 Stable Laws
139
§ 3.3 Stable Laws Recall from Exercise 2.3.23 the maps Tα : M1 (RN ) −→ M1 (RN ) given by the prescription ZZ x+y µ(dx)µ(dy), Tα µ(Γ) = 1Γ 1 2α RN ×RN
N and fixed points of Tα . That is, Fα (RN ) = let Fα (RN ) denote the set of non-trivial µ ∈ M1 (R ) \ {δ0 } : µ = Tα µ . If µ ∈ Fα (RN ) and µ2−n denotes the distrin n bution of x 2 α x under µ, then µ = µ?2 2−n for all n. Hence, by the result in Exercises 3.2.27, µ ∈ I(RN ), and so Fα (RN ) ⊆ I(RN ) for all α ∈ (0, ∞). In this section, I will study the L´evy systems associated with elements of Fα (RN ). § 3.3.1. General Results. Knowing that Fα (RN ) ⊆ I(RN ), we can phrase the N condition µ = Tα µ in terms of the associated L´evy systems. Namely, µ ∈N Fα (R ) 1 N −α if and only if µ ∈ I(R ) \ {δ0 } and `µ (ξ) = 2`µ 2 ξ for all ξ ∈ R . Next, using this and Exercise 3.2.24, we see that, for µ ∈ Fα (RN ), 0 if α > 2 n 2n 2 n `µ (ξ) = 2−n `µ (2 α ξ) = 2n( α −1) 2− α `µ (2 α ξ) −→ 1 − 2 (ξ, Cµ ξ)RN if α = 2
as n → ∞. Thus, we have already recovered the results in Exercises 2.3.21 and 2.3.23. I next will examine Fα (RN ) for α ∈ (0, 2) in greater detail. For this purpose, define Tˆα M for M ∈ M2 (RN ) to be the Borel measure determined by Z Z 1 ˆ (3.3.1) ϕ(y) Tα M (dy) = 2 ϕ(2− α y) M (dy) RN
RN
for Borel measurable ϕ : RN −→ [0, ∞). It is easy to check that Tˆα maps M2 (RN ) into itself. Lemma 3.3.2. For any α ∈ (0, 2), Mµ = Tˆα Mµ , and Z 1 1 η 1− α )mµ = η(y) − η(2 α y) y Mµ (dy). (1 − 2
( C = 0, µ N
µ ∈ Fα (R ) ∪ {δ0 } ⇐⇒
RN
In addition, if M ∈ M2 (RN ) \ {0} satisfies M = Tˆα M for some α ∈ (0, 2), then M ∈ Mβ (RN ) for all β > α but M ∈ / Mα (RN ). Proof: From the uniqueness of the L´evy system associated with an element of 2 I(RN ), it is clear that, for any µ ∈ I(RN ), MTα µ = Tˆα Mµ , CTα µ = 21− α Cµ , and Z 1 1 η η 1− α mµ + η(y) − η(2 α y) y Tˆα Mµ (dy). mTα µ = 2 RN
140
3 Infinitely Divisible Laws
2 Hence, µ ∈ Fα (RN ) ∪ {δ0 } if and only if Mµ = Tˆα Mµ , Cµ = 21− α Cµ , and, for any η satisfying (3.2.19),
1 1− α
(1 − 2
Z
)mηµ
1 η(y) − η(2 α y) y Mµ (dy).
= RN
In particular, when α ∈ (0, 2), Cµ = 0, and so the first assertion follows. The second assertion turns on the fact that, for all n ∈ Z+ , n
M = Tˆα M =⇒ M B(0, 2− α ) \ B(0, 2−
n+1 α
1 ) = 2n M B(0, 1) \ B(0, 2− α ) .
1 From this we see that κ ≡ M B(0, 1) \ B(0, 2− α ) > 0 unless M = 0 and that P∞ β the M -integral of |y|β over B(0, 1) is bounded below by 2−1 κ n=0 2n(1− α ) and P∞ β above by κ n=0 2n(1− α ) .
Theorem 3.3.3. µ ∈ F2 (RN ) if and only if µ = γ0,C for some non-negative definite, symmetric C ∈ Hom(RN ; RN ) \ {0}. If α ∈ (1, 2), then µ ∈ Fα (RN ) if and only if µ ∈ I(RN ) and `µ (ξ) equals √
−1
1−2
Z
ξ, y
1 1− α
RN
M (dy)
1
2− α <|y|≤1
+
Z
√
e
−1(ξ,y)RN
−1−
√
−1 1[0,1] (|y|) ξ, y
RN
M (dy)
RN
for some M ∈
T
β>α
Mβ (RN ) \ Mα (RN ) satisfying M = Tˆα M . If α ∈ (0, 1),
then µ ∈ Fα (RN ) if and only if µ ∈ I(RN ) and `µ (ξ) equals Z
√
e
−1(ξ,y)RN
− 1 M (dy)
RN
for some M ∈
T
N M (R ) \ Mα (RN ) satisfying M = Tˆα M . Finally, µ ∈ β β>α
F1 (RN ) if and only if µ ∈ I(RN ) and either µ = δm for some m ∈ RN \ {0} or `µ (ξ) =
√
−1 m, ξ
Z
RN
+ RN
√ √ e −1(ξ,y)RN − 1 − −1 1[0,1] (|y|) ξ, y RN M (dy)
T N N ˆ for some m ∈ RN and M ∈ β∈(1,2] Mβ (R ) \ M1 (R ) satisfying M = T1 M and Z y M (dy) = 0. 1 2 <|y|≤1
§ 3.3 Stable Laws
141
Proof: The first assertion requires no comment. When α ∈ (0, 2), the “if” 1 assertions can be proved by checking that, in each case, `µ (ξ) = 2`µ (2− α ξ). When α ∈ [1, 2), the “only if” assertion follows immediately from Lemma 3.3.2 with η = 1B(0,1) , and when α ∈ (0, 1), it follows from that lemma combined with the observation that Z Z 1 1− α ˆ y M (dy). y M (dy) = M = Tα M =⇒ 1 − 2 1 {2− α <|y|≤1}
B(0,1)
§ 3.3.2. α-Stable Laws. The most studied elements of Fα (RN ) are the αstable laws: those µ ∈ I(RN )\{δ0 } such that `µ (tξ) = tα `µ (ξ) for all t ∈ (0, ∞), not just for t = 2. Equivalently, if µ ∈ M1 (RN ) is α-stable if and only if µ ∈ I(RN ) \ {δ0 } and, for all non-negative, Borel measurable functions ϕ, Z Z 1 ϕ(y) µt (dy) = ϕ(t α y) µ(dy), t ∈ (0, ∞), RN
RN
where µbt (ξ) = et`µ (ξ) . Thus, there are no α-stable laws if α > 2, and µ is 2-stable if and only if µ = γ0,C for some C 6= 0. To examine the α-stable laws when α ∈ (0, 2), I will need the computations contained in the following lemmas. Lemma 3.3.4. Assume that M ∈ M2 (RN ) and that α ∈ (0, 2), and define the finite Borel measure ν on SN −1 by Z 2 −|y| 1 y |y| e M (dy) ϕ |y| hϕ, νi = Γ(2 − α) RN \{0}
for bounded, Borel measurable ϕ : SN −1 −→ C. Then, M satisfies Z Z (3.3.5) ϕ(ty) M (dy) = tα ϕ(y) M (dy), t ∈ (0, ∞) RN
RN
for all ϕ ∈ Cc RN \ {0}; R if and only if Z
Z
Z
ϕ(y) M (dy) = RN
ϕ(rω) SN −1
(0,∞)
dr
r1+α
! ν(dω)
for all ϕ ∈ Cc RN \ {0}; R . Proof: The “if” assertion is obvious. In addition, the “only if” assertion y will follow once I prove it for ϕ’s such that ϕ(y) = ϕ1 |y| ϕ2 (|y|), where ϕ1 ∈ C SN −1 ; [0, ∞) and ϕ2 ∈ Cc (0, ∞); R . Given ϕ1 ∈ C SN −1 ; [0, ∞) , determine the Borel measure ρ on (0, ∞) by Z y ϕ2 (|y|)|y|2 M (dy) hϕ2 , ρi = ϕ1 |y| RN \{0}
142
3 Infinitely Divisible Laws
for ϕ2 ∈ Cc (0, ∞); R . Then (3.3.5) implies that Z e
−tr
ρ(dr) = t
α−2
Z
(0,∞)
e−r ρ(dr) = tα−2 Γ(2 − α)hϕ, νi
(0,∞)
for t ∈ (0, ∞). Hence, since Z r1−α e−tr dr = Γ(2 − α)tα−2 ,
t ∈ (0, ∞),
(0,∞)
uniqueness of the Laplace transform (cf. Exercise 1.2.12) implies that ρ(dr) = hϕ1 , νir1−α dr, and therefore that Z RN \{0}
y ϕ1 |y|
Z M (dy) = (0,∞)
ϕ2 (r) ρ(dr) = hϕ1 , νi r2
Z ϕ1 (r) (0,∞)
dr
r1+α
.
Lemma 3.3.6. Let µ ∈ I(RN ). Then µ is 2-stable if and only if µ = γ0,C for some symmetric, non-negative definite C 6= 0; µ is α-stable for some α ∈ (0, 1) if and only if there is a finite, non-negative Borel measure ν 6= 0 on SN −1 such that ! Z Z dr √−1(ξ,rω) N R − 1 1+α ν(dω); `µ (ξ) = e r SN −1 (0,∞)
µ is 1-stable if and only if there exists a finite, non-negative, Borel measure ν on SN −1 and an m ∈ RN satisfying Z N −1 |m| + ν(S ) > 0 and ω ν(dω) = 0 SN −1
such that `µ (ξ) equals √
−1 ξ, m RN Z Z +
√ dr √−1(ξ,rω) N R − 1 − −11[0,1] (r) ξ, rω RN 2 e r (0,∞)
SN −1
! ν(dω);
and µ is α-stable for some α ∈ (1, 2) if and only if there is a finite, non-negative, Borel measure ν 6= 0 on SN −1 such that `µ (ξ) equals √
−1 1−α
Z ξ, ω SN −1
Z + SN −1
RN
ν(dω)
√ dr √−1(ξ,rω) N R − 1 − −11[0,1] (r) ξ, rω RN 1+α e r (0,∞)
Z
! ν(dω).
§ 3.3 Stable Laws
143
Proof: The sufficiency part of each case is easy to check directly or as a consequence of Theorem 3.3.3. To prove the necessity, first check that if µ is α-stable and therefore `µ (tξ) = tα `µ (ξ), then M must have the scaling property in (3.3.5) and therefore have the form described in Lemma 3.3.4. Second, when M has this form, simply check that in each case the result in Theorem 3.3.3 translates into the result here. In the following, C+ denotes the open upper half-space {ζ ∈ C : Im(ζ) > 0} in C, and C+ denotes its closure {ζ√ ∈ C : Im(ζ) ≥ 0}. In addition, given ζ ∈ C , where argζ is 0 if ζ = 0 and is the and α ∈ (0, 2), we take ζ α ≡ |ζ|α e −1αargζ √ unique θ ∈ (−π, π] such that ζ = |ζ|e −1θ if ζ 6= 0.
Lemma 3.3.7. If α ∈ (0, 1), then √
Z
e
−1ζr
−1
r1+α
(0,∞)
Γ(1 − α dr = α
ζ √ −1
α for ζ ∈ C+ .
In particular,
Z aα ≡ (0,∞)
and
Γ(2−α) α(α−1)
cos απ 2
if α ∈ (1, 2)
cos r − 1 dr = − Γ(1−α) cos απ 2 r1+α π α −2
Z bα ≡ (0,∞)
απ Γ(1 − α) sin r sin dr = 2 α r1+α
if α ∈ (0, 1) if α = 1
if α ∈ (0, 1).
Proof: Let fα (ζ) denote the integral on the left-hand side of the first equation. Clearly fα is continuous on C+ and analytic on C+ . In addition, fα (ξ) = ξ αfα (1) for ξ ∈ (0, ∞), and Re fα (1) < 0. Hence, there exist c > 0 and θ ∈ 0, π2 such √ that fα (ξ) = −ce −1θ ξ α for ξ ∈ (0, ∞). Since ζ ∈ C+ 7−→ ζ α ∈ C is the unique continuous extension of ξ ∈ (0,√∞) 7−→ ξ α ∈ (0, ∞) to C+ that is analytic on C+ , we know that fα (ζ) = −ce −1θ ζ α for ζ ∈ C+ . In addition,
√ fα ( −1) =
Z (0,∞)
1 e−r − 1 dr = − 1+α α r
Z (0,∞)
r−α e−r dr = −
Γ(1 − α) . α
and θ = − απ Hence, c = Γ(1−α) 2 . α When α ∈ (0, 1), the values of aα and bα follow immediately from the evaluation of fα (1). When α ∈ (1, 2), one can find the value of aα by first observing that Z Z cos r − 1 cos(ξr) − 1 α dr for ξ ∈ (0, ∞), dr = ξ 1+α r1+α r (0,∞) (0,∞)
144
3 Infinitely Divisible Laws
and then differentiating this with respect to ξ to get Z α (0,∞)
cos r − 1 dr = − r1+α
Z (0,∞)
sin r dr = −bα−1 . rα
To evaluate a1 , simply note that
π Γ(2 − α) cos απ 2 =− . α&1 2 1−α α
a1 = lim aα = − lim α&1
Theorem 3.3.8. Let µ ∈ I(RN ). If α ∈ (0, 2) \ {1}, then µ is α-stable if and only if there exists a finite, non-negative, Borel measure ν 6= 0 on SN −1 such that α Z (ξ, ω)RN √ ν(dω). `µ (ξ) = (−1)1(0,1) (α) −1 SN −1
On the other hand, µ is 1-stable if and only if there exist an m ∈ RN and a finite, non-negative, Borel measure ν on SN −1 such that |m| + ν(SN −1 ) > 0, Z ω ν(dω) = 0, SN −1
and `µ (ξ) =
√
√ −1 ξ, m RN − −1
where ζ log ζ = ζ log |ζ| +
√
Z ξ, ω SN −1
RN
log ξ, ω
RN
ν(dω),
−1ζargζ for ζ ∈ C.
Proof: When α ∈ (0, 1), the conclusion is a simple application of the corresponding results in Lemmas 3.3.6 and 3.3.7. When α ∈ (1, 2), one has to massage the corresponding expression in Lemma 3.3.6. Specifically, begin with the observation that √ Z i dr h √ √ −1ξ + e −1ξr − 1 − −1ξ1[0,1] (r)r 1+α r 1−α (0,∞) ! √ Z i dr h √ √ −1sgn(ξ) −1sgn(ξ)r α − 1 − −1sgn(ξ)1[0,1] (r)r 1+α + = |ξ| e 1−α r (0,∞)
for ξ ∈ R. Thus, we can write the expression for `µ (ξ) as Z SN −1
ξ, ω N α gα sgn(ξ, ω)RN ν(dω), R
§ 3.3 Stable Laws
145
where (cf. Lemma 3.3.7)
√ i dr −1 − 1 ∓ −11[0,1] (r)r 1+α ± gα (±1) = e 1−α r (0,∞) √ Z √ dr −1 . sin r − 1[0,1] (r)r 1+α ± = aα ± −1 1−α r (0,∞) Z
√
√ ± −1r
h
Next use integration by parts over the intervals (0, 1] and [1, ∞) to check that Z sin r − 1[0,1] (r)r (0,∞)
aα−1 α
Hence, since
dr 1 1 + = 1+α α−1 α r
Z (0,∞)
aα−1 1 cos r − 1 . + dr = α α α−1 r
Γ(2−α) sin απ = − α(α−1) 2 ,
gα (±1) =
Γ(2 − α) ∓ απ e 2 , α(α − 1)
and therefore α Γ(2 − α) sgn(x, ω)RN ξ, ω RN = α(α − 1)
gα
(ξ, ω)RN √ −1
α .
1−α ν. Thus, all that we need to do is replace the ν in Theorem 3.3.8 by Γ(1−α) Turning to the case α = 1, note that, because of the mean zero condition on ν,
Z SN −1
Z
h
√
e
−1(ξ,ω)RN r
(0,∞)
−1−
√
−11[0,1] (r)r ξ, ω
i dr
RN
r2
! ν(dω)
! i dr h √ = lim e −1(ξ,ω)RN r − 1 1+α ν(dω) α%1 SN −1 r (0,∞) α Z (ξ, ω)RN Γ(1 − α) √ ν(dω) = − lim α%1 α −1 SN −1 Z √ α 1 ξ, ω RN − ξ, ω RN ν(dω) = −1 lim α%1 1 − α SN −1 Z √ ξ, ω RN log ξ, ω RN ν(dω), = − −1 Z
Z
SN −1
where I have used (1 − α)Γ(1 − α) = Γ(2 − α) −→ 1. I close this section with a couple of examples of particularly important stable laws.
146
3 Infinitely Divisible Laws
Corollary 3.3.9. For any α ∈ (0, 2], µ is a symmetric and α-stable law if and only if there is a finite, non-negative, symmetric, Borel measure ν 6= 0 on SN −1 such that Z ξ, ω N α ν(dω). `µ (ξ) = − R SN −1
Moreover, µ is a rotationally invariant, α-stable law if and only if `µ (ξ) = −t|ξ|α for some t ∈ (0, ∞). Proof: If µ is 2-stable, then µ = γ0,C for some C 6= 0 and is therefore symmetric. In addition, by defining ν on SN −1 so that Z y 1 γ0,C (dω), |y|2 ϕ hϕ, νi = |y| 2 RN
we see that `µ (ξ) = −
1 ξ, Cξ RN = 2
Z SN −1
ξ, ω N 2 ν(dω). R
If α ∈ (0, 2) \ {1}, then, for every non-zero, symmetric ν on SN −1 , Z − SN −1
(ξ, ω
α Z α (ξ, ω)RN απ 1(0,1) (α) √ ν(dω) ν(dω) = (−1) csc RN 2 −1 SN −1
is `µ (ξ) for a symmetric, α-stable µ. Conversely, if µ is symmetric and α-stable for some α ∈ (0, 1), then, because `µ (ξ) = `µ (−ξ), the associated ν in Theorem 3.3.8 can be chosen to be symmetric, in which case `µ (ξ) equals (−1)
Z
1(0,1) (α)
SN −1
(ξ, ω)RN √ −1
α
Z απ ξ, ω N α ν(dω). ν(ω) = − cos R 2 SN −1
To handle the case when α = 1, first suppose that ν 6= 0 on SN −1 is symmetric. Then Z Z ξ, ω N ν(dω) = 2 − ξ, ω RN ν(dω) R SN −1
=
{ω:(ξ,ω)RN <0}
Z
2 √ π −1
ξ, ω
RN
log ξ, ω
RN
+ ξ, −ω
RN
log ξ, −ω
RN
ν(dω)
{ω:(ξ,ω)RN <0}
√
1 = − −1 π
Z ξ, ω SN −1
RN
log ξ, ω
RN
ν(dω),
which is `µ (ξ) for a symmetric, 1-stable µ. Conversely, if µ is symmetric and 1-stable, one can use `µ (ξ) = `µ (−ξ) to see that m = 0 and ν is symmetric in
Exercises for § 3.3
147
the expression for `µ (ξ) in Theorem 3.3.8. Hence, by the preceding calculation, `µ (ξ) has the desired form. Finally, if µ is a rotationally invariant, α-stable law, then `µ (ξ) is a rotationally invariant function of ξ and therefore the preceding leads to Z
Z
α |ξ|α ω, ω 0 ν(dω 0 ) λSN −1 (dω) = −t|ξ|α ,
`µ (ξ) = − SN −1
SN −1
where λSN −1 is normalized surface measure on SN −1 and
t = ν(SN −1 )
Z
Z
SN −1
SN −1
e, ω N α λSN −1 (dω) R
for any e ∈ SN −1 . Conversely, by taking ν to be an appropriate multiple of λSN −1 , one sees that, for any t ∈ (0, ∞), −t|ξ|α is `µ (ξ) for a symmetric, αstable µ.
Exercises for § 3.3 Exercise 3.3.10. Given α ∈ (0, 2), define Sα ν for finite, non-negative, Borel 1 measures ν on B(0, 1) \ B(0, 2− α ) by
Sα ν(Γ) =
X m∈Z
2−m
Z
m
1Γ (2 α y) ν(dy), RN
and show that this map is one-to-one and onto the set of M ∈ M2 (RN ) satisfying (cf. (3.3.1)) M = Tˆα M . Conclude that, for each α ∈ (0, 2), Fα (RN ) contains lots of elements! Exercise 3.3.11. Here are a few further properties of elements of Fα (RN ). (i) Show that there is µ ∈ Fα (RN ) such that µ {y : (e, y)RN < 0} = 0 for some e ∈ SN −1 if and only if α ∈ (0, 1). Hint: Reduce to the case when N = 1, and look at Exercise 3.2.24. N −1 (ii) If µ ∈ F1 (RN ), show that, , µ {y : (e, y)RN < 0} > for every e ∈ S 0 ⇐⇒ µ {y : (e, y)RN > 0} > 0. (iii) If α ∈ (1, 2), show that for each > 0 there is a µ ∈ Fα (R) such that µ (−∞, −] = 0. Exercise 3.3.12. Take N = 1. This exercise is about an important class of stable laws known as one-sided stable laws: stable laws that are supported on [0, ∞). (i) Show that there exists a one-sided α-stable law only if α ∈ (0, 1).
148
3 Infinitely Divisible Laws
(ii)If α ∈ α(0, 1), show that µ is a one-sided α-stable law if and only if `µ (ξ) = ξ for some t ∈ (0, ∞). −t √−1
(iii) Let α ∈ (0, 1), and use νtα to denote the one-sided α-stable law with `νtα (ξ) = α −t √ξ−1 . Show that √
Z e
−1ζy
νtα (dy)
= exp −t
[0,∞)
ζ √ −1
α for ζ ∈ C with Im(ζ) ≥ 0.
In particular, use Exercise 1.2.12 to conclude that νtα is characterized by the facts that it is supported on [0, ∞) and its Laplace transform is given by Z
α
e−λy νtα (dy) = e−tλ ,
λ ≥ 0.
[0,∞)
Exercise 3.3.13. Given α ∈ (0, 2], let µα t denote the symmetric α-stable law, described in Corollary 3.3.9, with `µαt (ξ) = −t|ξ|α . Clearly µ2t = γ0,2tI . When α ∈ (0, 2), show that Z α α µt = γ0,2τ I νt2 (dτ ), [0,∞) α
where νt2 is the one-sided α2 -stable law in part (iii) of the preceding exercise. This representation is an example of subordination, and, as we will see in Exercise 3.3.17, can be used to good effect.
Exercise 3.3.14. Because their Fourier transforms are rapidly decreasing, we know that each of the measures νtα in part (iii) of Exercise 3.3.11 admits a smooth density with respect to Lebesgue measure λR on R. In this exercise, we examine these densities. (i) For α ∈ (0, 1), set hα t =
(3.3.15)
and show that
∞
Z
dνtα dλR
for t ∈ (0, ∞),
α
−tλ e−λτ hα , t (τ ) dτ = e
0 1
1
−α α −α h1 (t τ ). and that hα t (τ ) ≡ t
λ ∈ [0, ∞),
Exercises for § 3.3
149
(ii) Only when α = 12 is an explicit expression for hα 1 readily available. To find this expression, first note that, by the uniqueness of the Laplace transform (cf. 1
Exercise 1.2.12) and (i), h12 is uniquely determined by Z ∞ 1 2 e−λ τ h12 (τ ) dτ = e−λ , λ ∈ [0, ∞). 0
Next, show that Z ∞ 1 2 a2 1 π 2 e−2ab τ − 2 e−( τ +b τ ) dτ = b 0
Z
∞
and
3
τ − 2 e−(
a2 τ
1
+b2 τ )
dτ =
0
π 2 e−2ab a
2
for all (a, b) ∈ (0, ∞) , and conclude from the second of these that 1
1(0,∞) (τ )e− 4τ . h1 (τ ) = √ 3 4πτ 2 1 2
(3.3.16)
1
1
Hint: To prove the first identity, try the change of variables x = aτ − 2 − bτ 2 , and get the second by differentiating the first with respect to a.
Exercise 3.3.17. In this exercise we will discuss the densities of the symmetric stable laws µα t for α ∈ (0, 2) (cf. Exercise 3.3.13). Once again, we know that each µα admits a smooth density with respect to Lebesgue measure λRN on RN . t Further, it is clear that this density is symmetric and that α 1 1 dµ dµα t 1 (t− α x) (x) = t− α dλRN dλRN
for t ∈ (0, ∞).
(i) Referring to Exercise 3.3.14 and using Exercise 3.3.12, show that Z ∞ |x|2 α 1 dµα 1 −N 2 e− 4τ h 2 (τ ) dτ. τ (x) = (3.3.18) N dλRN (4π) 2 0 1
(ii) Because we have an explicit expression for h12 , we can use (3.3.18) to get an
explicit expression for
dµ11 dλRN
. In fact, show that
N 2tN dµ1t (t, x) ∈ (0, ∞) × RN , (x) = πtR (x) ≡ N +1 , dλRN ωN (t2 + |x|2 ) 2 −1 N +1 is the surface area of SN in RN +1 . The function where ωN = 2π 2 Γ N2+1 R π1 is the density for what probabilists call the Cauchy distribution. For N general N ’s, (t, x) ∈ (0, ∞) × RN 7−→ πtR (x) is what analysts call the Poisson kernel for the right half-space in RN +1 . That is (cf. Exercise 10.2.22), if f ∈ Cb (RN ; R), then Z N (t, x) uf (t, x) = f (x − y) πtR (y) dy
(3.3.19)
RN
is the unique, bounded harmonic extension of f to the right half-space.
150
3 Infinitely Divisible Laws
(iii) Given α ∈ (0, 2), show that kf k2α
ZZ ≡ RN ×RN
e
−|y−x|α
Z f (x)f (y) dxdy = RN
|fˆ(ξ)|2 µα 1 (dξ)
for f ∈ L1 (RN ; C). This can be used to prove that k · kα determines a Hilbert norm on Cc (RN ; C).
Chapter 4 L´ evy Processes
Although analysis was the engine that drove the proofs in Chapter 3, probability theory can do a lot to explain the meaning of the conclusions drawn there. Specifically, in this chapter I will develop an intuitively appealing way of thinking about a random variable X whose distribution is infinitely divisible, an X for √−1 (ξ,X) P N R equals which E e
exp
√
−1 ξ, m) −
1 2
ξ, Cξ
Z + RN
RN
h √ √ i −1 (ξ,y)RN − 1 − −1 1[0,1] |y| ξ, y RN M (dy) e
for some m ∈ RN , some symmetric, non-negative definite C ∈ Hom(RN ; RN ), and L´evy measure M ∈ M2 (RN ). In most of this chapter I will deal with the case when there is no Gaussian component. That is, I will be assuming that C = 0. Because it is distinctly different, I will treat the Gaussian component separately in the final section. However, I begin with some general comments that apply to the considerations in the whole chapter. The key idea, which seems to have been L´evy’s, is to develop a dynamic picture of X. To understand the origin of his idea, denote by µ ∈ I(RN ) the distribution of X, and define `µ accordingly, as in Theorem 3.2.7. Then, for each t ∈ [0, ∞), there is a unique µt ∈ I(RN ) for which µbt = et`µ , and so µs+t = µs ? µt for all s, t ∈ [0, ∞). L´evy’s idea was to associate with {µt : t ≥ 0} a family of random variables {Z(t) : t ≥ 0} that would reflect the structure of {µt : t ≥ 0}. Thus, Z(0) = 0 and, for each (s, t) ∈ [0, ∞), Z(s + t) − Z(s) should be independent of {Z(τ ) : τ ∈ [0, s]} and have distribution µt . In other words, {Z(t) : t ≥ 0} should be the continuous parameter analog of the sums of independent, identically distributed random variables. Indeed, given any τ > 0, let {Xm : m ≥ 0} be a sequence of independent random variables with distribution µτ . Then {Z(nτP ) : n ≥ 0} should have the same distribution as {Sn : n ≥ 0}, where Sn = 1≤m≤n Xm . This observation suggests that one should think about t Z(t) as a evolution that, when one understands its dynamics, will reveal information about Z(1) and therefore µ. 151
152
4 L´evy Processes
For reasons that should be obvious now, an evolution {Z(t) : t ∈ [0, ∞)} of the sort described above used to be called a process with independent, homogeneous increments, the term “process” being the common one for continuous families of random variables and the adjective “homogeneous” referring to the fact that the distribution of the increment Z(t) − Z(s) for 0 ≤ s < t depends only on the length t − s of the time interval over which it is taken. In more recent times, a process with independent, homogeneous increments is said to be a L´ evy process, and so I will adopt this more modern terminology. Assuming that the family {Z(t) : t ∈ [0, ∞)} exists, notice that we already know what the joint distribution of {Z(tk ) : k ∈ N} must be for any choice of 0 = t0 < · · · < tk < · · · . Indeed, Z(0) = 0 and
P Z(tk ) − Z(tk−1 ) ∈ Γk , 1 ≤ k ≤ K =
K Y
µtk −tk−1 (Γk )
k+1
for any K ∈ Z+ and Γ1 , . . . , ΓK ∈ BRN . Equivalently, P Z(tk ) ∈ Γk , 0 ≤ k ≤ K equals X Y Z Z Y K k K 1Γ0 (0) · · · 1Γk yj µtk −tk−1 (dyk ) (RN )K
k=1
j=1
k=1
for any K ∈ Z+ and Γ0 , . . . , ΓK ∈ BRN . My goal is this chapter is to show that each µ ∈ I(RN ) admits a L´evy process {Zµ (t) : t ≥ 0} and that the construction of the associated L´evy process improves our understanding of µ. Unfortunately, before I can carry out this program, I need to deal with a few technical, bookkeeping matters. § 4.1 Stochastic Processes, Some Generalities Given an index A with some nice structure and a family {X(α) : α ∈ A} of random variables on a probability space (Ω, F, P) taking values in some measurable space (E, B), it is often helpful to think about {X(α) : α ∈ A} in terms of the map ω ∈ Ω 7−→ X( · , ω) ∈ E A . For instance, if A is linearly ordered, then ω X( · , ω) can be thought of as a random evolution. More generally, when probabilists want to indicate that they are thinking about {X(α) : α ∈ A} as the map ω X( · , ω), they call {X(α) : α ∈ A} a stochastic process on A with state space (E, B). The distribution of a stochastic process is the probability measure X∗ P on1 (E A , B A ) obtained by pushing P forward under the map ω X( · , ω). Hence two stochastic processes {X(α) : α ∈ A} and {Y (α) : α ∈ A} on (E, B) have the same distribution if and only if P X(αk ) ∈ Γk , 0 ≤ k ≤ K = P Y (αk ) ∈ Γk , 0 ≤ k ≤ K Recall that BA is the σ-algebra over E A that is generated by all the maps ψ ∈ E A 7−→ ψ(α) ∈ E as α runs over A. 1
§ 4.1 Stochastic Processes, Some Generalities
153
for all K ∈ Z+ , {α0 , . . . , αK } ⊆ A, and Γ0 , . . . , ΓK ∈ B. As long as A is countable, there are no problems because E A is a reasonably tame object and B A contains lots of its subsets. However, when A is uncountable, E A is a ridiculously large space and B A will be too meager to contain many of the subsets in which one is interested. The point is that for B to be in B A there must (cf. Exercise 4.1.11) be a countable subset {αk : k ∈ N} of A such that one can determine whether or not ψ ∈ B by knowing {ψ(αk ) : k ∈ N}. Thus [0,∞) (cf. Exercise 4.1.11), for instance, C [0, ∞); R ∈ / BR . Probabilists expended a great deal of effort to overcome the problem raised in the preceding paragraph. For instance, using a remarkable piece of measure theoretic reasoning, J.L. Doob2 proved that in the important case when A = [0, ∞) and E = R, one can always make a modification, what he called the “separable modification,” so that sets like C [0, ∞); R become measurable. However, in recent times, probabilists have tried to simplify their lives by constructing their processes in such a way that these unpleasant measurability questions never arise. That is, if they suspect that the process should have some property that is not measurable with respect to B A , they avoid constructions based on general principles, like Kolmogorov’s Extension Theorem (cf. part (iii) of Exercise 9.1.17), and instead adopt a construction procedure that produces the process with the desired properties already present. The rest of this chapter contains important examples of this approach, and the rest of this section contains a few technical preparations. § 4.1.1. The Space D(RN ). Unless its L´evy measure M is zero, a L´evy process for µ ∈ I(RN ) cannot be constructed so that it has continuous paths. In fact, if M 6= 0, then t Zµ (t) will be almost never continuous. Nonetheless, {Zµ (t) : t ≥ 0} can be constructed so that its paths are reasonably nice. Specifically, its paths can be made to be right-continuous everywhere and have no oscillatory discontinuities. For this reason, I introduce the space D(RN ) of paths ψ : [0, ∞) −→ RN such that ψ(t) = ψ(t+) ≡ limτ &t ψ(τ ) for each t ∈ [0, ∞) and ψ(t−) ≡ limτ %t ψ(τ ) exists in RN for each t ∈ (0, ∞). Equivalently, ψ(0) = ψ(0+), and, for each t ∈ (0, ∞) and > 0, there is a δ ∈ (0, t) such that sup{|ψ(t)−ψ(τ )| : τ ∈ (t, t+δ)} < and sup{|ψ(t−)−ψ(τ )| : τ ∈ (t−δ, t)} < . The following lemma presents a few basic properties possessed by elements of −n D(RN ). In its statement, for n ∈ N and τ ∈ (0, ∞), bτ c+ : m∈ n = min{m2 + n − + −n −n Z and m ≥ 2 τ } and bτ cn = bτ cn − 2 = max{m2 : m ∈ N and m < 2n τ }. In addition, for 0 ≤ a < b, (4.1.1)
kψk[a,b] ≡ sup |ψ(t)| t∈[a,b]
2
See Chapter II of Doob’s Stochastic Processes, Wiley (1953).
154
4 L´evy Processes
is the uniform norm of ψ [a, b], and X K var[a,b] (ψ) = sup |ψ(tk ) − ψ(tk−1 )| : K ∈ Z+ (4.1.2)
k=1
and a = t0 < t1 < · · · < tK = b is the total variation of ψ [a, b]. Lemma 4.1.3. r > 0, the set
If ψ ∈ D(RN ), then, for each t > 0, kψk[0,t] < ∞, and for each J(t, r, ψ) ≡ {τ ∈ (0, t] : |ψ(τ ) − ψ(τ −)| ≥ r}
is finite subset of (0, t]. In addition, there exists an n(t, r, ψ) ∈ N such that, for every n ≥ n(t, r, ψ) and m ∈ Z+ ∩ (0, 2n ], ψ m2−n t − ψ (m − 1)2−n t ≥ r =⇒ m2−n = τ + for some τ ∈ J(t, r, ψ). t n
Finally, kψk[0,t] = lim max |ψ(m2−n t)| : m ∈ N ∩ [0, 2n ] n→∞
and var[0,t] (ψ) = lim
n→∞
X
ψ m2−n t − ψ (m − 1)2−n t .
m∈Z+ ∩[0,2n ]
Proof: Begin by noting that it suffices to treat the case when t = 1, since one can always reduce to this case by replacing ψ with τ ψ(tτ ). If kψk[0,1] were infinite, then we could find a sequence {τn : n ≥ 1} ⊆ [0, 1] such that |ψ(τn )| −→ ∞, and clearly, without loss in generality, we could choose this sequence so that τn −→ τ ∈ [0, 1] and {τn : n ≥ 1} is either strictly decreasing or strictly increasing. But, in the first case this would contradict right-continuity, and in the second it would contradict the existence of left limits. Thus, kψk[0,1] must be finite. Essentially the same reasoning shows that J(1, r, ψ) is finite. If it were not, then we could find a sequence {τn : n ≥ 0} of distinct points in (0, 1] such that |ψ(τn ) − ψ(τn −)| ≥ r, and again we could choose them so that they were either strictly increasing or strictly decreasing. If they were strictly increasing, then τn % τ for some τ ∈ (0, 1] and, for each n ∈ Z+ , there would exist a τn0 ∈ (τn−1 , τn ) such that |ψ(τn ) − ψ(τn0 )| ≥ 2r , which would contradict the existence of a left limit at τ . Similarly, right-continuity would be violated if the τn ’s were decreasing. Although it has the same flavor, the proof of the existence of n(1, r, ψ) is a bit trickier. Let 0 < τ1 < · · · τK ≤ 1 be the elements of J(1, r, ψ). If n(1, r, ψ)
§ 4.1 Stochastic Processes, Some Generalities
155
failed to exist, then we could choose a subsequence {(mj , nj ) : j ≥ 1} from Z+ × N so that 1} is strictly increasing and tj ≡ mj 2−nj ∈ (0, 1] {nj : j ≥−n satisfies ψ tj − ψ tj − 2 j ≥ r for all j ∈ Z+ , but tj 6= bτk c+ nj for any j ∈ Z+ and 1 ≤ k ≤ K. If tj = t infinitely often for some t, then we would have the contradiction that t ∈ / J(1, r, ψ) and yet |ψ(t) − ψ(t−)| ≥ r. Hence, I will assume that the tj ’s are distinct. Further, without loss in generality, I assume that {tj : j ≥ 1} is a subset of one of the intervals (0, τ1 ), (τk−1 , τk ) for some 2 ≤ k ≤ K, or of (τK , 1]. Finally, I may and will assume that either tj % t ∈ (0, 1] or that tj & t ∈ [0, 1). But, since |ψ(tj ) − ψ(tj − 2−nj )| ≥ r, tj % t contradicts the existence of ψ(t−). Similarly, if tj & t and tj − 2−nj ≥ t for infinitely many j 0 s, then we get a contradiction with right-continuity at t. Thus, the only remaining case is when tj & t and tj − 2−nj < t ≤ tj for all but a finite number of j’s, in which case we get the contradiction that t ∈ / J(1, r, ψ) and yet |ψ(t) − ψ(t−)| = lim ψ(tj ) − ψ tj − 2−nj ≥ r. j→∞
To prove the assertion about kψk[0,1] , simply observe that, by monotonicity, the limit exists and that, by right-continuity, for any t ∈ [0, 1], ≤ lim max ψ(m2−n ) ≤ kψk[0,1] . |ψ(t)| = lim ψ btc+ n n n→∞
n→∞ 0≤m≤2
The assertion about var[0,1] (ψ) is proved in essentially the same manner, although now the monotonicity comes from the triangle inequality and the first equality in the preceding must be replaced by |ψ(t)−ψ(t−)| = limn→∞ |ψ(btc+ n )− − ψ(btcn )|. I next give D(RN ) the topological structure corresponding to uniform convergence on compacts, or, equivalently, the topological structure for which ρ(ψ, ψ 0 ) ≡
∞ X
2−n
n=1
kψ − ψ 0 k[0,n] 1 + kψ − ψ 0 k[0,n]
is a metric. Because it is not separable (cf. Exercise 4.1.10), this topological structure is less than ideal. Nonetheless, the metric ρ is complete. To see that it is, first observe that |ψ(τ −)| ≤ kψk[0,t] for all 0 < τ ≤ t. Thus, if sup`>k ρ(ψ` , ψk ) −→ 0 as k → ∞, then there exist paths ψ : [0, ∞) −→ RN and ψ˜ : (0, ∞) −→ RN such that sup |ψk (τ ) − ψ(τ )| −→ 0 τ ∈[0,t]
and
˜ )| −→ 0 sup |ψk (τ −) − ψ(τ τ ∈(0,t]
for each t > 0. Therefore, if t ≥ τn & τ , then
lim |ψ(τ ) − ψ(τn )| ≤ 2kψ − ψk k[0,t] + lim |ψk (τ ) − ψk (τn )| ≤ 2kψ − ψk k[0,t]
n→∞
n→∞
156
4 L´evy Processes
for all k ∈ Z+ , and so ψ is right-continuous. Essentially the same argument ˜ ) for τ > 0, which means, of course, that ψ ∈ D(RN ) shows that ψ(τ −) = ψ(τ and that supτ ∈(0,t] |ψk (τ −) − ψ(τ −)| −→ 0 for each t > 0. One might think that I would take the measurable structure on D(RN ) to be the one given by the Borel field BD(RN ) determined by uniform convergence on compacts. However, this is not the choice I will make. Instead, the measurable structure I choose for D(RN ) is the one that D(RN ) inherits as a subset of (RN )[0,∞) . That is, I take for D(RN ) the measurable structure given by the σalgebra FD(RN ) = σ {ψ(t) : t ∈ [0, ∞)} , the σ-algebra generated by the maps ψ ∈ D(RN ) 7−→ ψ(t) ∈ RN as t runs over [0, ∞). The reason for my insisting on this choice is that I want two D(RN )-valued stochastic processes {X(t) : t ≥ 0} and {Y(t) : t ≥ 0} to induce the same measure on D(RN ) if they have the same distribution. Seeing as (cf. Exercise 4.1.11) FD(RN ) $ BD(RN ) , this would not be true were I to choose the Borel structure. Because FD(RN ) 6= BD(RN ) , FD(RN ) -measurability does not follow from topological properties like continuity. Nonetheless, many functions related to the topology of D(RN ) are FD(RN ) -measurable. For example, the last part of Lemma 4.1.3 proves that both ψ kψk[0,t] , which is continuous, and ψ var[0,t] (ψ), which is lower semicontinuous, are both FD(RN ) -measurable for all t ∈ [0, ∞). In the next subsection, I will examine other important functions on D(RN ) and will show that they, too, are FD(RN ) -measurable. § 4.1.2. Jump Functions. Let M∞ (RN ) be the space of non-negativeBorel measures M on RN with the properties that M ({0}) = 0 and M B(0, r){ < ∞ for all r > 0. A jump function is a map t ∈ [0, ∞) 7−→ j(t, · ) ∈ M∞ (RN ) ¯ j(0, ∆) = 0, t with the property that, for each ∆ ∈ BRN with 0 ∈ / ∆, j(t, ∆) is a non-decreasing, piecewise constant element of D(RN ) such that j(t, ∆) − j(t−, ∆) ∈ {0, 1} for each t > 0. Lemma 4.1.4. A map t j(t, · ) is a non-zero jump function if and only if there exists a set ∅ 6= J ⊂ (0, ∞) that is finite or countable and a set {yτ : τ ∈ J} ⊆ RN \ {0} such that {τ ∈ J ∩ (0, t] : |yτ | ≥ r} is finite for each (t, r) ∈ (0, ∞)2 and X (4.1.5) j(t, · ) = 1[τ,∞) (t)δyτ . τ ∈J
In particular, if t j(t, · ) is a jump function and t > 0, then, either j(t, · ) = j(t−, · ) or j(t, · ) − j(t−, · ) = δy for some y ∈ RN \ {0}. Proof: It should be obvious that if J and {yτ : τ ∈ J} satisfy the stated conditions, then the t j(t, · ) given by (4.1.5) is a jump function. To go in the other direction, suppose that t j(t, · ) is a jump function, and, for each r > 0, set fr (t) = j t, RN \ B(0, r) . Because t fr (t) is a non-decreasing, piecewise constant, right-continuous function satisfying fr (0) = 0 and fr (t) − fr (t−) ∈
§ 4.1 Stochastic Processes, Some Generalities
157
{0, 1} for each t > 0, it has at most a countable number of discontinuities, and at most fr (t) of them can occur in any interval (0, t]. Furthermore, if fr has a discontinuity at τ , then j τ, B(0, r) − j τ −, B(0, r) = 0, and so the measure ντ = j(τ, · ) − j(τ −, · ) is a {0, 1}-valued probability measure on RN that assigns mass 0 to B(0, r). Hence (cf. Exercise 4.1.15) fr (τ ) 6= fr (τ −) =⇒ ντ = δy for some yτ ∈ RN \ B(0, r). From these considerations, it follows easily that if J(r) = {τ ∈ (0, ∞) : fr (τ ) 6= fr (τ −)} and if, for each τ ∈ J(r), yτ ∈ RN \B(0, r) is chosen so that j(τ, · ) − j(τ −, · ) = δyτ , then J(r) ∩ (0, t] is finite for all t > 0 and X j t, · B(0, r){ = 1[τ,∞) (t)δyτ . τ ∈J(r)
S
Thus, if J = r>0 J(r), then J is at most countable, {(τ, yτ ) : τ ∈ J} has the required finiteness property, and (4.1.5) holds. The reason for my introducing jump functions is that every element ψ ∈ D(RN ) determines a jump function t j(t, · , ψ) by the prescription j(t, Γ, ψ) = (4.1.6)
X
1Γ ψ(τ ) − ψ(τ −) ,
τ ∈J(t,ψ)
where J(t, ψ) ≡ {τ ∈ (0, t] : ψ(τ ) 6= ψ(τ −)}, for Γ ⊆ RN \ {0}. S To check that j(t, · , ψ) is well defined and is a jump function, take J(ψ) = t>0 J(t, ψ) and yτ = ψ(τ ) − ψ(τ −) when τ ∈ J(ψ), note that, by Lemma 4.1.3, J(ψ) is at most countable and that {(τ, yτ ) : τ ∈ J(ψ)} has the finiteness required in Lemma 4.1.4, and observe that (4.1.5) holds when j(t, · ) = j(t, · , ψ) and J = J(ψ). Because it will be important for us to know that the distribution of a D(RN )valued stochastic process determines the distribution of the jump functions for its paths, we will make frequent use of the following lemma. Lemma 4.1.7. If ϕ : RN −→ R is a BRN -measurable function that vanishes in a neighborhood of 0, then ϕ is j(t, · , ψ)-integrable for all (t, ψ) ∈ [0, ∞) × D(RN ), and Z (t, ψ) ∈ [0, ∞) × D(RN ) 7−→ ϕ(y) j(t, dy, ψ) ∈ R RN
is a B[0,∞) × FD(RN ) -measurable function that, for each ψ, is right-continuous and piecewise constant as a function of t. Finally, for all Borel measurable R ϕ : RN −→ [0, ∞), (t, ψ) ∈ [0, ∞) × D(RN ) 7−→ RN ϕ(y)j(t, dy, ψ) ∈ [0, ∞] is B[0,∞) × FD(RN ) -measurable. Proof: The final assertion is an immediate consequence of the earlier one plus the Monotone Convergence Theorem. Let r > 0 be given. If ϕ is a Borel measurable function that vanishes on B(0, r), then it is immediate from the first part of Lemma 4.1.3 that ϕ is
158
4 L´evy Processes
j(t, · ,Rψ)-integrable for all (t, ψ) ∈ [0, ∞) × D(RN ) and, for each ψ ∈ D(RN ), t ϕ(y) j(t, dy, ψ) is right-continuous and piecewise constant. Thus, it RN suffices to show that, for each t ∈ (0, ∞), Z (*) ψ ϕ(y) j(t, dy, ψ) is FD(RN ) -measurable. RN
Moreover, it suffices to do this when t = 1 and ϕ is continuous, since rescaling time allows one to replace t by 1 and the set of ϕ’s for which (*) is true is closed under pointwise convergence. But, by the second part of Lemma 4.1.3, we know that 2n X ϕ ψ m2−n − ψ (m − 1)2−n = m=1
X
− ϕ ψ bτ c+ n − ψ bτ cn
τ ∈J(1,r,ψ)
for n ≥ n(1, r, ψ), and therefore Z RN
2n X ϕ(y) j(1, dy, ψ) = lim ϕ ψ m2−n − ψ (m − 1)2−n . n→∞
m=1
Here are some properties of a path ψ ∈ D(RN ) which are determined by its relationship to its jump function. First, it should be obvious that ψ ∈ C(RN ) ≡ C [0, ∞); RN if and only if j(t, · , ψ) = 0 for all t > 0. At the opposite extreme, say that a ψ is an absolutely pure jump path if and only if (cf. R § 3.2.2) j(t, · , ψ) ∈ M1 (RN ) and ψ(t) = y j(t, dy, ψ) for all t > 0. Among the absolutely pure jump paths are those that are the piecewise constant paths: those absolutely pure jump ψ’s for which j(t, · , ψ) ∈ M0 (RN ), t > 0. Because of Lemma 4.1.7, each of these properties is FD(RN ) -measurable. In particular, if {Z(t) : t ≥ 0} is a D(RN )-valued stochastic process whose paths almost surely have any one of these properties, then the paths of every D(RN )-valued stochastic process with the same distribution as {Z(t) : t ≥ 0} will almost surely possess that property. Finally, I need to address the question of when a jump function is the jump function for some ψ ∈ D(RN ). Theorem 4.1.8. Let t j(t, · ) be a non-zero jump function, and set j Γ (t, dy) ¯ and if ψ ∆ (t) = = / ∆ R 1Γ (y)j(t, dy) for ∆Γ ∈ BRN . If ∆ ∈ BRN with 0 ∈ y j(t, dy), then ψ is a piecewise constant element of D(RN ), j(t, · , ψ ∆ ) = ∆ N j ∆ (t, · ), and j(t, · , ψ−ψ ∆ ) = j R \∆ (t, · ) = j(t, · )−j ∆ (t, · ) for any ψ ∈ D(RN ) whose jump function is t j(t, · ). Finally, suppose that {ψm : m ≥ 0} ⊆ N D(R ) and a non-decreasing Ssequence {∆m : m ≥ 0} ⊆ BRN satisfy the ∞ conditions that RN \ {0} = m=0 ∆m and, for each m ∈ N, 0 ∈ / ∆m and ∆m j(t, · , ψm ) = j (t, · ), t ≥ 0. If ψm −→ ψ uniformly on compacts, then j(t, · , ψ) = j(t, · ), t ≥ 0.
Exercises for § 4.1
159
Proof: Throughout the proof I will use the notation introduced in Lemma 4.1.4. ¯ we know that Assuming that 0 ∈ / ∆, X j ∆ (t, · ) = 1[τ,∞) (t)1∆ (yτ )δyτ , τ ∈J
where, for each t > 0, there are only finitely many non-vanishing terms. At the same time, X X ψ ∆ (t) = 1[τ,∞) (t)1∆ (yτ )yτ and j(t, · , ψ−ψ ∆ ) = 1[τ,∞) (t)1RN \∆ (yτ )δyτ τ ∈J
τ ∈J
if j(t, · , ψ) = j(t, · ). Thus, all that remains is to prove the final assertion. To this end, suppose that j(t, · , ψ) 6= j(t−, · , ψ). Since kψ − ψm k[0,t] −→ 0, there exists an m such that ψm (t) 6= ψm (t−) and therefore that j(t, · ) − j(t−, · ) = δy for some y ∈ ∆m . Since this means that ψn (t) − ψn (t−) = y for all n ≥ m, it follows that ψ(t) − ψ(t−) = y and therefore that j(t, · , ψ) − j(t−, · , ψ) = δy = j(t, · ) − j(t−, · ). Conversely, suppose that j(t, · ) 6= j(t−, · ) and choose m so that j(t, · ) − j(t−, · ) = δy for some y ∈ ∆m . Then ψn (t) − ψn (t−) = y for all n ≥ m. Thus, since this means that ψ(t) − ψ(t−) = y, we again have that j(t, · , ψ) − j(t−, · , ψ) = δy = j(t, · ) − j(t−, · ). After combining these, we see that j(t, · , ψ) − j(t−, · , ψ) = j(t, · ) − j(t−, · ) for all t > 0, from which it is an easy step to j(t, · ) = j(t, · , ψ) for all t ≥ 0. Exercises for § 4.1 Exercise 4.1.9. When dealing with uncountable collections of random variables, it is important to understand what functions are measurable with respect to them. To be precise, suppose that {Xi : i ∈ I} is a non-empty collection of functions on some space Ω with values in some measurable space (E, B), and let F = σ {Xi : i ∈ I} be the σ-algebra over Ω which they generate. Show that + A ∈ F if and only if there is a sequence {im : m ∈ Z+ } ⊆ I and an Γ ∈ B Z such that A = ω ∈ Ω : Xi1 (ω), . . . , Xim (ω), . . . ∈ Γ . More generally, if f : Ω −→ R, show that f is F-measurable if and only if there + + is a sequence {im : m ∈ Z+ } ⊆ I and a F Z -measurable F : E Z −→ R such that f (ω) = F Xi1 (ω), . . . , Xim (ω), . . . . Hint: Make use of Exercise 1.1.12. Exercise 4.1.10. Let e ∈ SN −1 , set ψt (τ ) = 1[t,∞) (τ )e for t ∈ [0, 1], and show that kψt − ψs k[0,1] = 1 for all s 6= t from [0, 1]. Conclude from this that D(RN ) is not separable in the topology of uniform convergence on compacts.
160
4 L´evy Processes
Exercise 4.1.11. Using Exercise 4.1.9, show that a function ϕ : D(RN ) −→ R is FD(RN ) -measurable if and only if there exists an (RN )N -measurable function Φ : (RN )N −→ R and a sequence {tk : k ∈ N} ⊆ [0, ∞) such that ϕ(ψ) = Φ ψ(t0 ), . . . , ψ(tk ), . . . , ψ ∈ D(RN ). Next, define ψt as in Exercise 4.1.10, and use that exercise together with the preceding to show that the open set {ψ ∈ D(RN ) : ∃ t ∈ [0, 1] kψ − ψt k[0,1] < 1} is not FD(RN ) -measurable. Conclude that BD(RN ) % FD(RN ) . Similarly, conclude that neither D(RN ) nor C(RN ) is a measurable subset of (RN )[0,∞) . On the other hand, as we have seen, C(RN ) ∈ FD(RN ) . Exercise 4.1.12. Show that Z (4.1.13) var[0,t] (ψ) ≥ |y| j(t, dy, ψ),
(t, ψ) ∈ [0, ∞) × D(RN ).
RN
Hint: This is most easily seen from the representation of j(t, · , ψ) in terms of point masses at the discontinuities of ψ. One can use this representation to show that, for each r > 0, Z X var[0,t] (ψ) ≥ ψ(τ ) − ψ(τ −) = |y| j(t, dy, ψ), (t, ψ) ∈ [0, ∞). |y|≥r
τ ∈J(t,r,ψ)
Exercise 4.1.14. If ψ is an absolutely pure jump path, show that var[0,t] (ψ) = R |y| j(t, dy, ψ) and therefore that ψ has locally bounded variation. Conversely, if ψ ∈ C(RN ) has locally bounded variation, show that ψ is an absolutely pure R jump path if and only if var[0,t] (ψ) = |y| j(t, dy, ψ). Finally, if ψ ∈ D(RN ) R and j(t, · , ψ) ∈ M1 (RN ) for all t ≥ 0, set ψc (t) ≡ ψ(t) − y j(t, dy, ψ) and show that ψc ∈ C(RN ) and Z var[0,t] (ψ) = var[0,t] (ψc ) + |y| j(t, dy, ψ). Exercise 4.1.15. If ν ∈ M1 (RN ), show that ν(Γ) ∈ {0, 1} for all Γ ∈ BRN if and only if ν = δy for some y ∈ RN . Hint: Begin by showing that it suffices to handle the case when N = 1. Next, assuming that N = 1, show that ν is compactly supported, let m be its mean value, and show that ν = δm . § 4.2 Discontinuous L´ evy Processes In this section I will construct the L´evy processes corresponding to those µ ∈ I(RN ) with no Gaussian component. That is, √ −1 ξ, mµ RN µ ˆ(ξ) = exp (4.2.1) Z h √ √ i −1(ξ,y) − 1 − −1 1[0,1] (|y|) ξ, y RN Mµ (dy) . + e RN
§ 4.2 Discontinuous L´evy Processes
161
Because they are the building blocks out of which all such processes are made, I will treat separately the case when µ is a Poisson measure πM for some M ∈ M0 (RN ) and will call the corresponding L´evy process the Poisson process associated with M . § 4.2.1. The Simple Poisson Process. I begin with the case when P∞ N 1= 1 −1 and M = δ1 , for which πM is the simple Poisson measure e m=0 m! δm √
whose Fourier transform is exp e −1ξ − 1 . To construct the Poisson process associated with δ1 , start with a sequence {τm : m ≥ 1} of independent, unit exponential random variables on a probability space (Ω, F, P). That is, ! n X + P {ω : τ1 (ω) > t1 , . . . , τn (ω) > tn } = exp − tm m=1
for all n ∈ Z+ and (t1 , . . . , tn ) ∈ Rn . Without loss in generality, I may and will assume that τm (ω) > 0 for all m ∈ Z+ and ω ∈ Ω. InPaddition, by The Strong ∞ Law of Large Numbers, I may and will assume that m=1 τm (ω) = ∞ for all Pn ω ∈ Ω. Next, set T0 (ω) = 0 and Tn (ω) = m=1 τm (ω), and define (4.2.2) N (t, ω) = max{n ∈ N : Tn (ω) ≤ t} =
∞ X
1[Tn (ω),∞) (t) for t ∈ [0, ∞).
n=1
Clearly t N (t, ω) is a non-decreasing, right-continuous, piecewise constant, Nvalued path that starts at 0 and, whenever it jumps, jumps by +1. In particular, N ( · , ω) ∈ D(RN ), N (t, ω) − N (t−, ω) ∈ {0, 1} for all t ∈ (0, ∞), and (cf. (4.1.6)) j t, · , N ( · , ω) = N (t, ω)δ1 . Because P N (t) = n = P Tn ≤ t < Tn+1 , P N (t) = 0 = P(τ1 > t) = e−t , and, when n ≥ 1 (below |Γ| denotes the Lebesgue measure of Γ ∈ BRn ) Z Z Pn+1 P N (t) = n , = · · · e− m=1 τm dτ1 · · · dτn+1 = e−t |B|, A
Pn Pn+1 where A = (τ1 , . . . , τn+1 ) ∈ (0, ∞)n+1 : τ ≤ t < τ and m m m=1 m=1 Pn B = (τ1 , . . . , τn ) ∈ (0, ∞)n : τ ≤ t . By making the change of m m=1 Pm variables sm = j=1 τj and remarking that the associated Jacobian is 1, one sees that |B| = |C|, where C = (s1 , . . . , sn ) ∈ Rn : 0 < s1 < · · · < sn ≤ t . n Since |C| = tn! , we have shown that the P-distribution of N (t) is the Poisson measure πtδ1 . In particular, πδ1 is the P-distribution of N (1). I now want to use the same sort of calculation to show that {N (t) : t ∈ [0, ∞)} is a simple Poisson process, that is, a L´evy process for πδ1 . (See Exercise 4.2.18 for another, perhaps preferable, approach.)
162
4 L´evy Processes
Lemma 4.2.3. For any (s, t) ∈ [0, ∞), the P-distribution of the increment N (s + t) − N (s) is πtδ1 . In addition, for any K ∈ Z+ and 0 = t0 < t1 < · · · < tK , the increments {N (tk ) − N (tk−1 ) : 1 ≤ k ≤ K} are independent. Proof: What I have to show is that, for all K ∈ Z+ , 0 = n0 ≤ · · · ≤ nK , and 0 = t0 < t1 < · · · < tK , P N (tk ) − N (tk−1 ) = nk − nk−1 , 1 ≤ k ≤ K K Y e−(tk −tk−1 ) (tk − tk−1 )nk −nk−1 , (nk − nk−1 )!
=
k=1
which is equivalent to checking that K Y (tk − tk−1 )nk −nk−1 ; P N (tk ) = nk , 1 ≤ k ≤ K = e−tK (nk − nk−1 )! k=1
and, since the case when nK = 0 is trivial, I will assume that nK ≥ 1. In fact, because neither side is changed if one removes those nk ’s for which nk = nk−1 , I will assume that 0 = n0 < · · · < nK . Begin by noting that P N (tk ) = nk , 0 ≤ k ≤ K = P Tnk ≤ tk < Tnk+1 , 1 ≤ k ≤ K Z Z PnK +1 = · · · e− m=1 τm dτ1 · · · dτnK +1 = e−tK |B|, A
where ( A=
nK +1
(τ1 , . . . , τnK +1 ) ∈ (0, ∞)
:
nk X
τm ≤ tk <
m=1
nX k +1
) τm , 1 ≤ k ≤ K
m=1
and ( B=
(τ1 , . . . , τnK ) ∈ (0, ∞)nK : tk−1 <
nk X
) τm ≤ tk : 1 ≤ k ≤ K
.
m=1
Pm To compute |B|, make the change of variables sm = j=1 τj to see that |B| = |C|, where C = (s1 , . . . , snK ) ∈ RnK : tk−1 < snk−1 +1 < · · · < snk ≤ tk for 1 ≤ k ≤ K . Finally, for 1 ≤ k ≤ K, set Ck = (snk−1 +1 , . . . , snk ) ∈ Rnk −nk−1 : tk−1 < snk−1 +1 < · · · < snk ≤ tk ,
§ 4.2 Discontinuous L´evy Processes
163
and check that |C| =
Y
|Ck | =
k∈S
Y (tk − tk−1 )nk −nk−1 . (nk − nk−1 )!
k∈S
The simple Poisson process {N (t) : t ≥ 0} is aptly named. It starts at 0, waits a unit exponential holding time before jumping to 1, sits at 1 for another, independent, unit exponential holding time before jumping to 2, etc. Thus, since πδ1 is the distribution of this process at time 1, we now have an appealing picture of the way in which simple Poisson random variables arise. Given α ∈ [0, ∞), I will say that a D(R)-valued process whose distribution is the same as that of {N (αt) : t ≥ 0} is a simple Poisson process run at rate α. § 4.2.2. Compound Poisson Processes. I next want to build a Poisson process associated with a general M ∈ M0 (RN ). If M = 0, there is nothing to do, since the corresponding process will simply sit at 0 for all time. If M 6= 0, I write it as αν, where α = M (RN ) and ν = M α . After augmenting the probability space if necessary, I introduce a sequence {Xn : n ≥ 1} of mutually independent, ν-distributed, random variables that are independent of the unit exponential random variables {τm : m ≥ 1} out of which I built the simple Poisson process {N (t) : t ≥ 0} in the preceding subsection. Further, since M ({0}) = 0, I may and will assume that none of the Xn ’s is ever 0. Finally, set
(4.2.4)
X
ZM (t, ω) =
Xn (ω),
1≤n≤N (αt,ω)
with the understanding that a sum over the empty set is 0. Clearly, the process {ZM (t) : t ≥ 0} is nearly as easily understood as is the simple Poisson process. Like the simple Poisson process, its paths are rightcontinuous, start at 0, and are piecewise constant. Further, its holding times and jumps are all independent of one another. The difference is that its holding times are now α-exponential random variables (i.e., exponential with mean value 1 α ) and its jumps are random variables with distribution ν. In particular,
(4.2.5)
j t, · , ZM ( · , ω) =
X 1≤n≤N (αt,ω)
δXn (ω) =
∞ X
1[Tn (ω),∞) (t)δXn (ω) .
n=1
I now want to check that {ZM (t) : t ≥ 0} is a L´evy process for πM and, as such, deserves to be called a Poisson process associated with M : the one with . That is, I want to show that, for rate M (RN ) and jump distribution M M (RN ) each 0 = t0 < t1 < · · · tK , the random variables ZM (tk ) − ZM (tk−1 ), 1 ≤ k ≤ K,
164
4 L´evy Processes
are mutually independent and that the kth one has distribution π(tk −tk−1 )M . Equivalently, I need to check that, for any ξ1 , . . . , ξK ∈ RN , !# " K K X Y √ P −1 ξk , ZM (tk ) − ZM (tk−1 ) RN = π[ E exp τk M (ξk ), k=1
k=1
where τk = tk − tk−1 . But, because of our independence assumptions, the above expectation is equal to X P N (αtk ) − N (αtk−1 ) = nk − nk−1 , 1 ≤ k ≤ K nK ≥···≥n1 ≥0
K √ X × EP exp −1
X
ξ k , Xm
RN
k=1 nk−1 +1<m≤nk
=
X
K K n −n Y Y e−ατk τk k k−1 νˆ(ξk )nk −nk−1 = π[ τk M (ξk ). (nk − nk−1 )!
nK ≥···≥n1 ≥0 k=1
k=1
Any stochastic process {Z(t) : t ≥ 0} with right-continuous, piecewise constant paths and the same distribution as the process {ZM (t) : t ≥ 0} just constructed is called a Poisson process associated with M . Here is a beautiful and important procedure for transforming one Poisson process into another. 0
Lemma 4.2.6. Suppose that F : RN −→ RN is a Borel measurable function 0 that takes the origin in RN into the origin in RN , and, for M ∈ M0 (RN ), define 0 M F ∈ M0 (RN ) by M F (Γ) = M F −1 Γ \ {0} for Γ ∈ BRN 0 . If {Z(t) : t ≥ 0} is a Poisson process associated with πM and Z F (4.2.7) Z (t, ω) = F (y) j t, dy, Z( · , ω) for (t, ω) ∈ [0, ∞) × Ω, RN
then {ZF (t) : t ≥ 0} is a Poisson process associated with πM F . Moreover, if, for each i in an index set I, Fi : RN −→ RNi is a Borel measurable satisfying N Fi (0) = 0 and, for each most one i ∈ I for which Fi (y) 6= 0, Fy ∈ R , there is at i then the processes {Z (t) : t ≥ 0} : i ∈ I are mutually independent. Proof: In proving the first part, I will, without loss in generality, assume that (cf. (4.2.4)) Z = ZM . But then, by (4.2.5), X ZF (t, ω) = F Xn (ω) , 1≤n≤N (αt,ω)
§ 4.2 Discontinuous L´evy Processes
165
from which the first assertion follows immediately from the same computation with which I just showed that {ZM (t) : t ≥ 0} is a Poisson process associated with M . To prove the second assertion, I begin by observing that it suffices to treat the case when I = {1, 2}. To see this, suppose that we know the result in that case, and let n > 2 and a set {i1 , . . . , in } of distinct elements from I be given. By taking F1 = (Fi1 , . . . , Fin−1 ), F2 = Fin , and applying the assumed result, we would have that {ZFin (t) : t ≥ 0} is independent of ZFi1 (t), . . . , ZFin−1 (t) : t ≥ 0 . Hence, F proceeding by induction, we would be able to show that the processes {Z im (t) : t ≥ 0} : 1 ≤ m ≤ n are independent. Now assume that I = {1, 2}. What I have to check is that, for any K ∈ Z+ , 0 = t0 < t1 < · · · < tK , and {(ξk1 , ξk2 ) : 1 ≤ k ≤ K} ⊆ RN1 × RN2 , " K h √ X P −1 ξk1 , ZF1 (tk ) − ZF1 (tk−1 RN1 E exp k=1
i + ξk2 , ZF2 (tk ) − ZF2 (tk−1 ) RN2 " P
=E
√
exp
−1
#
!#
K X
ξk1 , ZF1 (tk )
F1
− Z (tk−1 ) RN1
k=1
" P
×E
exp
√
−1
K X
!# ξk2 , ZF2 (tk )
F2
− Z (tk−1 ) RN2
.
k=1
For this purpose, take F : RN −→ RN1 +N2 to be given by F (y) = F1 (y), F2 (y) , and set ξk = (ξk1 , ξk2 ). Then the first expression in the preceding equals " # K √ X F F P −1 ξk , Z (tk ) − Z (tk−1 RN1 +N2 E exp k=1
=
K Y
h √ i EP exp −1 ξk , ZF (tk − tk−1 ) RN1 +N2 ,
k=1
since {ZF (t) : t ≥ 0} has independent, homogeneous increments. Hence, it suffices to observe that, for any t > 0 and ξ = (ξ 1 , ξ 2 ), Z √ h i EP exp ξ, ZF (t) RN1 +N2 = exp t e −1(ξ,F (y))RN1 +N2 − 1 M (dy) RN Z √ −1(ξ1 ,F1 (y))RN1 − 1 M (dy) = exp t e RN Z √ −1(ξ2 ,F2 (y))RN2 − 1 M (dy) × exp t e RN h i h i = EP exp ξ 1 , ZF1 (t) RN1 EP exp ξ 2 , ZF2 (t) RN2 .
166
4 L´evy Processes
As an essentially immediate consequence of Lemma 4.2.6 and Theorem 4.1.8, we have the following important conclusion. If {Z(t) : t ≥ 0} is aPoisson process associated with πM , BRN \{0} , j t, ∆, Z( · ) : t ≥ 0 is a simple Poisson process Moreover, if y j t, dy, Z and M ∆ (Γ) = M (∆ ∩ Γ) for Γ ∈ BRN , ∆ ∆ then {Z (t) : t≥ 0} is the Poisson process associated with M ∆ and j t, Γ, Z∆ = j t, Γ ∩ ∆, Z for all (t, Γ) ∈ [0, ∞) × BRN . Finally, if {∆i : i ∈ I} is a family of disjoint Borel subsets of RN \ {0}, then boththe Poisson processes mutually ∆i {Z (t) : t ≥ 0} : i ∈ I as well as the jump processes {j t, ∆i , Z : t ≥ 0} : i ∈ I are mutually independent. Theorem 4.2.8. then, for each ∆ ∈ run at rate M (∆). Z ∆ Z (t) =
The result in Theorem 4.2.8 says that the jumps of a Poisson process can be decomposed into a family of mutually independent, simple Poisson processes run at rates determined by the M -measure of the jump sizes. The next result can be thought of as a re-assembly procedure that complements this decomposition result. Theorem 4.2.9. If {Zk (t) : t ≥ 0} : 1 ≤ k ≤ K are mutually independent Poisson processes associated with {Mk : 1 ≤ k ≤ K} ⊆ M0 (RN ), then ( ) K K X X Z(t) ≡ Zk (t) : t ≥ 0 is a Poisson process associated with M ≡ Mk . k=1
k=1
Next, suppose that the Mk ’s are mutually singular in the sense that, for each k, there exists a ∆k ∈ BRN \{0} with the properties that ∆k ∩ ∆` = ∅ and Mk ∆k { = 0 = M` (∆k ) for ` 6= k. Then, for P-almost every ω ∈ Ω, K X j t, · , Z( · , ω) = j t, · , Zk ( · , ω) ,
t ∈ [0, ∞).
k=1
Equivalently, for P-almost every ω ∈ Ω and all t ≥ 0, there is at most one k such that Zk (t, ω) 6= Zk (t−, ω). Proof: Clearly, {Z(t) : t ≥ 0} starts at 0 and has independent increments. In addition, for any s, t ∈ [0, ∞) and ξ ∈ RN , K i i Y h √ h √ EP e −1(ξ,Zk (s+t)−Zk (s))RN EP e −1(ξ,Z(s+t)−Z(s))RN = k=1
=
K Y
Z exp t
√
e
−1(ξ,y)RN
− 1 Mk (dy)
RN
k=1
Z = exp t RN
√
e
−1(ξ,y)RN
− 1 M (dy) .
§ 4.2 Discontinuous L´evy Processes
167
Now assume that the Mk ’s are as in the final part of the statement, and choose ∆k ’s accordingly. Without loss in generality, I will assume that RN \ {0} = SK k=1 ∆k . Also, because the assertion depends only on the joint distribution of the processes involved, I may and will assume that Z Zk (t) = y j t, dy, Z for 1 ≤ k ≤ K, ∆k
PK since then Z(t) = k=1 Zk (t), and, by Theorem 4.2.8, the Zk ’s are independent and the kth one is a Poisson process associated with Mk . But with this choice, another application of Theorem 4.2.8 shows that j t, Γ, Zk = j t, Γ ∩ ∆k , Z , and therefore K X j t, Γ, Z = j t, Γ, Zk , t ∈ [0, ∞). k=1
Because the paths of a Poisson process are piecewise constant, they certainly have finite variation on each compact time interval. The first part of the next lemma provides an estimate of that variation. The estimate in the second part will be used in § 4.2.5. Lemma 4.2.10. M0 (RN ), then
If {Z(t) : t ≥ 0} is a Poisson process associated with M ∈ E var[0,t] (Z) = t
Z
P
|y| M (dy). RN
In addition, if
R RN
¯ |y| M (dy) < ∞ and Z(t) = Z(t) −
2 N 2t 2 ¯ ¯ [0,t] ≥ R ≤ N t EP |Z(t)| = 2 P kZk R R2
R RN
Z
y M (dy), then
|y|2 M (dy).
RN
Proof: Again I will assume that (cf. (4.2.4)) Z = ZM , in which case X var[0,t] (Z) = |Xm |. 1≤m≤N (αt)
Hence (cf. the notation used in § 4.1.1) EP var[0,t] (Z) = EP N (αt) EP |X1 | = αt
Z
Z |y| ν(dy) = t
RN
|y| M (dy). RN
Turning to the second part, begin by observing that −n ¯ ¯ P kZk[0,t] > R = lim P max Z m2 t > R n→∞ 1≤m≤2n − 12 −n ¯ ≤ N lim sup P max n e, Z m2 t RN > N R . n→∞ e∈SN −1
1≤m≤2
168
4 L´evy Processes
Next, given e ∈ SN −1 and n ≥ 1, write X −n ¯ ¯ −n t) − Z((` ¯ e, Z(m2 t) RN = e, Z(`2 − 1)2−n t) RN , 1≤`≤m
and apply Kolmogorov’s Inequality to conclude that 2 −n − 12 ¯ ¯ . P max n e, Z m2 t RN > N R ≤ N R−2 EP e, Z(t) RN 1≤m≤2
R ¯ M (t)|2 = t N |y|2 M (dy). To this Thus, we will be done once I check that EP |Z R R 2 2 2 ¯ end, first note that EP |Z(t)| = EP |Z(t)|2 −α t |m|2 , where m = RN y ν(dy). ¯ m = Xm − m, then EP |Z(t)|2 equals At the same time, if X 2 2 X X P P ¯ E X m = E Xm + |m|2 EP N (αt)2 1≤m≤N (αt) 1≤m≤N (αt) ¯ 1 |2 + |m|2 α2 t2 + αt = αtEP |X1 |2 + α2 t2 |m|2 . = αtEP |X R Thus, since αEP |X1 |2 = RN |y|2 M (dy), the desired equality follows. § 4.2.3. Poisson Jump Processes. Rather than attempting to construct more general L´evy processes directly, I will first construct their jump processes and then construct them out of their jumps. With this idea in mind, given a probability space (Ω, F, P), I will say that (t, ω) j(t, · , ω) is a Poisson jump process associated with M ∈ M∞ (RN ) if, for each ω ∈ Ω, t j(t, · , ω) is a jump func+ tion, and for each n ∈ Z and collection Sn {∆1, . . . , ∆n } of mutually disjoint Borel subsets sets of RN satisfying 0 ∈ / i=1 ∆i , {j(t, ∆i ) : t ≥ 0} : 1 ≤ i ≤ n are mutually independent, simple Poisson processes, the ith of which is run at rate M (∆i ) for each 1 ≤ i ≤ n. By starting with simple functions and passing to limits, one can easily check that Z (t, ω) ∈ [0, ∞) × Ω 7−→ ϕ(y) j(t, dy, ω) ∈ [0, ∞]
is measurable for every Borel measurable function ϕ : RN −→ [0, ∞]. Therefore, 0 if F : RN −→ RN is a Borel measurable function, and, for T > 0, Z Ω(T ) ≡ ω : |F (y)| j(T, dy, ω) < ∞ , then both the set Ω(T ) and the function Z 0 (t, ω) ∈ [0, T ] × Ω(T ) F (y) j(t, y, ω) ∈ RN are measurable. Note that if |F (y)| vanishes for y’s in a neighborhood of 0, then Ω(T ) = Ω for all T > 0. My goal in this subsection is to prove the following existence result.
§ 4.2 Discontinuous L´evy Processes
169
Theorem 4.2.11. For each M ∈ M∞ (RN ) there exists an associated Poisson jump process. (See § 9.2.2 for another approach.) Proof: Set A0 = RN \B(0, 1) and Ak = B(0, 2−k+1 )\B(0, 2−k ) for k ∈ Z+ , and define Mk(dy) = 1Ak (y) M (dy). Next, choose mutually independent Poisson processes {Zk (t) : t ≥ 0} : k ∈ N so that the kth one is associated with Mk , and set jk (t, · , ω) = j t, · , Zk ( · , ω) . Without loss in generality, I may and will assume that jk (t, Ak {, ω) = 0 for P all (t, ω) ∈ [0, ∞) × Ω and k ∈ N. In addition, m by Theorem 4.2.9, if Z(m) (t) = k=0 Zk (t), then we know that, for P-almost every ω ∈ Ω, m X j (m) (t, · , ω) ≡ j t, · , Z(m) ( · , ω) = jk (t, · , ω),
t ≥ 0.
k=0
Hence, I may and will assume that t
j(t, · , ω) ≡
∞ X
jk (t, · , ω)
k=1
is a jump function for all ω ∈ Ω. Finally, suppose that {∆i : 1 ≤ i ≤ n} ⊆ BRN Sn Sn are disjoint and that 0 ∈ / i=1 ∆i . Choose m ∈ N so that ( 1 ∆i )∩ B(0, 2−m ) = ∅, and note that, P-almost surely, j(t, ∆i , ω) = j (m) (t, ∆i , ω) for all t ≥ 0 and 1 ≤ i ≤ n. Hence, the required independence property is a consequence of the last part of Theorem 4.2.8. In preparation for the next section, I prove the following. 0
Lemma 4.2.12. Let F : RN −→ RN be a Borel measurable function satisfying 0 F (0) = 0 and 0 ∈ / F −1 RN \ B(0, r) for any r > 0. For any M ∈ M∞ (RN ), 0 M F ∈ M∞ (RN ). Moreover, if {j(t, · ) : t ≥ 0} is a Poisson jump process associated with M and j F (t, Γ, ω) ≡ j t, F −1 (Γ\{0}), ω , then {j F (t, · ) : t ≥ 0} is a Poisson jump process associated with (cf. Lemma 4.2.6) M F . Finally, if 0 0∈ / F −1 (RN \ {0}) and Z Z ZF (t, ω) ≡ y j F (t, dy, ω) = F (y) j(t, dy, ω), 0
then M F ∈ M0 (RN ), {ZF (t) : t ≥ 0} is a Poisson process associated with M F , and j t, · , ZF ( · , ω) = j F (t, · , ω). Proof: To prove the first assertion, suppose that {∆1 , . . . , ∆n } are disjoint Sn 0 Borel subsets of RN and that 0 ∈ / i=1 ∆i . Then {F −1 (∆1 ), . . . , F −1 (∆n )} satisfy the same conditions as subsets of RN , and therefore, since j F (t, ∆i , ω) = F −1 j t, F (∆i ), ω), {j (t, ∆i ) : t ≥ 0} : 1 ≤ i ≤ n has the required properties.
170
4 L´evy Processes 0
Turning to the second assertion, first note that M F ∈ M0 (RN ) is an immedi 0 ate consequence of 0 ∈ / F −1 (RN \ {0}) and that the equality j t, · , ZF ( · , ω) = j F (t, · , ω) is a trivial application of the final part of Theorem 4.1.8. To prove that {ZF (t) : t ≥ 0} is a Poisson process associated with M F , use Theorem 4.2.8 to see that {j F (t, · ) : t ≥ 0} has the same distribution as the jump process for a Poisson process {Z(t) : t ≥ 0} associated with M F . Hence, R since Z(t) = y j(t, dy, Z), {ZF (t) : t ≥ 0} has the same distribution as {Z(t) : t ≥ 0}.
§ 4.2.4. L´ evy Processes with Bounded Variation. Although the contents of the previous section provide the machinery with which to construct a L´evy process for any µ with Fourier transform given by (4.2.1), for reasons made clear in the next lemma, I will treat the special case when M ∈ M1 (RN ) here and will deal with M ∈ M2 (RN ) \ M1 (RN ) in the following subsection. Lemma 4.2.13. Let {j(t, · ) : t ≥ 0}R be a Poisson jump process associated with M ∈ M2 (RN ), and set V (t, ω) = |y| j(t, dy, ω). Then V (t) < ∞ almost surely or V (t) = ∞ almost surely for all t > 0, depending on whether M is or is not in M1 (RN ). (See Exercise 4.3.11 to see that the same conclusion holds for any M ∈ M∞ (RN ).) R Proof: Since |y|>1 |y| j(t, dy, ω) < ∞ for all (t, ω) ∈ [0, ∞) × Ω, the question R is entirely about the finiteness of V0 (t, ω) ≡ B(0,1) |y| j(t, dy, ω). To study this −k+1 ) \ B(0, 2−k ), F (y) = |y|1 k Ak (y), and Vk (t, ω) = Rquestion, set Ak = B(0, 2 |y| j(t, dy, ω) for k ≥ 1. Clearly, the processes {V (t) : t ≥ 0} : k ∈ Z+ k Ak are mutually independent. In addition, for each k, t Vk (t) is non-decreasing and, by the second part of Lemma 4.2.12, {Vk (t) : t ≥ 0} is a Poisson process associated with M Fk . Thus, by Lemma 4.2.10,
ak ≡ EP Vk (t) = t
Z
|y| M (dy) and bk ≡ Var Vk (t) = t
Ak
Z
|y|2 M (dy).
Ak
From the first of these, it follows that
#
"Z P
|y| j(t, dy) =
E
B(0,1)
∞ X
E Vk (t) =
Z
P
|y| M (dy), B(0,1)
k=1
which finishes the case when M ∈ M1 (RN ). When M ∈ M2 (RN ) \ M1 (RN ), set V¯k (t) = Vk (t) − tak . Then, for each t > 0, {V¯k (t) : k ∈ Z+ } is a sequence of mutually independent random variables with mean value 0. Furthermore, ∞ X k=1
∞ X Var V¯k (t) = t bk = t k=1
Z
|y|2 M (dy) < ∞. B(0,1)
§ 4.2 Discontinuous L´evy Processes
171
P∞ ¯ Hence, by Theorem 1.4.2, k=1 Vk (t) converges P-almost P P∞ surely. But, when ∞ M ∈ / M1 (RN ), k=1 ak = ∞, and so, for each t > 0, k=1 Vk (t) must diverge P-almost surely. Before stating the main result of the subsection, I want to introduce the notion of a generalized Poisson measure. Namely, if M ∈ M1 (RN ) \ M0 (RN ) and πM is the element of I(RN ) whose Fourier transform is given by Z √ −1(ξ,y)RN − 1 M (dy) , exp e
R or, equivalently, π d M is given by (4.2.1) with m = B(0,1) y M (dy), then I will call πM the generalized Poisson measure for M . Similarly, if {Z(t) : t ≥ 0} is a L´evy process for a generalized Poisson measure πM , I will say that it is a generalized Poisson process associated with M .
Theorem 4.2.14. Suppose that M ∈ M1 (RN ) and that {j(t, · ) : t ≥ 0} is a Poisson jump process associated with M . Set N = {ω : ∃t > 0 j(t, · , ω) ∈ / N M1 (R )}, and define (t, ω) ZM (t, ω) so that R y j(t, dy, ω) if ω ∈ /N ZM (t, ω) = 0 if ω ∈ N . Then P(N ) = 0 and {ZM (t) : t ≥ 0} is a (possibly generalized) Poisson process associated with M . In particular, t ZM (t, ω) is absolutely pure jump for all ω ∈ Ω, and {j(t, · , ZM ) : t ≥ 0} is a Poisson jump process associated with M . Finally, if µ ∈ I(RN ) has Fourier transform given by (4.2.1), then ! ) ( Z y M (dy) + ZM (t) : t ≥ 0 t m− B(0,1)
is a L´evy process for µ. Proof: That P(N ) = 0 follows from Lemma 4.2.13. To prove that {ZM (t) : t ≥ 0} is a L´evy process for πM , set Z Z(r) (t, ω) = y j(t, dy, ω) |y|>r (r)
for r > 0. By Lemma 4.2.12, {Z (t) : t ≥ 0} is a Poisson process associated with M (r) (dy) ≡ 1(r,∞) (y) M (dy). In addition, if ω ∈ / N , then Z(r) ( · , ω) −→ ZM ( · , ω) uniformly on compacts, from which it is easy to check that {ZM (t) : t ≥ 0} is a Poisson process associated with M and that the process in the last assertion is a L´evy process for the µ whose Fourier transform is given by (4.2.1) with this M . Finally, by the last part of Theorem 4.1.8, j t, · , ZM ( · , ω) = j(t, · , ω) when ω ∈ / N , from which it is clear that {j(t, · , ZM ) : t ≥ 0} is a Poisson jump process associated with M . § 4.2.5. General, Non-Gaussian L´ evy Processes. In this subsection I will complete the construction of L´evy processes with no Gaussian component.
172
4 L´evy Processes
Theorem 4.2.15. For each m ∈ RN and M ∈ M2 (RN ) there is a L´evy process for the µ ∈ I(RN ) whose Fourier transform is given by (4.2.1). Moreover, if {Z(t) : t ≥ 0} is such a process, then {j(t, · , Z) : t ≥ 0} is a Poisson jump process associated with M . Finally, if, for r ∈ (0, 1], Z Z Z(r) (t) = y j(t, dy, Z) − t y M (dy), |y|>r
r<|y|≤1
then ! P
sup Z(τ ) − τ m − Z(r) (τ ) ≥
≤
τ ∈[0,t]
N 2t 2
Z
|y|2 M (dy). B(0,r)
Proof: Without loss in generality, I will assume that m = 0. By Theorem 4.2.11, we know that there is a Poisson jump process {j(t, · ) : t ≥ 0} associated with M . Take ¯j(t, dy, ω) = j(t, dy, ω) − t1 (y)M (dy), B(0,1)
and define (r)
Z
Z
y ¯j(t, dy, ω),
(t, ω) =
(t, ω) ∈ [0, ∞) × Ω,
|y|>r
for r ∈ (0, 1]. By Theorem 4.2.14, we know that {Z(r) (t) : t ≥ 0} is a L´evy process for µ(r) , where ! Z i h √ √ (r) (ξ) = exp µd e −1(ξ,y)RN − 1 − −1 1[0,1] (y) ξ, y RN M (dy) . |y|>r
Furthermore, by the second part of Lemma 4.2.10, we know that, for 0 < r < r0 ≤ 1, Z N 2t 0 |y|2 M (dy). (*) P kZ(r ) − Z(r) k[0,t] ≥ ≤ 2 0 r<|y|≤r
Hence, if 1 ≥ rm & 0 is chosen so that Z |y|2 M (dy) ≤ 2−m , B(0,rm )
then P sup kZ(rn ) − Z(rm ) k[0,t] ≥ n>m
1 m
≤
X
P kZ(rn+1 ) − Z(rn ) k[0,t] ≥ (m + 1)−2
n≥m
≤ N 2t
∞ X
(n + 1)4 2−n ,
n=m
§ 4.2 Discontinuous L´evy Processes
173
and therefore, by the first part of the Borel–Cantelli Lemma, 1 = 1. P ∃m ∀n ≥ m kZ(rn ) − Z(rm ) k[0,t] ≤ m+1
We now know that there is a P-null set N such that, for any ω ∈ / N , there exists a Z( · , ω) ∈ D(RN ) to which {Z(rm ) ( · , ω) : n ≥ 0} converges uniformly on compacts. Thus, if we take Z(t, ω) = 0 for (t, ω) ∈ [0, ∞) × N , then it is an easy matter to check that {Z(t) : t ≥ 0} is a L´evy process for the µ ∈ I(RN ) whose Fourier transform is given by (4.2.1) with m = 0. In addition, since, by Theorem 4.1.8, we know that t j(t, · , ω) is the jump function for t Z(t, ω) when ω ∈ / N , it is clear that {j(t, · , Z) : t ≥ 0} is a Poisson jump process associated with M . Finally, to prove the estimate in the concluding assertion, observe that, for ω ∈ / N , the path t Z(r) (t, ω) used in our construction coincides with the path described in the statement. Thus, the desired estimate is an easy consequence of the one in (*) above. Corollary 4.2.16. Let µ ∈ I(RN ) with Fourier transform given by (4.2.1), and suppose that {Z(t) : t ≥ 0} is a L´evy process for µ. Then, depending on whether or not M ∈ M1 (RN ), either P-almost all or P-almost none of the paths t Z(t) has locally bounded variation. Moreover, if M ∈ M1 (RN ), then, P-almost surely, ! Z y M (dy) is an absolutely pure jump path. t Z(t) − t m − B(0,1)
Proof: From Theorem 4.2.14, we already know that t Z(t) − tm is almost surely an absolutely pure jump path if M ∈ M1 (RN ), and so t Z(t) is almost surely of locally bounded variation. Conversely, if t Z(t) has locally bounded variation with positive probability, then, by (4.1.13), j t, · , Z ∈ M1 (RN ) with positive probability. But then, since {j t, · , Z : t ≥ 0} is a Poisson jump process associated with M , it follows from Lemma 4.2.13 that M ∈ M1 (RN ). Corollary 4.2.17. Let µ and {Z(t) : t ≥ 0} be as in Corollary 4.2.16. Given ¯ set ∆ ∈ BRN with 0 ∈ / ∆, Z Z ∆ ∆ ∆ y M ∆ (dy). Z (t) = y j(t, dy, Z), M (dy) = 1∆ (y)M (dy), and m = ∆
B(0,1)
Then {Z∆ (t) : t ≥ 0} is a Poisson process associated with M ∆ , {Z(t) − Z∆ (t) : t ≥ 0} is a L´evy process for the element of I(RN ) whose Fourier transform is √ −1 ξ, m − m∆ RN exp Z h √ √ i −1(ξ,y)RN − 1 − −11[0,1] |y| ξ, y RN M (dy) , + e RN \∆
and {Z(t) − Z (t) : t ≥ 0} is independent of {j t, · , Z∆ : t ≥ 0}, and therefore of {Z∆ (t) : t ≥ 0} as well. ∆
174
4 L´evy Processes
Proof: That {Z∆ (t) : t ≥ 0} is a Poisson process associated with M ∆ is an immediate consequence of Lemma 4.2.12. Next, define Z(r) (t) as in Theorem 4.2.15. Then, for all r ∈ (0, 1], Z Z (r) ∆ Z (t) − Z (t) = 1RN \∆ (y)y j(t, dy) − t y M (dy). |y|>r
r<|y|≤1
In particular, this means that {Z(r) (t)−Z∆ (t) : t ≥ 0} has independent, homogeneous increments and (cf. Theorem 4.1.8) is independent of {j t, · , Z∆ : t ≥ 0}. Thus, since, as r & 0, Z(r) (t) −→ Z(t) − tm in probability, it follows that {Z(t) − Z∆ (t) : t ≥ 0} is independent of {j(t, · , Z∆ ) : t ≥ 0}. In addition, √ √ √ (r) ∆ ∆ ∆ ∆ e− −1t(ξ,m−m )RN EP e −1(ξ,Z(t)−Z (t))RN = lim EP e −1(ξ,Z (t)−Z (t)+tm )RN r&0 Z i h √ √ e −1(ξ,y)RN − 1 − −11[0,1] |y| ξ, y RN M (dy) = lim exp r&0
(∆∪B(0,r)){
Z = exp RN \∆
h
e
√ −1(ξ,y)RN
−1−
√
−11[0,1]
! i |y| ξ, y RN M (dy) .
Hence, it follows that {Z(t) − Z∆ (t) : t ≥ 0} is a L´evy process for the specified element of I(RN ). Exercises for § 4.2 Exercise 4.2.18. Here is another proof that the process {N (t) : t ≥ 0} in § 4.2.1 has independent, homogeneous increments. Refer to the notation used there. (i) Given n ∈ Z+ and measurable functions f : [0, ∞)n+1 7−→ [0, ∞) and g : [0, ∞)n −→ R, show that EP f (τ1 , . . . , τn+1 ), τn+1 > g(τ1 , . . . , τn ) + = EP e−g(τ1 ,... ,τn ) f τ1 , . . . , τn , τn+1 + g(τ1 , . . . , τn )+ . (ii) Let K ∈ Z+ , 0 = n0 ≤ n1 ≤ · · · ≤ nK , and 0 = t0 ≤ t1 < · · · < tK = s be given, and set A = {N (tk ) = nk , 1 ≤ k ≤ K}. Show that A = B ∩ {τnK +1 > s − TnK }, where B ∈ σ {τ1 , . . . , τnK } , and apply (i) to see that P(A) = EP e(s−TnK ) , B . (iii) Let n ∈ Z+ and t > 0 be given, and set h(ξ) = P(Tn−1 > ξ). Referring to (ii) and again using (i), show that P A ∩ {N (s + t) − N (s) < n} = EP h(t + s − TnK +1 ), B ∩ {τnK +1 > s − TnK } = EP e−(s−TnK ) h(t − τnK +1 ), B = EP h(t − τnK +1 ) EP e−(s−TnK ) , B = P N (t) < n P(A).
Exercises for § 4.2
175
Exercise 4.2.19. Let {N (t) : t ≥ 0} be a simple Poisson process, and show that limt→∞ N t(t) = 1 P-almost surely.
Hint: First use The Strong Law of Large Numbers to show that limn→∞ 1 P-almost surely. Second, use 2 N (t) − N (n) ≥ ≤ P N (1) ≥ n ≤ 2 2 P sup n t n≤t≤n+1
to see that
N (n) n
=
N (t) N (btc) = 0 P-almost surely. − lim t→∞ btc t
Exercise 4.2.20. Assume that µ ∈ I(R) has its Fourier transform given by (4.2.1), and let {Z(t) : t ≥ 0} be a L´evy process for µ. Using Exercise 3.2.25, show that t R Z(t) is non-decreasing if and only if M ∈ M1 (R), M (−∞, 0) = 0, and m ≥ [−1,1] y M (dy). Exercise 4.2.21. Let {j(t, · ) : t ≥ 0} be a Poisson jump process associated with some M ∈ M∞ (RN ), and suppose that F : RN −→ R is a Borel measurable, M -integrable function that vanishes at 0. (i) Let N be the set of ω ∈ Ω for which there is a t > 0 such that F is not j t, · , ω)-integrable, and show that P(N ) = 0. (ii) Show that (cf. Lemma 4.2.6) M F ∈ M1 (R) and that, in fact, Z Z F |y| M (dy) = |F (y)| M (dy) < ∞. Next, define F
R
Z (t, ω) =
F (y) j(t, dy, ω)
0
if ω ∈ /N if ω ∈ N ,
and show that {Z F (t) : t ≥ 0} is a (possibly generalized) Poisson process associated with M F . (iii) Show that Z F (t) = t→∞ t lim
Z F (y) M (dy)
P-almost surely.
Hint: Begin by using Lemma 4.2.10 to show that it suffices to handle F ’s that vanish in a neighborhood of 0. When F vanishes in a neighborhood of 0, use Lemma 4.2.12 to see that {Z F (t) : t ≥ 0} is a Poisson process associated with M F . Finally, use the representation of a Poisson process in terms of a simple Poisson process and independent random variables, and apply The Strong Law of Large Numbers together with the result in Exercise 4.2.19.
176
4 L´evy Processes
Exercise 4.2.22. Let {Z(t) : t ≥ 0} be a L´evy process for the µ ∈ I(RN ) with ¯ Fourier transform given by (4.2.1), and set Z(t) = Z(t) − tm. Show that for all ¯ [0,t] ≥ R is dominated by t times R ∈ [1, ∞) and t ∈ (0, ∞), P kZk 4N R2
Z
|y|2 M (dy) +
B(0,1)
2 R
Z √ 1<|y|≤ R
|y| M (dy) + M B(0,
¯ Hint: Write Z(t) = Z1 (t) + Z2 (t) + Z3 (t), where Z Z y j(t, dy, Z) and Z (t) = Z2 (t) = 3 √
√ |y|> R
1<|y|≤ R
√
R){ .
y j(t, dy, Z).
Then, P kZk[0,t] ≥ R ≤ P kZ1 k[0,t] ≥
R 2
+ P kZ2 k[0,t] ≥
R 2
+ P kZ3 k[0,t] 6= 0 .
Apply the estimates in Lemma 4.2.10 to control the first two terms on the right, and use √ √ N P j t, RN \ B(0, R), Z 6= 0 = 1 − e−tM (R \B(0, R))
to control the third. Exercise 4.2.23. Let ν be a locally finite Borel measure on RN . A Poisson point process with intensity measure ν is a random, locally finite, purely atomic measure-valued random variable ω P ( · , ω) with the properties that, for any bounded Γ ∈ BRN , P (Γ) is a Poisson random variable with mean value ν(Γ) and, for any n ≥ 2 and family {Γ1 , . . . , Γn } of mutually disjoint, bounded, Borel subsets of RN , {P (Γ1 ), . . . , P (Γn )} are mutually independent. The purpose of this exercise is to show how one always construct such a Poisson point process for any ν. y (i) Define F : RN −→ RN so that F (0) = 0 and F (y) = |y| 2 for y 6= 0. Clearly, −1 F is one-to-one and onto, and both F and F are Borel measurable. Assuming that ν({0}) = 0, show that M ≡ F∗ ν ∈ M∞ (RN ) and that ν = (F −1 )∗ M .
(ii) Continue to assume that ν({0}) = 0, let {j(t, · ) : t ≥ 0} be a Poisson jump process associated with the M in (i), and set P ( · , ω) = (F −1 )∗ j(1, · , ω). Show that ω P ( · , ω) is a Poisson point process with intensity ν. (vi) In order to handle ν’s that charge 0, suppose ν({0}) > 0. Choose a point x ∈ RN for which ν({x}) = 0, define ν 0 (Γ) = ν(x + Γ), note that ν 0 ({0}) = 0, and construct a Poisson point process ω P 0 ( · , ω) with intensity measure ν 0 . 0 Finally, define P (Γ, ω) = P (Γ − x, ω), and check that ω P ( · , ω) is a Poisson point process with intensity measure ν.
§ 4.3 Brownian Motion, the Gaussian L´evy Process
177
Exercise 4.2.24. Let M ∈ M2 (RN ) be given, and assume that there exists a decreasing sequence {rn : n ≥ 0} ⊆ (0, 1] with rn & 0 such that Z m = lim y M (dy) n→∞
rn <|y|≤1
exists. Let µ ∈ I(RN ) have Fourier transform given by (4.2.1) with this m and M . If {Z(t) : t ≥ 0} is a L´evy process for µ, set Z Zn (t, ω) = y j t, dy, Z( · , ω) , |y|>rn
and show that limn→∞ P kZ − Zn k[0,t] ≥ = 0 for all t ≥ 0 and > 0. Thus, after passing to a subsequence {nm : m ≥ 0} if necessary, one sees that, P-almost surely, Z Z(t, ω) = lim y j t, dy, Z( · , ω) , m→∞
|y|>rnm
where the convergence is uniform on finite time intervals. In particular, one can say that P-almost all the paths t Z(t, ω) are “conditionally pure jump.” § 4.3 Brownian Motion, the Gaussian L´ evy Process What remains of the program in this chapter is the construction of a L´evy process for the standard, normal distribution γ0,I , the infinitely divisible law |ξ|2
whose Fourier transform is e− 2 . Indeed, if {Zγ0,I (t) : t ≥ 0} is such a process and {Zµ (t) : t ≥ 0} is a L´evy process for the µ ∈ I(RN ) whose Fourier transform is given by (4.2.1), and if {Zγ0,I (t) : t ≥ 0} is independent of {Zµ (t) : t ≥ 0}, 1 then it is an easy matter to check that C 2 Zγ0,I (t) + Zµ (t) will be a L´evy process for γ0,C ? µ, whose Fourier transform is
exp
√
−1 ξ, m RN − 12 ξ, Cξ RN Z h √ √ i + e −1(ξ,y)RN − 1 − −1 1[0,1] (|y|) ξ, y RN M (dy) . RN
Because one of its earliest applications was as a mathematical model for the motion of “Brownian particles,” 1 such a L´evy process for γ0,1 is called a Brownian motion. In recognition of its provenance, I will adopt this terminology and will use the notation {B(t) : t ≥ 0} instead of {Zγ0,I (t) : t ≥ 0}. 1
R. Brown, an eighteenth century English botanist, observed the motion of pollen particles in a dilute gas. His observations were interpreted by A. Einstein as evidence for the kinetic theory of gases. In his famous 1905 paper, Einstein took the first steps in a program, eventually completed by N. Wiener in 1923, to give a mathematical model of what Brown had seen.
178
4 L´evy Processes
Before getting into the details, it may be helpful to think a little about what sorts of properties we should expect the paths t B(t) will possess. For this N purpose, set Mn = n δ − 12 + δ − 12 , and recall that we have seen already −n n that πMn =⇒γ0,I . Since a Poisson process associated with Mn has nothing but 1 jumps of size n− 2 , if one believes that the L´evy process for γ0,I should be, in some sense, the limit of such Poisson processes, then it is reasonable to guess that its paths will have jumps of size 0. That is, they will be continuous. Although the prediction that the paths of {B(t) : t ≥ 0} will be continuous is correct, it turns out that, because it is based on the Central Limit Theorem, the heuristic reasoning just given does not lead to the easiest construction. The problem is that The Central Limit Theorem gives convergence of distributions, not random variables, and therefore one should not expect the paths, as opposed to their distributions, of the approximating Poisson processes to converge. For this reason, it is easier to avoid The Central Limit Theorem and work with Gaussian random variables from the start, and that is what I will do here. The Central Limit approach is the content of § 9.3.
§ 4.3.1. Deconstructing Brownian Motion. My construction of Brownian motion is based on an idea of L´evy’s; and in order to explain L´evy’s idea, I will begin with the following line of reasoning. Assume that {B(t) : t ≥ 0} is a Brownian motion in RN . That is, {B(t) : t ≥ 0} starts at 0, has independent increments, any increment B(s + t) − B(s) has distribution γ0,tI , and the paths t B(t) are continuous. Next, given n ∈ N, let t Bn (t) be the polygonal path obtained from t B(t) by linear interpolation during each time interval [m2−n , (m + 1)2−n ]. Thus, Bn (t) = B(m2−n ) + 2n t − m2−n
B (m + 1)2−n − B(m2−n )
for m2−n ≤ t ≤ (m + 1)2−n . The distribution of {B0 (t) : t ≥ 0} is very easy to understand. Namely, if Xm,0 = B(m) − B(m − 1) for m ≥ 1, then the N X Pm,0 ’s are independent, standard normal R -valued random variables, B0 (m) = 1≤m≤n Xm,0 , and B0 (t) = (m − t)B0 (m − 1) + (t − m + 1)B0 (m) for m − 1 ≤ t ≤ m. To understand the relationship between successive Bn ’s, observe that Bn+1 (m2−n ) = Bn (m2−n ) for all m ∈ N and that
n Xm,n+1 ≡ 2 2 +1 Bn+1 (2m − 1)2−n−1 − Bn (2m − 1)2−n−1 ! B m2−n + B (m − 1)2−n n +1 −n−1 B (2m − 1)2 − = 22 2 h n = 2 2 B (2m − 1)2−n−1 − B (m − 1)2−n i − B m2−n − B (2m − 1)2−n−1 ,
§ 4.3 Brownian Motion, the Gaussian L´evy Process
179
and therefore {Xm,n+1 : m ≥ 1} is again a sequence of independent standard normal random variables. What is less obvious is that {Xm,n : (m, n) ∈ Z+ ×N} is also a family of independent random variables. In fact, checking this requires us to make essential use of the fact that we are dealing with Gaussian random variables. In preparation for proving the preceding independence assertion, say that G ⊆ L2 (P; R) is a Gaussian family if G is a linear subspace and each element of G is a centered (i.e., mean value 0), R-valued Gaussian random variable. My interest in Gaussian families at this point is that the linear span G(B) of ξ, B(t) RN : t ≥ 0 and ξ ∈ RN is one. To see this, simply note that, for any 0 = t0 < t1 < · · · tn and ξ1 , . . . , ξn ∈ RN , ! n n n X X X ξm , B(tm ) RN = ξm , B(t` ) − B(t`−1 ) RN , m=1
`=1
m=`
RN
which, as a linear combination of independent centered Gaussians, is itself a centered Gaussian. The crucial fact about Gaussian families is the content of the next lemma. Lemma 4.3.1. Suppose that G ⊆ L2 (P; R) is a Gaussian family. Then the closure of G in L2 (P; R) is again a Gaussian family. Moreover, for any S ⊆ G, S is independent of S ⊥ ∩ G, where S ⊥ is the orthogonal complement of S in L2 (P; R). Proof: The first assertion is easy since, as I noted in the introduction to Chapter 3, Gaussian random variables are closed under convergence in probability. Turning to the second part, what I must show is that if X1 , . . . , Xn ∈ S and X10 , . . . , Xn0 ∈ S ⊥ ∩ G, then (cf. part (ii) of Exercise 1.1.13) # # " n # " n " n n √ Y √ Y √ Y Y √ 0 0 0 0 −1 ξm Xm −1 ξm Xm P −1 ξm Xm P −1 ξm Xm P E e =E e e E e m=1
m=1
m=1
m=1
0 for any choice of {ξm : 1 ≤ m ≤ n} ∪ {ξm : 1 ≤ m ≤ n} ⊆ R. But the expectation value on the left is equal to !2 n X 1 0 0 ξm Xm + ξm Xm exp − EP 2 m=1 !2 !2 n n X X 1 1 0 0 ξm Xm ξm Xm − EP = exp − EP 2 2 m=1 m=1 # # " n " n Y √ Y √ 0 0 e −1 ξm Xm , = EP e −1 ξm Xm EP
m=1
m=1
180
4 L´evy Processes
0 0 since EP [Xm Xm 0 ] = 0 for all 1 ≤ m, m ≤ n. Armed with Lemma 4.3.1, we can now check that {Xm,n : (m, n) ∈ Z+ ×N} is independent. Indeed, since, for all (m, n) ∈ Z+ × N and ξ ∈ RN , ξ, Xm,n RN a member of the Gaussian family G(B), all that we have to do is check that, for each (m, n) ∈ Z+ × N, ` ∈ N, and (ξ, η) ∈ (RN )2 , EP ξ, Xm,n+1 RN η, B(`2−n ) RN = 0.
But, since, for s ≤ t, B(s) is independent of B(t) − B(s), EP ξ, B(s) RN η, B(t) RN = EP ξ, B(s) RN η, B(s) RN = s ξ, η RN and therefore n 2− 2 −1 EP ξ, Xm,n+1 RN η, B(`2−n ) RN h i = EP ξ, B (2m − 1)2−n−1 N η, B(`2−n ) N R R i 1 P h −n ξ, B m2 + B (m − 1)2−n N η, B(`2−n ) N − E 2 R R m ∧ ` + (m − 1) ∧ ` = 0. = 2−n ξ, η RN m − 12 ∧ ` − 2
§ 4.3.2. L´ evy’s Construction of Brownian Motion. L´evy’s idea was to invert the reasoning given in the preceding subsection. That is, start with a family {Xm,n : (m, n) ∈ Z+ × N} of independent N (0, I)-random variables. Next, define {Bn (t) : t ≥ 0} inductively Bn (t) is linear on each P so that t −n −n interval [(m − 1)2 , m2 ], B0 (m) = 1≤`≤m X`,0 , m ∈ N, Bn+1 (m2−n ) = Bn (m2−n ) for m ∈ N, and n Bn+1 (2m − 1)2−n = Bn (2m − 1)2−n−1 + 2− 2 −1 Xm,n+1 for m ∈ Z+ .
If Brownian motion exists, then the distribution of {Bn (t) : t ≥ 0} is the distribution of the process obtained by polygonalizing it on each of the intervals [(m − 1)2−n , m2−n ], and so the limit limn→∞ Bn (t) should exist uniformly on compacts and should be Brownian motion. To see that this procedure works, one must first verify that the preceding definition of {Bn (t) : t ≥ That 0} gives a process with the correct distribution. is, we need to show that Bn (m+1)2−n −Bn m2−n : m ∈ N is a sequence of independent N (0, 2−n I)-random variables. But, since this sequence is contained in the Gaussian family spanned by {Xm,n : (m, n) ∈ Z+ × N}, Lemma 4.3.1 says that we need only show that h EP ξ, Bn (m + 1)2−n − Bn m2−n N R i 0 0 −n × ξ , Bn (m + 1)2 − Bn m0 2−n = 2−n ξ, ξ 0 RN δm,m0 RN
§ 4.3 Brownian Motion, the Gaussian L´evy Process
181
for ξ, ξ 0 ∈ RN and m, m0 ∈ N. When n = 0, this is obvious. Now assume that it is true for n, and observe that Bn+1 (m2−n ) − Bn+1 (2m − 1)2−n−1 Bn (m2−n ) − Bn (m − 1)2−n n − 2− 2 −1 Xm,n+1 = 2
and Bn+1 (2m − 1)2−n−1 − Bn+1 (m − 1)2−n Bn (m2−n ) − Bn (m − 1)2−n n + 2− 2 −1 Xm,n+1 . = 2
Using these expressions and the induction hypothesis, it is easy to check the required equation. Second, and more challenging, we must show that, P-almost surely, these processes are converging uniformly on compact time intervals. For this purpose, consider the difference t Bn+1 (t) − Bn (t). Since this path is linear on each interval [m2−n−1 , (m + 1)2−n−1 ], Bn+1 (m2−n−1 ) − Bn (m2−n−1 ) max Bn+1 (t) − Bn (t) = max t∈[0,2L ]
1≤m≤2L+n+1
n
= 2− 2 −1
max
1≤m≤2L+n
n
L+n 2X
|Xm,n+1 | ≤ 2− 2 −1
14 |Xm,n+1 |4 .
m=1
Thus, by Jensen’s Inequality,
L+n 2X
n EP kBn+1 − Bn k[0,2L ] ≤ 2− 2 −1
14 n−L−4 EP |Xm,n+1 |4 = 2− 4 CN ,
m=1
1 where CN ≡ EP |X1,0 |4 4 < ∞. Starting from the preceding, it is an easy matter to show that there is a measurable B : [0, ∞) × Ω −→ RN such that B(0) = 0, B( · , ω) ∈ C [0, ∞); RN ) for each ω ∈ Ω, and kBn − Bk[0,t] −→ 0 both P-almost surely and in L1 (P; R) −n −n for every t ∈ [0, ∞). Furthermore, since ) P-almost surely B(m2 )−n= Bn (m2 −n 2 for all (m, n) ∈ N , it is clear that B (m + 1)2 − B(m2 ) : m ≥ 0 is a sequence of independent N (0, 2−n I)-random variables for all n ∈ N. Hence, by continuity, it follows that {B(t) : t ≥ 0} is a Brownian motion. We have now completed the task described in the introduction to this section. However, before moving on, it is only proper to recognize that, clever as his method is, L´evy was not the first to construct a Brownian motion. Instead, it
182
4 L´evy Processes
was N. Wiener who was the first. In fact, his famous2 1923 article “Differential Space” in J. Math. Phys. #2 contains three different approaches. § 4.3.3. L´ evy’s Construction in Context. There are elements of L´evy’s construction that admit interesting generalizations, perhaps the most important of which is Kolmogorov’s Continuity Criterion. Theorem 4.3.2. Suppose that {X(t) : t ∈ [0, T ]} is a family of random variables taking values in a Banach space B, and assume that, for some p ∈ [1, ∞), C < ∞, and r ∈ (0, 1], 1 1 EP kX(t) − X(s)kpB p ≤ C|t − s| p +r for all s, t ∈ [0, T ].
˜ Then, there exists a family {X(t) : t ∈ [0, T ]} of random variables such that ˜ ˜ ω) ∈ B is X(t) = X(t) P-almost surely for each t ∈ [0, T ] and t ∈ [0, T ] 7−→ X(t, continuous for all ω ∈ Ω. In fact, for each α ∈ (0, r), " P
E
sup 0≤s
˜ − X(s)k ˜ kX(t) B (t − s)α
!p # p1
1
5CT p +r−α . ≤ (1 − 2−r )(1 − 2α−r )
Proof: First note that, by rescaling time, it suffices to treat the case when T = 1.
Given n ≥ 0, set Mn = max1≤m≤2n X(m2−n ) − X (m − 1)2−n B , and observe that ! p1 2n X 1
p
X(m2−n ) − X (m − 1)2−n ≤ C2−rn . EP Mnp p ≤ EP B m=1
Next, let t Xn (t) be the polygonal path obtained by linearizing t each interval [(m − 1)2−n , m2−n ], and check that
X(t) on
max kXn+1 (t) − Xn (t)kB
−n −n X (m − 1)2 − X(m2 )
= max n X (2m − 1)2−n−1 −
≤ Mn+1 . 1≤m≤2
2
t∈[0,1]
B
h
i 12 p
≤ C2−rn , and so there exists a Hence, EP supt∈[0,1] kXn+1 (t) − Xn (t)kpB ˜ : [0, 1] × Ω −→ B such that t ˜ ω) is continuous for all measurable X X(t, ω ∈ Ω and " # p1 C2−rn ˜ − Xn (t)kp . ≤ EP sup kX(t) B 1 − 2−r t∈[0,1] 2
Wiener’s article is remarkable, but I must admit that I have never been convinced that it is complete. Undoubtedly, my doubts are more a consequence of my own ineptitude than of his.
§ 4.3 Brownian Motion, the Gaussian L´evy Process
183
Moreover, because, for each t ∈ [0, 1], kX(τ ) − X(t)kB −→ 0 in probability as ˜ τ → t, it is easy to check that, for each t ∈ [0, 1], X(t) = X(t) P-almost surely. −n−1 To prove the final estimate, note that for 2 ≤ t − s ≤ 2−n one has that ˜ − X(s)k ˜ ˜ ˜ kX(t) B ≤ kX(t) − Xn (t)kB + kXn (t) − Xn (s)kB + kXn (s) − X(s)kB ˜ ) − Xn (τ )kB + 2n (t − s)Mn , ≤ 2 sup kX(τ τ ∈[0,1]
and therefore that ˜ − X(s)k ˜ kX(t) B ˜ ) − Xn (τ )kB + 2n 2(α−1)n Mn . ≤ 22α(n+1) sup kX(τ (t − s)α τ ∈[0,1]
But, by the estimates proved above, this means that " P
E
sup 0≤s
˜ − X(s)k ˜ kX(t) B (t − s)α
!p # p1
≤C
≤
∞ X 2α(n+1) 2−rn αn −rn + 2 2 2 1 − 2−r n=0
5C . (1 − 2−r )(1 − 2α−r )
Corollary 4.3.3. If {B(t) : t ≥ 0} is an RN -valued Brownian motion, then, B(t) is P-almost surely H¨ older continuous of order α. for each α ∈ 0, 21 , t In fact, for each T ∈ (0, ∞),
P
E
sup 0≤s
|B(t) − B(s)| < ∞. (t − s)α
Proof: In view of Theorem 4.3.2, all that we have to do is note that, for each n ∈ Z+ , there is a Cn < ∞ such that EP |B(t) − B(s)|2n ≤ Cn |t − s|n . § 4.3.4. Brownian Paths Are Non-Differentiable. Having shown that Brownian paths are H¨ older continuous of every order strictly less than 12 , I will close this section by showing that they are nowhere H¨ older continuous of any order strictly greater than 12 . In particular, this will prove Wiener’s famous result that Brownian paths are nowhere differentiable. The proof that follows is due to A. Devoretzky.
Theorem 4.3.4. Let {B(t) : t ≥ 0} be an RN -valued Brownian motion. Then, for each α > 12 ,
|B(t) − B(s)| <∞ P ∃s ∈ [0, ∞) lim t&s (t − s)α
= 0.
184
4 L´evy Processes
Proof: Because {B(T + t) − B(T ) : t ≥ 0} is a Brownian motion for each T ∈ [0, ∞), it suffices for us to show that
|B(t) − B(s)| <∞ P ∃s ∈ [0, 1) lim t&s (t − s)α
= 0.
To this end, note that, for every L ∈ Z+ ,
|B(t) − B(s)| < ∞ t&s (t − s)α
∃s ∈ [0, 1) lim
∞ [ ∞ \ ∞ [ n L−1 [ \ B ⊆
m+`+1 n
−B
m+` n
≤
M nα
.
M =1 ν=1 n=ν m=0 `=0
Thus, it enough to show that there is a choice of L such that lim nP B
n→∞
`+1 n
−B
` n
≤
0 ≤ ` < L = 0.
M nα ,
But P B
`+1 n
−B
` n
≤
L M
= γ0, n1 I B 0, nα
M nα ,
=
0≤`
(2π)
Z 1 B(0,M n 2 −α )
e
|y|2 − 2
!L dy
1
≤ Cn( 2 −α)N L .
Hence, we need only take L so that (α − 12 )N L > 1.
In spite of their being non-differentiable, “differentials” of Brownian paths display remarkable regularity properties. To wit, I make the following simple observation. In its statement, k · kH.S. denotes the Hilbert–Schmidt norm on Hom(RN ; RN ). Theorem 4.3.5. If {B(t) : t ≥ 0} is an RN -valued Brownian motion, then, for each T ∈ (0, ∞)
[nt]
X
lim sup ∆ B ⊗ ∆ B − tI m,n m,n
n→∞ t∈[0,T ]
m=1
= 0 P-almost surely,
H.S.
m−1 . In particular, P-almost no Brownian path where ∆m,n B ≡ B m n n −B has locally bounded variation.
§ 4.3 Brownian Motion, the Gaussian L´evy Process
185
N Proof: Let (e1 , . . . , eN ) be an orthonormal basis for R , and set Xi (k, n) = ei , ∆k,n B RN . Then, what we have to show is that m X m (*) lim sup Xi (k, n)Xj (k, n) − δi,j = 0 P-almost surely. n→∞ 1≤m≤nT n k=1
To this end, note that, for each n ∈ Z+ and 1 ≤ i ≤ N , {Xi (k, n) : k ≥ 1} are mutually independent N (0, n−1 )-random variables. Hence, for each 1 ≤ i ≤ N , {Xi (k, n)2 − n−1 : k ≥ 1} are independent random variables with mean value 0 and variance 3n−2 , and therefore, by (1.4.22) and the second inequality in (1.3.2), m 4 X E max Xi (k, n)2 − n1 1≤m≤nT k=1 4 12M4 T 2 X 2 1 , ≤ 4E Xi (k, n) − n ≤ n2 1≤k≤nT
where M4 is the fourth moment of X1 (1, 1)2 − 1, and so the Borel–Cantelli Lemma can be used to check (*) when i = j. When i 6= j, the argument is essentially the same, only, because Xi (k, n)Xj (k, n) has mean value 0, there is no need to subtract off its mean. To prove the final assertion, note that if ψ ∈ C [0, T ]; R has bounded variation, then [nT ] X 2 m−1 = 0. − ψ lim ψ m n n n→∞
m=1
§ 4.3.5. General L´ evy Processes. Our original reason for constructing Brownian motion was to complete the program of constructing all the L´evy processes. In this subsection, I will do that. Throughout this subsection, µ ∈ I(RN ) has Fourier transform √ −1 ξ, m RN − 12 ξ, Cξ RN exp (4.3.6) Z h √ √ i −1(ξ,y)RN − 1 − −11[0,1] (|y|) ξ, y RN M (dy) , + e
where m ∈ RN , C ∈ Hom(RN ; RN ) is symmetric and non-negative definite, and M ∈ M2 (RN ). In addition, I will use µ0 to denote γm,C and µ1 to denote the element of I(RN ) whose Fourier transform is Z h √ √ i exp e −1(ξ,y)RN − 1 − −11[0,1] (|y|) ξ, y RN M (dy) .
Thus, µ = µ0 ? µ1 .
186
4 L´evy Processes
Theorem 4.3.7. There is a L´evy process {Z(t) : t ≥ 0} for each µ ∈ I(RN ). Furthermore, if µ0 and µ1 are as in the preceding discussion and if {Z(t) : t ≥ 0} is a L´evy process for µ = µ0 ? µ1 , then there exist independent L´evy processes {Z0 (t) : t ≥ 0} and {Z1 (t) : t ≥ 0} for µ0 and µ1 , respectively, such that Z(t) = Z0 (t) + Z1 (t), t ≥ 0, P-almost surely. In fact, if, for r ∈ (0, 1], Z Z Z(r) (t) = y j(t, dy, Z) − t y M (dy), |y|>r
r<|y|≤1
then, for each t ∈ (0, ∞), N 2t P kZ(r) − Z1 k[0,t] ≥ ≤ 2
Z
|y|2 M (dy). B(0,r)
Proof: Let {B(t) : t ≥ 0} be a Brownian motion and {Z1 (t) : t ≥ 0} an 1 independent L´evy process for µ1 , and define Z0 (t) = tm + C 2 B(t) and Z(t) = Z0 (t)+Z1 (t). As I pointed out in the introduction to this section, {Z0 (t) : t ≥ 0} is a L´evy process for µ0 and {Z(t) : t ≥ 0} is a L´evy process for µ. Furthermore, because t Z0 (t) is continuous, j(t, · , Z) = j(t, · , Z1 ). Hence, by the last part of Theorem 4.2.15, we know that the last part of the present theorem holds for this choice of {Z(t) : t ≥ 0}. Finally, since every L´evy process for µ will have the same distribution as this one, there is nothing more to do.
Corollary 4.3.8. Let {Z(t) : t ≥ 0} be a L´evy process for µ. Then t Z(t) is P-almost surely continuous if and only if M = 0 and is P-almost surely of locally bounded variation if and only if C = 0 and M ∈ M1 (RN ). Finally, t Z(t) is P-almost surely R an absolutely pure jump path if and only if C = 0, M ∈ M1 (RN ), and m = B(0,1) y M (dy).
Proof: Let Z(t) = Z0 (t) + Z1 (t) be the decomposition described in Theorem 4.3.7, and let {j(t, · ) : t ≥ 0} be the jump process for {Z(t) : t ≥ 0}. If M = 0, then Z1 (t) = 0, t ≥ 0, P-almost surely, and so t Z(t) = Z0 (t) is continuous P-almost surely. Conversely, if t Z(t) is continuous P-almost surely, then j(t, · ) = 0, t ≥ 0, P-almost surely. Hence, since {j(t, · ) : t ≥ 0} is a Poisson jump process associated with M , we see that M = 0. Next, suppose that C = 0. Then Z(t) = Z1 (t) + tm, t ≥ 0, P-almost surely and therefore, by Corollary 4.2.16, t Z(t) has locally bounded variation P-almost surely N if and only if M ∈ M1 (R ) and is P-almost Rsurely an absolutely pure jump path if and only if M ∈ M1 (RN ) and m = B(0,1) y M (dy). Thus, all that remains is to show that C = 0 if t Z(t) P-almost surely has locally bounded variation. But, Z(t) has locally bounded variation P-almost surely, then, R if t by (4.1.13), |y|j(t, dy) < ∞, t ≥ 0, P-almost surely and therefore, by Lemma 4.2.13, M ∈ M1 (RN ), which, by Corollary 4.2.16, implies that t Z1 (t) has locally bounded variation P-almost surely. Since this means that t Z0 (t) must
Exercises for § 4.3
187
also have locally bounded variation P-almost surely, and, since {Z0 (t) : t ≥ 0} 1 has the same distribution as {tm + C 2 B(t) : t ≥ 0}, Theorem 4.3.5 shows that this is possible only if C = 0.
Remark 4.3.9. Recall the linear functional Aµ introduced in (3.2.10). As I showed in Lemma 3.2.14, the action of Aµ on ϕ decomposes into a local part and a non-local part, which, with 20-20 hindsight, we can write as, respectively, m, ∇ϕ(0) RN + 12 Trace C∇2 ϕ(0) Z h i and ϕ(y) − ϕ(0) − 1[0,1] (|y|) y, ∇ϕ(0) RN M (dy).
In terms of this decomposition, Corollary 4.3.8 is saying that the local part of Aµ governs the continuous part of {Z(t) : t ≥ 0} and that the non-local part governs the discontinuous part. Exercises for § 4.3 Exercise 4.3.10. This exercise deals with a few elementary facts about Brownian motion. (i) Let {X(t) : t ≥ 0} be an RN -valued stochastic process satisfying X(0, ω) = 0 N and X( · , ω) ∈ C(RN ) for all ω ∈ Ω, and showthat {X(t) : t ≥ 0} is an R -valued Brownian motion if and only if the span of ξ, X(t) RN : t ≥ 0 & ξ ∈ RN } is a Gaussian family with the property that, for all t, t0 ∈ [0, ∞) and ξ, ξ 0 ∈ RN , h i EP ξ, X(t) RN ξ 0 , X(t0 ) RN = t ∧ t0 (ξ, ξ 0 )RN . (ii) Assuming that {B(t) : t ≥ 0} is an RN -valued Brownian motion, show that {OB(t) : t ≥ 0} is also an RN -valued Brownian motion for any orthogonal transformation O on RN . That is, the distribution of Brownian motion is invariant under rotation. (See Theorem 8.3.14 for a significant generalization.) (iii) Assuming that {B(t) : t ≥ 0} is an RN -valued Brownian motion, show that 1 {λ− 2 B(λt) : t ≥ 0} is also an RN -Brownian motion for each λ ∈ (0, ∞). This is called the Brownian scaling invariance property.
Exercise 4.3.11. This exercise introduces the time inversion invariance property of Brownian motion. (i) Suppose that {B(t) : t ≥ 0} is an RN -valued Brownian motion, and set 1 X(t) = tB t for t > 0. As an application of (i) in Exercise 4.3.10, show that {X(t) : t > 0} has the same distribution as {B(t) : t > 0}, and conclude from ˜ ω) = 0 and, for this that limt&0 X(t) = 0 P-almost surely. In particular, if B(0, t ∈ (0, ∞), tB 1t , ω when limτ →0 τ B τ1 , ω = 0 ˜ B(t, ω) = 0 otherwise, N ˜ show that {B(t) : t ≥ 0} is an R -valued Brownian motion.
188
4 L´evy Processes
(ii) As a consequence of part (i), prove the Brownian Strong Law of Large Numbers: limt→∞ t−1 B(t) = 0. Exercise 4.3.12. Let {B(t) : t ≥ 0} be an RN -valued Brownian motion. (i) As an application of Theorem 1.4.13, show that, for any e ∈ SN −1 and T ∈ (0, ∞), ! P
sup e, B(t) ≥ R t∈[0,T ]
R2 ≤ 2P e, B(T ) RN ≥ R ≤ 2e− 2T ,
and conclude that R2 P kBk[0,T ] ≥ R ≤ 2N e− 2N T .
(4.3.13)
(ii) Now assume that N = 1, and set B ∗ (t) = max as in part (i), τ ∈[0,t] B(τ ). Just use Theorem 1.4.13 to show that P B ∗ (1) ≥ a ≤ 2P B(1) ≥ a for all a > 0. By examining its proof, one sees that the inequality in Theorem 1.4.13 comes from not knowing how far over a the partial sums jump when they first exceed level a. Thus, because we are now dealing with “continuous partial sums,” one should suspect that the inequality can be made an equality. To verify this suspicion, let Γn () denote the set of ω such that |B(t, ω) − B(s, ω)| < for all 0 ≤ s < t ≤ 1 with t − s ≤ 2−n , and show that, for 0 < < a, {B(1) ≥ a} ∩ Γn () 2n −1 [ −n −n −n ⊆ max B(`2 ) < a − ≤ B(m2 ) & B(1) − B(m2 ) > 0 , m=1
0≤`<m
and conclude that P {B(1) ≥ a} ∩ Γn () ≤ 12 P B ∗ (1) ≥ a − for all n∈ N. Now let n → ∞ and then & 0 to arrive at P B ∗ (1) ≥ a ≥ 2P B(1) ≥ a . (iii) By combining the preceding with Brownian scaling invariance, arrive at r (4.3.14)
∗
P B (t) ≥ a = 2P B(t) ≥ a =
2 π
Z
∞
1 at− 2
e−
x2 2
dx.
This beautiful result, which is sometimes called the reflection principle for Brownian motion, seems to have appeared first in L. Bachelier’s now famous 1900 thesis, where he used what is now called “Brownian motion” to model price fluctuations on the Paris Bourse. More information about the reflection principle can be found in § 8.6.3.
Exercises for § 4.3
189
Exercise 4.3.15. Let {B(t) : t ≥ 0} be an R-valued Brownian motion. The goal of this exercise is to prove the Brownian Law of the Iterated Logarithm:
lim q
t→∞
B(t) = 1 = lim q t&0 2t log(2) t 2t log(2) t−1 B(t)
P-almost surely.
Begin by checking that the second equality follows from the first applied to the ˜ time inverted process {B(t) : t ≥ 0} described in (i) of Exercise 4.3.11. Next, observe that B(n) = 1 P-almost surely lim q n→∞ 2n log(2) n
is just the Law of the Iterated Logarithm for standard normal random variables. Thus, all that remains is to show that
B(n) B(t) = 0 P-almost surely, −q lim sup q n→∞ t∈[n,n+1] 2t log(2) t 2n log(2) n which can be checked by a combination of the Strong Law for Brownian motion, the estimate in (4.3.13), and the easy half of the Borel–Cantelli Lemma. Exercise 4.3.16. Given a stochastic process {X(t) : t ≥ 0}, the stochastic ˜ process {X(t) : t ≥ 0} is said to be a modification of {X(t) : t ≥ 0} if, for ˜ each t ∈ [0, ∞), X(t) = X(t) P-almost surely. Further, given a stochastic process {X(t) : t ≥ 0} with values in a metric space (E, ρ), one says that {X(t) : t ≥ 0} is stochastically continuous if, as t → s, X(t) −→ X(s) in probability for each s ∈ [0, ∞). (i) Show that the simple Poisson process {N (t) : t ≥ 0} is stochastically continuous. Thus, stochastic continuity does not imply path continuity. (ii) Let Q denote the set of rational real numbers. Show that an RN -valued, stochastically continuous stochastic process {X(t) : t ≥ 0} admits a continuous modification if and only if, for each T > 0, t ∈ [0, T ] ∩ Q 7−→ X(t) is uniformly continuous. Conclude that a stochastically continuous process {X(t) : t ≥ 0} admits a continuous modification if and only if there exists a µ ∈ M1 C(RN ) such that the distribution of {X(t) : t ≥ 0} under P is the same as the distribution of {ψ(t) : t ≥ 0} under µ. Equivalently, a stochastically continuous process {X(t) : t ≥ 0} admits a continuous modification if and only if there exists a continuous stochastic process {Y (t) : t ≥ 0}, not necessarily on the same probability space, with the same distribution as {X(t) : t ≥ 0}.
190
4 L´evy Processes
Exercise 4.3.17. It is important to realize that the insistence in Theorem 4.3.2 that the pth moment of |X(t) − X(s)| be dominated by |t − s| to a power strictly greater than p is essential. To see this, recall the simple Poisson process {N (t) : t ≥ 0} in § 5.2.1, and set X(t) = N (t) − t. The paths of this process are right-continuous but definitely not continuous. On the other hand, show that 2 EP N (t) − N (s) − (t − s) ≤ t − s for 0 ≤ s < t. More generally, knowing 2 that E |X(t) − X(s)| is dominated by |t − s| is not enough to conclude that there is a continuous modification of t X(t). Exercise 4.3.18. There is an important extension of Theorem 4.3.2 to processes that have a multidimensional parametrization. Let B be a Banach space and {X(x) : x ∈ [0, T ]ν } a family of B-valued random variables with the property that 1 ν EP kX(y) − X(x)kpB p ≤ C|y − x| p +r
for some p ∈ [1, ∞), r > 0, and C < ∞. Show that there exists a family ˜ ˜ {X(x) : x ∈ [0, T ]ν } with the properties that x ∈ [0, T ]ν 7−→ X(x, ω) ∈ B ν ˜ is continuous for all ω, and, for each x ∈ [0, T ] , X(x, ω) = X(x, ω) P-almost surely. Further, show that, for each α ∈ (0, r), there is a universal K(ν, r, α) < ∞ such that ˜ ˜ ν kX(y) − X(x)k B ≤ K(ν, r, α)CT p +r−α . EP sup α |y − x| x,y∈[0,T ]ν y6=x
Hint: First rescale time to reduce to the case when T = 1. Now assume that 2 T = 1. Given n ∈ N, take Sn to be the set of pairs (m, m0 ) ∈ {0, . . . , 2n }N P ν such that m0i ≥ mi for all 1 ≤ i ≤ ν and i=1 (m0i − mi ) = 1, note that Sn has no more than ν2(n+1)ν elements, set Mn = max kX(m0 2−n ) − X(m2−n )kB : (m, m0 ) ∈ Sn , 1
Xn (x) denote the nth and show that EP [Mn ] ≤ C2ν ν p 2−rn . Next, let x dyadic multiliniarization of x X(x), the one that is multilinear on each dyadic QN cube i=1 [(mi − 1)2−n , mi 2−n ] for (m1 , . . . , mN ) ∈ {1, . . . , 2n }N . As in the proof of Theorem 4.3.2, argue that kXn+1 − Xn ku,B ≤ Mn+1 , and conclude ˜ that there exists an (x, ω) X(x, ω) that is continuous in x for each ω and is P-almost surely equal to X(x, · ) for each x. Finally, to derive the H¨older 1 continuity estimate, observe that kXn (y) − Xn (x)kB ≤ 2n ν 2 |y − x|Mn , and proceed as in the proof of the corresponding part of Theorem 4.3.2.
Exercise 4.3.19. In this exercise we will examine a couple of the implications that Theorem 4.3.5 has about any Riemann–Stieltjes type integration theory
Exercises for § 4.3
191
involving Brownian paths. For simplicity, I will restrict my attention to the onedimensional case. Thus, let {B(t) : t ≥ 0} be an R-valued Brownian motion. Because t B(t) is continuous, one knows that any function ψ : [0, 1] −→ R of bounded variation is Riemann–Stieltjes integrable on [0, 1] with respect to B [0, 1]. However, as the following shows, almost no Brownian path is Riemann– Stieltjes with respect to itself. Namely, using Theorem 4.3.5, show that P-almost surely, lim
n→∞
n X
B
m=1 n X
lim
n→∞
whereas lim
n→∞
m−1 n
m n
B
B
B
m n
−B
m−1 n
=
B(1)2 − 1 , 2
m n
−B
m−1 n
=
B(1)2 + 1 , 2
= B(1)2 .
m=1
n X
B
2m−1 2n
B
m n
−B
m−1 n
m=1
Exercise 4.3.20. Say that a D(RN )-valued process {Z(t) : t ≥ 0} is a L´evy process if Z(0) = 0 and it has independent, homogeneous increments. Show that every L´evy process is a L´evy process for some µ ∈ I(RN ). Exercise 4.3.21. Let {j(t, · ) : t ≥ 0} be a Poisson jump process associated with some M ∈ M∞ (RN ). In Lemma 4.2.13, we showed that when M ∈ M2 (RN ), R then |y| j(t, dy) < ∞, t ≥ 0, with positive probability only if M ∈ M1 (RN ). In this exercise, weR will show that the same is true for any M ∈ M∞ (RN ). That is, assuming that |y| j(t, dy) < ∞, t ≥ 0, with positive probability, it is to be shown that M ∈ M1 (RN ). Here are some steps that you might want to follow. R (i) As an application of Kolmogorov’s 0–1 Law, show that |y| j(t, dy) < ∞ with positive probability implies it is finite with probability 1. R (ii) Let N be the set of ω ∈ Ω for which there is aRt > 0 such that |y| j(t, dy, ω) = ∞. By (i), P(N ) = 0. Define Z(t, ω) = y j(t, dy, ω) for ω ∈ / N and Z(t, ω) = 0 for ω ∈ N , and show that {Z(t) : t ≥ 0} is a L´evy process with absolutely pure jump paths. (iii) Applying Theorem 4.1.8, first show that {Z(t) : t ≥ 0} is a L´evy process for a µ with L´evy measure M , and then apply Corollary 4.3.8 to conclude that M ∈ M1 (RN ). Exercise 4.3.22. Corollary 4.3.3 can be sharpened. In fact, L´evy showed that if {B(t) : t ≥ 0} is an R-valued Brownian motion, then P
lim
sup
δ&0 0
|B(t) − B(s)| √ = 2 L(δ)
= 1,
192
4 L´evy Processes
p where L(δ) ≡ δ log δ −1 . Notice that, on the one hand, this result is in the direction that one should expect: we know (cf. Theorem 4.3.4) that Brownian paths are almost never H¨ older continuous of any order greater than 12 . On the other hand, the Brownian Law of the Iterated Logarithm (cf. Exercise q 4.3.15) might make one guess that their true modulus of continuity ought to be δ log(2) δ −1 ,
not L(δ). However, that guess is wrong because it fails to take into account the difference between a question about what is true at a single time as opposed to what is true simultaneously for all times. The purpose of this exercise is to show how the considerations in § 4.3.3 can be used to get a statement that is related to but far less refined than L´evy’s. The result to be proved here says only that |B(t) − B(s)| ≤K =1 (4.3.23) P lim sup δ&0 0
for some K < ∞. (i) First show that it suffices to prove that there exists a K < ∞ such that P lim
sup
δ&0 0
|B(t) − B(s)| ≤ K = 1 L(δ)
and that this will follow from (*)
∞ X n=0
P
sup 2−n−1 ≤t−s≤2−n
|B(t) − B(s)| >K L(2−n−1 )
! < ∞.
(ii) Define the polygonal approximation {Bn (t) : t ≥ 0} as in § 4.3.1, set Mn = max1≤m≤2n |B(m2−n ) − B((m − 1)2−n )|, and show that
2kB − Bn k[0,1] Mn |B(t) − B(s)| for 2−n−1 ≤ t − s ≤ 2−n . + ≤ −n−1 L(2−n ) L(2−n−1 ) L(2 ) P∞ P∞ p (iii) Set C = n=0 (n + 1)2−n , show that n=m L(2−n )−1 ≤ CL(2−m )−1 for all m ≥ 0, and, arguing as in the proof of Theorem 4.3.2, conclude that, for any R > 0, ∞ X P kB − Bn k[0,1] ≥ R ≤ P Mm+1 ≥ C −1 RL(2−m−1 )−1 . m=n
(iv) Show that, for all R > 0 and n ∈ N, −1 2 P Mn ≥ RL(2−n )−1 ≤ 2n(1−2 R ) , and combine this with (ii) and (iii) to prove that (*) holds for some K < ∞.
Chapter 5 Conditioning and Martingales
Up to this point I have been dealing with random variables that are either themselves mutually independent or are built out of other random variables that are. For this reason, it has not been necessary for me to make explicit use of the concept of conditioning, although, as we will see shortly, this concept has been lurking silently in the background. In this chapter I will first give the modern formulation of conditional expectations and then provide an example of the way in which conditional expectations can be used. Let (Ω, F, P) be a probability space, and suppose that A ∈ F is a set having positive P-measure. For reasons that are most easily understood when Ω is finite and P is uniform, the ratio P(B|A) ≡
P(A ∩ B) , P(A)
B ∈ F,
is called the conditional probability of B given A. As one learns in an elementary course, the introduction of conditional probabilities makes many calculations much simpler; in particular, conditional probabilities help to clarify dependence relations between the events represented by A and B. For example, B is independent of A precisely when P(B|A) = P(B) or, in words, when the condition that A occurs does not change the probability that B occurs. Thus, it is unfortunate that the na¨ıve definition of conditioning as described above does not cover many important situations. For example, suppose that X and Y are random variables and that one wants to talk about the conditional probability that Y ≤ b given that X = a. Unless one is very lucky and P(X = a) > 0, dividing by P(X = a) is not going to do the job. As this example illustrates, it is of great importance to generalize the concept of conditional probability to include situations when the event on which one is conditioning has P-measure 0, and the next section is devoted to Kolmogorov’s elegant solution to the problem of doing so. § 5.1 Conditioning In order to appreciate the idea behind Kolmogorov’s solution, imagine someone told you the conditional probability that the event B occurs given that the event A occurs. Obviously, since you have no way of saying anything about the 193
194
5 Conditioning and Martingales
probability of B when A does not occur, she has provided you with incomplete information about B. Thus, before you are satisfied, you should demand to know also what is the conditional probability of B given that A does not occur. Of course, this second piece of information is relevant only if A is not certain, in which case P(A) < 1 and therefore P B A{ is well defined. More generally, suppose that P = {A1 , . . . , AN } (N here may be either finite or countably infinite) is a partition of Ω into elements of F having positive P-measure. Then, in order to have complete information about the probability of B ∈ F relative to P, one has to know the entire list of the numbers P B An , 1 ≤ n ≤ N . Next, suppose that one attempts to describe this list in a way that does not depend explicitly on the positivity of the numbers P(An ). For this purpose, consider the function N X ω ∈ Ω 7−→ f (ω) ≡ P B An 1An (ω). n=1
Clearly, f is not only F-measurable, it is measurable with respect to the σalgebra σ(P) over Ω generated by P. In particular (because the only σ(P)measurable set of P-measure 0 is empty), f is uniquely determined by its Pintegrals EP [f, A] over sets A ∈ σ(P). Moreover, because, for each B ∈ σ(P) and n, either An ⊆ B or B ∩ An = ∅, we have that N X EP f, A = P B ∩ An = n=1
X
P An ∩ B = P A ∩ B .
{n:An ⊆B}
Hence, the function f is uniquely determined by the properties that it is σ(P)measurable and that EP f, A = P A ∩ B for every A ∈ σ(P). The beauty of this description is that it makes perfectly good sense even if some of the An ’s have P-measure 0, except in that case the description does not determine f pointwise but merely up to a σ(P)-measurable P-null set (i.e., a set of P-measure 0), which is the very least one should expect to pay for dividing by 0. § 5.1.1. Kolmogorov’s Definition. With the preceding discussion in mind, one ought to find the following formulation reasonable. Namely, given a subσ-algebra Σ ⊆ F and a (−∞, ∞]-valued random variable X whose negative part X − ≡ −(X ∧ 0) is P-integrable, I will say that the random variable XΣ is a conditional expectation of X given Σ if XΣ is (−∞, ∞]-valued and − is P-integrable, and Σ-measurable, XΣ (5.1.1) EP XΣ , A = EP X, A for every A ∈ Σ. Obviously, having made this definition, my first duty is to show that such an XΣ always exists and to discover in what sense it is uniquely determined. The latter problem is dealt with in the following lemma.
§ 5.1 Conditioning
195
Lemma 5.1.2. Let Σ be a sub-σ-algebra of F, and suppose that XΣ and YΣ are a pair of (−∞, ∞]-valued Σ-measurable random variables for which XΣ− and YΣ− are both P-integrable. Then EP XΣ , A ≤ EP YΣ , A
for every A ∈ Σ,
if and only if XΣ ≤ YΣ (a.s., P). Proof: Without loss in generality, I may and will assume that Σ = F and will therefore drop the subscript Σ; and, since the “if” implication is completely trivial, I will discuss only the minimally less trivial “only if” assertion. Thus, suppose that P-integrals of Y dominate those of X and yet that X > Y on a set of positive P-measure. We could then choose an M ∈ [1, ∞) so that P(A) ∨ P (B) > 0, where 1 and B ≡ X = ∞ and Y ≤ M }. A ≡ X ≤ M and Y ≤ X − M
But if P(A) > 0, then EP X, A ≤ EP Y, A ≤ EP X, A −
1 M P (A),
which, because EP X, A is a finite number, is impossible. At the same time, if P(B) > 0, then ∞ = EP X, B ≤ EP Y, B ≤ M < ∞, which is also impossible.
Theorem 5.1.3. Let Σ be a sub-σ-algebra of F and X a (−∞, ∞]-valued random variable for which X − is P-integrable. Then there exists a conditional expectation value XΣ of X. Moreover, if Y is a second (−∞, ∞]-valued random variable and Y ≥ X (a.s., P), then Y − is P-integrable and YΣ ≥ XΣ (a.s., P) for any YΣ that is a conditional expectation value of Y given Σ. In particular, if X = Y (a.s., P), then {YΣ 6= XΣ } is a Σ-measurable, P-null set.1 Proof: In view of Lemma 5.1.2, it suffices for me to handle the initial existence statement. To this end, let G denote the class of X satisfying EP [X − ] < ∞ for which an XΣ exists, and let G + denote the non-negative elements of G. If {Xn : n ≥ 1} ⊆ G + is non-decreasing and, for each n ∈ Z+ , Xn Σ denotes a conditional expectation of Xn given Σ, then 0 ≤ Xn Σ ≤ Xn+1 Σ (a.s., P), and therefore I can arrange that 0 ≤ Xn Σ ≤ Xn+1 Σ everywhere. In par ticular, if X and XΣ are the pointwise limits of the Xn ’s and Xn Σ ’s, respectively, then the Monotone Convergence Theorem guarantees that XΣ is a 1 Kolmogorov himself, and most authors ever since, have obtained the existence of conditional expectation values as a consequence of the Radon–Nikodym Theorem. Because I find projections more intuitively appealing, I prefer the approach given here.
196
5 Conditioning and Martingales
conditional expectation of X given Σ. Hence, we now know that G + is closed under non-decreasing, pointwise limits, and therefore we will know that G + contains all non-negative random variables X as soon as we show that G contains all bounded X’s. But if X is bounded (and is therefore an element of L2 (P; R)) and LΣ = L2 (Ω, Σ, P; R) is the subspace of L2 (P; R) consisting of its Σ-measurable elements, then the orthogonal projection XΣ of X onto LΣ is a Σ-measurable random variable that is P-square integrable and satisfies (5.1.1). So far I have proved that G + contains all non-negative, F-measurable X’s. Furthermore, if X is non-negative, then (by Lemma 5.1.2) XΣ ≥ 0 (a.s., P) and so XΣ is P-integrable precisely when X itself is. In particular, I can arrange and P-integrable. to make XΣ take its values in [0, ∞) when X is non-negative Finally, to see that X ∈ G for every X with EP X − < ∞, simply consider X + and X − separately, apply the preceding to show that X ± Σ ≥ 0 (a.s., P) and that X − Σ is P-integrable, and check that the random variable X + Σ − X − Σ when X ± Σ ≥ 0 and X − Σ < ∞ XΣ ≡ 0 otherwise is a conditional expectation of X given Σ. Convention. Because it is determined only up to a Σ-measurable P-null set, one cannot, in general, talk about the conditional expectation of X as a function. Instead, the best that one can do is say that the conditional expectation of X is the equivalence class of Σ-measurable XΣ ’s that satisfy (5.1.1), and I will adopt the notation EP [X|Σ] to denote this equivalence class. On the other hand, because one is usually interested only in P-integrals of conditional expectations, it has become common practice to ignore, for the most part, the distinction between the equivalence class EP [X|Σ] and the members of that equivalence class. Thus (just as one would when dealing with the Lebesgue spaces) I will abuse notation by using EP [X|Σ] to denote a generic element of the equivalence class EP [X|Σ] and will be more precise only when EP [X|Σ] contains some particularly distinguished member. For example, recall the random variables Tn entering the definition of the simple Poisson process {N (t) : t ∈ (0, ∞)} in § 4.2.1. It is then clear (cf. part (i) in Exercise 1.1.9) that we can take h i EP 1{n} N (t) σ T1 , . . . , Tn = 1[0,t] Tn e−(t−Tn ) , and one would be foolish to take any other representative. More generally, I will always take non-negative representatives of EP [X|Σ] when X itself is nonnegative and R-valued representatives when X is P-integrable. Finally, for historical reasons, it is usual to distinguish the case when X is the indicator function 1B of a set B ∈ F and to call EP [1B |Σ] the conditional probability of B given Σ and to write P(B|Σ) instead of EP [1B |Σ]. Of course, representatives of P(B|Σ) will always be assumed to take their values in [0, 1].
§ 5.1 Conditioning
197
Once one has established the existence and uniqueness of conditional expectations, there is a long list of more or less obvious properties that one can easily verify. The following theorem contains some of the more important items that ought to appear on such a list. Theorem 5.1.4. Let Σ be a sub-σ-algebra of F. If X is a P-integrable random variable and C ⊆ Σ is a π-system (cf. Exercise 1.1.12) that generates Σ, then Y = EP X Σ (a.s., P) ⇐⇒ Y ∈ L1 (Ω, Σ, P; R) and EP Y, A = EP X, A for A ∈ C ∪ {Ω}. Moreover, if X is any (−∞, ∞]-valued random variable that satisfies EP [X − ] < ∞, then each of the following relations holds P-almost surely: (5.1.5)
P E X Σ ≤ EP |X| Σ ;
(5.1.6)
h i EP X T = EP EP X Σ T
when T is a sub-σ-algebra of Σ; and, when X is R-valued and P-integrable, EP −X Σ = −EP X Σ . Next, let Y be a second (−∞, ∞]-valued random variable with EP [Y − ] < ∞. Then, P-almost surely, EP αX + βY Σ = αEP X Σ + βEP Y Σ
for each α, β ∈ [0, ∞),
and EP Y X Σ = Y EP X Σ
(5.1.7)
if Y is Σ-measurable and (XY )− is P-integrable. Finally, suppose that {Xn : n ≥ 1} is a sequence of (−∞, ∞]-valued random variables. Then, P-almost surely, EP Xn Σ % EP X Σ
(5.1.8)
if EP [X1− ] < ∞ and Xn % X (a.s., P);
and, more generally, P
(5.1.9) E
lim Xn Σ ≤ lim EP Xn Σ
n→∞
n→∞
if Xn ≥ 0 (a.s., P) for each n ∈ Z+ .
198
5 Conditioning and Martingales
Proof: To prove the first assertion, note that the set of A ∈ Σ for which EP [X, A] = EP [Y, A] is (cf. Exercise 1.1.12) a λ-system that contains C and therefore Σ. Next, clearly (5.1.5) is just an application of Lemma 5.1.2, while (5.1.6) and the two equations that follow it are all expressions of uniqueness. As for the next equation, one can first reduce to the case when X and Y are both non-negative. Then one can use uniqueness to check it when Y is the indicator function of an element of Σ, use linearity to extend it to simple Σ-measurable functions, and complete the job by taking monotone limits. Finally, (5.1.8) is an immediate application of the Monotone Convergence Theorem, whereas (5.1.9) comes from the conjunction of inf Xn Σ ≤ inf EP Xn Σ n≥m n≥m
P
E with (5.1.8).
(a.s., P),
m ∈ Z+ ,
It probably will have occurred to most readers that the properties discussed in Theorem 5.1.4 give strong evidence that, for fixed ω ∈ Ω, X 7−→ EP [X|Σ](ω) behaves like an integral (in the sense of Daniell) and therefore ought to be expressible in terms of integration with respect to a probability measure Pω . Indeed, if one could actually talk about X 7−→ EP [X|Σ](ω) for a fixed (as opposed to P-almost every) ω ∈ Ω, then there is no doubt that such a Pω would have to exist. Thus, it is reasonable to ask whether there are circumstances in which one can gain sufficient control over all the P-null sets involved to really make sense out of X 7−→ EP [X|Σ](ω) for fixed ω ∈ Ω. Of course, when Σ is generated by a countable partition P, we already know what to do. Namely, when ω ∈ A ∈ P, we can take ( 0 if P(A) = 0 P E [X|Σ](ω) = EP [X, A] if P(A) > 0. P(A)
Even when Σ does not arise in this way, one can often find a satisfactory representation of conditional expectations as expectations. A quite general statement of this sort is the content of Theorem 9.2.1 in Chapter 9. § 5.1.2. Some Extensions. For various applications it is convenient to have two extensions of the basic theory developed in § 5.1.1. Specifically, as I will now show, the theory is not restricted to probability (or even finite) measures and can be applied to random variables that take their values in a separable Banach space. Thus, from now on, µ will be an arbitrary (non-negative) measure on (Ω, F) and E, k·kE will be a separable Banach space; and I begin by reviewing a few elementary facts about µ-integration for E-valued random variables.2 2 The integration that I outline below is what functional analysts call the Bochner integral for Banach space–valued functions. There is a more subtle and intricate theory due to Pettis, but Bochner’s theory seems adequate for most probabilistic considerations.
§ 5.1 Conditioning
199
A function X : Ω −→ E is said to be µ-simple if X is F-measurable, X takes only finitely many values, and µ X 6= 0 < ∞, in which case its integral with respect to µ is the element of E given by Z X µ E [X] = X(ω) µ(dω) ≡ x µ(X = x). Ω
x∈E\{0}
Notice that another description of Eµ [X] is as the unique element of E with the property that
µ E [X], x∗ = Eµ hX, x∗ i for all x∗ ∈ E ∗ (I use E ∗ to denote the dual of E and hx, x∗ i to denote the action of x∗ ∈ E ∗ on x ∈ E), and therefore that the mapping taking µ-simple X to Eµ [X] is linear. Next, observe that ω ∈ Ω 7−→ kX(ω)kE ∈ R is F-measurable if X : Ω −→ E is F-measurable. In particular, for F-measurable X : Ω −→ E, I will set ( 1 if p ∈ [1, ∞) Eµ kXkpE p kXkLp (µ;E) = if p = ∞ inf M : µ kXkE > M = 0
and will write X ∈ Lp (µ; E) when kXkLp (µ;E) < ∞. Also, I will say the X : Ω −→ E is µ-integrable if X ∈ L1 (µ; E); and I will say that X is locally µ-integrable if 1A X is µ-integrable for every A ∈ F with µ(A) < ∞. The definition of µ-integration for an E-valued X is completed in the following lemma. Lemma 5.1.10. For each µ-integrable X : Ω −→ E there is a unique element Eµ [X] ∈ E satisfying EP [X], x∗ = EP [hX, x∗ ] for all x∗ ∈ E ∗ . In particular, the mapping X ∈ L1 (µ; E) 7−→ Eµ [X] ∈ E is linear and satisfies
µ
E [X] ≤ Eµ kXkE . (5.1.11) E Finally, if X ∈ Lp (µ; E), where p ∈ [1, ∞), then there is a sequence {Xn : n ≥ 1} of E-valued, µ-simple functions with the property that kXn − XkLp (µ;E) −→ 0. Proof: Clearly uniqueness, linearity, and (5.1.11) all follow immediately from the given characterization of Eµ [X]. Thus, all that remains is to prove existence and the final approximation assertion. In fact, once the approximation assertion is proved, then existence will follow immediately from the observation that, by (5.1.11), Eµ [X] can be taken equal to limn→∞ Eµ [Xn ] if kX − Xn kL1 (µ;E) −→ 0. To prove the approximation assertion, I begin with the case when µ is finite and M = supω∈Ω kX(ω)kE < ∞. Next, choose a dense sequence {x` : ` ≥ 1} in E, set A0,n = ∅, and let o n for (`, n) ∈ Z+ × Z+ . A`,n = ω : kX(ω) − x` kE < n1
200
5 Conditioning and Martingales
Then, for each n ∈ Z+ there exists an Ln ∈ Z+ with the property that ! Ln [ 1 µ Ω\ A`,n < p . n `=1
Hence, if Xn : Ω −→ E is defined so that when 1 ≤ ` ≤ Ln and ω ∈ A`,n \
Xn (ω) = x`
`−1 [
Ak,n
k=0
and Xn (ω) = 0 when ω ∈ /
SLn 1
A`,n , then Xn is µ-simple and
M + µ(E) . n In order to handle the general case, let X ∈ Lp (µ; E) and n ∈ Z+ be given. We can then find an rn ∈ (0, 1] with the property that Z 1 , kX(ω)kpE µ(dω) ≤ (2n)p kX − Xn kLp (µ;E) ≤
Ω(rn ){
where
o n for r ∈ (0, 1]. Ω(r) ≡ ω : r ≤ kX(ω)kE ≤ 1r Since, for any r ∈ (0, 1], rp µ Ω(r) ≤ kXkpLp (µ;E) , we can apply the preceding to the restrictions of µ and X to Ω(rn ) and thereby find a µ-simple Xn : Ω(rn ) −→ E with the property ! p1 Z 1 p . ≤ kX(ω) − Xn (ω)kE µ(dω) 2n Ω(rn )
Hence, after extending Xn to Ω by taking it to µ-simple Xn for which kX − Xn kLp (µ;E) ≤ n1 . Given an F-measurable X : Ω −→ E and a B I will use, depending on the context, either Z Eµ X, B or X dµ or B
be 0 off of Ω(rn ), we arrive at a ∈ F for which 1B X ∈ L1 (µ; E),
Z X(ω) µ(dω) B
to denote the quantity Eµ [1B X]. Also, when discussing the spaces Lp (µ; E), I will adopt the usual convention of blurring the distinction between a particular F-measurable X : Ω −→ E belonging to Lp (µ; E) and the equivalence class of those F-measurable Y ’s that differ from X on a µ-null set. Thus, with this convention, k · kLp (µ;E) becomes a bona fide norm (not just a seminorm) on Lp (µ; E) with respect to which Lp (µ; E) becomes a normed vector space. Finally, by the same procedure with which one proves the Lp (µ; R) spaces are complete, one can prove that the spaces Lp (µ; E) are complete for any separable Banach space E.
§ 5.1 Conditioning
201
Theorem 5.1.12. Let (Ω, F, µ) be a σ-finite measure space and X : Ω −→ E a locally µ-integrable function. Then µ X 6= 0 = 0 ⇐⇒ Eµ X, A = 0 for A ∈ F with µ(A) < ∞. Next, assume that Σ is a sub-σ-algebra for which µ Σ is σ-finite. Then, for each locally µ-integrable X : Ω −→ E, there is a µ-almost everywhere unique locally µ-integrable, Σ-measurable XΣ : Ω −→ E such that (5.1.13)
Eµ XΣ , A = Eµ X, A
for every A ∈ Σ with µ(A) < ∞.
In particular, if Y : Ω −→ E is a second locally µ-integrable function, then, for all α, β ∈ R, αX + βY Σ = αXΣ + βYΣ (a.e., µ). Finally,
XΣ ≤ kXkE E Σ
(5.1.14)
(a.e., µ).
Hence, not only does (5.1.13) continue to hold for any A ∈ Σ with 1A X ∈ L1 (µ; E), but also, for each p ∈ [1, ∞], the mapping X ∈ Lp (µ; E) 7−→ XΣ ∈ Lp (µ; E) is a linear contraction. Proof: Clearly, it is only necessary to prove the “⇐=” part of the first assertion. Thus, suppose that µ(X 6= 0) > 0. Then, because E is separable and therefore (cf. Exercise 5.1.19) E ∗ with the weak* topology is also separable, there exists an > 0 and a x∗ ∈ E ∗ with the property that µ X, x∗ ≥ > 0, from which it follows (by σ-finiteness) that there is an A ∈ F for which µ(A) < ∞ and D
h
E i Eµ X, A , x∗ = Eµ X, x∗ , A 6= 0.
I turn next to the uniqueness and other properties of XΣ . But it is obvious that uniqueness is an immediate consequence of the first assertion and that linearity follows from uniqueness. As for (5.1.14), notice that if x∗ ∈ E ∗ and kx∗ kE ∗ ≤ 1, then
Eµ XΣ , x∗ , A = Eµ X, x∗ , A ≤ Eµ kXkE , A = Eµ kXkE Σ , A for every A ∈ Σ with µ(A) < ∞. Hence, at least when µ is a probability
measure, Theorem 5.1.3 implies that XΣ , x∗ ≤ kXkE Σ (a.e., µ) for each element x∗ from the unit ball in E ∗ ; and so, because E ∗ with the weak* topology is separable, (5.1.14) follows in this case. To handle µ’s that are not probability measures, note that either µ(Ω) = 0, in which case everything is trivial, or µ(Ω) ∈ (0, ∞), in which case we can renormalize µ to make it a probability
202
5 Conditioning and Martingales
measure, or µ(Ω) = ∞, in which case we can use the σ-finiteness of µ Σ to reduce ourselves to the countable, disjoint union of the preceding cases. Finally, to prove the existence of XΣ , I proceed as in the last part of the preceding paragraph to reduce myself to the case when µ is a probability measure P. Next, suppose that X is simple, let R denote its range, and note that X XΣ ≡ xP X = x Σ x∈R
has the required properties. In order to handle general X ∈ L1 (P; E), I use the approximation result in Lemma 5.1.10 to find a sequence {Xn : n ≥ 1} of simple functions that tend to X in L1 (P; E). Then, since (Xn )Σ − (Xm )Σ = Xn − Xm Σ (a.s., P) and therefore, by (5.1.14),
(Xn )Σ − (Xm )Σ 1 ≤ kXn − Xm L1 (P;E) , L (P;E) 1 we exists a Σ-measurable XΣ ∈ L (P; E) to which the sequence know that there (Xn )Σ : n ≥ 1 converges; and clearly XΣ has the required properties. Referring to the setting in the second part of Theorem 5.1.12, I will extend the convention introduced following Theorem 5.1.3 and call the µ-equivalence class of XΣ ’s satisfying (5.1.13) the µ-conditional expectation of X given Σ, will use Eµ [X|Σ] to denote this µ-equivalence class, and will, in general, ignore the distinction between the equivalence class and a generic representative of that class. In addition, if X : Ω −→ E is locally µ-integrable, then, just as in Theorem 5.1.4, the following are essentially immediate consequences of uniqueness: Eµ Y X Σ = Y Eµ X Σ (a.e., µ) for Y ∈ L∞ (Ω, Σ, µ; R),
and
h i Eµ X T = Eµ Eµ X Σ T
(a.e., µ)
whenever T is a sub-σ-algebra of Σ for which µ T is σ-finite. Exercises for § 5.1 Exercise 5.1.15. As the proof of existence in Theorem 5.1.4 makes clear, the operation X ∈ L2 (P; R) 7−→ EP [X|Σ] is just the operation of orthogonal projection from L2 (P; R) onto the space L2 (Ω, Σ, P; R) of Σ-measurable elements of L2 (P; R). For this reason, one might be inclined to think that the concept of conditional expectation is basically a Hilbert space notion. However, as this exercise shows, that inclination should be resisted. The point is that, although conditional expectation is definitely an orthogonal projection, not every orthogonal projection is a conditional expectation!
Exercises for § 5.1
203
(i) Let L be a closed linear subspace of L2 (P; R), and let ΣL = σ {X : X ∈ L} be the σ-algebra over Ω generated by X ∈ L. Show that L = L2 Ω, ΣL , P; R if and only if 1 ∈ L and X + ∈ L whenever X ∈ L. Hint: To prove the “if” assertion, let X ∈ L be given, and show that h i + Xn ≡ n X − α1 ∧ 1 ∈ L for every α ∈ R and n ∈ Z+ . Conclude that Xn % 1(α,∞) ◦ X must be an element of L. (ii) Let Π be an orthogonal projection operator on L2 (P; R), set L = Range(Π), and let Σ = ΣL , where ΣL is defined as in part (i). Show that ΠX = EP [X|Σ] (a.s., P) for all X ∈ L2 (P; R) if and only if Π1 = 1 and (*)
Π X ΠY = (ΠX)(ΠY )
for all
X, Y ∈ L∞ (P; R).
Hint: Assume that Π1 = 1 and that (*) holds. Given X ∈ L∞ (P; R), use induction to show that 2 kΠXknL2n (P) ≤ kXkn−1 L∞ (P) kXkL (P)
and
ΠX
n
= Π X(ΠX)n−1
n for all n ∈ Z+ . Conclude that kΠXkL∞ (P) ≤ kXkL∞ (P) and that ΠX ∈ L, n ∈ Z+ , for every X ∈ L∞ (P; R). Next, using the preceding together with Weierstrass’s Approximation Theorem, show that (ΠX)+ ∈ L, first for X ∈ L∞ (P; R) and then for all X ∈ L2 (P; R). Finally, apply (i) to arrive at L = L2 Ω, Σ, P; R . (iii) To emphasize the point being made here, consider once again a closed linear subspace L of L2 (P; R), and let ΠL be orthogonal projection onto L. Given X ∈ L2 (P; R), recall that ΠL X is characterized as the unique element of L for which X − ΠL X ⊥ L, and show that EP [X|ΣL ] is the unique element of L2 (Ω, ΣL , P; R) with the property that X − EP X ΣL ⊥ f Y1 , . . . , Yn for all n ∈ Z+ , f ∈ Cb Rn ; R , and Y1 , . . . , Yn ∈ L. In particular, ΠL X = EP [X|ΣL ] if and only if X −ΠL X is perpendicular not only to all linear functions of the Y ’s in L but even to all nonlinear ones. Exercise 5.1.16. In spite of the preceding, there is a situation in which orthogonal projection coincides with conditioning. Namely, suppose that G is a closed Gaussian family in L2 (P; R), and let L be a closed, linear subspace of G. As an application of Lemma 4.3.1, show that, for any X ∈ G, the orthogonal projection ΠL X of X onto L is a conditional expectation value of X given the σ-algebra ΣL generated by the elements of L.
204
5 Conditioning and Martingales
Exercise 5.1.17. Because most projections are not conditional expectations, it is an unfortunate fact of life that, for the most part, partial sums of Fourier series cannot be interpreted as conditional expectations. Be that as it may, there are special cases in which such an interpretation is possible. To see this, take Ω = [0, 1), F = B[0,1) , and P to be the restriction of Lebesgue measure to [0, 1). Next, for n ∈ N, take Fn to be the σ-algebra generated by√ those f ∈ C([0, 1); C) −n for k ∈ Z, that are periodic with period 2 . Finally, set ek (x) = exp −1k2πx and use elementary Fourier analysis to show that, for each n ∈ N, ek2n : k ∈ Z is an orthonormal basis for L2 (Ω, Fn , P; C). In particular, conclude that, for every f ∈ L2 (P; C), X f, ek2n L2 ([0,1);C) ek2n , EP f Fn = EP [f ] + k∈Z
where the convergence is in L2 ([0, 1]; C). (Also see Exercise 5.2.45.) Exercise 5.1.18. Let (Ω, F, µ) be a measure space and Σ a sub-σ-algebra of F with the property that µ Σ is σ-finite. Next, let E be a separable Hilbert 0 space, p ∈ [1, ∞], X ∈ Lp (µ; E), and Y a Σ-measurable element of Lp (µ; E) (p0 is the H¨older conjugate of p). Show that h i µ-almost surely. Eµ Y, X E Σ = Y, Eµ X Σ E
Hint: First observe that it suffices to check that h h i i Eµ Y, X E = Eµ Y, Eµ X Σ . E
Next, choose an orthonormal basis {en : n ≥ 0} for E, and justify the steps in ∞ X Eµ Y, en E en , X E Eµ Y, X E = 1
=
∞ X
h i Eµ Y, en E Eµ en , X E Σ = Eµ Y, Eµ [X|Σ] E .
1
Exercise 5.1.19. Let E be a separable Banach space, and show that, for each R > 0, the closed ball BE ∗ (0, R) with the weak* topology is a compact metric space. Conclude from this that the weak* topology on E ∗ is second countable and therefore separable. Hint: Choose a countable, dense subset {xn : n ≥ 1} in the unit ball BE (0, 1), and define
ρ(x∗ , y ∗ ) =
∞ X n=1
2−n hxn , x∗ − y ∗ i for x∗ , y ∗ ∈ BE ∗ (0, R).
§ 5.2 Discrete Parameter Martingales
205
Show that ρ is a metric for the weak* topology on BE ∗ (0, R). Next, choose {xnm : m ≥ 1} so that xn1 = x1 and xnm+1 = xn if n is the first n > nm such that xn is linearly independent of {x1 , . . . , xn−1 }. Given a sequence {x∗` : ` ≥ 1} in BE ∗ (0, R), use a diagonalization argument to find a subsequence {x∗`k : k ≥ 1} such that am = limk→∞ hxnm , x∗`k i exists for each m ≥ 1. Now define f on the PM PM span S of {xnm : m ≥ 1} so that f (x) = m=1 αm am if x = m=1 αm xnm , note that f (x) = limk→∞ hx, x∗`k i for x ∈ S, and conclude that f is linear on S and satisfies the estimate |f (x)| ≤ RkxkE there. Since S is dense in E, there is a unique extension of f as a bounded linear functional on E satisfying the same estimate, and so there exists an x∗ ∈ BE ∗ (0, R) such that hx, x∗ i = limk→∞ hx, x∗`k i for all x ∈ S. Finally, check that this convergence continues to hold for all x ∈ E, and conclude that x∗`k −→ x∗ in the weak* topology.
Exercise 5.1.20. The purpose of this exercise is to show that Bochner’s theory of integration for Banach space functions relies heavily on the assumption that the Banach space be separable. In particular, the approximation procedure on which the proof of Lemma 5.1.10 fails in the absence of separability. To see this, consider the Banach space `∞ (µ; R) of uniformly bounded sequences x = (x0 , . . . , xn , . . . ) ∈ RN with kxk`∞ (N;R) = supn≥0 |xn |. Next, let {Xn : n ≥ 0} be a sequence of mutually independent, {−1, 1}-valued, Bernoulli random with ∞ mean value 0 on some probability space (Ω, F, P), and define X : Ω −→ ` (N; R) by X(ω) = X0 (ω), . . . , Xn (ω), . . . . Show that, for any simple function Y : Ω −→ `∞ (N; R), P kX − Yk`∞ (N;R) < 14 = 0. Hint: For any α ∈ R, show that P |Xn − α| < 14 ≤ 12 and therefore that P kX − Ak`∞ (N;R) < 14 = 0 for any A ∈ `∞ (N; R).
§ 5.2 Discrete Parameter Martingales In this section I will introduce an interesting and useful class of stochastic processes that unifies and simplifies several branches of probability theory as well as other branches of analysis. From the analytic point of view, what I will be doing is developing an abstract version of differentiation theory (cf. Theorem 6.1.8). Although I will want to make some extensions in § 5.3, I start in the following setting. (Ω, F, P) is a probability space and {Fn : n ∈ N} is a nondecreasing sequence of sub-σ-algebra’s of F. Given a measurable space (E, B), : n ∈ N} of E-valued random variables is Fn : say that the family {X n n ∈ N -progressively measurable if Xn is Fn -measurable for each n ∈ N. random variables is said Next, a family {Xn : n ∈ N} of (−∞, ∞]-valued to be a P-submartingale with respect to Fn : n ∈ N if it is Fn : n ∈ N -progressively measurable, EP [Xn− ] < ∞, and, for each n ∈ N, X n ≤ |F ] (a.s., P). It is said to be a P-martingale with respect to Fn : EP [Xn+1 n n ∈ N if {Xn : n ∈ N} is an Fn : n ∈ N -progressively measurable family of
206
5 Conditioning and Martingales
R-valued, P-integrable random variables satisfying Xn = EP [Xn+1 |Fn ] (a.s., P) for each n ∈ N. In the future, I will abbreviate these statements by saying that the triple Xn , Fn , P is a submartingale or a martingale. Examples. The most trivial example of a submartingale is provided by a non- decreasing sequence {an : n ≥ 0}. That is, if Xn ≡an , n ∈ N, then Xn , Fn , P is a submartingale on any probability space Ω, F, P relative to any non-decreas ing Fn : n ∈ N . More interesting examples are those which follow.1 (i) Let {Yn : n ≥ 1} be a sequence of mutually independent (−∞, ∞]-valued ran- ∞, n ∈ N, set F0 = {∅, Ω}, Fn = σ {Y1 , . . . , Yn } dom variables with EP [Yn− ]
For a much more interesting and complete list of examples, the reader might want to consult J. Neveu’s Discrete-parameter Martingales, North–Holland (1975).
§ 5.2 Discrete Parameter Martingales
207
Theorem 5.2.1 (Doob’s Inequality). Assume that Xn , Fn , P is a submartingale. Then, for every N ∈ Z+ and α ∈ (0, ∞), 1 P (5.2.2) P max Xn ≥ α ≤ E XN , max Xn ≥ α . 0≤n≤N 0≤n≤N α
In particular, if the Xn ’s are non-negative, then, for each p ∈ (1, ∞), (5.2.3)
P
E
sup Xnp n∈N
p1
≤
1 p sup EP Xnp p . p − 1 n∈N
Proof: To prove (5.2.2), set A0 = {X0 ≥ α} and An = Xn ≥ α but max Xm < α 0≤m
for n ∈ Z+ .
Then the An ’s are mutually disjoint and An ∈ Fn for each n ∈ N. Thus, X N N X EP Xn , An P max Xn ≥ α = P An ≤ 0≤n≤N α n=0 n=0 N X EP XN , An 1 = EP XN , max Xn ≥ α . ≤ 0≤n≤N α α n=0
Now assume that the Xn ’s are non-negative. Given (5.2.2), (5.2.3) becomes an easy application of Exercise 1.4.18. Doob’s inequality is an example of what analysts call a weak-type inequality. To be more precise, it is a weak-type 1–1 inequality. The terminology derives from the fact that such an inequality follows immediately from an L1 -norm, or strong-type 1–1, inequality between the objects under consideration; but, in general, it is strictly weaker. In order to demonstrate how powerful such a result can be, I will now apply Doob’s Inequality to prove a theorem of Marcinkewitz. Because it is an argument to which we will return again, the reader would do well to become comfortable with the line of reasoning that allows one to pass from a weak-type inequality, like Doob’s, to almost sure convergence results. Corollary 5.2.4. Let X be an R-valued random variable and p ∈ [1, ∞). If X ∈ Lp (P; R), then, for any non-decreasing sequence Fn : n ∈ N of sub-σalgebras of F, " ∞ # _ P P (a.s., P) and in Lp (P; R) as n → ∞. E X Fn −→ E X Fn 0
In particular, if X is Lp (P; R).
W∞ 0
Fn -measurable, then EP [X|Fn ] −→ X (a.s., P) and in
208
5 Conditioning and Martingales
W∞ Proof: Without loss in generality, assume that F = 0 Fn . Given X ∈ L1 (P; R), set Xn = EP [X|Fn ] for n ∈ N. The key to my proof will be the inequality 1 (5.2.5) P sup |Xn | ≥ α ≤ EP |X|, sup |Xn | ≥ α , α ∈ (0, ∞); α n∈N n∈N
and, since, by (5.1.5), |Xn | ≤ EP [|X| |Fn ] (a.s., P), while proving (5.2.5) I may and will assume that X and all the Xn ’s are non-negative. But then, by (5.2.2), 1 P P sup Xn > α ≤ E XN , sup Xn > α α 0≤n≤N 0≤n≤N 1 = EP X, sup Xn > α α 0≤n≤N
for all N ∈ Z+ , and therefore (5.2.5) follows when N → ∞ and one takes right limits in α. As my first application of (5.2.5), note that {Xn : n ≥ 0} is uniformly Pintegrable. Indeed, because |Xn | ≤ EP [|X| |Fn ], we have from (5.2.5) that h h i i sup EP |Xn |, |Xn | ≥ α ≤ sup EP |X|, |Xn | ≥ α n∈N n∈N P ≤ E |X|, sup |Xn | ≥ α −→ 0 n∈N
as α → ∞. Thus, we will know that the asserted convergence takes place in L1 (P; R) as soon as we show that it happens P-almost surely. In addition, if X ∈ Lp (P; R) for some p ∈ (1, ∞), then, by (5.2.5) and Exercise 1.4.18, we see that |Xn |p : n ∈ N is uniformly P-integrable and, therefore, that Xn −→ X in Lp (µ; R) as soon as it does (a.s., P). In other words, everything comes down to checking the P-almost sure convergence for X ∈ L1 (P; R). To prove the P-almost sure convergence, let G be the set of X ∈ L1 (P; R) for which Xn −→ X (a.s., P). Clearly, X ∈ G if X ∈ L1 (P; R) is Fn -measurable for some n ∈ N, and, therefore, G is dense in L1 (P; R). Thus, all that remains is to prove that G is closed in L1 (P; R). But if {X (k) : k ≥ 1} ⊆ G and X (k) −→ X in L1 (P; R), then, by (5.2.5), P sup Xn − X ≥ 3α n≥N
≤P
sup Xn − Xn(k) ≥ α + P sup Xn(k) − X (k) ≥ α
n≥N
n≥N
+ P X (k) − X ≥ α
(k) 2 (k) (k)
≥α X −X + P sup Xn − X ≤ L1 (P) α n≥N
§ 5.2 Discrete Parameter Martingales
209
for every N ∈ Z+ , α ∈ (0, ∞), and k ∈ Z+ . Hence, by first letting N → ∞ and then k → ∞, we see that lim P sup Xn − X ≥ 3α = 0 for every α ∈ (0, ∞); N →∞
n≥N
and this proves that X ∈ G. Before moving on to more sophisticated convergence results, I will spend a little time showing that Corollary 5.2.4 is already interesting. In order to introduce my main application, recall my preliminary discussion of conditioning when I was attempting to explain Kolmogorov’s idea at the beginning of this chapter. As I said there, the most easily understood situation occurs when one conditions with respect to a sub-σ-algebra Σ that is generated by a countable partition P. Indeed, in that case one can easily verify that X EP X, A P 1A , (5.2.6) E X Σ = P(A) A∈P
where it is understood that EP X, A ≡0 P(A)
when P(A) = 0.
Unfortunately, even when F is countably generated, Σ need not be (cf. Exercise 1.1.18). Furthermore, just because Σ is countably generated, it will be seldom true that its generators can be chosen to form a countable partition. (For example, as soon as Σ contains an uncountable number of atoms, such a partition cannot exist.) Nonetheless, if Σ is any countably generated σ-algebra, then we can find a sequence {Pn : n ≥ 0} of finite partitions with the properties that ! ∞ [ Pn and σ Pn−1 ⊆ σ Pn , n ∈ Z+ . Σ=σ 0
In fact, simply choose a countable generating sequence {An : n ≥ 0} for Σ and take Pn to be the collection of distinct sets of the form B0 ∩ · · · ∩ Bn , where Bm ∈ {Am , Am {} for each 0 ≤ m ≤ n. Theorem 5.2.7. Let Σ be a countably generated sub-σ-algebra of F, and choose {Pn : n ≥ 0} to be a sequence of finite partitions as above. Next, given p ∈ [1, ∞) and a random variable X ∈ Lp (P; R), define Xn for n ∈ N by the right-hand side of (5.2.6) with P = Pn . Then Xn −→ EP [X|Σ] both P-almost surely and in Lp (P; R). Moreover, even if Σ is not countably generated, for each separable, closed subspace L of Lp (P; R) there exists a sequence {Pn : n ∈ N} of finite partitions such that X EP X, A 1A −→ EP X Σ (a.s., P) and in Lp (P; R) P(A) A∈Pn
for every X ∈ L.
210
5 Conditioning and Martingales
Proof: To prove the first part, simply set Fn = σ Pn , identify the Xn in (5.2.6) as EP [X|Fn ], and finally apply Corollary 5.2.4. As for the second part, let Σ(L) be the σ-algebra generated by EP [X|Σ] : X ∈ L , note that Σ(L) is countably generated and that EP X Σ = EP X Σ(L)
(a.s., P)
for each X ∈ L,
and apply the first part with Σ replaced by Σ(L). Theorem 5.2.7 makes it easy to transfer the usual Jensen’s Inequality to conditional expectations. Corollary 5.2.8 (Jensen’s Inequality). Let C be a closed, convex subset of RN , X a C-valued, P-integrable random variable, and Σ a sub-σ-algebra of F. Then there is a C-valued representative XΣ of P E X1 Σ .. P E XΣ ≡ . . EP XN Σ In addition, if g : C −→ [0, ∞) is continuous and concave, then EP g(X) Σ ≤ g XΣ
(a.s., P).
Finally, if f : C −→ R is continuous, convex, and bounded above and if X is a C-valued, P-integrable random variable, then f (X) is P-integrable and (5.2.9)
f EP [X|Σ] ≤ EP f (X)|Σ] (a.s., P).
(See Exercise 6.1.15 for Banach space–valued random variables.) Proof: By the classical Jensen’s Inequality, Y ≡ g(X) is P-integrable. Hence, by the second part of Theorem 5.2.7, we can find finite partitions Pn , n ∈ N, so that X EP [X, A] 1A −→ EP [X|Σ] Xn ≡ P(A) A∈Pn
and
X EP g(X), A 1A −→ EP g(X) Σ Yn ≡ P(A) A∈Pn
P-almost surely. Furthermore, again by the classical Jensen’s Inequality, EP [X, A] ∈C P(A)
and
P EP g(X), A E [X, A] ≤g P(A) P(A)
§ 5.2 Discrete Parameter Martingales
211
for all A ∈ F with P(A) > 0. Hence, if Λ ∈ Σ denotes the set of ω for which Xn (ω) lim ∈ RN +1 n→∞ Yn (ω) exists, v is a fixed element of C, limn→∞ Xn (ω) if ω ∈ Λ XΣ (ω) ≡ v if ω ∈ / Λ, and
Y (ω) ≡
limn→∞ Yn (ω) if ω ∈ Λ v if ω ∈ / Λ,
then XΣ is a C-valued representative of EP [X|Σ], Y is a representative of P E [g(X)|Σ], and Y (ω) ≤ g XΣ (ω) for every ω ∈ Ω. Turning to the final assertion, begin by observing that once one knows that f (X) ∈ L1 (P; R), the concluding inequality follows immediately by applying the first part to the non-negative, concave function M − f , where M ∈ R is an upper bound of f . Thus, what remains to be shown is that f − (X) ∈ L1 (P; R). To and this end, set fn = (−n) ∨ f for n ≥ 1. Then fn is bounded and convex, P P so, by the preceding with Σ = {∅, Ω}, we know that fn E [X] ≤ E fn (X) . Writing fn = f+ − fn− , this shows that EP fn− (X) ≤ M + − f E P [X] when n ≥ −f E P [X] . Finally, note that fn− = n ∧ f − , and conclude that f − (X) ∈ L1 (P; R). Corollary 5.2.10. Let I be a non-empty, closed interval in R ∪ {+∞} (i.e., either I ⊂ R is bounded on the right or I ∩ R is unbounded on the right and I includes the point +∞). Then every I-valued random variable X with Pintegrable negative part admits an I-valued representative of EP [X|Σ]. Furthermore, if f : I −→ R ∪ {+∞} is a continuous, convex function and either f is bounded above and X ∈ L1 (P; R) or f is bounded below and to the left (i.e., f is bounded on each interval of the form I ∩ (−∞, a] with a ∈ I ∩ R), then f − (X) ∈ L1 (P; R) and (5.2.9) holds. In particular, for each p ∈ [1, ∞),
P
E X Σ p ≤ kXkLp (P;R) . L (P;R) Finally, if either Xn , Fn , P is anI-valued martingale and f satisfies the preceding conditions or if Xn , Fn , P is an I-valued submartingale and f is continuous, non-decreasing, convex, and bounded below, then f (Xn ), Fn , P is a submartingale. Proof: In view of Corollary 5.2.8, we know that an I-valued representative of EP [X|Σ] exists when X is P-integrable, and the general case follows after a trivial truncation procedure.
212
5 Conditioning and Martingales
In the case when X is P-integrable and f is bounded above, f (X) ∈ L1 (P; R) and (5.2.9) are immediate consequences of the last part of Corollary 5.2.8. To handle the case when f is bounded below and to the left, first observe that either f is non-increasing everywhere or there is an a ∈ I ∩ R with the property that f is non-increasing to the left of a and non-decreasing to the right of a. Next, let an I-valued X with X − ∈ L1 (P) be given, and set Xn = X ∧ n. Then there exists an m ∈ Z+ such that Xn is I-valued for all n ≥ m; and clearly, by the preceding, we know that (*) f EP Xn Σ ≤ EP f (Xn ) Σ (a.s., P) for all n ≥ m. Moreover, in the case when f is non-increasing, f (X n ) : n ≥ m is bounded below and non-increasing; and, in the other case, f (Xn ) : n ≥ m ∨ a is bounded below and non-decreasing. Hence, in both cases, the desired conclusion follows from (*) after an application of the version of the Monotone Convergence Theorem in (5.1.8). To complete the proof, simply note that in either of the two cases given, the results just proved justify P-almost surely. EP f (Xn ) Fn−1 ≥ f EP Xn Fn−1 ≥ f Xn−1 § 5.2.2. Doob’s Stopping Time Theorem. Perhaps the most far-reaching contribution that Doob made to martingale theory is his observation that one can “stop” a martingale without destroying the martingale property. Later, L. Snell showed that the analogous result is true for submartingales. In order to state their results here, I need to introduce the notion of a stopping time in this setting. Namely, I will say that thefunction ζ : Ω −→ N ∪ {∞} is a stopping time relative to {Fn : n ≥ 0} if ω : ζ(ω) = n ∈ Fn for each n ∈ N. In addition, given a stopping time ζ, I use Fζ to denote the σ-algebra of A ∈ F such that A ∩ {ζ = n} ∈ Fn , n ∈ Z+ . Notice that Fζ1 ⊆ Fζ2 if ζ1 ≤ ζ2 . In addition, if {Xn : n ∈ N} is Fn : n ∈ N -progressively measurable, check that the random variable Xζ given by Xζ (ω) = Xζ(ω) (ω) is Fζ -measurable on {ζ < ∞}. Doob used stopping times to give a mathematically rigorous formulation of the W.C. Field’s assertion that “you can’t cheat an honest man.” That is, consider a gambler who is trying to beat the system. Assuming that he is playing a fair game, it is reasonable to say his gain Xn after n plays will evolve as a martingale. More precisely, if Fn contains the history of the game up to and including the nth play, then Xn , Fn , P will be a martingale. In the context of this model, a stopping time can be thought of as a feasible (i.e., one that does not require the gift of prophesy) strategy that the gambler can use to determine when he should stop playing in order to maximize his gains. When couched in these terms, the next result predicts that there is no strategy with which the gambler can alter his expected gain.
§ 5.2 Discrete Parameter Martingales
213
Theorem 5.2.11 (Doob’s Stopping Time Theorem). For any submartin gale (martingale) Xn , Fn , P that is P-integrable and any stopping time ζ, Xn∧ζ , Fn , P is again a P-integrable submartingale (martingale). Proof: Let A ∈ Fn−1 . Then, since A ∩ {ζ > n − 1} ∈ Fn−1 , EP Xn∧ζ , A = EP Xζ , A ∩ {ζ ≤ n − 1} + EP Xn , A ∩ {ζ > n − 1} ≥ EP Xζ , A ∩ {ζ ≤ n − 1} + EP Xn−1 , A ∩ {ζ > n − 1} = EP X(n−1)∧ζ , A ; and, in the case of martingales, the inequality in the preceding can be replaced by an equality. Closely related to Doob’s Stopping Time Theorem is an important variant due to G. Hunt. In order to facilitate the proof of Hunt’s result, I begin with an easy but seminal observation of Doob’s. Lemma 5.2.12 (Doob’s Decomposition). For each n ∈ N let Xn be an Fn -measurable, P-integrable random variable. Then, up to a P-null set, there is at most one sequence {An : n ≥ 0} ⊆ L1 (P; R) such that A0 = 0, An is Fn−1 + measurable for each n ∈ Z , and Xn − An , Fn , P is a martingale. Moreover, if (Xn , Fn , P) is an integrable submartingale, then such a sequence {An : n ≥ 0} exists, and An−1 ≤ An P-almost surely for all n ∈ Z+ . Proof: To prove the uniqueness assertion, suppose that {An : n ≥ 0} and {Bn : n ≥ 0} are two such sequences, and set ∆n = Bn − An . Then ∆0 = 0, ∆n is Fn−1 -measurable for each n ∈ Z+ , and (∆n , Fn , P) is a martingale. But this means that ∆n = EP [∆n | Fn−1 ] = ∆n−1 for all n ∈ Z+ , and so ∆n = 0 for all n ∈ N. Now suppose that (Xn , Fn , P) is an integrable submartingale. To prove the asserted existence result, set A0 ≡ 0 and An = An−1 + EP Xn − Xn−1 Fn−1 ∨ 0
for n ∈ Z+ .
Theorem 5.2.13 (Hunt). Let Xn , Fn , P be a P-integrable submartingale. Given bounded stopping times ζ and ζ 0 satisfying ζ ≤ ζ 0 , (5.2.14)
Xζ ≤ EP Xζ 0 Fζ
(a.s., P),
and the inequality can be replaced by equality when Xn , Fn , P is a martingale. (Cf. Exercise 5.2.39 for unbounded stopping times.) Proof: Choose {An : n ∈ N} for (Xn , Fn , P) as in Lemma 5.2.12, and set Yn = Xn − An for n ∈ N. Then, because Aζ ≤ Aζ 0 and Aζ is Fζ -measurable, EP Xζ 0 Fζ ≥ EP Yζ 0 + Aζ Fζ = EP Yζ 0 Fζ + Aζ .
214
5 Conditioning and Martingales
Hence, it suffices to prove that equality holds in (5.2.14) when Xn , Fn , P is a martingale. To this end, choose N ∈ Z+ to be an upper bound for ζ 0 , let Γ ∈ Fζ be given, and note that N X EP XN , Γ ∩ {ζ = n} EP XN , Γ = n=0
=
N X
EP Xn , Γ ∩ {ζ = n} = EP Xζ , Γ .
n=0
Similarly, since Γ ∈ Fζ ⊆ Fζ 0 , EP [XN , Γ] = EP [Xζ 0 , Γ]. § 5.2.3. Martingale Convergence Theorem. My next goal is to show that, even when they are not given in the form covered by Corollary 5.2.4, martingales want to converge. If for no other reason, such a result has got to be more difficult because one does not know ahead of time what, if it exists, the limit ought to be. Thus, the reasoning will have to be more subtle than that used in the proof of Corollary 5.2.4. I will follow Doob and base my argument on the idea that, in some sense, a martingale has got to be nearly constant and that a submartingale is the sum of a martingale and a non-decreasing process. In order to make mathematics out of this idea, I need to introduce a somewhat novel criterion for convergence of real numbers. Namely, given a sequence {xn : n ≥ 0} ⊆ R and a numbers −∞ < a < b < ∞, say that {xn : n ≥ 0} upcrosses the interval [a, b] at least N times if there exist integers 0 ≤ m1 < n1 < · · · < mN < nN such that xmi ≤ a and xni ≥ b for each 1 ≤ i ≤ N and that it upcrosses [a, b] precisely N times if it upcrosses [a, b] at least N but does not upcross [a, b] at least N + 1 times. Notice that limn→∞ xn < limn→∞ xn if and only if there exist rational numbers a < b such that {xn : n ≥ 0} upcrosses [a, b] at least N times for every N ∈ Z+ . Hence, {xn : n ≥ 0} converges in [−∞, ∞] if and only if it upcrosses [a, b] at most finitely often for each pair of rational numbers a < b. 2 Theorem 5.2.15 (Doob’s Martingale Convergence Theorem). Suppose that Xn , Fn , P is a P-integrable submartingale. For −∞ < a < b < ∞, let U[a,b] (ω) denote the precise number of times that {Xn (ω) : n ≥ 0} upcrosses [a, b]. Then EP (Xn − a)+ P . (5.2.16) E U[a,b] ≤ sup b−a n∈N
In particular, if (5.2.17)
sup EP Xn+ < ∞,
n∈N
2 In the notes to Chapter VII of his Stochastic Processes, Wiley (1953), Doob gives a thorough account of the relationship between his convergence result and earlier attempts in the same direction. In particular, he points out that, in 1946, S. Anderson and B. Jessen formulated and proved a closely related convergence theorem.
§ 5.2 Discrete Parameter Martingales
215
then there exists a P-integrable random variable X to which {Xn : n ≥ 0} converges P-almost surely. (See Exercises 5.2.36 and 5.2.38 for other derivations.) −a)+ , and note that (by Corollary 5.2.10) Yn , Fn , P is Proof: Set Yn = (Xnb−a a P-integrable submartingale. Next, let N ∈ Z+ be given, set ζ00 = 0, and, for k ∈ Z+ , define 0 ζk = inf{n ≥ ζk−1 : Xn ≤ a} ∧ N
and ζk0 = inf{n ≥ ζk : Xn ≥ b} ∧ N.
Proceeding by induction, it is an easy matter to check that all the ζk ’s and (N ) ζ 0 ’s are stopping times. Moreover, if U[a,b] (ω) is the precise number of times k Xn∧N (ω) : n ≥ 0 upcrosses [a, b], then (N )
U[a,b] ≤
N X
N X 0 Yζk0 − Yζk = YN − Y0 − Yζk − Yζk−1
k=1
≤ YN −
k=1 N X
0 Yζk − Yζk−1 .
k=1
0 0 ≥ 0 for all ≤ ζk and therefore, by (5.2.14), EP Yζk − Yζk−1 Hence, since ζk−1 (N )
k ∈ Z+ , we see that EP [U[a,b] ] ≤ EP [YN ], and clearly (5.2.16) follows from this after one lets N → ∞. Given (5.2.16), the convergence result is easy. Namely, if (5.2.17) is satisfied, then (5.2.16) implies that there is a set Λ of full P-measure such that U[a,b] (ω) < ∞ for all rational a < b and ω ∈ Λ; and so, by the remark preceding the statement of this theorem, for each ω ∈ Λ, {Xn (ω) : n ≥ 0} converges to some X(ω) ∈ [−∞, ∞]. Hence, we will be done as soon as we know that EP [|X|, Λ] < ∞. But EP |Xn | = 2EP Xn+ − EP Xn ≤ 2EP Xn+ − EP X0 ,
n ∈ N,
and therefore Fatou’s Lemma plus (5.2.17) shows that X is P-integrable.
The inequality in (5.2.16) is quite famous and is known as Doob’s Upcrossing Inequality. Remark 5.2.18. The argument in the proof of Theorem 5.2.15 is so slick that it is easy to miss the point that makes it work. Namely, the whole proof turns 0 on the inequality EP [Yζk − Yζk−1 ] ≥ 0. At first sight, this inequality seems to be 0 wrong, since one is inclined to think that Yζk < Yζk−1 . However, Yζk need be 0 less than Yζk−1 only if ζk < N , which is precisely what, with high probability, the submartingale property is preventing from happening.
216
5 Conditioning and Martingales
Corollary 5.2.19. Let Xn , Fn , P be a martingale. Then there exists an X ∈ L1 (P; R) such that Xn = EP [X|Fn ] (a.s., P) for each n ∈ N if and only if the sequence {Xn : n ≥ 0} is uniformly P-integrable. In addition, if p ∈ (1, ∞], then there is an X ∈ Lp (P; R) such that Xn = EP [X|Fn ] (a.s., P) for each n ∈ N if and only if {Xn : n ≥ 0} is a bounded subset of Lp (P; R). Proof: Because of Corollary 5.2.4 and (5.2.3), I need only check the “if” statement in the first assertion. But, if {Xn : n ≥ 0} is uniformly P-integrable, then (5.2.17) holds and therefore Xn −→ X (a.s., P) for some P-integrable X. Moreover, uniform integrability together with almost sure convergence implies convergence in L1 (P; R), and therefore, by (5.1.5), for each m ∈ N, Xm = lim EP Xn Fm = EP X Fm (a.s., P). n→∞
Just as Corollary 5.2.4 led us to an intuitively appealing way to construct conditional expectations, so does Doob’s Theorem gives us an appealing approximation procedure for Radon–Nikodym derivatives. Theorem 5.2.20 (Jessen). Let P and Q be a pair of probability measures on the measurable space (Ω, F) and Fn : n ∈ N a non-decreasing sequence of subσ-algebras whose union generates F. For each n ∈ N, let Qn,a and Qn,s denote, respectively, the absolutely continuous and singular parts of Qn ≡ Q Fn dQ . Also, let Qa and Qs be the with respect to Pn ≡ P Fn , and set Xn = dPn,a n a absolutely and singular continuous parts of Q with respect to P, and set Y = dQ dP . Then Xn −→ Y (a.s., P). In particular, Q ⊥ P if and only if Xn −→ 0 (a.s., P). Moreover, if Qn Pn for each n ∈ N, then Q P if and only if {Xn : n ≥ 0} is uniformly P-integrable, in which case Xn −→ Y in L1 (P; R) as well as P-almost surely. Finally, if Qn ∼ Pn (i.e., Pn Qn as well as Qn Pn ) for each n ∈ N and G ≡ limn→∞ Xn ∈ (0, ∞) , then Qa (A) = Q(A ∩ G) for all A ∈ F, and therefore Q(G) = 1 ⇐⇒ Q P and Q(G) = 0 ⇐⇒ Q ⊥ P.
Proof: Without loss in generality, I will assume throughout that all the Xn ’s P P a as well as Y ≡ dQ dP take values in [0, ∞); and clearly, E [Xn ], n ∈ N, and E [Y ] are all dominated by 1. First note that n o for A ∈ Fn . Qn,s (A) = sup Q(A ∩ B) : B ∈ Fn and P(B) = 0
Hence, Qn,s Fn−1 ≥ Qn−1,s for each n ∈ Z+ , and so EP Xn , A = Qn,a (A) ≤ Qn−1,a (A) = EP Xn−1 , A for all n ∈ Z+ and A ∈ Fn−1 . In other words, −Xn , Fn , P is a non-positive submartingale. Moreover, in the case when Qn Pn , n ∈ N, the same argument
§ 5.2 Discrete Parameter Martingales
217
shows that Xn , Fn , P is a non-negative martingale. Thus, in either case, there is a non-negative, P-integrable random variable X with the property that Xn −→ X (a.s., P). In order to identify X as Y , use Fatou’s Lemma to see that, for any m ∈ N and A ∈ Fm , EP X, A ≤ lim EP Xn , A = lim Qn,a (A) ≤ Q(A); n→∞
n→∞
S∞ and therefore EP [X, A] ≤ Q(A), first for A ∈ 0 Fm and then for every A ∈ F. In particular, by choosing B ∈ F so that Qs (B) = 0 = P B{ , we have that EP X, A = EP X, A ∩ B ≤ Q(A ∩ B) = Qa (A) = EP Y, A for all A ∈ F, which means that X ≤ Y (a.s., P). On the other hand, if Yn = EP [Y |Fn ] for n ∈ N, then EP Yn , A = Qa (A) ≤ Qn,a (A) = EP Xn , A for all A ∈ Fn , and therefore Yn ≤ Xn (a.s., P) for each n ∈ N. Thus, since Yn −→ Y and Xn −→ X P-almost surely, this means that Y ≤ X (a.s., P). Next, assume that Qn Pn for each n ∈ N and therefore that Xn , Fn , P is a non-negative martingale. If {Xn : n ≥ 0} is uniformly P-integrable, then Xn −→ Y in L1 (P; R) and therefore Qs (Ω) = 1 − EP [Y ] = 0. Hence, Q P when {Xn : n ≥ 0} is uniformly P-integrable. Conversely, if Q P, then it is easy to see that Xn = EP [Y |Fn ] for each n ∈ N, and therefore, by Corollary 5.2.4, that {Xn : n ≥ 0} is uniformly P-integrable. Finally, assume that Qn ∼ Pn for each n ∈ N. Then, the Xn ’s can be chosen dPn . Hence, if Pa and Ps are to take their values in (0, ∞) and Yn ≡ X1n = dQ n the absolutely continuous and singular parts of P relative to Q and if Y ≡ Q a limn→∞ Yn , then Y = dP dQ and so Pa (A) = E [Y, A] for all A ∈ F. Thus, when 1 on G and B ∈ F is chosen so that Ps (B) = 0 = Q(B{), then, since Y = X P P E [X, C ∩ G] = E [X, C] for all C ∈ F, it is becomes clear that Q(A ∩ G = EQ XY, A ∩ G = EPa X, A ∩ G = EP X, A ∩ G ∩ B = EP X, A ∩ B = Qa (A ∩ B) = Qa (A)
for all A ∈ F. § 5.2.4. Reversed Martingales and De Finetti’s Theory. For some applications it is important to know what happens if one runs a submartingale or martingale backwards. Thus, again let (Ω, F, P) be a probability space, only this time suppose that Fn : n ∈ N is a sequence of sub-σ-algebras that is non-increasing. Given a sequence {Xn : n ≥ 0} of (−∞, ∞]-valued random variables, I will say that the triple Xn , Fn , P is either a reversed submartingale or a reversed martingale if, for each n ∈ N, Xn is Fn -measurable and either Xn− ∈ L1 (P; R) and Xn+1 ≤ EP [Xn | Fn+1 ] or Xn ∈ L1 (P; R) and Xn+1 = EP [Xn | Fn+1 ].
218
5 Conditioning and Martingales
Theorem 5.2.21. If (Xn , Fn , P) is a reversed submartingale, then (5.2.22)
P sup Xn ≥ R n∈N
1 P ≤ E X0 , sup Xn ≥ R , R n∈N
R ∈ (0, ∞).
In particular, if (Xn , Fn , P) is a non-negative reversed submartingale and X0 ∈ L1 (P; R), then {Xn : n ≥ 0} is uniformly P-integrable and (5.2.23)
sup Xn
n∈N
≤
Lp (P;R)
p
X0 p L (P;R) p−1
when p ∈ (1, ∞).
Moreover, if (Xn , Fn , P) is a reversed martingale, then (|Xn |, Fn , P is a re, Fn , P) is a reversed submartingale and versed submartingale. Finally, if (XnT ∞ X0 ∈ L1 (P; R), then there is a F∞ ≡ n=0 Fn -measurable X : Ω −→ [−∞, ∞] to which Xn converges P-almost surely. In fact, X will be P-integrable if supn≥0 EP [|Xn |] < ∞; and if (Xn , Fn , P) is either a non-negative reversed submartingale or a reversed martingale with X0 ∈ Lp (P; R) for some p ∈ [1, ∞), then Xn −→ X in Lp (P; R). Proof: More or less everything here follows immediately from the observation that (Xn , Fn , P) is a reversed submartingale or a reversed martingale if and only if, for each N ∈ Z+ , (XN −n∧N , FN −n∧N , P) is a submartingale or a martingale. Indeed, by this observation and (5.2.2) applied to (XN −n∧N , FN −n∧N , P),
P
max Xn > R
0≤n≤N
1 P ≤ E X0 , max Xn > R 0≤n≤N R
for every N ≥ 1. When N → ∞, the left-hand side of the preceding tends to P (supn∈N Xn > R) and EP X0 , max Xn > R = EP X0+ , max Xn > R − EP X0− , max Xn > R 0≤n≤N 0≤n≤N 0≤n≤N + − P P P −→ E X0 , sup Xn > R − E X0 , sup Xn > R = E X0 , sup Xn > R , n∈N
n∈N
n∈N
since X0+ is non-negative, and therefore the Monotone Convergence Theorems applies, and X0− is integrable, and therefore Lebesgue’s Dominated Convergence Theorem applies. Thus (5.2.22) follows after one takes right limits in R. Starting from (5.2.22) and applying Exercise 1.4.18, (5.2.23) follows for non-negative, reversed submartingales. Moreover, because it is obvious that (|Xn |, Fn , P) is a reversed submartingale when (Xn , Fn , P) is a reversed martingale, (5.2.23) holds for reversed martingales as well.
§ 5.2 Discrete Parameter Martingales
219
Next, suppose that (Xn , Fn , P) is a non-negative, reversed submartingale or a reversed martingale. Then P P P sup E |Xn |, |Xn | ≥ R ≤ sup E |X0 |, |Xn | ≥ R ≤ E |X0 |, sup |Xn | ≥ R , n∈N
n∈N
n∈N
which, by (5.2.22), tends to 0 as R → ∞. Thus, {Xn : n ≥ 0} is uniformly P-integrable. It remains to prove the convergence assertions, and again the key is the same observation about reversing time to convert reversed submartingales into submartingales. However, before seeing how it applies, first say that {xn : n ≥ 0} downcrosses [a, b] at least N times if there exist 0 ≤ m1 < n1 < · · · < mN < nN such that xmi ≥ b and xni ≤ a for each 1 ≤ i ≤ N . Clearly, the same argument that I used for upcrossings applies to downcrossings and shows that {xn : n ≥ 0} converges in [−∞, ∞] if and only if it downcrosses [a, b] finitely often for each rational pair a < b. In addition, {xn : 0 ≤ n ≤ N } downcrosses [a, b] the same (N ) number of times as {xN −n : 0 ≤ n ≤ N } upcrosses it. Hence, if D[a,b] (ω) is the number of times {Xn∧N : n ≥ 0} downcrosses [a, b], then this observation (N ) together with the estimate in the proof of Theorem 5.2.15 for EP [U[a,b] ] show that (N ) EP (X0 − a)+ P . E D[a,b] ≤ b−a Starting from here, the argument used to prove Theorem 5.2.15 shows that there exits a F∞ -measurable X : Ω −→ [−∞, ∞] to which {Xn : n ≥ 0} converges P-almost surely. Once one has this almost sure convergence result, the rest of the theorem is an easy application of standard measure theory and the uniform integrability estimates proved above. An important application of reversed martingales is provided by De Finetti’s theory of exchangeable random variables. To describe his theory, let Σ denote the group of all finite permutations of Z+ . That is, an element π of Σ is an isomorphism of Z+ that moves only a finite number of integers. Alternatively, S∞ Σ = m=1 Σm , where Σm is the group of isomorphisms π of Z+ with the property that n = π(n) for all n > m. Next, let (E, B) be a measurable space, and, for + + each π ∈ Σ, define Sπ : E Z −→ E Z so that if x = x1 , . . . , xn , . . . . Sπ x = xπ(1) , . . . , xπ(n) , . . . +
+
Obviously, each Sπ is a B Z -measurable isomorphism from E Z onto itself. Also, if + for m ∈ Z+ , Am ≡ B ∈ BZ : B = Sπ B for all π ∈ Σm +
then the Am ’s form a non-increasing sequence of sub-σ-algebras of B Z , and ∞ \ m=1
+ Am = A∞ ≡ B ∈ B Z : B = Sπ B for all π ∈ Σ .
220
5 Conditioning and Martingales
Now suppose that {Xn : n ≥ 1} is a sequence of E-valued random variables on + the probability space (Ω, F, P), and set X(ω) = X1 (ω), . . . , Xn (ω), . . . ) ∈ E Z . The Xn ’s are said to be exchangeable random variables if X has the same P-distribution as Sπ X for every π ∈ Σ. The central result of De Finetti’s theory is De Finetti’s Strong Law, which states that, for any g : E −→ R satisfying g ◦ X1 ∈ L1 (P; R), n 1X g ◦ Xm , EP g ◦ X1 X−1 (A∞ ) = lim n→∞ n 1
(5.2.24)
where the convergence is P-almost sure and in L1 (P; R). To prove (5.2.24), observe that, for any 1 ≤ m ≤ n, EP [g ◦ Xm | X−1 (An )] = EP [g ◦ X1 | X−1 (An )], which immediately leads to # " n n X −1 −1 1 X 1 P P g ◦ Xm . g ◦ Xm X (An ) = E g ◦ X1 X (An ) = E n m=1 n m=1
Hence, (5.2.24) follows as an application of Theorem 5.2.21. De Finetti’s Strong Law makes it important to get a handle on the σ-algebra X−1 (A∞ ). In particular, one would like to know when X−1 (A∞ ) is trivial in the sense that each of its elements has probability 0 or 1, in which case (5.2.24) self-improves to the statement that n
(5.2.25)
1X g ◦ Xm = EP [g ◦ X1 ] lim n→∞ n 1
P-almost surely and in L1 (P; R).
The following lemma is the crucial step toward gaining an understanding of X−1 (A∞ ). T∞ Lemma 5.2.26. Refer to the preceding, and let T = m=1 σ {Xn : n ≥ m} be the tail σ-algebra determined by {Xn : n ≥ 1}. Then T ⊆ X−1 (A∞ ) and X−1 (A∞ ) is contained in the completion of T with respect to P. In particular, for each F ∈ L1 (P; R), (a.s., P). (5.2.27) EP F X−1 (A∞ ) = EP F T Proof: The inclusion T ⊆ X−1 (A∞ ) is obvious. Thus, what remains to be proved is that, for any F ∈ L1 (P; R), EP [F | X−1 (A∞ )] is, up to a P-null set, T -measurable. To this end, begin by observing that it suffices to check this for N } -measurable for some N ∈ Z+ . Indeed, since F ’s that are σ {Xn : 1 ≤ m ≤ X−1 (A∞ ) ⊆ σ {Xn : n ≥ 1} , we know that i h EP F X−1 (A∞ ) = EP EP F σ {Xn : n ≥ 1} X−1 (A∞ ) i h = lim EP EP F σ {Xm : 1 ≤ m ≤ N } X−1 (A∞ ) . N →∞
§ 5.2 Discrete Parameter Martingales
221
Now suppose that F is σ {Xm : 1 ≤ m ≤ N } -measurable. Then there exists a g : E N −→ R such that F = g X1 , . . . , XN ). If N = 1, then, because Pn limn→∞ n1 m=1 g ◦ Xm is T -measurable, (5.2.24) says that E P [F | X−1 (A∞ )] is T -measurable. To get the same conclusion when N ≥ 2, I want to apply the same reasoning, only now with E replaced by E N . To be precise, define
) Z+ : B = Sσ B for all π ∈ Σ(N ) , where A(N ∞ = B ∈B Σ(N ) = π ∈ Σ : π(`N + m) = π(`N + 1) + m − 1 for all ` ∈ N and 1 ≤ m < N of length N . is the group of finite permutations that transform Z+ in blocks −1 (N ) N P By (5.2.24) applied with E replacing E, we find that E F X (A∞ ) = (N ) −1 −1 EP F T P-almost surely. Hence, since X (A∞ )1⊆ X (A∞ ), (5.2.27) holds for every σ {Xn : 1 ≤ n ≤ N } -measurable F ∈ L (P; R). The best known consequence of Lemma 5.2.26 is the Hewitt–Savage 0– 1 Law, which says that X−1 (A∞ ) is trivial if the Xn ’s are independent and identically distributed. Clearly, their result is an immediate consequence of Lemma 5.2.26 together with Kolmogorov’s 0–1 Law. Seeing as the Strong Law of Large Numbers follows from (5.2.24) combined with the Hewitt–Savage 0–1 Law, one might think that (5.2.24) represents an extension of the strong law. However, that is not really the case, since it can be shown that X−1 (A∞ ) is trivial only if the Xn ’s are independent. On the other hand, the derivation of the strong law via (5.2.24) extends without alteration to the Banach space setting (cf. part (ii) of Exercise 6.1.16). § 5.2.5. An Application to a Tracking Algorithm. In this subsection I will apply the considerations in § 5.2.1 to the analysis of a tracking algorithm. The origin of this algorithm is an idea which Jan Mycielski introduced as a model for learning. However, the treatment here derives from a variation, suggested by Roy O. Davies, of Mycielski’s model. Because I do not understand learning theory, I prefer to think of Mycielski’s algorithm as a tracking algorithm. Let (E, B) be a measurable space for which there exists a nested sequence {Pk : k ≥ 0} of finite or countable partitions such that P0 = {E} and B = S∞ ˘ be the “parent” of Q in the sense σ ( k=0 Pk ). Given k ≥ 1 and Q ∈ Pk , let Q ˘ that Q is the unique element of Pk−1 which contains Q. Also, for each x ∈ E and k ≥ 0, use Qk (x) to denote the unique Q ∈ Pk such that Q 3 x. Further, let µ be a probability measure on (E, B) with the property that, for some θ ∈ (0, 1), ˘ for each Q ∈ S∞ Pk 0 < µ(Q) ≤ (1 − θ)µ(Q) k=0 Next, let (Ω, F, P) be a probability space on which there exists a sequence {Xn : n ≥ 1} of mutually independent E-valued random variables with distribution µ. In addition, let {Zn : n ≥ 1} be a sequence of E-valued random variables with the property that, for each n ≥ 1, Zn is independent of σ {Xm : 1 ≤ m ≤ n} , let νn be the distribution of Zn , and assume that
222
5 Conditioning and Martingales
n < ∞ for some r ∈ (1, ∞). Finally, deνn µ with Kr ≡ supn≥1 dν dµ r L (µ;R)
fine {Yn : n ≥ 1} by the prescription that Yn (ω) = Xm (ω) if Xm (ω) is the first element of {X1 (ω), . . . , Xn (ω)} which is “closest” to Zn (ω) in the sense that, Z ∈ / Q (ω) for 1 ≤ j < m, Xm (ω) ∈ Qk Zn (ω) , and for some k ≥ 0, Xj (ω) k n / Qk+1 Zn (ω) for m < j ≤ n. Xj (ω) ∈ The goal here is to show that the Yn ’s “search out” the Zn ’s in the sense that, for any B-measurable f : E −→ R, (5.2.28) lim P |f (Yn ) − f (Zn )| ≥ = 0 for all > 0. n→∞
At least in the case when νn = µ, Mycielski has an alternative, in some sense simpler, derivation of (5.2.28). The strategy which I will use is the following. For each k ≥ 1 and f ∈ L1 (µ; R), define fk : E −→ R so that Z 1 f (y) µ(dy). fk (x) = µ(Qk (x)) Qk (x) Obviously fk Yn (ω) = fk Zn (ω) if Yn (ω) ∈ Qk Zn (ω) . Moreover, as I will / Qk (Zn ) = 0 for each k ≥ 0. Thus, the key step is show below, limn→∞ P Yn ∈ to show that lim sup P |f (Yn ) − fk (Yn )| ≥ = 0 for all > 0. k→∞ n≥1
Notice that, because fk = Eµ f σ(Pk ) , this would be obvious from Corollary 5.2.4 if the Yn were replaced by Xn . Thus, the problem comes down to showing that the distributions of Yn ’s are uniformly sufficiently close to µ. For each n ≥ 1, define Πn (z, Γ) = =
n ∞ X X k=0 j=1 ∞ X n
∆
k=0
j−1 n−j 1 − µ Qk (z) µ (Qk (z) \ Qk+1 (z)) ∩ Γ 1 − µ Qk+1 (z) µ (Qk (z) \ Qk+1 (z)) ∩ Γ , Qk+1 (z) µ Qk (z) \ Qk+1 (z)
n ˘ n . Then where ∆n (Q) ≡ 1 − µ(Q) − 1 − µ(Q) ZZ 1B (z, y) Πn (z, dy)νn (dz). (5.2.29) P (Zn , Yn ) ∈ B = B
In particular, if µn is the distribution of Yn , then Z ∞ ˘ \ Q) ∩ Γ X X µ (Q n . µn (Γ) = Πn (z, Γ) νn (dz) = ∆ (Q)νn (Q) ˘ \ Q) ν(Q k=0 Q∈Pk+1
§ 5.2 Discrete Parameter Martingales
223
In addition, because Q` (z) \ Q`+1 (z) ∩ Qk (z) = ∅ if ` < k and is equal to Q` (z) \ Q`+1 (z) when ` ≥ k, ∞ X ∆n Q`+1 (z)) Πn z, Qk (z) = `=k
= lim
L→∞
n n n 1 − µ(QL+1 (z)) − 1 − µ(Qk (z) = 1 − 1 − µ(Qk (z)) .
Thus, if r0 is the H¨older conjugate of r, then / Qk (Zn ) = P Yn ∈
Z 1−µ(Qk (z))
n
Z νn (dz) ≤ Kr
10 r nr0 , µ(dz) 1 − µ(Qk (z))
and so, by Lebesgue’s Dominated Convergence Theorem, / Qk (Zn ) = 0 for all k ≥ 0. (5.2.30) lim P Yn ∈ n→∞
Given an f ∈ L1 (µ; R) and Q ∈ 1 Af (Q) = µ(Q)
S∞
k=0
Z f dµ
Pk , set ( 0
0
and M f (Q) = sup A|f |(Q ) : Q ⊆ Q ∈
Q
∞ [
) Pk
.
k=0
Clearly, x ∈ Q =⇒ M f (Q) ≤ f ∗ (x) ≡ sup A|f | Qk (x) , k≥0
and, because Af Qk (x) = Eµ f σ(Pk ) (x), Doob’s Inequality (5.2.3) implies p kf kLp (µ;R) for all p ∈ (1, ∞]. that kf ∗ kLp (µ;R) ≤ p−1
Lemma 5.2.31. For any f ∈ L1 (µ; R), Z Z −1 f ∗ dνn . (5.2.32) |f | dµn ≤ θ 0
In particular, if q ∈ [1, ∞) and f ∈ Lqr (µ; R), then (5.2.33)
kf kLq (µn ;R) ≤
rKr θ
q‘
kf kLqr0 (µ;R) .
Proof: Without loss in generality, I will assume throughout that f ≥ 0. To prove (5.2.32), first note that Z Z ∞ X X 1 n f dµ ∆ (Q)νn (Q) f dµn = ˘ \ Q) Q\Q ˘ µ(Q k=0 Q∈Pk+1 X X ˘ ∆n (Q)νn (Q)M f (Q), ≤ θ−1 k=0 Q∈Pk+1
224
5 Conditioning and Martingales
since
Z
1 ˘ µ(Q \ Q)
˘ ≤ θ−1 M f (Q). ˘ f dµ ≤ θ−1 Af (Q)
˘ Q\Q
Next, for each k ≥ 0, X Q∈Pk+1
n ˘ 1 − µ(Q) νn (Q)M f (Q)
X
˘ = ∆n (Q)νn (Q)M f (Q)
Q∈Pk+1
˘ n νn (Q)M f (Q) ˘ 1 − µ(Q)
X
−
Q∈Pk+1
X n n ˘ − 1 − µ(Q) νn (Q)M f (Q) 1 − µ(Q) νn (Q)M f (Q)
X
=
Q∈Pk+1
≤
Q∈Pk
X n n 1 − µ(Q) νn (Q)M f (Q) − 1 − µ(Q) νn (Q)M f (Q),
X Q∈Pk+1
Q∈Pk
and therefore K X X
Z f dµn ≤ lim
θ
K→∞
n 1 − µ(Q) νn (Q)M f (Q)
Q∈Pk+1
k=0
−
n 1 − µ(Q) νn (Q)M f (Q)
X
Q∈Pk
X
= lim
K→∞
n 1 − µ(Q) νn (Q)M f (Q) ≤
Z
f ∗ dνn .
Q∈PK+1
Given (5.2.32), (5.2.33) is an easy application of H¨older’s Inequality and the estimate coming from (5.2.3) on the Lp (µ; R)-norm of f ∗ in terms of that of f . Namely, Z
q
f dµn ≤ θ
−1
Z
q ∗
(f ) dνn ≤ Kr
≤ θ−1 Kr
Z
0 q ∗ r
(f )
10 r dµ
rKr r0 kf kqLqr0 (µ;R) . kf q kLr0 (µ;R) = 0 θ r −1
Theorem 5.2.34. For each B-measurable f : E −→ R, (5.2.28) holds. More0 over, if q ∈ (1, ∞) and f ∈ Lqr (µ; R), then (5.2.35)
lim EP |f (Yn ) − f (Zn )|q = 0 for each p ∈ [1, q).
n→∞
(See Exercise 6.1.19 for a related result.)
§ 5.2 Discrete Parameter Martingales
225
Proof: It is easy to prove Indeed, given δ > 0, choose (5.2.28) from (5.2.35). R R > 0 so that µ |f | ≥ R < δ, and set f = f 1[−R,R] (f ). Then, by (5.2.35), limn→∞ P |f R (Yn ) − f R (Zn )| ≥ = 0 for all > 0. Hence, lim P |f (Yn ) − f (Zn )| ≥ 3 n→∞ ≤ lim µn |f − f R | ≥ + lim νn |f − f R | ≥ . n→∞
n→∞
By H¨older’s Inequality, 1 1 νn |f − f R | ≥ ≤ Kr µ |f − f R | ≥ r0 < Kr δ r0 ,
and, by (5.2.33) with q = 1,
1 rKr rKr 10 δr . µ |f − f R | ≥ r0 < µn |f − f R | ≥ ≤ θ θ
The proof of (5.2.35) follows the strategy outlined earlier. That is, 1 EP |f (Yn ) − f (Zn )|p p
1 ≤ kf − fk kLp (µn ;R) + EP |fk (Yn ) − fk (Zn )|p p + kfk − f kLp (νn ;R) .
By (5.2.33), kf − fk kLp (µn ;R) ≤
rKr θ
p1
kf − fk kLpr0 (µ;R) ,
and, by H¨older’s Inequality, 1
kf − fk kLp (νn ;R) ≤ Krp kf − fk kLpr0 (µ;R) .
Since, by Corollary 5.2.4, kf − fk kLpr0 (µ;R) −→ 0 as k → ∞, all that remains is 1 to show that, for each k ≥ 0, EP |fk (Yn ) − fk (Zn )|p p −→ 0. But
1 1 / Qk (Zn ) p EP |fk (Yn ) − fk (Zn )|p p = EP |fk (Yn ) − fk (Zn )|p , Yn ∈ 1−1 1 / Qk (Zn ) p q . ≤ EP |fk (Yn ) − fk (Zn )|q q P Yn ∈ By (5.2.30), the final factor tends to 0 as n → ∞. Hence, since, by H¨older’s Inequality and (5.2.33), 1 EP |fk (Yn ) − fk (Zn )|q q ≤ kfk kLq (µn ;R) + kfk kLq (νn ;R) 1 1 1 1 r q r q + 1 Krq kf kLqr0 (µ;R) , + 1 Krq kfk kLqr0 (µ;R) ≤ ≤ θ θ
the proof is complete.
226
5 Conditioning and Martingales Exercises for § 5.2
Exercise 5.2.36. In this exercise I will outline a quite independent derivation of the convergence assertion in Doob’s Martingale Convergence Theorem. The key observations here are first that, given Doob’s Inequality (cf. (5.2.2)), the result is nearly trivial for martingales having uniformly bounded second moments and second that everything can be reduced to that case. (i) Let Mn , Fn , P be a martingale which is L2 -bounded (i.e., supn∈N EP [Mn2 ] < ∞). Note that h 2 2 i EP Mn2 − EP Mm−1 = EP Mn − Mm−1
for
1 ≤ m ≤ n;
and starting from this, show that there is an M ∈ L2 (P; R) such that Mn −→ M 2 in L (P; R). Next apply (5.2.5) to the submartingale Mn∨m − Mm , Fn , P to show that, for every > 0, P
sup Mn − Mm ≥
n≥m
≤
i 1 P h E M − Mm −→ 0
as
m → ∞,
and conclude that Mn −→ M (a.s., P). (ii) Let Xn , Fn , P be a non-negative submartingale with the property that supn∈N EP [Xn2 ] < ∞, define the sequence {An : n ∈ N} accordingly, as in Lemma 5.2.12, and set Mn = Xn − An , n ∈ N. Then Mn , Fn , P is a martingale, and clearly both Mn and An are P-square integrable for each n ∈ N. In fact, check that 2 = EP Mn − Mn−1 Xn + Xn−1 EP Mn2 − Mn−1 2 2 − EP An − An−1 Xn + Xn−1 ≤ EP Xn2 − Xn−1 , = EP Xn2 − Xn−1 and therefore that EP Mn2 ≤ EP Xn2 and
EP A2n ≤ 4EP Xn2
for every n ∈ N.
Finally, show that there exist M ∈ L2 (P; R) and A ∈ L2 P; [0, ∞) such that Mn −→ M , An % A, and, therefore, Xn −→ X ≡ M + A both P-almost surely and in L2 (P; R). (iii) Let Xn , Fn , P be a non-negative martingale, set Yn = e−Xn , n ∈ N, use Corollary 5.2.10 to see that Yn , Fn , P is a uniformly bounded, non-negative, submartingale, and apply part (ii) to conclude that {Xn : n ≥ 0} converges P-almost surely to a non-negative X ∈ L1 (P; R).
Exercises for § 5.2
227
(iv) Let Xn , Fn , P be a martingale for which (5.2.37)
sup EP Xn < ∞. n∈N
± ± Fm ∨0 for n ∈ N. Show that Y ± For each m ∈ N, define Yn,m = EP Xn∨m ≥ n+1,m ± + ± ± Yn,m (a.s., P), define Ym = limn→∞ Yn,m , check that both Ym , Fm , P and Ym− , Fm , P are non-negative martingales with EP Y0+ +Y0− ≤ supn∈N EP |Xn | , and note that Xm = Y m+ − Ym− (a.s., P) for each m ∈ N. In other words, every martingale Xn , Fn , P satisfying (5.2.37 ) admits a Hahn decomposition3 as the difference of two non-negative martingales whose sum has expectation value dominated by the left-hand side of (5.2.37). Finally, use this observation together with (iii) to see that every such martingale converges P-almost surely to some X ∈ L1 (P; R).
(v) By combining the final assertion in (iv) together with Doob’s Decomposition in Lemma 5.2.12, give another proof of the convergence assertion in Theorem 5.2.15. Exercise 5.2.38. In this exercise we will develop another way to reduce Doob’s Martingale Convergence Theorem to the case of L2 -bounded martingales. The technique here is due to R. Gundy and derives from the ideas introduced by Calder´on and Zygmund in connection with their famous work on weak-type 1–1 estimates for singular integrals. measurable, [0, R]-valued (i) Let {Zn : n ∈ N} be a Fn : n ∈ N -progressively , F , P is a submartingale. Next, choose sequence with the property that −Z n n {An : n ∈ N} for −Zn , Fn , P as in Lemma 5.2.12, note that An ’s can be chosen so that 0 ≤ An − An−1 ≤ R for all n ∈ Z+ , and set Mn = Zn + An , n ∈ N. Check that Mn , Fn , P is a non-negative martingale with Mn ≤ (n + 1)R for each n ∈ N. Next, show that 2 = EP Mn − Mn−1 Zn + Zn−1 EP Mn2 − Mn−1 2 + EP An − An−1 Zn + Zn−1 = EP Zn2 − Zn−1 2 + 2R EP An − An−1 , ≤ EP Zn2 − Zn−1 and conclude that EP [A2n ] ≤ EP [Mn2 ] ≤ 3REP [Z0 ] for all n ∈ N. (ii) Let Xn , Fn , P be a non-negative martingale. Show that, for each R ∈ (R) (R) (R) (R) (0, ∞), Xn = Mn − An + ∆n , n ∈ N, where Mn , Fn , P is a non-negative (R) (R) 2 ≤ 3R EP X0 ; An : n ∈ N is a martingale satisfying supn≥0 EP Mn (R)
non-decreasing sequence of random variables with the properties that A0 3
This useful observation was made by Klaus Krickeberg.
≡ 0,
228
5 Conditioning and Martingales
(R) 2 (R) (R) ≤ 3REP X0 ; and ∆n : An is Fn−1 -measurable, and supn≥1 EP An n ∈ N is a Fn : n ∈ N -progressively measurable sequence with the property that 1 P ∃n ∈ N ∆(R) 6= 0 ≤ EP X0 . n R (R)
(R)
(R)
Hint: Set Zn = Xn ∧ R and ∆n = Xn − Zn for n ∈ N, apply part (i) (R) to Zn : n ∈ N , and use Doob’s Inequality to estimate the probability that (R) ∆n 6= 0 for some n ∈ N. (iii) Let Xn , Fn , P be any martingale. Using (ii) above and part (iv) of Exer(R) (R) (R) cise 5.2.36, show that, for each R ∈ (0, ∞), Xn = Mn + Vn + ∆n , n ∈ N, (R) (R) 2 ≤ 12 REP |Xn | ; where Mn , Fn , P is a martingale satisfying EP Mn (R) (R) (R) Vn : n ∈ N is a sequence of random variables satisfying V0 ≡ 0, Vn is Fn−1 -measurable, and !2 n X Vm(R) − V (R) ≤ 12REP |Xn | EP m−1 1
for n ∈ Z+ ; and {∆n : ∈ N} is an sequence satisfying
Fn : n ∈ N -progressively measurable
2 P ∃ 0 ≤ m ≤ n ∆(R) = 6 0 ≤ EP |Xn | . m R
The preceding representation is called on–Zygmund decomposi the Calder´ tion of the martingale Xn , Fn , P . (iv) Let Xn , Fn , P be a martingale that satisfies (5.2.37), and use part (iii) above together with part (i) of Exercise 5.2.36 to show that, for each R ∈ (0, ∞), 2 times the {Xn : n ≥ 0} converges off of a set whose P-measure is no more than R P supremum over n ∈ N of E [|Xn |]. In particular, when combined with Lemma 5.2.12, the preceding line of reasoning leads to the advertised alternate proof of the convergence result in Theorem 5.2.15.
Exercise 5.2.39. In this exercise we will extend Hunt’s Theorem (cf. Theorem 5.2.13) to allow unbounded stopping times. To this end, let Xn , Fn , P be a uniformly P-integrable submartingale on the probability space (Ω, F, P), and set Mn = Xn − An , n ∈ N, where {An : n ∈ N} is the sequence produced in Lemma 5.2.12. After checking that Mn , Fn , P is a uniformly P-integrable martingale, show that, for any stopping time ζ: Xζ = EP [M∞ |Fζ ] + Aζ (a.s., P), where X∞ , M∞ , and A∞ are, respectively, the P-almost sure limits of {Xn : n ≥ 0}, {Mn : n ≥ 0}, and {An : n ≥ 0}. In particular, if ζ and ζ 0 are a pair of stopping times and ζ ≤ ζ 0 , conclude that Xζ ≤ EP [Xζ 0 |Fζ ] (a.s., P).
Exercises for § 5.2
229
Exercise 5.2.40. There are times when submartingales converge even though they are not bounded in L1 (P; R). For example, suppose that (Xn , Fn , P) is a submartingale for which there exists a non-decreasing function ρ : R 7−→ R with the properties that ρ(R) ≥ R for all R and Xn+1 ≤ ρ Xn (a.e., P) for each n ∈ N. (i) Set ζR (ω) = inf n ∈ N : Xn (ω) ≥ R for R ∈ (0, ∞), and note that sup Xn∧ζR ≤ X0 ∨ ρ(R)
(a.e., P).
n∈N
In particular, if X0 is P-integrable, show that {Xn (ω) : n ≥ 0} converges in R for P-almost every ω for which the sequence {Xn (ω) : n ≥ 0} is bounded above. + Hint: After observing that supn∈N EP [Xn∧ζ ] < ∞ for every R ∈ (0, ∞), conR clude that, for each R ∈ (0, ∞), {Xn : n ≥ 0} converges P-almost everywhere on {ζR = ∞}. (ii) Let {Yn : n ≥ 1} be a sequence of mutually independent, P-integrable random variables, assume that EP [Yn ] ≥ 0 for n ∈ N and supn∈N kYn+ kL∞ (P;R) < Pn ∞, and set Sn = 1 Ym . Show that {Sn : n ≥ 0} is either P-almost surely unbounded above or P-almost surely convergent in R. (iii) Let Fn : n ∈ N be a non-decreasing sequence of sub-σ-algebras and An an element of Fn for each n ∈ N. Show that the set of ω ∈ Ω for which either ∞ X
1An (ω) < ∞ but
n=0
or
∞ X
∞ X
P An Fn−1 (ω) = ∞
n=1
1An (ω) = ∞ but
n=0
∞ X
P An Fn−1 (ω) < ∞
n=1
has P-measure 0. In particular, note that this gives another derivation of the second part of the Borel–Cantelli Lemma (cf. Lemma 1.1.3). Exercise 5.2.41. For each n ∈ N, let (En , Bn ) be a measurable space and µn and νn a pair of probability measures on (En , Bn ) with the property that Theorem, Q which says that (cf. Exercise 1.1.14) νn µ Qn . Prove Kakutani’s Q Q either n∈N νn ⊥ n∈N µn or n∈N νn n∈N µn . Hint: Set Y Y Y Y En , F = Bn , P = µn , and Q = νn . Ω= n∈N
n∈N
n∈N
n∈N
Qn ( 0 Bm ), where πn is the natural projection from Ω onto Next, take Fn = Q n 0 Em , set Pn = P Fn and Qn = Q Fn , and note that πn−1
n
Xn (x) ≡
Y dQn (x) = fm (xm ), dPn 0
x ∈ Ω,
230
5 Conditioning and Martingales
dνn . In particular, when νn ∼ µn for each n ∈ N, use Kolwhere fn ≡ dµ n mogorov’s 0–1 Law (cf. Theorem 1.1.2) to see that Q(G) ∈ {0, 1}, where G ≡ limn→∞ Xn ∈ (0, ∞)}, and combine this with the last part of Theorem 5.2.20 to conclude that Q 6⊥ P =⇒ Q P. Finally, to remove the assumption that νn ∼ µn for all n’s, define ν˜n on (En , Bn ) by ν˜n = 1 − 2−n−1 νn + 2−n−1 µn , ˜ ≡Q ˜n , and use the preceding to complete check that ν˜n ∼ µn and Q Q n∈N ν the proof.
Exercise 5.2.42. Let (Ω, F) be a measurable space and Σ a sub-σ-algebra of F. Given a pair of probability measures P and Q on (Ω, F), let XΣ and YΣ be non-negative Radon–Nikodym derivatives of, respectively, PΣ ≡ P Σ and QΣ ≡ Q Σ with respect to PΣ + QΣ , and define P, Q Σ =
Z
1
1
XΣ2 YΣ2 d(P + Q).
(i) Show that if µ is any σ-finite measure on (Ω, Σ) with the property that PΣ µ and QΣ µ, then the number P, Q Σ given above is equal to Z
dPΣ dµ
12
dQΣ dµ
12
dµ.
Also, check that PΣ ⊥ QΣ if and only if P, Q Σ = 0. (ii) Suppose that Fn : n ∈ N is a non-decreasing sequence of sub-σ-algebras of F, and show that (P, Q)Fn −→ (P, Q)W∞ Fn . 0
(iii) Referring to part (ii), assume that Q Fn P Fn for each n ∈ N, let Xn be a non-negative Radon–Nikodym derivative Fn , √ to P W∞of Q Fn with respect W∞ P Xn −→ 0 and show that Q 0 Fn is singular to P 0 Fn if and only if E as n → ∞.
(iv) Let {σn }∞ 0 ⊆ (0, ∞), and, for each n ∈ N, let µn and νn be Gaussian measures on R with variance σn2 . If an and bn are the mean values of, respectively, µn and νn , show that Y
νn ∼
n∈N
depending on whether
Y n∈N
P∞ 0
µn
or
Y n∈N
νn ⊥
Y
µn
n∈N
σn−2 (bn − an )2 converges or diverges.
Exercise 5.2.43. Let {Xn : n ∈ Z+ } be a sequence of identically distributed, mutually independent, integrable, mean value P 0, R-valued random variables on n the probability space (Ω, F, P), and set Sn = 1 Xm for n ∈ Z+ . In Exercise
Exercises for § 5.2
231
1.4.28 we showed that limn→∞ |Sn | < ∞ P-almost surely. Here we will show that
(5.2.44)
lim |Sn | = 0 P-almost surely. n→∞
As was mentioned before, this result was proved first by K.L. Chung and W.H. Fuchs. The basic observation behind the present proof is due to A. Perlin, who noticed that, by the Hewitt–Savage 0–1 Law, limn→∞ |Sn | = L P-almost surely for some L ∈ [0, ∞). Thus, the problem is to show that L = 0, and we will do this by an simple argument invented by A. Yushkevich.
(i) Assuming that L > 0, use the Hewitt–Savage 0–1 Law to show that P |Sn − x| <
L 3
i.o. = 0
for any x ∈ R,
where “i.o.” stands for “infinitely often” and means here “for infinitely many n’s.” Hint: Set ρ = L3 . Begin by observing that, because {Sm+n − Sm : n ∈ Z+ } has the same P-distribution as {Sn : n ∈ Z+ }, P(|Sm+n − Sm | < 2ρ i.o.) = 0 for any m ∈ Z+ . Thus, since |Sm+n − x| ≥ |Sm+n − Sm | − |Sm − x|, P(|Sn − x| < ρ i.o.) ≤ P(|Sm − x| ≥ ρ) for any m ∈ Z+ . Moreover, by the Hewitt–Savage 0–1 Law, P(|Sn − x| < ρ i.o.) ∈ {0, 1}. Hence, either P(|Sn − x| < ρ i.o.) = 0, or one has the contradiction that P(|Sm − x| < ρ) = 0 for all m ∈ Z+ and yet P(|Sn − x| < ρ i.o.) = 1.
(ii) Still assuming that L > 0, argue that P |Sn − L| <
L 3
i.o. ∨ P |Sn + L| <
L 3
i.o. = 1,
which, in view of (i), is a contradiction. Conclude that (5.2.44) holds. (iii) Knowing (5.2.44) and the Hewitt–Savage 0–1 Law, show that, for each x ∈ R and > 0, one has the dichotomy P |Sn − x| < = 0 for all n ≥ 1 or
P |Sn − x| < i.o. = 1.
Exercise 5.2.45. of reversed martingales. Here is a rather frivolous application Let (Ω, F, P), Fn : n ∈ N , and {ek : k ∈ Z be as in part (v) of Exercise 5.1.17. Next, take Sm = {(2k + 1)2m : k ∈ Z} for each m ∈ N, and, for 2 f ∈ L [0, 1); C , set ∆m (f ) =
X `∈Sm
f, e`
e, L2 ([0,1);C) `
232
5 Conditioning and Martingales
where the convergence is in L2 (([0, 1]; C). By Exercise 5.1.17, n X ∆m (f ). f − EP f Fn+1 = m=0
After noting that Fn : n ∈ N is non-increasing, use the convergence result for reversed martingales in Theorem 5.2.21 to see that the expansion f = f, 1
+ L2 ([0,1);C)
∞ X
∆m (f )
m=0
converges both almost everywhere as well as in L2 ([0, 1); C).4
4
When f is a function with the property that (f, e` )L2 ([0,1);C) = 0 for all ` ∈ Z\{2m : m ∈ N}, the preceding almost everywhere convergence result can be interpreted as saying that the Fourier series of f converges almost everywhere, a result that was discovered originally by Kolmogorov. The proof suggested here is based on fading memories of a conversation with N. Varopolous. Of course, ever since L. Carleson’s definitive theorem on the almost every convergence of the Fourier series of an arbitrary square integrable function, the interest in this result of Kolmogorov is mostly historical.
Chapter 6 Some Extensions and Applications of Martingale Theory
Many of the results obtained in § 5.2 admit easy extensions to both infinite measures and Banach space–valued random variables. Furthermore, in many applications, these extensions play a useful, and occasionally essential, role. In the first section of this chapter, I will develop some of these extensions, and in the second section I will show how these extensions can be used to derive Birkhoff’s Individual Ergodic Theorem. The final section is devoted to Burkholder’s Inequality for martingales, an estimate that is second in importance only to Doob’s Inequality. § 6.1 Some Extensions Throughout the discussion that follows, (Ω, F, µ) will be a measure space and Fn : n ∈ N will be a non-decreasing sequence of sub-σ-algebras with the property that µ F0 is σ-finite. In particular, this means that the conditional expectation of a locally µ-integrable random variable given Fn is well defined (cf. Theorem 5.1.12) even if the random variable takes values in a separable Banach space E. Thus, I will say that the sequence Xn ; n ∈ N of E-valued random variables is a µ-martingale with respect to Fn : n ∈ N , or, more briefly, that the triple Xn , Fn , µ is a martingale, if {Xn : n ∈ N} is Fn : n ∈ N progressively measurable, each Xn is locally µ-integrable, and Xn−1 = Eµ Xn Fn−1 (a.e., µ) for each n ∈ Z+ . Furthermore, whenE = R, I will say that {Xn : n ∈ N} is a µ-submartingale with respect to Fn : n ∈ N (equivalently, the triple (Xn , Fn , µ) is a submartingale) if {Xn : n ∈ N} is Fn : n ∈ N -progressively measurable, each Xn is locally µ-integrable, and Xn−1 ≤ Eµ Xn Fn−1 (a.e., µ) for each n ∈ Z+ . § 6.1.1. Martingale Theory for a σ-Finite Measure Space. Without any real effort, I can now prove the following variants of each of the basic results in § 5.2. 233
234
6 Some Extensions and Applications
Theorem 6.1.1. Let Xn , Fn , µ be an R-valued µ-submartingale. Then, for each N ∈ N and A ∈ F0 on which XN is µ-integrable, 1 (6.1.2) µ max Xn ≥ α ∩ A ≤ Eµ XN , max Xn ≥ α ∩ A 0≤n≤N 0≤n≤N α
for all α ∈ (0, ∞); and so, when all the Xn ’s are non-negative, for every p ∈ (1, ∞) and A ∈ F0 , p1 Eµ sup |Xn |p , A ≤
1 p sup Eµ |Xn |p , A p . p − 1 n∈N n∈N Furthermore, for each stopping time ζ, Xn∧ζ , Fn , µ is a submartingale or a martingale depending on whether Xn , Fn , µ is a submartingale or a martingale. In addition, for any pair of bounded stopping times ζ ≤ ζ 0 , Xζ ≤ Eµ Xζ 0 Fζ (a.e., µ),
and the inequality is an equality in the martingale case. Finally, given a < b and A ∈ F0 , Eµ (Xn − a)+ , A µ , E U[a,b] , A ≤ sup b−a n∈N
where U[a,b] (ω) denotes the precise number of times that {Xn (ω) : n ≥ 1} upcrosses [a, b] (cf. the discussion preceding Theorem 5.2.15), and therefore sup Eµ Xn+ , A < ∞ for every A ∈ F0 with µ(A) < ∞ n∈N
=⇒ Xn −→ X
(a.e., µ),
W∞ where X is 0 Fn -measurable and locally µ-integrable. In fact, in the case of W∞ martingales, there is a 0 Fn -measurable, locally µ-integrable X such that Xn = Eµ X Fn (a.e., µ) for all n ∈ N if and only if {Xn : n ≥ 0} is uniformly µ-integrable on each A ∈ F0 with µ(A) < ∞, in which case X is µ-integrable if and only if Xn −→ X in L1 (µ; R). On the other hand, when p ∈ (1, ∞), X ∈ Lp (µ; R) if and only if {Xn : n ≥ 0} is bounded in Lp (µ; R), in which case Xn −→ X in Lp (µ; R). Proof: Obviously, there is no problem unless µ(Ω) = ∞. However, even then, each of these results follows immediately from its counterpart in § 5.2 once one makes the following trivial observation. Namely, given Ω0 ∈ F0 with µ(Ω0 ) ∈ (0, ∞), set F 0 = F[Ω0 ],
Fn0 = Fn [Ω0 ],
Xn0 = Xn Ω0 ,
and P =
µ F0 . µ(Ω0 )
§ 6.1 Some Extensions
235
Then Xn0 , Fn0 , P0 is asubmartingale or a martingale depending on whether the original Xn , Fn , µ was a submartingale or a martingale. Hence, when µ(Ω) = ∞, simply choose aSsequence {Ωk : k ≥ 1} of mutually disjoint, µ-finite ∞ elements of F0 so that Ω = 1 Ωk , work on each Ωk separately, and, at the end, sum the results. I will now spend a little time seeing how Theorem 6.1.1 can be applied to give a simple proof of the Hardy–Littlewood Maximal Inequality. To state their result, define the maximal function Mf for f ∈ L1 (RN ; R) by Z 1 |f (y)| dy, x ∈ RN , Mf (x) = sup Q3x |Q| Q
where Q is used to denote a generic cube (6.1.3)
Q=
N Y
[aj , aj + r)
with a = (a1 , . . . , aN ) ∈ RN and r > 0.
j=1
As is easily checked, Mf : RN −→ [0, ∞] is lower semicontinuous and therefore certainly Borel measurable. Furthermore, if we restrict our attention to nicely meshed families of cubes, then it is easy to relate Mf to martingales. More precisely, for each n ∈ Z, the nth standard dyadic partition of RN is the partition Pn of RN into the cubes N Y ki ki + 1 , k ∈ ZN . , (6.1.4) Cn (k) ≡ n n 2 2 i=1
These partitions are nicely meshed in the sense that the (n + 1)st is a refinement of the nth. Equivalently, if Fn denotes the σ-algebra over RN generated by the partition Pn , then Fn ⊆ Fn+1 . Moreover, if f ∈ L1 (RN ; R) and Z f nN f (y) dy for x ∈ Cn (k) and k ∈ ZN , Xn (x) ≡ 2 Cn (k)
then, for each n ∈ Z, Xnf = EλRN |f | Fn
(a.e., λRN ),
where λRN denotes Lebesgue measure on RN . In particular, for each m ∈ Z, f Xm+n , Fm+n , λRN , n ∈ N, is a non-negative martingale; and so, by applying (6.1.2) for each m ∈ Z and then letting m & −∞, we see that Z n o 1 (0) |f (y)| dy, α ∈ (0, ∞), (6.1.5) x : M f (x) ≥ α ≤ α {M(0) f ≥α}
236
6 Some Extensions and Applications
where
( (0)
M
f (x) = sup
1 |Q|
)
Z |f (y)| dy : x ∈ Q ∈ Q
[
Pn
n∈Z
and I have used |Γ| to denote λRN (Γ), the Lebesgue measure of Γ. At first sight, one might hope that it should be possible to pass directly from (6.1.5) to analogous estimates on the level sets of Mf . However, the passage from (6.1.5) to control on Mf is not as easy as it might appear at first: the “sup” in the definition of Mf involves many more cubes than the one in the definition of M(0) f . For this reason I will have to introduce additional families of meshed partitions. Namely, for each η ∈ {0, 1}N , set (−1)n η N + Cn (k) : k ∈ Z , Pn (η) = 3 × 2n
where Cn (k) is the cube described in (6.1.4). It is then an easy matter to check that, for each η ∈ {0, 1}N , Pn (η) : n ∈ Z is a family of meshed partitions of RN . Furthermore, if ) ( Z [ (η) 1 f (y) dy : x ∈ Q ∈ Pn (η) , x ∈ RN , M f (x) = sup |Q| Q n∈Z
then exactly the same argument that (when η = 0) led us to (6.1.5) can now be used to get Z n o 1 N (η) f (y) dy (*) x ∈ R : M f (x) ≥ α ≤ α {M(η) f ≥α}
for each η ∈ {0, 1}N and α ∈ (0, ∞). Finally, if Q is given by (6.1.3) and r ≤ 3 12n , then it is possible to find an η ∈ {0, 1}N and a C ∈ Pn (η) for which Q ⊆ C. (To see this, first reduce to the case when N = 1.) Hence,
max
η∈{0,1}N
M(η) f ≤ Mf ≤ 6N
max
η∈{0,1}N
M(η) f.
After combining this with the estimate in (*), we arrive at the following version of the Hardy–Littlewood Maximal Inequality: n o (12)N Z |f (y)| dy. (6.1.6) x ∈ RN : Mf (x) ≥ α ≤ α RN
At the same time, (*) implies that max
η∈{0,1}N
(η)
M f p N ≤ L (R ;R)
p kf kLp (RN ;R) , p−1
p ∈ (1, ∞].
§ 6.1 Some Extensions
237
To check this, first note that it suffices to do so when f vanishes outside of the ball B(0, R) for some R > 0. Second, assuming that f = 0 off of B(0, R), observe that (*) implies that Z n o 1 f (y) dy. x ∈ B(0, R) : M(η) f (x) ≥ α ≤ α {M(η)∩B(0,R) f ≥α}
Next, even though the result in Exercise 1.4.18 was stated for probability measures, it applies equally well to any finite measure. Thus, we now know that (η)
kM
! p1
Z
(η)
f kLp (RN ;R) = lim
R→∞
(M
p
f ) (x) dx
B(0,R)
≤
p kf kLp (RN ;R) , p−1
and so we can repeat the argument just made to obtain (6.1.7)
N
Mf p N ≤ (12) p kf kLp (RN ;R) L (R ;R) p−1
for p ∈ (1, ∞].
In this connection, notice that there is no hope of getting this sort of estimate when p = 1, since it is clear that lim |x|N Mf (x) > 0 |x|→∞
whenever f does not vanish λRN -almost everywhere. The inequality in (6.1.6) plays the same role in classical analysis as Doob’s Inequality plays in martingale theory. For example, by essentially the same argument as I used to pass from Doob’s Inequality to Corollary 5.2.4, we obtain the following version of famous Lebesgue Differentiation Theorem. Theorem 6.1.8. For each f ∈ L1 RN ; R), Z 1 f (y) − f (x) dy = 0 lim B&{x} |B| B (6.1.9)
for λRN -almost every x ∈ RN , where, for each x ∈ RN , the limit is taken over balls B that contain x and tend to x in the sense that their radii shrink to 0. In particular, Z 1 f (y) dy for λRN -almost every x ∈ RN . f (x) = lim B&{x} |B| B
Proof: I begin with the observation that, for each f ∈ L1 (RN ; R), Z 1 ˜ f (y) dy ≤ κN Mf (x), x ∈ RN , Mf (x) ≡ sup B3x |B| B
238
6 Some Extensions and Applications
2N with ΩN = B(0, 1) . Second, notice that (6.1.9) for every where κn = Ω N x ∈ RN is trivial when f ∈ Cc (RN ; R). Hence, all that remains is to check that if fn −→ f in L1 (RN ; R) and if (6.1.9) holds for each fn , then it holds for f . To this end, let > 0 be given and check that, because of the preceding and (6.1.6), Z f (y) − f (x) dy ≥ x : lim 1 B&{x} |B| B n o ˜ ≤ x : M(f − fn )(x) ≥ 3 Z 1 fn (y) − fn (x) dy ≥ + x : lim 3 B&{x} |B| B n o + x : fn (x) − f (x) ≥ 3 3 ≤ 1 + (12)N κN kf − fn kL1 (RN )
for every n ∈ Z+ . Hence, after letting n → ∞, we get (6.1.9) f . Although applications like Lebesgue’s Differentiation Theorem might make one think that (6.1.6) is most interesting because of what it says about averages over small cubes, its implications for large cubes are also significant. In fact, as I will show in § 6.2, it allows one to prove Birkhoff’s Individual Ergodic Theorem (cf. Theorem 6.2.7), which may be viewed as a result about differentiation at infinity. The link between ergodic theory and the Hardy–Littlewood Inequality is provided by the following deterministic version of the Maximal Ergodic Lemma (cf. Lemma 6.2.1). Namely, let ak : k ∈ ZN be a summable subset of [0, ∞), and set X 1 aj+k , n ∈ N and k ∈ ZN , S n (k) = N (2n) j∈Qn where Qn = j ∈ ZN : −n ≤ ji < n for 1 ≤ i ≤ N . By applying (6.1.6) and (6.1.7) to the function f given by (cf. (6.1.4)) f (x) = ak when x ∈ C0 (k), we see that (12)N X N ak , α ∈ (0, ∞) (6.1.10) card k ∈ Z : sup S n (k) ≥ α ≤ α n∈Z+ N k∈Z
and
! p1 (6.1.11)
X k∈ZN
sup |S n (k)|p n∈Z+
(12)N p ≤ p−1
! p1 X
|ak |p
for p ∈ (1, ∞].
k∈ZN
The inequality in (6.1.10) is called Hardy’s Inequality. Actually, Hardy worked in one dimension and was drawn to this line of research by his passion
§ 6.1 Some Extensions
239
for the game of cricket. What Hardy wanted to find is the optimal order in which to arrange batters to maximize the average score per inning. Thus, he worked with a non-negative sequence {ak : k ≥ 0} in which ak represented the expected number of runs scored by player k, and what he showed is that, for each α ∈ (0, ∞), k ∈ N : sup S n (k) ≥ α + n∈Z
is maximized when {ak : k ≥ 0} is non-increasing, from which it is an easy application of Markov’s Inequality to prove that ∞ X k ∈ N : sup S n (k) ≥ α ≤ 1 ak , α n∈Z+ 0
α ∈ (0, ∞).
Although this sharpened result can also be obtained as a corollary the Sunrise Lemma,1 Hardy’s approach remains the most appealing. § 6.1.2. Banach Space–Valued Martingales. I turn next to martingales with values in a separable Banach space. Actually, everything except the easiest aspects of this topic becomes extremely complicated and technical very quickly, and, for this reason, I will restrict my attention to those results that do not involve any deep properties of the geometry of Banach spaces. In fact, the only general theory with which I will deal is contained in the following. Theorem 6.1.12. Let E be a separable Banach space and X , F , µ an En n valued martingale. Then kXn kE , Fn , µ is a non-negative submartingale and therefore, for each N ∈ Z+ and all α ∈ (0, ∞), (6.1.13)
µ
sup kXn kE ≥ α
≤
0≤n≤N
1 µ E kXN kE , α
sup kXn kE ≥ α . 0≤n≤N
In particular, for each p ∈ (1, ∞], (6.1.14)
sup kXn kE
n∈N
Lp (µ;E)
≤
p sup kXn kLp (µ;E) . p − 1 n∈N
Finally, if Xn = Eµ [X | Fn ], where X ∈ Lp (µ; E) for some p ∈ [1, ∞), then " ∞ # _ Xn −→ Eµ X Fn both (a.e., µ) and in Lp (µ; E). 0
1
See Lemma 3.4.5 in my A Concise Introduction to the Theory of Integration, Third Edition, Birkhauser (1998).
240
6 Some Extensions and Applications
Proof: The fact kXn kE , Fn , µ is a submartingale is an easy application of the inequality in (5.1.14); and, given this fact, the inequalities in (6.1.13) and (6.1.14) follow from the corresponding inequalities in Theorem 6.1.1. While proving the convergence statement, I may and will assume that F = W∞ p µ 0 Fn . Now let X ∈ L (µ; E) be given, and set Xn = E [X|Fn ], n ∈ N. Because of (6.1.13) and (6.1.14), we know (cf. the proofs of Corollary 5.2.4 and Theorem 6.1.8) that the set of X for which Xn −→ X (a.e., µ) is a closed subset of Lp (µ; E). Moreover, if X is µ-simple, then the µ-almost everywhere convergence of Xn to X follows easily from the R-valued result. Hence, we now know that Xn −→ X (a.s, µ) for each X ∈ L1 (µ; E). In addition, because of (6.1.14), when p ∈ (1, ∞), the convergence in Lp (µ; E) follows by Lebesgue’s Dominated Convergence Theorem. Finally, to prove the convergence in L1 (µ; E) when X ∈ L1 (µ; E), note that, by Fatou’s Lemma, kXkL1 (µ;E) ≤ lim kXn kL1 (µ;E) , n→∞
whereas (5.1.14) guarantees that kXkL1 (µ;E) ≥ lim kXn kL1 (µ;E) . n→∞
Hence, because kXn kE − kXkE − kXn − XkE ≤ 2kXkE , the convergence in L1 (µ; E) is again an application of Lebesgue’s Dominated Convergence Theorem. Going beyond the convergence result in Theorem 6.1.12 to get an analog of Doob’s Martingale Convergence Theorem is hard. For one thing, a na¨ıve analog is not even true for general separable Banach spaces, and a rather deep analysis of the geometry of Banach spaces is required in order to determine exactly when it is true. (See Exercise 6.1.18 for a case in which it is.) Exercises for § 6.1 Exercise 6.1.15. In this exercise we will develop Jensen’s Inequality in the Banach space setting. Thus, (Ω, F, P) will be a probability space, C will be a closed, convex subset of the separable Banach space E, and X will be a C-valued element of L1 (P; E). (i) Show that there exists a sequence {Xn : n ≥ 1} of C-valued, simple functions that tend to X both P-almost surely and in L1 (P; E). (ii) Show that EP [X] ∈ C and that EP g(X) ≤ g EP [X] for every continuous, concave g : C −→ [0, ∞).
Exercises for § 6.1
241
(iii) Given a sub-σ-algebra Σ of F, follow the argument in Corollary 5.2.8 to show that there exists a sequence {Pn }∞ 0 of finite, Σ-measurable partitions with the property that X EP [X, A] 1A −→ EP [X|Σ] P(A)
both P-almost surely and in L1 (P; E).
A∈Pn
In particular, conclude that there is a representative XΣ of EP [X|Σ] that is C-valued and satisfies EP g(X) Σ ≤ g XΣ (a.s., P) for each continuous, convex g : C −→ [0, ∞). Exercise 6.1.16. Again let (Ω, F, P) be a probability space and E be a separable Banach space. Further, suppose that {FTn : n ≥ 0} is a non-increasing se∞ quence of sub-σ-algebras of F, and set F∞ = 0 Fn . Finally, let X ∈ L1 (P; E). (i) Show that EP X Fn −→ EP [X|F∞ ] both P-almost surely and in Lp (P; E) for any p ∈ [1, ∞) with X ∈ Lp (P; E). Hint: Use (6.1.13) and the approximation result in Theorem 5.1.10 to reduce to the case when X is simple. When X is simple, get the result as an application of the convergence result for R-valued, reversed martingales in Theorem 5.2.21. (ii) Using part (i) and following the line of reasoning suggested at the end of § 5.2.4, give a proof of The Strong Law of Large Numbers for Banach space– valued random variables.2 (See Exercises 6.2.18 and 9.1.18 for entirely different approaches.) Exercise 6.1.17. As we saw in the proof of Theorem 6.1.8, the Hardy– Littlewood maximal function can be used to dominate other quantities of interest. As a further indication of its importance, I will use it in this exercise to prove the analog of Theorem 6.1.8 for a large class of approximate identities. R That is, let ψ ∈ L1 (RN ; R) with RN ψ(x) dx = 1 be given, and set ψt (x) = t−N ψ xt , t ∈ (0, ∞) and x ∈ RN .
Then {ψt : t > 0} forms an approximate identity in the sense that, as tempered distributions, ψt −→ δ0 as t & 0. In fact, because kψt ? f kLp (RN ;R) ≤ kψkL1 (RN ;R) kf kLp (RN ;R) , 2
t ∈ (0, ∞) and p ∈ [1, ∞],
This proof, which seems to have been the first, of the Strong Law for Banach spaces was given by E. Mourier in “El´ ements al´ eatoires dans un espace de Banach,” Ann. Inst. Poincar´ e 13, pp. 166–244 (1953).
242
6 Some Extensions and Applications
and
Z ψ(y) f (x − ty) dy,
ψt ? f (x) = RN
it is easy to see that, for each p ∈ [1, ∞),
lim ψt ? f − f Lp (RN ;R) = 0
t&0
first for f ∈ Cc (RN ; R) and then for all f ∈ Lp (RN ; R). The purpose of this exercise is to sharpen the preceding under the assumption that ψ(x) = α |x| ,
x ∈ RN \ {0} for some α ∈ C 1 (0, ∞); R with Z A≡ rN |α0 (r)| dr < ∞. (0,∞)
Notice that when α is non-negative and non-increasing, integration by parts shows that A = N . (i) Let f ∈ Cc (RN ; R) be given, and set f˜(r, x) =
1 |B(x, r)|
Z f (y) dy
for r ∈ (0, ∞) and x ∈ RN .
B(x,r)
Using integration by parts and the given hypotheses, show that ψt ? f (x) =
− N1
Z
rN α0 (r) f˜(tr, x) dr,
(0,∞)
and conclude that ψt ? f (x) ≤
A N
˜ (x), Mf
˜ is the quantity introduced at the beginning of the proof of Theorem where Mf 6.1.8. In particular, conclude that there is a constant KN ∈ (0, ∞), depending only on N ∈ Z+ , such that Mψ f (x) ≡ sup ψt ? f (x) ≤ KN A Mf (x),
x ∈ RN .
t∈(0,∞)
(ii) Starting from the conclusion in (i), show that (12)N KN Akf kL1 (RN ) {x : Mψ f (x) ≥ R} ≤ , R
f ∈ L1 (RN ; R),
Exercises for § 6.1
243
and that for p ∈ (1, ∞], N
Mψ f p N ≤ (12) KN A p kf kLp (RN ;R) , f ∈ Lp (RN ; R). L (R ;R) p−1 Finally, proceeding as in the proof of Theorem 6.1.8, use the first of these to prove that, for f ∈ L1 (RN ; R) and Lebesgue almost every x ∈ RN , lim ψt ? f (x) − f (x) t&0
Z
ψt (y) f (x − y) − f (x) dy = 0.
≤ lim
t&0
RN
Two of the most familiar examples to which the preceding applies are the 2 N Gauss kernel gt (x) = (2πt)− 2 exp − |x|2 and the Poisson kernel (cf. (3.3.19)) N ΠR t . In both these cases, A = N .
Exercise 6.1.18. Let E be a separable Hilbert space and (Xn , F, P) an Evalued martingale on some probability space (Ω, F, P) satisfying the condition sup EP kXn k2E < ∞. n∈Z+
W∞ Proceeding as in (i) of Exercise 5.2.36, first prove that there is a 1 Fn -measurable X ∈ L2 (P; E) to which {Xn : n ≥ 1} converges in L2 (P; E), next check that Xn = EP X Fn (a.s., P) for each n ∈ Z+ , and finally apply the last part of Theorem 6.1.12 to see that Xn −→ X P-almost surely. Exercise 6.1.19. This exercise deals with a variation, proposed by Jan Mycielski, on the sort of search algorithm discussed in § 5.2.5. Let G be a non-empty, bounded, open subset of RN with the property that λRN B(x, r) ∩ G ≥ αΩN rd for some α > 0 and all x ∈ G and 0 < r ≤ diam(G), and define µ on (G, BG ) λ (Γ∩G) by µ(Γ) = RλNN (G) . Next, let (Ω, F, P) be a probability space on which there R exists sequences {Xn : n ≥ 1} and {Zn : n ≥ 1} of G-valued random variables with the properties that the Xn ’s are mutually independent and have distribution µ, Zn is independent of {X 1 , . . . , Xn } and has distribution νn µ for each
n < ∞ for some r ∈ (1, ∞). Without loss n ≥ 1, and Kr ≡ supn≥1 dν dµ r L (µ;R)
in generality, assume that n 6= n0 =⇒ Xn (ω) 6= Xn0 (ω) for all ω ∈ Ω. For each n ≥ 1, let Yn (ω) be the last element of {X1 (ω), . . . , Xn (ω)} which is closest to Zn (ω). That is, if Σn is the permutation group on {1, . . . , n} and, for π ∈ Σn , An (π) = ω : |Xπ(m) (ω) − Zn (ω)| < |Xπ(m−1) (ω) − Zn (ω)| : for 2 ≤ m ≤ n , then Yn = Zπ(n) on An (π). Show that for all Borel measurable f : G −→ R, |f (Yn ) − f (Zn )| −→ 0 in P-probability. Here are some steps that you might want to follow.
244
6 Some Extensions and Applications
(i) Given f ∈ L1 (µ; R), show that ) ( Z 1 |f | dµ ≤ α−1 Mf (x) MG f (x) ≡ sup |B(x, r) ∩ G| B(x,r)∩G r>0
and therefore that there is a C < ∞ such that kMG f kLp (µ;R) ≤ for all p ∈ (1, ∞].
Cp p p−1 kf kL (µ;R)
(ii) Given n ≥ 1 and z ∈ G, set An (z) = ω : |Xm (ω) − z| < |Xm−1 (ω) − z| : for 2 ≤ m ≤ n , and show that E f (Yn ) = n! P
Z
EP f (Xn ), An (z) νn (dz).
Next, for n ≥ 2, set rn (ω) = |Xn−1 (ω) − z|, and show that "Z # P P E f (Xn ), An (z) = E f dµ, An−1 (z) ≤ MG f (z)P An (z) , B(z,rn )
and conclude from this that E f (Yn ) ≤ P
Z MG f dνn .
(iii) Given the conclusion drawn at the end of (ii), proceed as in the derivation of Theorem 5.2.34 from Lemma 5.2.31 to get the desired result. § 6.2 Elements of Ergodic Theory Among the two or three most important general results about dynamical systems is D. Birkhoff’s Individual Ergodic Theorem. In this section, I will present a generalization, due to N. Wiener, of Birkhoff’s basic theorem. The setting in which I will prove the Ergodic Theorem will be the following. (Ω, be a σ-finite measure space on which there exits a semigroup kF, µ) will N Σ : k ∈ N of measurable, µ-measure preserving transformations. That is, for each k ∈ NN , Σk is an F-measurable map from Ω into itself, Σ0 is the identity map, Σk+` = Σk ◦ Σ` for all k, ` ∈ NN , and µ(Γ) = µ (Σk )−1 (Γ) for all k ∈ N and Γ ∈ F. Further, E will be a separable Banach space with norm k · kE , and, given a function F : Ω −→ E, I will be considering the averages 1 X F ◦ Σk (ω), n ∈ Z+ , An F (ω) ≡ N n + k∈Qn
N where Q+ : kkk∞ < n and kkk∞ ≡ max1≤j≤N kj . My n is the cube k ∈ N goal (cf. Theorem 6.2.7) is to show that, for each p ∈ [1, ∞) and F ∈ Lp (µ; E), {An F : n ≥ 1} converges µ-almost everywhere. In fact, when either µ is finite or p ∈ (1, ∞), I will show that the convergence is also in Lp (µ; E).
§ 6.2 Elements of Ergodic Theory
245
§ 6.2.1. The Maximal Ergodic Lemma. Because he was thinking in terms of dynamical systems and therefore did not take full advantage of measure theory, Birkhoff’s own proof of his theorem is rather cumbersome. Later, F. Riesz discovered a proof which has become the model for all later proofs. Specifically, he introduced what is now called the Maximal Ergodic Inequality, which is an inequality that plays the same role here that Doob’s Inequality played in the derivation of Corollary 5.2.4. In order to cover Wiener’s extension of Birkhoff’s theorem, I will derive a multiparameter version of the Maximal Ergodic Inequality, which, as the proof shows, is really just a clever application of Hardy’s Inequality.1 Lemma 6.2.1 (Maximal Ergodic Lemma). For each n ∈ Z+ and p ∈ [1, ∞], An is a contraction on Lp (µ; E). Moreover, for each F ∈ Lp (µ; E), (6.2.2)
(24)N kF kL1 (µ;E) , µ sup kAn F kE ≥ λ ≤ λ n≥1
λ ∈ (0, ∞),
or
sup kAn F kE
n≥1
(6.2.3)
≤
Lp (µ)
(24)N p kF kLp (µ;E) , p−1
depending on whether p = 1 or p ∈ (1, ∞). Proof: First observe that, because kAn F kE ≤ An kF kE , it suffices to prove all of these assertions in the case when E = R and F is non-negative. Thus, I will restrict myself to this case. Since F ◦ Σk has the same distribution as F itself, the first assertion is trivial. To prove (6.2.2) and (6.2.3), let n ∈ Z+ be given, apply (6.1.10) and (6.1.11) to ak (ω) ≡
F ◦ Σk (ω) if k ∈ Q+ 2n if k ∈ / Q+ 2n ,
0
and conclude that n o + k Cn (ω) ≡ card k ∈ Qn : max Am F ◦ Σ (ω) ≥ λ 1≤m≤n
N
≤
1
(12) λ
X
F ◦ Σk (ω)
k∈Q+ 2n
The idea of using Hardy’s Inequality was suggested to P. Hartman by J. von Neumann and appears for the first time in Hartman’s “On the ergodic theorem,” Am. J. Math. 69, pp. 193–199 (1947).
246
6 Some Extensions and Applications
and X k∈Q+ n
max
1≤m≤n
k
Am F ◦ Σ (ω)
p
≤
(12)N p p−1
p X
F ◦ Σk (ω)
p
.
k∈Q+ 2n
Hence, by Tonelli’s Theorem,
X
k
max Am F ◦ Σ
µ
≥λ
1≤m≤n
k∈Q+ n
Z =
Cn (ω) µ(dω)
Z (12)N X F ◦ Σk f dµ ≤ λ + k∈Q2n
and, similarly, X Z k∈Q+ n
max
1≤m≤n
Am F ◦ Σ
k
p
dµ ≤
(12)N p p−1
p X Z
F ◦ Σk
p
dµ.
k∈Q+ 2n
Finally, since the distributions of max1≤m≤n Am F ◦ Σk and F ◦ Σk do not depend on k ∈ NN , the preceding lead immediately to µ
max Am F ≥ λ
1≤m≤n
and
max Am F
1≤m≤n
Lp (µ)
≤
(24)N kF kL1 (µ) λ
N
2 p (12)N p kF kLp (µ) ≤ p−1
for all n ∈ Z+ . Thus, (6.2.2) and (6.2.3) follow after one lets n → ∞. Given (6.2.2) and (6.2.3), I adopt again the strategy used in the proof of Corollary 5.2.4. That is, I must begin by finding a dense subset of each Lp -space on that the desired convergence results can be checked by hand, and for this purpose I will have to introduce the notion of invariance. A set Γ ∈ F is said to be invariant, and I write Γ ∈ I if Γ = (Σk )−1 (Γ) for every k ∈ NN . As is easily checked, I is a sub-σ-algebra of F. In addition, it is clear that Γ ∈ F is invariant if Γ = (Σej )−1 (Γ) for each 1 ≤ j ≤ N , where {ei : 1 ≤ i ≤ N } is the standard orthonormal basis in RN . Finally, if I is the µ-completion of I relative to F in the sense that Γ ∈ I if and only if Γ ∈ F and ˜ ∈ I such that µ(Γ∆Γ) ˜ = 0 (A∆B ≡ (A\B)∪(B \A) is the symmetric there is Γ difference between the sets A and B), then an F-measurable F : Ω −→ E is I-measurable if and only if F = F ◦ Σk (a.e., µ) for each k ∈ NN . Indeed, one
§ 6.2 Elements of Ergodic Theory
247
need only check this equivalence for indicator functions of sets. But if Γ ∈ F ˜ = 0 for some Γ ˜ ∈ I, then and µ(Γ∆Γ) ˜ + µ(Γ∆Γ) ˜ = 0, µ Γ∆(Σk )−1 (Γ) ≤ µ (Σk )−1 (Γ∆Γ) and so Γ ∈ I. Conversely, if Γ ∈ I, set [ ˜= Γ (Σk )−1 (Γ), k∈NN
˜ ∈ I and µ(Γ∆Γ) ˜ = 0. and check that Γ Lemma 6.2.4. Let I(E) be the subspace of I-measurable elements of L2 (µ; E). Then, I(E) is a closed linear subspace of L2 (µ; E). Moreover, if ΠI(R) denotes orthogonal projection from L2 (µ; R) onto I(R), then there exists a unique linear contraction ΠI(E) : L2 (µ; E) −→ I(E) with the property that ΠI(E) (af ) = aΠI(R) f for a ∈ E and f ∈ L2 (µ; R). Finally, for each F ∈ L2 (µ; E),
(6.2.5)
An F −→ ΠI(E) F
(a.e., µ) and in L2 (µ; E).
Proof: I begin with the case when E = R. The first step is to identify the orthogonal complement I(R)⊥ of I(R). To this end, let N denote the subspace of L2 (µ; R) consisting of elements having the form g − g ◦ Σej for some g ∈ L2 (µ; R) ∩ L∞ (µ; R) and 1 ≤ j ≤ N . Given f ∈ I(R), observe that f, g − g ◦ Σej L2 (µ;R) = f, g L2 (µ;R) − f ◦ Σej , g ◦ Σej L2 (µ;R) = 0. Hence, N ⊆ I(R)⊥ . On the other hand, if f ∈ L2 (µ; R) and f ⊥ N , then it is clear that f ⊥ f − f ◦ Σej for each 1 ≤ j ≤ N and therefore that
f − f ◦ Σej 2 2 L (µ;R)
2 2 = kf kL2 (µ;R) − 2 f, f ◦ Σej L2 (µ;R) + f ◦ Σej L2 (µ;R) = 2 kf k2L2 (µ;R) − f, f ◦ Σej L2 (µ;R) = 2 f, f − f ◦ Σej L2 (µ;R) = 0. Thus, for each 1 ≤ j ≤ N , f = f ◦ Σej µ-almost everywhere; and, by induction on kkk∞ , one concludes that f = f ◦ Σk µ-almost everywhere for all k ∈ NN . In other words, we have now shown that I(R) = N ⊥ or, equivalently, that N = I(R)⊥ . Continuing with E = R, next note that if f ∈ I(R), then An f = f (a.e., µ) for each n ∈ Z+ . Hence, (6.2.5) is completely trivial in this case. On the other hand, if g ∈ L2 (µ; R) ∩ L∞ (µ; R) and f = g − g ◦ Σej , then X X nN An f = g ◦ Σk − g ◦ Σk+ej , {k∈Q+ n :kj =0}
{k∈Q+ n :kj =n−1}
248
6 Some Extensions and Applications
and so, with p ∈ {2, ∞},
2kgkLp (µ;R)
An f p −→ 0 ≤ L (µ;R) n
as n → ∞.
Hence, in this case also, (6.2.5) is easy. Finally, to complete the proof for E = R, simply note that, by (6.2.3) with p = 2 and E = R, the set of f ∈ L2 (µ; R) for which (6.2.5) holds is a closed linear subspace of L2 (µ; R) and that we have already verified (6.2.5) for f ∈ I(R) and f from a dense subspace of I(R)⊥ . Turning to general E’s, first note that ΠI(E) F is well defined for µ-simple F ’s. P` Indeed, if F = 1 ai 1Γi for some {ai : 1 ≤ i ≤ `} ⊆ E and {Γi : 1 ≤ i ≤ `} of mutually disjoint elements of F with finite µ-measure, then ΠI(E) F =
` X
ai ΠI(R) 1Γi
1
and so
ΠI(E) F 2 2 ≤ L (µ;E)
Z
` X
!2 kai kE ΠI(R) 1Γi
dµ
1
= ΠI(R)
` X 1
! 2
kai kE 1Γi
2
L (µ;R)
≤
` X
kai k2E µ(Γi ) = kF k2L2 (µ;E) .
1
Thus, since the space of µ-simple functions is dense in L2 (µ; E), it is clear that ΠI(E) not only exists but is also unique. Finally, to check (6.2.5) for general E’s, note that (6.2.5) for E-valued, µsimple F ’s is an immediate consequence of (6.2.5) for E = R. Thus, we already know (6.2.5) for a dense subspace of L2 (µ; E), and so the rest is another elementary application of (6.2.3). § 6.2.2. Birkhoff ’s Ergodic Theorem. For any p ∈ [1, ∞), let Ip (E) denote the subspace of I-measurable elements of Lp (µ; E). Clearly Ip (E) is closed for every p ∈ [1, ∞). Moreover, since (6.2.6) µ(Ω) < ∞ =⇒ ΠI(E) F = Eµ F I ,
when µ is finite ΠI(E) extends automatically as a linear contraction from Lp (µ; E) onto Ip (E) for each p ∈ [1, ∞), the extension being given by the right-hand side of (6.2.6). However, when µ(E) = ∞, there is a problem. Namely, because µ I will seldom be σ-finite, it will not be possible to condition µ with respect to I. Be that as it may, (6.2.5) provides an extension of ΠI(E) . Namely, from (6.2.5) and Fatou’s Lemma, it is clear that, for each p ∈ [1, ∞),
ΠI(E) F p ≤ kF kLp (µ;E) , F ∈ Lp (µ; E) ∩ L2 (µ; E), L (µ;E) and therefore the desired existence of the extension follows by continuity.
§ 6.2 Elements of Ergodic Theory
249
Theorem 6.2.7 (Birkhoff ’s Individual Ergodic Theorem). For each p ∈ [1, ∞) and F ∈ Lp (µ; E), (6.2.8)
An F −→ ΠI(E) F
(a.e., µ).
Moreover, if either p ∈ (1, ∞) or p = 1 and µ(Ω) < ∞, then the convergence in (6.2.8) is also in Lp (µ; E). Finally, if µ(Γ) ∧ µ(Γ{) = 0 for every Γ ∈ I, then (6.2.8) can be replaced by Eµ [F ] µ(Ω) lim An F = n→∞ 0
if µ(Ω) ∈ (0, ∞) (a.e., µ), if µ(Ω) = ∞
and the convergence is in Lp (µ; E) when either p ∈ (1, ∞) or p = 1 and µ(Ω) < ∞. Proof: As I said above, the proof is now an easy application of the strategy used to prove Corollary 5.2.4. Namely, by (6.2.2), the set of F ∈ L1 (µ; E) for which (6.2.8) holds is closed and, by (6.2.5), it includes L1 (µ; E) ∩ L∞ (µ; E). Hence, (6.2.8) is proved for p = 1. On the other hand, when p ∈ (1, ∞), (6.2.3) applies and shows first that the set of F ∈ Lp (µ; E) for which (6.2.8) holds is closed in Lp (µ; E) and second that µ-almost everywhere convergence already implies convergence in Lp (µ; E). Hence, we have proved that (6.2.8) holds and that the convergence is in Lp (µ; E) when p ∈ (1, ∞). In addition, when µ(Γ) ∧ µ(Γ{) = 0 for all Γ ∈ I, it is clear that the only elements of Ip (E) are µ-almost everywhere constant, which, in the case when µ(Ω) < ∞, means (cf. µ [F ] , and, when µ(Ω) = ∞, means that Ip (E) = {0} (6.2.6)) that ΠI(E) F = Eµ(Ω) for all p ∈ [1, ∞). In view of the preceding, all that remains is to discuss the L1 (µ; E) convergence in the case when p = 1 and µ(Ω) < ∞. To this end, observe that, because the An ’s are all contractions in L1 (µ; E), it suffices to prove L1 (µ; E) convergence for E-valued, µ-simple F ’s. But L1 (µ; E) convergence for such F ’s reduces to showing that An f −→ ΠI(R) f in L1 (µ; R) for non-negative f ∈ L∞ (µ; R). Finally, if f ∈ L1 µ; [0, ∞) , then
An f kL1 (µ) = kf kL1 (µ) = ΠI(R) f kL1 (µ;R) ,
n ∈ Z+ ,
where, in the last equality, I used (6.2.6); and this, together with (6.2.8), implies (cf. the final step in the proof of Theorem 6.1.12) convergence in L1 (µ). I will say that semigroup Σk : k ∈ NN is ergodic on (Ω, F, µ) if, in addition to being µ-measure preserving, µ(Γ) ∧ µ(Γ{) = 0 for every invariant Γ ∈ I.
250
6 Some Extensions and Applications
Classic Example. In order to get a feeling for what the Ergodic Theorem is saying, take µ to be Lebesgue measure on the interval [0, 1) and, for a given α ∈ (0, 1), define Σα : [0, 1) −→ [0, 1) so that Σα (ω) ≡ ω + α − [ω + α] = ω + α mod 1. If α is rational and m is the smallest element of Z+ with the property that mα ∈ Z+ , then it is clear that, for any F on [0, 1), F ◦ Σα = F if and only if F 1 . Hence, if F ∈ L2 [0, 1); C and has period m Z √ c` (F ) ≡ F (ω)e− −1 2π`ω dω, ` ∈ Z, [0,1)
then elementary Fourier analysis leads to the conclusion that, in this case, √ X lim An F (ω) = cm` (F )e −1 2m`πω for Lebesgue-almost every ω ∈ [0, 1). n→∞
`∈Z
On the other hand, if α is irrational, then Σkα : k ∈ N} is µ-ergodic on [0, 1). To see this, suppose that F ∈ I(C). Then (cf. the preceding and use Parseval’s Identity) X
2 c` (F ) − c` (F ◦ Σα ) 2 . 0 = F − F ◦ Σα L2 ([0,1);C) = `∈Z
But, clearly,
√
c` (F ◦ Σα ) = e
−1 2π`α
c` (F ),
` ∈ Z,
and so (because α is irrational) c` (F ) = 0 for each ` 6= 0. In other words, the only elements of I(C) are µ-almost everywhere constant. Thus, for each irrational α ∈ (0, 1), p ∈ [1, ∞), separable Banach space E, and F ∈ Lp [0, 1); E , Z lim An F = F (ω) dω Lebesgue-almost everywhere and in Lp (µ; E). n→∞
[0,1)
Finally, notice that the situation changes radically when one moves from [0, 1) to [0, ∞) and again takes µ to be Lebesgue measure and α ∈ (0, 1) to be irrational. If I extend the definition of Σα by taking Σα (ω) = bωc + Σα (ω − bωc) for ω ∈ [0, ∞), then it is clear that invariant functions are those that are constant on each R bωc+1 interval [m, m+1) and that, Lebesgue-almost surely, An f (ω) −→ bωc f (η) dη. On the other hand, if one defines Σα (ω) = ω + α, then every invariant set that has non-zero measure will have infinite measure, and so, now, every choice of α ∈ (0, 1) (not just irrational ones) will give rise to an ergodic system. In particular, one will have, for each p ∈ [1, ∞) and F ∈ Lp (µ; E), lim An F = 0
n→∞
Lebesgue-almost everywhere,
and the convergence will be in Lp (µ; E) when p ∈ (1, ∞).
§ 6.2 Elements of Ergodic Theory
251
§ 6.2.3. Stationary Sequences. For applications to probability theory, it is useful to reformulate these considerations in terms of stationary families of random variables. Thus, let (Ω, F, P) be a probability space and (E, B) be a measurable space (E need not be a Banach space). Given a family F = {Xk : k ∈ NN } of E-valued random variables on (Ω, F, P), I will say that F is Pstationary (or simply stationary) if, for each ` ∈ NN , the family F` ≡ Xk+` : k ∈ NN has the same (joint) distribution under P as F itself. Clearly, one can test for stationarity by checking that the distribution of Fej is the same as that of F for each 1 ≤ j ≤ N . In order to apply the considerations of § 6.2.1 to stationary families, note that all questions about the properties of F can be phrased in N terms of the following canonical setting . Namely, set E = E N and define µ N N on E, B N to be the image measure F∗ P. In other words, for each Γ ∈ B N , µ(Γ) = P F ∈ Γ . Next, for each ` ∈ NN , define Σ` : E −→ E to be the natural shift transformation on E given by Σ` (x)k = xk+` for all k ∈ NN . Obviously, stationarity of F is equivalent to the statement that {Σk : k ∈ NN } is µ-measure N preserving. Moreover, if I is the σ-algebra of shift invariant elements Γ ∈ B N −1 (i.e., Γ = Σk (Γ) for all k ∈ NN ), then, by Theorem 6.2.7, for any separable Banach space B, any p ∈ [1, ∞), and any F ∈ Lp (P; B), h i 1 X F ◦ Fk = EP F ◦ F F−1 (I) (a.s., P) and in Lp (P; B). lim N n→∞ n + k∈Qn
N In particular, when Σk : k ∈ NN is ergodic on E, B N µ , I will say that the family F is ergodic and conclude that the preceding can be replaced by 1 X F ◦ Fk = EP F ◦ F (a.s., P) and in Lp (P; B). (6.2.9) lim N n→∞ n + k∈Qn
So far I have discussed one-sided stationary families, that is, families indexed by NN . However, for various reasons (cf. Theorem 6.2.11) it is useful to know that one can usually embed a one-sided stationary family into a two-sided one. In terms of the semigroup of shifts, to the trivial observation that k this corresponds N NN the semigroup Σ : k ∈ N on E = E can be viewed as a sub-semigroup ˆ = E ZN . With these comments in of the group of shifts Σk : k ∈ ZN on E mind, I will prove the following. Lemma 6.2.10. Assume that E is a complete, separable, metric space and that F = {Xk : k ∈ NN } is a stationary family of E-valued random variables on the ˆ and ˆ F, ˆ P) probability space (Ω, F, P). Then there exists a probability space (Ω, N N ˆ ˆ a family F = Xk : k ∈ Z with the property that, for each ` ∈ Z , ˆ` ≡ X ˆ k+` : k ∈ NN F ˆ as F has under P. has the same distribution under P
252
6 Some Extensions and Applications
Proof: When formulated correctly, this theorem is an essentially trivial application of Kolmogorov’s Extension Theorem (cf. part (iii) of Exercise 9.1.17). Namely, for n ∈ N, set Λn = k ∈ ZN : kj ≥ −n for 1 ≤ j ≤ N , and define Φn : E Λ0 −→ E Λn so that Φn (x)k = xn+k
for x ∈ E Λ0 and k ∈ Λn , where n ≡ (n, . . . , n)
Next, take µ0 on E Λ0 to be the P-distribution of F and, for n ≥ 1, µn on E Λn to be (Φn )∗ µ0 . Using stationarity, one can easily check that, for each n ≥ 0 and k ∈ NN , µn is invariant under the obvious extension of Σk to E Λn . In particular, if one identifies E Λn+1 with E Λn+1 \Λn × E Λn , then µn+1 E Λn+1 \Λn × Γ = µn (Γ)
for all Γ ∈ BE Λn .
Hence the µn ’s are consistently defined on the spaces E Λn , and therefore Kolmogorov’s Extension Theorem applies and guarantees the existence of a unique N Borel probability measure µ on E Z with the property that N
µ EZ
\Λn
× Γ = µn (Γ)
for all n ≥ 0 and Γ ∈ BE Λn .
Moreover, since each µn is Σk -invariant for all k ∈ NN , it is clear that µ is also. N Thus, because Σk is invertible on E Z and Σ−k is its inverse, it follows that µ is invariant under Σk for all k ∈ ZN . ˆ = µ, ˆ = E ZN , Fˆ = B ˆ , P To complete the proof at this point, simply take Ω Ω ˆ k (ˆ and X ω) = ω ˆ k for k ∈ ZN . As an example of the advantage that Lemma 6.2.10 affords, I present the following beautiful observation made originally by M. Kac. Theorem 6.2.11. Let (E, B) be a measurable space and {Xk : k ∈ N} a stationary sequence of E-valued random variables on the probability space (Ω, F, P). Given Γ ∈ B, define the return time ρΓ (ω) = inf{k ≥ 1 : Xk (ω) ∈ Γ}. Then, EP ρΓ , X0 ∈ Γ = P Xk ∈ Γ for some k ∈ N . In particular, if {Xk : k ∈ N} is ergodic, then P X0 ∈ Γ > 0 =⇒ EP ρΓ , X0 ∈ Γ = 1. Proof: Set Uk = 1Γ ◦Xk for k ∈ N. Then {Uk : k ∈ N} is a stationary sequence of {0, 1}-valued random variables. Hence, by Lemma 6.2.10, we can find a probˆ on which there is a family {U ˆ F, ˆ P ˆk : k ∈ Z} of {0, 1}-valued ability space Ω,
§ 6.2 Elements of Ergodic Theory
253
ˆn , . . . , U ˆn+k , . . . random variables with the property that, for every n ∈ Z, U ˆ as (U0 , . . . , Uk , . . . ) has under P. In particular, has the same distribution under P ˆ U ˆ0 = 1 and P ρΓ ≥ 1, X0 ∈ Γ = P ˆ U ˆ−n = 1, U ˆ−n+1 = 0, . . . , U ˆ0 = 0 , P ρΓ ≥ n + 1, X0 ∈ Γ = P
n ∈ Z+ .
Thus, if λΓ (ˆ ω ) ≡ inf k ∈ N : U−k (ˆ ω) = 1 , then ˆ λΓ = n − 1 , P ρΓ ≥ n, X0 ∈ Γ = P
n ∈ Z+ ,
and so ˆ λΓ < ∞ . EP ρΓ , X0 ∈ Γ = P Now observe that ˆ λΓ > n = P ˆ U ˆ−n = 0, . . . , U ˆ0 = 0 = P X0 ∈ P / Γ, . . . , Xn ∈ /Γ , from which it is clear that ˆ λΓ < ∞ = P ∃k ∈ N Xk ∈ Γ . P Finally, assume that P∞{Xk : k ∈ N} is ergodic and that P(X0 ∈ Γ) > 0. Because, by (6.2.9), 0 1Γ Xk = ∞ P-almost surely, it follows that, P-almost surely, Xk ∈ Γ for some k ∈ N. It should be noticed that, although there are far more elementary proofs, when {Xn : n ≥ 0} is an irreducible, ergodic Markov chain on a countable state space E, then Kac’s theorem proves that the stationary measure at the state x ∈ E is the reciprocal of the expected time that the chain takes to return to x when it starts at x. § 6.2.4. Continuous Parameter Ergodic Theory. I turn now to the setting of continuously parametrized semigroups Thus, again of transformations. (Ω, F, µ) is a σ-finite measure space and Σt : t ∈ [0, ∞)N is a measurable semigroup of µ-measure preserving transformations on Ω. That is, Σ0 is the identity, Σs+t = Σs ◦ Σt , (t, ω) ∈ [0, ∞)N × Ω 7−→ Σt (ω) ∈ Ω is B[0,∞)N × F-measurable, and Σt ∗ µ = µ for every t ∈ [0, ∞)N . Next, given an F-measurable F with values in some separable Banach space E, let G(F ) be the set of ω ∈ Ω with the property that Z
F ◦ Σt (ω) dt < ∞ for all T ∈ (0, ∞). E [0,T )N
254
6 Some Extensions and Applications
Clearly, ω ∈ G(F ) =⇒ Σt (ω) ∈ G(F )
for every t ∈ [0, ∞)N .
In addition, if F ∈ Lp (µ; E) for some p ∈ [1, ∞), then ! Z Z
F ◦ Σt (ω) p dt µ(dω) = T N kF kp p Ω
and so F ∈
L (µ;E)
E
[0,T )N
[
< ∞,
Lp (µ; E) =⇒ µ G(F ){ = 0.
p∈[1,∞)
Next, for each T ∈ (0, ∞), define ( −N R T F ◦ Σt (ω) dt [0,T )N AT F (ω) = 0
if ω ∈ G(F ) if ω ∈ / G(F ),
and note that, as a consequence of the invariance of G(F ), AT F ◦ Σt = AT F ◦ Σt for all t ∈ [0, ∞)N . ˆ to denote the σ-algebra of Γ ∈ F with the property that Γ = Finally, use I t −1 (Σ ) (Γ) for each t ∈ [0, ∞)N , and say that Σt : t ∈ [0, ∞)N is ergodic if ˆ µ(Γ) ∧ µ(Γ{) = 0 for every Γ ∈ I. t Theorem 6.2.12. Let (Ω, F, µ) be a σ-finite measure space and Σ : t ∈ [0, ∞)N be a measurable semigroup of µ-measure preserving transformations on Ω. Then, for each separable Banach space E, p ∈ [1, ∞), and T ∈ (0, ∞), AT is a contraction on Lp (µ; E). Next, set ΠI(E) = ΠI(E) ◦ A1 , where ΠI(E) ˆ k N is defined in terms of Σ : k ∈ N as in Theorem 6.2.7. Then, for each p ∈ [1, ∞) and F ∈ Lp (µ; E), lim AT F = ΠI(E) F ˆ
(6.2.13)
T →∞
(a.e., µ).
Moreover, if p ∈ (1, ∞) or p = 1 and µ(Ω) < ∞, then the convergence is also in Lp (µ; E). In fact, if µ(Ω) < ∞, then ˆ (a.e., µ) and in Lp (µ : E). lim AT F = Eµ F I T →∞
Finally, if Σt : t ∈ [0, ∞)N is ergodic, then (6.2.13) can be replaced by lim AT F =
T →∞
Eµ [F ] µ(Ω)
(a.e., µ),
where it is understood that the ratio is 0 when the denominator is infinite.
§ 6.2 Elements of Ergodic Theory Proof: The first step is the observation that
(24)N kF kL1 (µ;E) , (6.2.14) µ sup AT F E ≥ λ ≤ λ T >0
255
λ ∈ (0, ∞)
and
sup AT F
T >0 E
(6.2.15)
≤
Lp (µ;E)
(24)N p kF kLp (µ;E) p−1
for p ∈ (1, ∞).
Indeed, because of (AT F ) ◦ Σt = AT (F ◦ Σt ), (6.2.14) is derived from (6.1.6) in precisely the same way as I derived (6.2.2) from (6.1.10), and (6.2.15) comes from (6.1.7) just as (6.2.3) came from (6.1.7). Given (6.2.14) and (6.2.15), we know that it suffices to prove (6.2.13) for a dense subset of L1 (µ; E). Thus, let F be a uniformly bounded element of L1 (µ; E) and set Fˆ = A1 F . Because Z
N
T AT F (ω) − nN An Fˆ (ω) ≤
F ◦ Σt (ω)kE dt E [0,n+1)N \[0,n)N
for n ≤ T ≤ n + 1,
lim
sup n→∞
n≤T ≤n+1
AT F − An Fˆ E
=0
for every p ∈ [1, ∞].
Lp (µ;R)
Hence, for F ∈ L1 (µ; E)∩L∞ (µ; E), (6.2.13) follows from (6.2.8). As for the case when µ(Ω) < ∞, all that we have to do is check that ΠI(E) F = Eµ F ˆ I (a.e., µ). ˆ However, from (6.2.13), it is easy to see that ΠI(E) F is measurable with respect ˆ ˆ to the µ-completion of I, and so it suffices to show that ˆ Eµ F, Γ = Eµ A1 F, Γ for all Γ ∈ I. ˆ then But, if Γ ∈ I, µ
E A1 F, Γ =
Z
Eµ F ◦ Σt , Γ dt
[0,1)N
Z =
−1 Eµ F ◦ Σt , Σt (Γ) dt = Eµ [F, Γ].
[0,1)N
Finally, assume that Σt : t ∈ [0, ∞)N is µ-ergodic. When µ(Ω) < ∞, the asserted result follows immediately from the preceding; and when µ(Ω) = ∞, it follows from the fact that ΠI(E) F is measurable with respect to the µ-completion ˆ ˆ of I.
256
6 Some Extensions and Applications Exercises for § 6.2
Exercise 6.2.16. Given an irrational α ∈ (0, 1) and an ∈ (0, 1), let Nn (α, ) be the number of 1 ≤ m ≤ n with the property that α − ` ≤ for some ` ∈ Z. m 2m
As an application of the considerations in the Classic Example given at the end of § 6.1, show that Nn (α, ) ≥ . lim n n→∞ Hint: Let δ ∈ 0, 2 be given, take f equal to the indicator function of [0, δ) ∪ Pn (1 − δ, 1), and observe that Nn (α, ) ≥ k=1 f ◦ Σkα (ω) so long as 0 ≤ ω ≤ 2 − δ. Exercise 6.2.17. Assume that µ(Ω) < ∞ and that Σk : k ∈ NN is ergodic. Given a non-negative F-measurable function f , show that
lim An f < ∞ on a set of positive µ-measure =⇒ f ∈ L1 (µ; R)
n→∞
Eµ [f ] (a.e., µ). n→∞ µ(Ω) Exercise 6.2.18. Let F = Xk : k ∈ NN be a stationary family of random variables on the probability space (Ω, F, P) with values in the measurable space NN (E, B), and let I denote the σ-algebra of shift invariant Γ ∈ BE . =⇒ lim An f =
(i) Take T ≡
\
σ Xk : kj ≥ n for all 1 ≤ j ≤ N ,
n≥0
N the tail σ-algebra determined by X : k ∈ N . Show that F−1 (I) ⊆ T , and k conclude that Xk : k ∈ NN is ergodic if T is P-trivial (i.e., P(Γ) ∈ {0, 1} for all Γ ∈ T ). (ii) By combining (i), Kolmogorov’s 0–1 Law, and the Individual Ergodic Theorem, give another derivation of The Strong Law of Large Numbers for independent, identically distributed, integrable random variables with values in a separable Banach space. Exercise 6.2.19. Let Xk : k ∈ N be a stationary, ergodic sequence of Rvalued, integrable random variables on (Ω, F, P). Using the reasoning suggested in Exercise 1.4.28, prove Guivarc’h’s lemma: n−1 X Xk < ∞. EP X1 = 0 =⇒ lim n→∞ k=0
§ 6.3 Burkholder’s Inequality
257
§ 6.3 Burkholder’s Inequality Given a martingale Xn , Fn , P with X0 = 0 and a sequence {σn : n ≥ 0} of bounded functions with the property that σn is Fn -measurable for n ≥ 0, determine {Yn : n ≥ 0} byY0 = 0 and Yn − Yn−1 = σn−1 (Xn − Xn−1 ) for n ≥ 1. It is clear that Yn , Fn , P is again a martingale. In addition, if the absolute values of all the σn ’s are bounded by some constant σ < ∞ and Xn is square P-integrable, then one can easily check that n n X X EP Yn2 = EP σn2 (Xn − Xn−1 )2 ≤ σ 2 EP (Xn − Xn−1 )2 = σ 2 EP Xn2 . m=1
m=1
On the other hand, it is not at all clear how to compare the size of Yn to that of Xn in any of the Lp spaces other than p = 2. The problem of finding such a comparison was given a definitive solution by D. Burkholder, and I will present his solution in this section. Actually, Burkholder solved the problem twice. His first solution was a beautiful adaptation of general ideas and results that had been developed over the years to solve related problems in probability theory and analysis and, as such, did not yield the optimal solution. His second approach is designed specifically to address the problem at hand and bears little or no resemblance to familiar techniques. It is entirely original, remarkably elementary and effective, but somewhat opaque. The approach is the outgrowth of many years of deep thinking that Burkholder devoted to the topic, and the reader who wants to understand the path that led him to it should consult the explanation that he wrote.1 § 6.3.1. Burkholder’s Comparison Theorem. Burkholder’s basic result is the following comparison theorem. Theorem 6.3.1 (Burkholder). Let Ω, F, P be a probability space, Fn : n ∈ N a non-decreasing sequence of sub-σ-algebras of F, and E and F a pair of (real or complex) separable Hilbert spaces. Next, suppose that Xn , Fn , P and Yn , Fn , P are, respectively, E- and F -valued martingales. If kY0 kF ≤ kX0 kE and kYn − Yn−1 kF ≤ kXn − Xn−1 kE , n ∈ Z+ , P-almost surely, then, for each p ∈ (1, ∞) and n ∈ N, (6.3.2)
Yn p ≤ Bp Xn Lp (P;E) , L (P;F )
where Bp ≡ (p − 1) ∨
1 . p−1
As I said before, the derivation of Theorem 6.3.1 is both elementary and mysterious. I begin with the trivial observation that, without loss in generality, 1
For those who want to know the secret behind this proof, Burkholder has revealed it in his article “Explorations in martingale theory and its applications” for the 1989 Saint-Flour Ecole d’Et´ e lectures published by Springer-Verlag, LNM 1464 (1991).
258
6 Some Extensions and Applications
I may assume that both E and F are complex Hilbert spaces, since we can always complexify them, and, in addition, that E = F , since, if that is not already the case, I can embed them in E ⊕ F . Thus, I will be making these assumptions throughout. The heart of the proof lies in the computations contained in the following two lemmas. Lemma 6.3.3. Let p ∈ (1, ∞) be given, set αp =
p2−p (p − 1)p−1 2−p
if p ∈ [2, ∞) if p ∈ (1, 2],
p
and define u : E 2 −→ R by (cf. (6.3.2)) u(x, y) = kykE − Bp kxkE Then kykpE − Bp kxkE
p
kykE + kxkE
≤ αp u(x, y),
p−1
.
(x, y) ∈ E 2 .
Proof: When p = 2, there is nothing to do. Thus, I will assume that p ∈ (1, ∞) \ {2}. Observe that it suffices to show that, for all (x, y) ∈ E 2 satisfying kxkE + kykE = 1, depending on whether p ∈ (2, ∞) or p ∈ (1, 2), p ≤ p2−p (p − 1)p−1 kykE − (p − 1)kxkE p (*) kykE − (p − 1)kxkE ≥ p2−p (p − 1)p−1 kykE − (p − 1)kxkE . Indeed, when p ∈ (2, ∞), (*) is precisely the result desired, and, when p ∈ (1, 2), (*) gives the desired result after one divides through by (p − 1)p and reverses the roles of x and y. I begin the verification of (*) by checking that 2−p
(**)
p
(p − 1)
p−1
>1
if p ∈ (2, ∞)
<1
if p ∈ (1, 2).
To this end, set f (p) = (p − 1) log(p − 1) − (p − 2) log p for p ∈ (1, ∞). Then f is strictly convex on (1, 2) and strictly concave on (2, ∞). Thus, f (1, 2) cannot achieve a maximum and, therefore, since limp&1 f (p) = 0 = f (2), f < 0 on (1, 2). Similarly, f (2, ∞) cannot achieve a minimum and, therefore, since f (2) = 0 while limp%∞ f (p) = ∞, we have that f > 0 on (2, ∞). Next, observe that proving (*) comes down to checking that, for s ∈ [0, 1], Φ(s) ≡ p
2−p
(p − 1)
p−1
p
p p
(1 − ps) − (1 − s) + (p − 1) s
≥0 ≤0
if p ∈ (2, ∞) if p ∈ (1, 2).
§ 6.3 Burkholder’s Inequality
259
To this end, note that, by (**), Φ(0) > 0 when p ∈ (2, ∞) and Φ(0) < 0 when p ∈ (1, 2). Also, for s ∈ (0, 1), h i Φ0 (s) = p (p − 1)p sp−1 + (1 − s)p−1 − p2−p (p − 1)p−1 and
h i Φ00 (s) = p(p − 1) (p − 1)p sp−2 − (1 − s)p−2 . In particular, we see that Φ p1 = Φ0 p1 = 0. In addition, depending on whether p ∈ (2, ∞) or p ∈ (1, 2), lims&0 Φ00 (s) is negative or positive, Φ00 is strictly increasing or decreasing on (0, 1), and lims%1 Φ00 (1) is positive or negative. Hence, there exists a unique t = tp ∈ (0, 1) with the property that < 0 if p ∈ (2, ∞) > 0 if p ∈ (2, ∞) Φ00 (0, t) and Φ00 (t, 1) > 0 if p ∈ (1, 2) < 0 if p ∈ (1, 2. Moreover, because Φ00 (t) = 0, it is easy to see that t ∈ 0, p1 . Now suppose that p ∈ (2, ∞) and consider Φ on each of the intervals p1 , 1 , 1 t, p , and 0, t separately. Because both Φ and Φ0 vanish at p1 while Φ00 > 0 on p1 , 1 , it is clear that Φ > 0 on p1 , 1 . Next, because Φ0 p1 = 0 and Φ00 t, p1 > 0, we know that Φ is strictly decreasing on t, p1 and therefore that Φ t, p1 > Φ p1 = 0. Finally, because Φ00 (0, t) < 0 while Φ(0) ∧ Φ(t) ≥ 0, we also know that Φ (0, t) > 0. The argument when p ∈ (1, 2) is similar, only this time all the signs are reversed.
Lemma 6.3.4. Again let p ∈ (1, ∞) be given, and define u : E × F −→ R as in Lemma 6.3.3. In addition, define the functions v and w on E 2 \ {0, 0} by p−2 v(x, y) = p kykE + kxkE kykE + (2 − p)kxkE and w(x, y) = p(1 − p) kykE + kxkE
p−2
kxkE .
Then, for (x, y) ∈ E 2 and (k, h) ∈ E 2 satisfying min ky + thkE ∧ kx + tkkE > 0 and khkE ≤ kkkE , t∈[0,1]
one has u(x + k, y + h) − u(x, y) ≤ v(x, y) Re
y kykF
x , k , h + w(x, y) Re kxk E F
E
when p ∈ [2, ∞) and y x , k , h −v(y, x) Re (p−1) u(x+k, y+h)−u(x, y) ≤ −w(y, x) Re kyk kxkE E E
when p ∈ (1, 2].
E
260
6 Some Extensions and Applications
Proof: Set Φ(t) = Φ t; (x, k), (y, h) ≡ ky + thkE − (p − 1)kx + tkkE
kx + tkkE + ky + thkE
p−1
,
and observe that (
u x + tk, y + th =
Φ t; (x, k), (y, h) −(p − 1)
−1
if p ∈ [2, ∞)
Φ t; (y, h), (x, k)
if p ∈ (1, 2).
Hence, it suffices for us to check that y+th x+tk , k , h + w(x + tk, y + th)Re Φ0 (t) = v(x + tk, y + th)Re ky+thk kx+tkkE E E
E
and prove that Φ00 t; (x, k), (y, h)
≤0
if p ∈ [2, ∞) and khkE ≤ kkkE
≥0
if p ∈ (1, 2] and khkE ≥ kkkE .
To prove the preceding, = y + th, Ψ(t) = kx(t)kE + set x(t) = x + tk, y(t) ky(t)kE , a(t) =
Re x(t),k
kx(t)kE
E
, and b(t) =
Re y(t),h
ky(t)kE
E
. One then has that
h i Φ0 (t) = pΨ(t)p−2 (1 − p)kx(t)kE a(t) + ky(t)kE + (2 − p)kx(t)kE b(t) h i = p (1 − p)Ψ(t)p−2 kx(t)kE a(t) + b(t) + Ψ(t)p−1 b(t) . In particular, the first expression establishes the required form for Φ0 (t). In addition, from the second expression, we see that −
2 Φ00 (t) = (p − 1)(p − 2) Ψ(t)p−3 kx(t)kE a(t) + b(t) p i h 2 2 E + (p − 1)Ψ(t)p−2 a(t) a(t) + b(t) + kx(t)k ky(t)kE b⊥ (t) + a⊥ (t) i h b⊥ (t)2 − Ψ(t)p−2 (p − 1) a(t) + b(t) b(t) + Ψ(t) ky(t)k E
2 = (p − 1)(p − 2) Ψ(t)p−3 kx(t)kE a(t) + b(t) + (p − 1)Ψ(t)p−2 kkk2E − khk2E + (p − 2)Ψ(t)p−1
b⊥ (t)2 ky(t)kE ,
p p where a⊥ (t) = kkk2E − a(t)2 and b⊥ (t) = khk2E − b(t)2 . Hence the required properties of Φ00 (t) have also been established.
§ 6.3 Burkholder’s Inequality
261
Proof of Theorem 6.3.1: Set Kn = Xn − Xn−1 and Hn = Yn − Yn−1 for n ∈ Z+ . I will assume that there is an > 0 with the property that
X0 (ω) − span{Kn (ω) : n ∈ Z+ } ≥ E and
Y0 (ω) − span{Hn (ω) : n ∈ Z+ } ≥ E for all ω ∈ Ω. Indeed, if this is not already the case, then I can replace E by R × E (or, when E is complex, C × E) and Xn (ω) and Yn (ω), respectively, by Xn() (ω) ≡ , Xn (ω) and Yn() (ω) ≡ , Yn (ω) , ()
()
for each n ∈ N. Clearly, (6.3.2) for each Xn and Yn implies (6.3.2) for Xn and Yn after one lets & 0. Finally, because there is nothing to do when the right-hand side of (6.3.2) is infinite, let p ∈ (1, ∞) be given, and assume that Xn ∈ Lp (P; E) for each n ∈ N. In particular, if u is the function defined in Lemma 6.3.3 and v and w are those defined in Lemma 6.3.4, then u(Xn , Yn ) ∈ L1 (P; R)
0
and v(Xn , Yn ), w(Xn , Yn ) ∈ Lp (P; R)
p is the H¨older conjugate of p. for all n ∈ N, where p0 = p−1 Note that, by Lemma 6.3.3, it suffices for us to show that An ≡ EP u Xn , Yn ≤ 0, n ∈ N. Since u X0 , Y0 ) ≤ 0 P-almost surely, there is no question that A0 ≤ 0. Next, assume that An ≤ 0, and, depending on whether p ∈ [2, ∞) or p ∈ (1, 2], use the appropriate part of Lemma 6.3.4 to see that i h An+1 ≤EP v(Xn , Yn )Re kYYnnkE , Hn+1 E i h P Xn + E w(Xn , Yn )Re kXn kE , Kn+1
E
or
i h An+1 ≤ − EP w(Yn , Xn )Re kYYnnkE , Hn+1 E i h P Xn . − E v(Yn , Xn )Re kXn kE , Kn+1 E
v(Xn , Yn ) kYYnnkE
But, since (cf. Exercise 5.1.18)
is Fn -measurable, E [Hn+1 |Fn ] = 0, and therefore P
i h = 0. EP v(Xn , Yn )Re kYYnnkE , Hn+1 E
Since the same reasoning shows that each of the other terms on the right-hand side vanishes, we have now proved that An+1 ≤ 0. As an immediate consequence of Theorem (6.3.2), we have the following answer to the question raised at the beginning of this section.
262
6 Some Extensions and Applications
Corollary 6.3.5. Suppose that (Xn , Fn , P) is a martingale with values in a separable (real or complex) Hilbert space E. Further, let F be a second separable, complex Hilbert space, and suppose that {σn : n ≥ 0} is a sequence of Hom(E; F )-valued random variables with the properties that σ0 is constant, σn is Fn -measurable for n ≥ 1, and kσn kop ≤ σ < ∞ (a.s., P) for some constant σ < ∞ and all n ∈ N. If kY0 kF ≤ σkX0 kE and Yn − Yn−1 = σn−1 (Xn − Xn−1 ) for n ≥ 1, then (Yn , Fn , P) is an F -valued martingale and, for each p ∈ (1, ∞), (cf. (6.3.2)) kYn kLp (P;F ) ≤ σBp kXn kLp (P;E) , n ∈ N. § 6.3.2. Burkholder’s Inequality. In many applications, the most useful form of Burkholder’s result is as a generalization to p 6= 2 of the obvious equality " n # X 2 P 2 E |Xn − X0 | = E |Xm − Xm−1 | . P
m=1
This is the form of his inequality which is best known and, as such, is called Burkholder’s Inequality. Notice that his inequality can be viewed as a vast generalization of Khinchine’s Inequality (2.3.27), although it applies only when p ∈ (1, ∞). Theorem 6.3.6 (Burkholder’s Inequality). Let Ω, F, P and Fn : n ∈ N be as in Theorem 6.3.1, and let Xn , Fn , P be a martingale with values in the separable Hilbert space E. Then, for each p ∈ (1, ∞),
(6.3.7)
1 sup Xn − X0 Lp (P;E) Bp n∈N ! p2 p1 ∞ X
Xn − Xn−1 2 ≤ EP E 1
≤ Bp sup Xn − X0 Lp (P;E) , n∈N
with Bp as in (6.3.2). Proof: Let F = `2 (N; E) be the separable Hilbert space of sequences y = x0 , . . . , xn , . . . ∈ E N satisfying kykF ≡
∞ X 0
! 12 kxn k2E
< ∞,
Exercises for § 6.3
263
and define Yn (ω) = (X0 (ω), X1 (ω) − X0 (ω), . . . , Xn (ω) − Xn−1 (ω), 0, 0, . . . ) ∈ F for ω ∈ Ω and n ∈ N. Obviously, Yn , Fn , P is an F -valued martingale. Moreover, kX0 kE = kY0 kF
and kXn − Xn−1 kE = kYn − Yn−1 kF ,
n ∈ N,
and therefore the right-hand side of (6.3.7) is implied by (6.3.2) while the lefthand side also follows from (6.3.2) when the roles of the Xn ’s and Yn ’s are reversed. Exercises for § 6.3 Exercise 6.3.8. Because it arises repeatedly in the theory of stochastic integration, one of the most frequent applications of Burkholder’s Inequality is to situations in which E is a separable Hilbert space and (Xn , Fn , P) is an E-valued martingale for which one has an estimate of the form
h 1 i 2p
P 2p
<∞ Kp ≡ sup E kXn − Xn−1 kE Fm−1
∞ + n∈Z
L
(P;R)
for some p ∈ [1, ∞). To see how such an estimate gets used, let F be a second separable Hilbert space and suppose that {σn : n ∈ N} is a sequence of Hom(E; F )-valued random variables with the properties that, for each n ∈ N, 1 2p < ∞. Set Y0 = 0 and σn is Fn -measurable and an ≡ EP kσn k2p op
Yn =
n X
σm−1 Xm − Xm−1
for n ∈ Z+ ,
m=1
and show that
1
Yn 2p ≤ (2p − 1)n 2 Kp L (P;F )
n−1 1 X 2p a n m=0 m
1 ! 2p
.
Exercise 6.3.9. Return to the setting in Exercise 5.2.45, and let λ[0,1) denote Lebesgue measure on [0, 1). Given f ∈ L2 (λ[0,1) ; C), show that, for each p ∈ (1, ∞),
1
f − (f, 1)L2 (λ ;C) p (p − 1)∧ [0,1) L ([0,1);C) p−1 ! p p1 Z ∞ X 2 2 ∆m (f ) dt ≤ [0,1)
m=0
≤ (p − 1) ∨
1
f − (f, 1)L2 (λ ;C) p . [0,1) L (λ[0,1) ;C) p−1
264
6 Some Extensions and Applications
For functions f with (f, e` )L2 (λ[0,1) ;C) = 0 unless ` = ±2m for some m ∈ N, this estimate is a case of a famous theorem proved by Littlewood and Paley in order to generalize Parseval’s Identity to cover p 6= 2. Unfortunately, the argument here is far too weak to give their inequality for general f ’s. Exercise 6.3.10. In connection with the preceding exercise, it is interesting to note that there is an orthonormal basis for L2 λ[0,1) ; R that, as distinguished from the trigonometric functions, can be nearly completely understood in terms of martingale analysis. Namely, recall the Rademacher functions {Rn : n ∈ Z+ } introduced in § 1.1.2. Next, use F to denote the set of all finite subsets F of Z+ , and define the Walsh function WF for F ∈ F by 1 if F = ∅ WF = Q m∈F Rm if F 6= ∅. Finally, set A0 = ∅ and An = {1, . . . , n} for n ∈ Z+ . (i) For each n ∈ N, let Fn be the σ-algebra generated by the partition k k+1 : 0 ≤ k < 2n . 2n , 2n Show that, for each n ∈ Z+ , WF : F ⊆ An is an orthonormal basis for the subspace L2 [0, 1), Fn , λ[0,1) ; R , and conclude from this that WF : F ∈ F forms an orthonormal basis for L2 λ[0,1) ; R . (ii) Let f ∈ L1 [0, 1); R be given, and set ! X Z f Xn = f (t) WF (t) dt WF for n ∈ N. F ⊆An
[0,1)
Using the result in (i), show that Xnf = Eλ[0,1) [f |Fn ] and therefore that Xnf , Fn , λ[0,1) is a martingale. In particular, Xnf −→ f both (a.e., λ[0,1) ) as well as in 1 L λ[0,1) ; R . (iii) Show that for each p ∈ (1, ∞) and f ∈ L1 λ[0,1) ; R with mean value 0, (p − 1) ∧ (p − 1)−1 kf kLp ([0,1);R) Z ∞ X X ≤ [0,1)
n=1
F ⊆An \An−1
!
Z
f (s) WF (s) ds [0,1)
p1 2 p2 WF (t) dt
≤ (p − 1) ∨ (p − 1)−1 kf kLp ([0,1);R) . Exercise 6.3.11. Although Burkholder’s Inequality is extremely useful, it does not give particularly good estimates in the case of martingales with bounded increments. For such martingales, the following line of reasoning, which was introduced by J. Azema in his thesis, is useful.
Exercises for § 6.3
265
(i) For any a ∈ R and x ∈ [−1, 1], show that eax ≤
1 + x a 1 − x −a e = cosh a + x sinh a. e + 2 2
(ii) Suppose that {Y1 , . . . , Yn } are [−1, 1]-valued random variables on the probability space (Ω, F, P) with the property that, for each 1 ≤ m ≤ n, EP Yj1 · · · Yjm = 0
for all 1 ≤ j1 < · · · < jm ≤ n.
Show that, for any {aj }n1 ⊆ R,
n n n X X Y 1 a2j , aj Yj ≤ cosh aj ≤ exp EP exp 2 j=1 j=1 j=1
and conclude that ! n 2 X R , P aj Yj ≥ R ≤ exp − Pn 2 j=1 a2j j=1
R ∈ [0, ∞).
(iii) Suppose that (Xn , Fn , P) is a bounded martingale with X0 ≡ 0, and set Dn ≡ kXn − Xn−1 kL∞ (P) . Show that R2 P (Xn ≥ R) ≤ exp − Pn 2 j=1 Dj2
! ,
R ∈ [0, ∞).
Chapter 7 Continuous Parameter Martingales
It turns out that many of the ideas and results introduced in § 5.2 can be easily transferred to the setting of processes depending on a continuous parameter. In addition, the resulting theory is intimately connected with L´evy processes, and particularly Brownian. In this chapter, I will give a brief introduction to this topic and some of the techniques to which it leads.1 § 7.1 Continuous Parameter Martingales There is a huge number of annoying technicalities which have to be addressed in order to give a mathematically correct description of the continuous time theory of martingales. Fortunately, for the applications which I will give here, I can keep them to a minimum. § 7.1.1. Progressively Measurable Functions. Let (Ω, F) be a measurable space and Ft : t ∈ [0, ∞) a non-decreasing family of sub-σ-algebras. I will say that a function X on [0, ∞)×Ω into a measurable space (E, B) is progressively measurable with respect to Ft : t ∈ [0, ∞) if X [0, T ] × Ω is B[0,T ] × FT measurable for every T ∈ [0, ∞). When E is a metric space, I will say that X : [0, ∞) × Ω −→ E is right-continuous if X(s, ω) = limt&s X(t, ω) for every (s, ω) ∈ [0, ∞) × Ω and will say that it is continuous if X( · , ω) is continuous for all ω ∈ Ω. Remark 7.1.1. The reader might have been expecting a slightly different definition of progressive measurability here. Namely, he might have thought that one would say that X is Ft : t ∈ [0, ∞) -progressively measurable if it is B[0,∞) × F-measurable and ω ∈ Ω 7−→ X(t, ω) ∈ E is Ft -measurable for each t ∈ [0, ∞). Indeed, in extrapolating from the discrete parameter setting, this would be the first definition at which one would arrive. In fact, it was the notion with which Doob and Itˆo originally worked; and such functions were said by them to be adapted to Ft : t ∈ [0, ∞) . However, it came to be realized that there are various problems with the notion of adaptedness. For example, even if X is adapted and f : E −→ R is a bounded, B-measurable function, the 1
A far more thorough treatment can be found in D. Revuz and M. Yor’s treatise Continuous Martingales and Brownian Motion, Springer-Verlag, Grundlehren der Mathematishen #293 (1999).
266
§ 7.1 Continuous Parameter Martingales
267
Rt function (t, ω) Y (t, ω) ≡ 0 f X(s, ω) ds ∈ R need not be adapted. On the other hand, if X is progressively measurable, then Y will be also. The following simple lemma should help to explain the virtue of progressive measurability and its relationship to adaptedness. Lemma 7.1.2. Let PM denote the set of A ⊆ [0, ∞) × Ω with the property that [0, t] × Ω ∩ A ∈ B[0,t] × Ft for every t ≥ 0. Then PM is a sub-σ-algebra of B[0,∞) ×F and X is progressively measurable if and only if it is PM-measurable. Furthermore, if E is a separable metric space and X : [0, ∞) × Ω −→ E is a right-continuous function, then X is progressively measurable if it is adapted. Proof: Checking that PM is a σ-algebra is easy. Furthermore, for any X : [0, ∞) × Ω −→ E, T ∈ [0, ∞), and Γ ∈ B, (t, ω) ∈ [0, T ] × Ω : X(t, ω) ∈ Γ = [0, T ] × Ω ∩ (t, ω) ∈ [0, ∞) × Ω : X(t, ω) ∈ Γ}, and so X is Ft : t ∈ [0, ∞) -progressively measurable if and only if it is PMmeasurable. Hence, the first assertion has been proved. Next, suppose that X is a right-continuous, adapted function. To see that X is progressively measurable, let t ∈ [0, ∞) be given, and define n Xnt (τ, ω) = X [2 2τn]+1 ∧ t, ω , for (τ, ω) ∈ [0, ∞) × Ω and n ∈ N.
Obviously, Xnt is B[0,t] ×Ft -measurable for every n ∈ N and Xnt (τ, ω) −→ X(τ, ω) as n → ∞ for every (τ, ω) ∈ [0, t] × Ω. Hence, X [0, t] × Ω is B[0,t] × Ft measurable, and so X is progressively measurable. § 7.1.2. Martingales: Definition and Examples. Given a probability space (Ω, F, P ) and a non-decreasing family of sub-σ-algebras Ft : t ∈ [0, ∞) , I will say with respect to that X : [0, ∞) × Ω −→ (−∞, ∞] is a submartingale Ft : t ∈ [0, ∞) or, equivalently, that X(t), Ft , P is a submartingale if X is a right-continuous, progressively measurable function with the properties that X(t)− is P-integrable for every t ∈ [0, ∞) and X(s) ≤ EP X(t) Fs (a.s., P) for all 0 ≤ s ≤ t < ∞. When both X(t), Ft , P and −X(t), Ft , P are submartingales, I will say either that X is a martingale with respect to Ft : t ∈ [0, ∞) or simply that X(t), Ft , P is a martingale. Finally, if Z : [0, ∞) × Ω −→ C is a rightcontinuous, progressively measurable function, then Z(t), Ft , P is said to be a (complex) martingale if both Re Z(t), Ft , P and Im Z(t), Ft , P are. The next two results show that L´evy processes provide a rich source of continuous parameter martingales.
268
7 Continuous Parameter Martingales
Theorem 7.1.3. Let µ ∈ I(RN ) with µ ˆ(ξ) = e`µ (ξ) , where `µ (ξ) equals √
−1 ξ, m RN − ξ, Cξ RN +
Z
√
e
−1(ξ,y)RN
− 1 − 1[0,1] (|y|) ξ, y
RN
RN
M (dy).
If (Ω, F, P) is a probability space and Z : [0, ∞) × Ω −→ RN is a B[0,∞) × Fmeasurable map with the properties that Z(0, ω) = 0 and Z( · , ω) ∈ D(RN ) for every ω ∈ Ω, then {Z(t) : t ≥ 0} is a L´evy process for µ if and only if, for each x ∈ RN ,
(7.1.4)
exp
√
−1(ξ, Z(t))RN − t`µ (ξ) , Ft , P is a martingale,
where Ft = σ {Z(τ ) : τ ∈ [0, t]} . Proof: If {Z(t) : t ≥ 0} is a L´evy process for µ, then, because Z(t) − Z(s) is independent of Fs and has characteristic function e(t−s)`µ (ξ) , h √ EP exp −1 ξ, Z(t) RN − t`µ (ξ)
= exp
√
= exp
√
−1 ξ, Z(s)
−1 ξ, Z(s)
RN
RN
i Fs
h √ −1 − s`µ (ξ) e(s−t)`µ (ξ) EP e − s`µ (ξ) .
i
ξ,Z(t)−Z(s)
RN
To prove the converse assertion, observe that the defining distributional property of a L´evy process for µ can be summarized as the statement that Z(0, ω) = 0 and, for each 0 ≤ s < t, Z(t) − Z(s) is independent of σ {Z(τ ) : τ ∈ [0, t]} and has distribution µt−s , where µ cτ = eτ `µ . Hence, since (7.1.4) implies that h √ i EP exp −1 ξ, Z(t) − Z(s) RN Fs = e(t−s)`µ (ξ) ,
ξ ∈ RN ,
there is nothing more to do. Another, and often more useful, way to capture the same result is to introduce the L´ evy operator Lµ ϕ(x) = (7.1.5)
1 Trace C∇2 ϕ(x) + m, ∇ϕ(x) RN 2 Z h i + ϕ(x + y) − ϕ(x) − 1[0,1] (|y|) y, ∇ϕ(x) RN M (dy) RN
for ϕ ∈ Cb2 (RN ; C).
§ 7.1 Continuous Parameter Martingales
269
Theorem 7.1.6. Assume that µ ∈ I(RN ) and that {Z(t) : t ≥ 0} is a L´evy 1,2 N process for µ. Then, for every F ∈ Cb [0, ∞) × R ; C , Z t F t, Z(t) − ∂τ + Lµ F τ, Z(τ ) dτ, Ft , P 0
is a martingale, where Ft = σ {Z(τ ) : τ ∈ [0, t]} and Lµ is the operator described in (7.1.5). Conversely, if Z is a progressively measurable function satisfying Z(0, ω) = 0 and Z( · , ω) ∈ D(RN ) for each ω ∈ Θ, and if Z t ϕ Z(t) − Lµ ϕ Z(τ ) dτ, Ft , P 0
is a martingale for each ϕ ∈ Cc∞ (RN ; R), then {Z(t) : t ≥ 0} is a L´evy process for µ. Proof: Begin by noting that it suffices to handle the case when F is the restriction to [0, ∞) × RN of an element of the Schwartz test function space S (R × RN ; C). Indeed, because kLµ ϕku ≤ CkϕkCb2 (RN ;C) for some C < ∞, the result for F ∈ Cb1,2 [0, ∞) × RN ; C follows, via an obvious approximation procedure, from the result for F ∈ S (R × RN ; C). Next observe that it suffices to treat F ∈ S (RN ; C). To see this, simply interpret the process t ∈ [0, ∞) 7−→ t, Zµ (t) ∈ RN +1 as a L´evy process for δ1 × µ. Now let ϕ ∈ S (RN ; C) be given. The key to proving the required result is the identity d ϕ?µ ˘t = (Lµ ϕ) ? µ ˘t , dt
(*)
where µ ˘t is the distribution of −x under µt , the measure determined by µbt = et`µ . The easiest way to check (*) is to work via Fourier transform and to use (3.2.10) to verify that d\ t`µ (−ξ) µ ϕ(ξ)et`µ (−ξ) , ϕ?µ ˘t (ξ) = `µ (−ξ)ϕ(ξ)e ˆ = Ld dt
which is equivalent to (*). To see how (*) applies, observe that EP ϕ Z(t) Fs = ϕ ? µ ˘t−s Z(s) , and therefore that, for any A ∈ Fs , Z t EP ϕ Z(t) , A] − EP ϕ Z(s) , A = EP (Lµ ϕ) ? µ ˘τ −s Z(s) , A dτ s Z t Z t P µ P µ = E L ϕ Z(τ ) , A dτ = E L ϕ Z(τ ) dτ, A , s
s
270
7 Continuous Parameter Martingales
which, after rearrangement, is the asserted martingale property. To prove the converse assertion, again begin with the observation that, by an easy approximation procedure, one can prove the martingale property for all ϕ ∈ Cb2 (RN ; C) as soon as one knows it for ϕ ∈ Cc∞ (RN ; R). In particular, one √ −1(ξ,x)RN , in which case Lµ ϕ = `µ (ξ)ϕ, and therefore, for can take ϕ(x) = e any A ∈ Fs , one gets that Z t i h √ u(τ ) dτ. u(t) ≡ EP exp −1(ξ, Z(t) RN N , A = u(s) + `µ (ξ) R
s
Since this means that u(t) = e(t−s)`µ (ξ) u(s), it follows that {Z(t) : t ≥ 0} satisfies (7.1.4) and is therefore a L´evy process for µ. As an immediate consequence of the preceding we have the following characterizations of the distribution of a L´evy process. In the statement that follows, Ft is the σ-algebra over D(RN ) generated by {ψ(τ ) : τ ∈ [0, t]}. Theorem 7.1.7. Given µ ∈ I(RN ), let Qµ ∈ M1 D(RN ) be the distribution of a L´evy process for µ. Then Qµ is the unique P ∈ M1 D(RN ) that satisfies either one of the properties: i h√ exp −1 ξ, ψ(t) RN + t`µ (ξ) , Ft , P
is a martingale with mean value 1 for each ξ ∈ RN , or
ϕ ψ(t) − ϕ(0) −
Z
t
L ϕ ψ(τ ) dτ, Ft , P µ
0
is a martingale with mean value 0 for each ϕ ∈ Cc∞ (RN ; R). § 7.1.3. Basic Results. In this subsection I run through some of the results from § 5.2 that transfer immediately to the continuous parameter setting. Lemma 7.1.8. Let the interval I and the function f : I −→ R ∪ {∞} be as in Corollary 5.2.10. If either X(t), Ft , P is an I-valued martingale or X(t), Ft , P is an I-valued submartingale and f is non-decreasing and bounded below, then f ◦ X(t), Ft , P is a submartingale. Proof: The fact that the parameter is continuous plays no role here, and so this result is already covered by the argument in Corollary 5.2.10. Theorem 7.1.9 (Doob’s Inequality). Let X(t), Ft , P be a submartingale. Then, for every α ∈ (0, ∞) and T ∈ [0, ∞), " # ! 1 P P sup X(t) ≥ α ≤ E X(T ), sup X(t) ≥ α . α t∈[0,T ] t∈[0,T ]
§ 7.1 Continuous Parameter Martingales
271
In particular, for non-negative submartingales and T ∈ [0, ∞), # p1
" sup X(t)p
EP
≤
t∈[0,T ]
1 p EP X(T )p p , p−1
p ∈ (0, ∞).
Proof: Because of Exercise 1.4.18, I need only prove the first assertion. To this end, let T ∈ (0, ∞) and apply Theorem 5.2.1 to the discrete n ∈ N be given, mT , P , and observe that parameter submartingale X 2n , F mT n 2
sup X
mT 2n
: 0≤m≤2
n
% sup X(t)
as n → ∞.
t∈[0,T ]
Theorem 7.1.10 (Doob’s Martingale Convergence Theorem). Assume that X(t), Ft , P is a P-integrable submartingale. If sup EP X(t)+ < ∞, t∈[0,∞)
W then there exists an F∞ ≡ t≥0 Ft -measurable X = X(∞) ∈ L1 (P; R) to which X(t) converges P-almost surely as t → ∞. Moreover, when X(t), Ft , P is either a non-negative submartingale or a martingale, the convergence takes place in 1 L (P; R) if and only if the family X(t) : t ∈ [0, ∞) is uniformly P-integrable, in which case X(t) ≤ EP [X | Ft ] or X(t) = EP [X | Ft ] (a.s., P) for all t ∈ [0, ∞), and 1 (7.1.11) P sup |X(t)| ≥ α ≤ EP |X|, sup |X(t)| ≥ α . α t≥0 t≥0
Finally, again when X(t), Ft , P is either a non-negative submartingale or a martingale, for each p ∈ (1, ∞) the family |X(t)|p : t ∈ [0, ∞) is uniformly Pintegrable if and only if supt∈[0,∞) kX(t)kLp (P) < ∞, in which case X(t) −→ X in Lp (P; R). Proof: To prove the initial convergence assertion, note that, by Theorem W 5.2.15 applied to the discrete parameter process X(n), Fn , P , there is an n∈N Fn measurable X ∈ L1 (P; R) to which X(n) converges P-almost surely. Hence, we need only check that limt→∞ X(t) exists in [−∞, ∞] P-almost surely. To (n) this end, define U[a,b] (ω) for n ∈ N and a < b to be the precise number of times that the sequence X 2mn , ω : m ∈ N upcrosses the interval [a, b] (cf. the (n) paragraph preceding Theorem 5.2.15), observe that U[a,b] (ω) is non-decreasing (n)
as n increases, and set U[a,b] (ω) = limn→∞ U[a,b] (ω). Note that if U[a,b] (ω) < ∞,
272
7 Continuous Parameter Martingales
then (by right-continuity), there is an s ∈ [0, ∞) such that either X(t, ω) ≤ b for all t ≥ s or X(t, ω) ≥ a for all t ≥ s. Hence, we will know that X(t, ω) converges in [−∞, ∞] for P-almost every ω ∈ Ω as soon as we show that EP U[a,b] < ∞ for every pair a < b. In addition, by (5.2.16), we know that P
sup E n∈N
h
(n) U[a,b]
i
EP (X(t) − a)+ < ∞, ≤ sup b−a t∈[0,∞)
and so the required estimate follows from the Monotone Convergence Theorem. Now assume that X(t), Ft , P is either a non-negative submartingale or a martingale. Given the preceding, it is clear that X(t) −→ X in L1 (P; P) if X(t) : t ∈ [0, ∞) is uniformly P-integrable. Conversely, suppose that X(t) −→ X in L1 (P; R). Then, for any T ∈ [0, ∞), |X(T )| ≤ lim EP |X(t)| FT = EP |X| FT .
(*)
t→∞
In particular, from Theorem 7.1.9, ! P
sup |X(t)| ≥ α t∈[0,T ]
" # 1 P ≤ E |X|, sup |X(t)| ≥ α α t∈[0,T ]
for every T ∈ (0, ∞). Hence, (7.1.11) follows when one lets T → ∞. But, again from (*), EP |X(T )|, |X(T )| ≥ α ≤ EP |X|, |X(T )| ≥ α ≤ EP |X|, sup |X(t)| ≥ α , t≥0
and therefore, since, by (7.1.11), P supt≥0 |X(t)| ≥ α −→ 0 as α → ∞, we can conclude that {X(t) : t ≥ 0} is uniformly P-integrable. Finally, if {X(T ) : T ≥ 0} is bounded in Lp (P; R) for some p ∈ (1, ∞), then, by the last part of Theorem 7.1.9, supt≥0 |X(t)|p is P-integrable and therefore X(t) −→ X in Lp (P; R). § 7.1.4. Stopping Times and Stopping Theorems. A stopping time relative to a non-decreasing family {Ft : t ≥ 0} of σ-algebras is a map ζ : Ω −→ [0, ∞] with the property that {ζ ≤ t} ∈ Ft for every t ≥ 0. Given a stopping time ζ, I will associate with it the σ-algebra Fζ consisting of those A ⊆ Ω such S∞ that A ∩ {ζ ≤ t} ∈ Ft for every t ≥ 0. Note that, because {ζ < t} = n=0 {ζ ≤ (1 − 2−n )t}, {ζ < t} ∈ Ft for all t ≥ 0. Here are a few useful facts about stopping times. Lemma 7.1.12. Let ζ be a stopping time. Then ζ is Fζ -measurable, and, for any progressively measurable function X with values in a measurable space (E, B), the function ω X(ζ, ω) ≡ X ζ(ω), ω is Fζ -measurable on {ζ < ∞} in
§ 7.1 Continuous Parameter Martingales
273
the sense that ω : ζ(ω) < ∞ & X(ζ, ω) ∈ Γ ∈ Fζ for all Γ ∈ B. In addition, f ◦ ζ is again a stopping time if f : [0, ∞] −→ [0, ∞] is a non-decreasing, rightcontinuous function satisfying f (τ ) ≥ τ for all τ ∈ [0, ∞]. Next, suppose that ζ1 and ζ2 are a pair of stopping times. Then ζ1 + ζ2 , ζ1 ∧ ζ2 , and ζ1 ∨ ζ2 are all stopping times, and Fζ1 ∧ζ2 ⊆ Fζ1 ∩ Fζ2 . Finally, for any A ∈ Fζ1 , A ∩ {ζ1 ≤ ζ2 } ∈ Fζ1 ∧ζ2 . Proof: Since {ζ ≤ s} ∩ {ζ ≤ t} = {ζ ≤ s ∧ t} ∈ Ft , it is clear that ζ is Fζ -measurable. Next, suppose that X is a progressively measurable function. To prove that X(ζ) is Fζ -measurable, begin by checking that ω : ζ(ω), ω ∈ A ∈ Ft for any A ∈ Bt × Ft . Indeed, this is obvious when A = [0, s] × B for s ∈ [0, t] and B ∈ Ft and, since these generate B[0,t] × Ft , follows in general. Now, for any t ≥ 0 and Γ ∈ B, A(t, Γ) ≡ (τ, ω) ∈ [0, ∞) × Ω : τ, X(τ, ω) ∈ [0, t] × Γ ∈ B[0,t] × Ft , and therefore {X(ζ) ∈ Γ} ∩ {ζ ≤ t} = ω : ζ(ω), ω ∈ A(t, Γ) ∈ Ft . As for f ◦ ζ when f satisfies the stated conditions, simply note that {f ◦ ζ ≤ t} = {ζ ≤ f −1 (t)} ∈ Ft , where f −1 (t) ≡ inf{τ : f (τ ) ≥ t} ≤ t. Next suppose that ζ1 and ζ2 are two stopping times. It is trivial to see that ζ1 ∧ ζ2 and ζ1 ∨ ζ2 are again stopping times. In addition, if Q denotes the set of rational numbers, then [ {ζ1 + ζ2 > t} = {ζ1 > t} ∪ {ζ1 ≥ qt & ζ2 > (1 − q)t} ∈ Ft . q∈Q∩[0,1]
Thus, ζ1 + ζ2 is a stopping time. To prove the final assertions, begin with the observation that if ζ1 ≤ ζ2 , then A ∩ {ζ2 ≤ t} = A ∩ {ζ1 ≤ t} ∩ {ζ2 ≤ t} ∈ Ft for all A ∈ Fζ1 and t ≥ 0, and therefore Fζ1 ⊆ Fζ2 . Next, for any ζ1 and ζ2 , {ζ1 ≤ ζ2 } ∈ Fζ2 since [ {ζ1 > ζ2 } ∩ {ζ2 ≤ t} = {ζ1 > qt} ∩ {ζ2 ≤ qt} ∈ Ft . q∈Q∩[0,1]
Finally, if A ∈ Fζ1 , then A ∩ {ζ1 ≤ ζ2 } ∩ {ζ1 ∧ ζ2 ≤ t} = A ∩ {ζ1 ≤ t} ∩ {ζ1 ≤ t ∧ ζ2 }, and therefore, since A ∩ {ζ1 ≤ t} ∈ Ft and {ζ1 ≤ t ∧ ζ2 } ∈ Ft∧ζ2 ⊆ Ft , we have that A ∩ {ζ1 ≤ ζ2 } ∈ Fζ1 ∧ζ2 . In order to prove the continuous parameter analog of Theorems 5.2.13 and 5.2.11, I will need the following uniform integrability result.
274
7 Continuous Parameter Martingales
Lemma 7.1.13. If X(t), Ft , P is either a martingale or a non-negative, integrable submartingale, then, for each T > 0, the set X(ζ) : ζ is a stopping time dominated by T is uniformly P-integrable. Furthermore, if, in addition, {X(t) : t ≥ 0} is uniformly 7.1.10) X(∞) = limt→∞ X(t) (a.s., P), P-integrable and (cf. Theorem then X(ζ) : ζ is a stopping time is uniformly P-integrable. Proof: Throughout, without loss in generality, I will assume that X(t), Ft , P is a non-negative, integrable submartingale. n for n ≥ 0. By Lemma 7.1.12, Given a stopping time ζ ≤ T , define ζn = [2 2ζ]+1 n ζn is again a stopping time. Thus, by Theorem 5.2.13 applied to the discrete parameter submartingale X(m2−n ), Fm2−n , P , X(ζn ) ≤ EP X 2−n ([2n T ] + 1) Fζn ≤ EP X(T + 1) Fζn ,
and so EP X(ζn ), X(ζn ) ≥ α ≤ EP X(T + 1), X(ζn ) ≥ α " P
≤E
X(T + 1),
sup
#
X(t) ≥ α .
t∈[0,T +1]
Starting from here, noting that ζn & ζ as n → ∞, and applying Fatou’s Lemma, we arrive at " # (*) EP X(ζ), X(ζ) > α ≤ EP X(T + 1), sup X(t) ≥ α . t∈[0,T +1]
Hence, since, by Theorem 7.1.9, P supt∈[0,T +1] X(t) ≥ α tends to 0 as α → ∞, this proves the first assertion. When {X(t) : t ≥ 0} is uniformly integrable, we can replace (*) by P P E X(ζ ∧ T ), X(ζ ∧ T ) > α ≤ E X(∞), sup X(t) ≥ α t≥0
for any stopping time ζ and T > 0. Hence, after another application of Fatou’s Lemma, we get P P E X(ζ), X(ζ) > α ≤ E X(∞), sup X(t) ≥ α . t≥0
At the same time, the first inequality in Theorem 7.1.9 can be replaced by 1 1 P P sup X(t) ≥ α ≤ E X(∞), sup X(t) ≥ α ≤ EP [X(∞)], α α t≥0 t≥0
and so the asserted uniform integrability follows. It turns out that in the continuous time context, Doob’s Stopping Time Theorem is most easily seen as a corollary of Hunt’s. Thus, I will begin with Hunt’s.
§ 7.1 Continuous Parameter Martingales
275
Theorem 7.1.14 (Hunt). Stopping Time Theorem, Hunt’s continuous pa rameter Let X(t), Ft , P be either a non-negative, integrable submartingale or a martingale. If ζ1 and ζ2 are bounded stopping times and ζ1 ≤ ζ2 , then P X(ζ1 ) ≤ E X(ζ2 ) Fζ1 , and equality holds in the martingale case. Moreover, when {X(t) : t ≥ 0} is uniformly P-integrable and X(∞) ≡ limt→∞ X(t), then the same result holds for arbitrary stopping times ζ1 ≤ ζ2 . Proof: Given ζ1 ≤ ζ2 ≤ T , define (ζi )n = 2−n [2n ζi ] + 1 for n ≥ 0, note that (ζi )n is a {Fm2−n : m ≥ 0}-stopping time and that Fζ1 ⊆ F(ζ1 )n , and apply Theorem 5.2.13 to the discrete parameter submartingale X(m2−n , Fm2−n , P in order to see that h h i i EP X (ζ1 )n , A ≤ EP X (ζ2 )n , A , A ∈ Fζ1 , with equality in the martingale case. Because of right-continuity and Lemma 7.1.13, X (ζi )n −→ X(ζi ) in L1 (P; R), and so we have now shown that X(ζ1 ) ≤ P E X(ζ2 ) Fζ1 , with equality in the martingale case. When {X(t) : t ≥ 0} is uniformly P-integrable and ζ1 ≤ ζ2 are unbounded, {X(ζi ∧ T ) : T ≥ 0} is uniformly P-integrable for i ∈ {1, 2}. Hence, for any A ∈ Fζ1 and 0 ≤ t ≤ T , EP X(T ∧ ζ1 ), A ∩ {ζ1 ≤ t} ≤ EP X(T ∧ ζ2 ), A ∩ {ζ1 ≤ t} , with equality in the martingale case. Letting first T and then t tend to infinity, one gets the same relationship for X(ζ1 ) and X(ζ2 ), initially with A ∩ {ζ1 < ∞} and then, trivially, with A alone. Theorem 7.1.15 (Doob’s Stopping Time Theorem). If X(t), Ft , P is either a non-negative, integrable submartingale or a martingale, then, for every stopping time ζ, X(t ∧ ζ), Ft , P is either an integrable submartingale or a martingale. Proof: Given 0 ≤ s < t and A ∈ Fs , note that A ∩ {ζ > s} ∈ Fs∧ζ and therefore, by Hunt’s Theorem applied to the stopping times s ∧ ζ and t ∧ ζ, that EP X(t ∧ ζ), A = EP X(ζ), A ∩ {ζ ≤ s} + EP X(t ∧ ζ), A ∩ {ζ > s} ≥ EP X(ζ), A ∩ {ζ ≤ s} + EP X(s ∧ ζ), A ∩ {ζ > s} = EP X(s ∧ ζ), A , where the inequality is an equality in the martingale case. To demonstrate just how powerful these results are, I give the following extension of the independent increment property of L´evy processes. In its statement, the maps δt : D(RN ) −→ D(RN ) for t ∈ [0, ∞) are defined so that δt ψ(τ ) = ψ(τ + t) − ψ(t), Ft = σ {ψ(τ ) : τ ∈ [0, t]} , ζ is a τ ∈ [0, ∞). Also, stopping time relative to Ft : t ∈ [0, ∞) , and δζ is the map on {ψ : ζ(ψ) < ∞} into D(RN ) given by δζ ψ = δζ(ψ) ψ.
276
7 Continuous Parameter Martingales
Theorem 7.1.16. Given µ ∈ I(RN ), let Qµ ∈ M1 D(RN ) be the distribution of the L´evy process for µ. Then, for each stopping time ζ and FD(RN ) × Fζ measurable functions F : D(RN ) × D(RN ) −→ [0, ∞), Z ZZ F δζ ψ, ψ Qµ (dψ) = 1[0,∞) ζ(ψ 0 ) F (ψ, ψ 0 ) Qµ (dψ)Qµ (dψ 0 ). {ζ<∞}
Proof: By elementary measure theory, all that we have to show is that, for each B ∈ Fζ contained in {ζ < ∞}, Qµ (δζ−1 Γ) ∩ B = Qµ (Γ)Qµ (B). Given B ∈ Fζ contained in {ζ < ∞} with Qµ (B) > 0, choose T > 0 so that Qµ (BT ) > 0 when BT = B ∩ {ζ ≤ T }, and define QT ∈ M1 D(RN ) so that QT (Γ) =
Qµ (δζ−1 Γ) ∩ BT
Qµ (BT )
.
If we show that QT = Qµ , then we will know that Qµ (δζ−1 Γ) ∩ B = lim Qµ (δζ−1 Γ) ∩ BT T →∞ µ
= Q (Γ) lim Qµ (BT ) = Qµ (Γ)Qµ (B) T →∞
and therefore will be done. By Theorem 7.1.6, checking that QT = Qµ comes down to showing that, for any 0 ≤ s < t, ξ ∈ RN , and A ∈ Fs ,
√ √ EQT e −1(x,ψ(t))RN −t`µ (ξ) , A = EQT e −1(x,ψ(s))RN −s`µ (ξ) , A . To this end, note that, by Theorem 7.1.14 applied to s + ζ ∧ T and t + ζ ∧ T ,
√ Qµ (BT )EQT e −1(x,ψ(t))RN −t`µ (ξ) , A i h √ √ µ = EQ e− −1(ξ,ψ(ζ))RN +ζ`µ (ξ) e −1(ξ,ψ(t+ζ))RN −(t+ζ)`µ (ξ) , (δζ−1 A) ∩ BT i h √ √ µ = EQ e− −1(ξ,ψ(ζ))RN +ζ`µ (ξ) e −1(ξ,ψ(s+ζ))RN −(s+ζ)`µ (ξ) , (δζ−1 A) ∩ BT √ = Qµ (BT )EQT e −1(x,ψ(s))RN −s`µ (ξ) , A , √
since ψ e− −1(ξ,ψ(ζ))RN +ζ`µ (ξ) 1A (δζ ψ)1BT (ψ) is Fs+ζ∧T -measurable. § 7.1.5. An Integration by Parts Formula. In this subsection I will derive a simple result that has many interesting applications.
Theorem 7.1.17. Suppose V : [0, ∞)×Ω −→ C is a right-continuous, progressively measurable function, and let |V |(t, ω) ∈ [0, ∞] denote the total variation var[0,t] V ( · , ω) of V ( · , ω) on the interval [0, t]. Then |V | : [0, ∞)×Ω −→ [0, ∞] is a non-decreasing, progressively measurable function that is right-continuous
§ 7.1 Continuous Parameter Martingales
277
on each interval [0, t) for which |V |(t, ω) < ∞. Next, suppose that X(t), Ft , P is a C-valued martingale with the property that, for each (t, ω) ∈ (0, ∞) × Ω, the product kX( · , ω)k[0,t] |V |(t, ω) < ∞, and define (R Y (t, ω) =
(0,t]
X(s, ω) V (ds, ω) if |V |(t, ω) < ∞
0
otherwise,
where, in the case when |V |(t, ω) < ∞, the integral is the Lebesgue integral of X( · , ω) on [0, t] with respect to the C-valued measure determined by V ( · , ω). If h i EP kXk[0,T ] |V |(T ) + V (0) < ∞ for all T ∈ (0, ∞), then X(t)V (t) − Y (t), Ft , P is a martingale. Proof: Without loss in generality, I will assume that both X and V are R valued. To see that |V | is Ft : t ∈ [0, ∞) -progressively measurable, simply observe that, by right-continuity, [2n t]
|V |(t, ω) = sup n∈N
X V
k+1 2n
∧ t, ω − V
k 2n , ω
;
k=0
and to see that |V |( · , ω) is right-continuous on [0, t) whenever |V |(t, ω) < ∞, recall that the magnitude of the jumps (from the right and left) of the variation of a function coincide with those of the function itself. I turn now to the second part. Certainly Y is Ft : t ∈ [0, ∞) -progressively measurable. In addition, because kX( · , ω)k[0,t] |V |(t, ω) < ∞ for all (t, ω) ∈ [0, ∞) × Ω, for any ω ∈ Ω one has that Z Y (t, ω) = 0
X(s, ω) V (ds, ω) for all t ∈ [0, ∞);
or Y (t, ω) = (0,t]
and so, in either case, Y ( · , ω) is right-continuous and Y (t, ω) − Y (s, ω) can be computed as [2n t]
lim
n→∞
X
X
k+1 2n
∧ t, ω
V
k+1 2n
∧ t, ω − V
k 2n
∨ s, ω
.
k=[2n s]
In fact, under the stated integrability condition, the convergence in the preceding takes place in L1 (P; R) for every t ∈ [0, ∞); and therefore, for any 0 ≤ s ≤ t < ∞
278
7 Continuous Parameter Martingales
and A ∈ Fs , EP Y (t) − Y (s), A [2n t]
= lim
n→∞
X
h EP X
k+1 2n
∧ t, ω
h EP X(t) V
k+1 2n
V
k+1 2n
∧ t, ω − V
k 2n
∨ s, ω
,A
i
k=[2n s]
[2n t]
= lim
n→∞
X
∧ t, ω − V
k 2n
∨ s, ω
,A
i
k=[2n s]
h
h i i = EP X(t) V (t) − V (s) , A = EP X(t)V (t) − X(s)V (s), A , and clearly this is equivalent to the asserted martingale property. We will make frequent practical applications of Theorem 7.1.17 later, but here I will show that it enables us to prove that there is an important dichotomy between continuous martingales and functions of bounded variation. However, before doing so, I need to make a small, technical digression. A function ζ : Ω −→ [0, ∞] is an extended stopping time relative to Ft : t ∈ [0, ∞) if {ζ < t} ∈ Ft for every t ∈ (0, ∞). Since {ζ < t} ∈ Ft for any stopping time ζ, it is clear that every stopping time is an extended stopping time. On the other hand, not every extended stopping time is a stopping time. To wit, if X : [0, ∞)× Ω −→ R is a right-continuous, progressively measurable function relative to σ X(τ ) : τ ∈ [0, t] : t ≥ 0 , then ζ = inf{t ≥ 0 : X(t) > 1} will always be an extended stopping time but will seldom be a stopping time. T Lemma 7.1.18. For each t ≥ 0, set Ft+ = τ >t Fτ . Then ζ : Ω −→ [0, ∞] is an extended stopping time if and only if it is a stopping time relative to {Ft+ : t ≥ 0}. Moreover, if X(t), Ft , P is either a non-negative, integrable submartingale or a martingale, then so is X(t), Ft+ , P . In particular, if ζ is an extended stopping time, then X(t ∧ ζ), Ft+ , P is a non-negative, integrable submartingale or a martingale. T Proof: The first assertion is immediate from {ζ ≤ t} = τ >t {ζ < τ }. To prove the second assertion, apply right-continuity and the first uniform integrability result in Lemma 7.1.13 to see that if 0 ≤ s < t and A ∈ Fs+ , then EP X(s), A = lim EP X(τ ), A ≤ EP X(t), A , τ &s
where the inequality is an equality in the martingale case. Theorem 7.1.19. Suppose that X(t), Ft , P is a continuous martingale, and let |X|(t, ω) = var[0,t] X( · , ω) denote the variation of X( · , ω) [0, t]. Then P ∃t > 0 0 < |X|(t, ω) < ∞ = 0. Equivalently, for P-almost every ω and all t > 0, either X(τ, ω) = X(0, ω) for τ ∈ [0, t] or |X|(t, ω) = ∞.
§ 7.1 Continuous Parameter Martingales
279
Proof: Without loss in generality, I will assume that X(0, ω) ≡ 0. Given R > 0, let ζR (ω) = sup{t ≥ 0 : |X|(t, ω) ≤ R}, and set XR (t) = X(t ∧ ζR ). Then ζR is an extended stopping time, and so, by Lemma 7.1.18, (XR (t), Ft+ , P) is a bounded martingale. Hence, by Theorem 7.1.17,
2
Z
XR (t) −
t
XR (τ ) XR (dτ ), Ft+ , P
0
is also a martingale, and so EP XR (t)2 = EP
t
Z
XR (τ ) XR (dτ ) .
0
On the other hand, since XR ( · ) is continuous, and therefore, by Fubini’s Theorem, ZZ Z t XR (t)2 = XR (dτ1 )XR (dτ2 ) = 2 XR (τ ) XR (dτ ), 0
[0,t]2
we also know that E XR (t)2 = 2EP
Z
P
t
XR (τ ) XR (dτ ) .
0
Hence, EP XR (t)2 = 0 for all t > 0, which means that XR ( · ) ≡ 0 P-almost surely. The preceding result leads immediately to the following analog of the uniqueness statement in Lemma 5.2.12. Corollary 7.1.20. Let X : Ω −→ R be a right-continuous, progressively measurable function. Then, up to a P-null set, there is at most one continuous, progressively measurable A : Ω −→ R such that A(0, ω) = 0, A( · , ω) is of locally bounded variation for P-almost every ω ∈ Ω, and X(t) − A(t), Ft , P is a martingale. The role of continuity here seems minor, but it is crucial. Namely, continuity was used in Theorem 7.1.19 only when I wanted to know that XR (t)2 = Rt 2 0 XR (τ ) XR (dτ ). On the other hand, it is critical. Namely, if {N (t) : t ≥ 0} is the simple Poisson process in § 4.2 and Ft = σ N (τ ) : τ ∈ [0, t] , then it is easy to check that N (t) − t, Ft , P is a martingale, all of whose paths are of locally bounded variation.
280
7 Continuous Parameter Martingales Exercises for § 7.1
Exercise 7.1.21. The definition of stopping times and their associated σalgebras that I have adopted is due to E.B. Dynkin. Earlier, less ubiquitous but more transparent, definitions appear in the work of Doob and Hunt under the name of optional stopping times. To explain these earlier definitions, let E be a complete, separable metric space and Ψ a non-empty collection of rightcontinuous paths ψ : [0, ∞) −→ E with the property that for all ψ ∈ Ψ and t ∈ [0, ∞), the stopped path ψ t given by ψ t (τ ) = ψ(t∧τ ) is again in Ψ. Similarly, given a function ζ : Ψ −→ [0, ∞], define ψ ζ so that ψ ζ (t) = ψ t ∧ ζ(ψ) . Finally, for each t ∈ [0, ∞), define the σ-algebras Ft over Ψ to be the one generated W by {ψ(τ ) : τ ∈ [0, t]}, and take F = t≥0 Ft . In terms of these quantities, an optional stopping time is an F-measurable map ζ : Ψ −→ [0, ∞] such that ζ(ψ) ≤ t =⇒ ζ(ψ) = ζ(ψ t ), and the associated σ-algebra is σ {ψ ζ (t) : t ≥ 0} . The goal of this exercise is to show that ζ is an optional stopping time if and only if it is a stopping time and that its associated σ-algebra is Fζ . (i) It is an easy matter (cf. Exercise 4.1.9) to check that f : Ω −→ R is F+ + measurable if and only if there exists a B Z -measurable F : E Z −→ R and a sequence {tm : m ∈ Z+ } such that f (ψ) = F ψ(t1 ), . . . , ψ(tm ), . . . , from which it is clear that an F-measurable f will be Ft -measurable for some t ∈ [0, ∞) if and only if f (ψ) = f (ψ t ). Use this to show that every optional stopping time is a stopping time. (ii) Show that ζ : Ψ −→ [0, ∞] is a stopping time relative to Ft : t ∈ [0, ∞) if and only if it is F-measurable and, for each t ∈ [0, ∞), {ψ : ζ(ψ) ≤ t} = {ψ : ζ(ψ t ) ≤ t}. In addition, if ζ is a stopping time, show that ζ(ψ) < ∞ =⇒ ζ(ψ) = ζ(ψ ζ ), and therefore that ζ(ψ) ≤ t =⇒ ζ(ψ) = ζ(ψ t ) for all t ∈ [0, ∞). Thus, ζ is an optional stopping time if and only if it is a stopping time. Hint: In proving the second part, check that {ζ = t} ∈ Ft , and conclude that 1{t} ζ(ψ) = 1{t} ζ(ψ t ) for all (t, ψ) ∈ [0, ∞) × Ψ. (iii) If ζ is a stopping time, show that Fζ = σ {ψ ζ (t) : t ≥ 0} . Besides having intuitive value, this shows that, at least in the situation here, Fζ is countably generated. Hint: Using right-continuity, first show that ψ ψ ζ is F-measurable. Next, given a B-measurable f : E −→ R and t ∈ [0, ∞), use (ii) to show that 1[0,t] ζ(ψ) f ψ ζ (τ ) = 1[0,t] ζ(ψ t ) f ψ(τ ∧ ζ(ψ t ) , τ ∈ [0, ∞), and conclude that σ {ψ ζ (t) : t ≥ 0} ⊆ Fζ . To prove the opposite inclusion, show that if f : Ψ −→ R is Fζ -measurable, then, for each t ∈ [0, ∞), 1{t} ζ(ψ) f (ψ) = 1{t} ζ(ψ t ) f (ψ t ), and thereby arrive at f (ψ) = f (ψ ζ ). Fi nally, use this together with Exercise 4.1.9 to show that f is σ {ψ ζ (t) : t ≥ 0} measurable.
Exercises for § 7.1
281
Exercise 7.1.22. Let (Ω, F, P) be a probability space and Ft : t ∈ [0, ∞) is non-decreasing family of sub-σ-algebras of F. Denoteby F and Ft the completions of F and Ft with respect to P. If X(t), Ft , P is a submartingale or martingale, show that X(t), Ft , P is also.
Exercise 7.1.23. Let µ ∈ I(RN ) be given as in Exercise 3.2.23, and extend `µ to CN accordingly. If {Z(t) : t ≥ 0} is a L´evy process for µ, show that (7.1.4) continues to hold for all ξ ∈ CN . Exercise 7.1.24. In Exercise 3.3.12, we discussed one-sided stable laws, and in Exercise 4.3.12 we showed that P maxτ ∈[0,t] B(τ ) ≥ a = 2P B(t) ≥ a , where {B(t) : t ≥ 0} is an R-valued Brownian motion. In this exercise, we will examine the relationship between these two. (i) Set ζ a (ψ) = inf{t ≥ 0 : ψ(t) ≥ a}, and show that the result in Exercise 4.3.12 can be rewritten as r W
(1)
a
ζ ≤t =
2 π
Z
∞ 1
e−
y2 2
dy.
at− 2
Now use the results in Exercise 3.3.14 (especially (3.3.16)) to conclude that the 1
1
W (1) -distribution of ζ a is ν 21 , the one-sided 12 -stable law “at time 2 2 a.” 22
a
(ii) Here is another, more conceptual way to understand the conclusion drawn in (i) that the W (1) -distribution is a one-sided 12 -stable law. Namely, begin by a a+b a b a showing that if ψ(0) = 0 and ζ (ψ) < ∞, then ζ (ψ) = ζ (ψ) + ζ δζ ψ . As an application of Theorem 7.1.16, conclude from this that if βa denotes the W (1) distribution of ζ a , then βa+b = βa ? βb . In particular, this means that β ≡ β1 ca = ea`β , where `β is the exponent appearing in is infinitely divisible and that β ˆ the L´evy–Khinchine formula for β.
(iii) Next, use Brownian scaling to see that, for all λ > 0, ζ λa has the same W (1) distribution as λ2 ζ a , and use this together with part (iii) of Exercise 3.3.12 to 1
see that the distribution of ζ 1 is νc2 for some c > 0. 1
(iv) Although we know from (i) that the constant c must be 2 2 , here is an 1 2 easier way to find it. Use Exercise 7.1.23 to see that eλψ(t)− 2 λ t , Ft , W (1) for every λ ∈ R, and apply Doob’s Stopping Time Theorem and the fact that (1) 1 2 a W (1) (ζ a < ∞) = 1 to verify the identity EW e− 2 λ ζ = e−λa for λ > 0. 1
√
Hence, the Laplace transform of νc2 is e− 2λ , which, by the calculation in part 1 (iii) of Exercise 3.3.12, means that c = 2 2 . Of course, this calculation makes the preceding parts of this exercise unnecessary. Nonetheless, it is interesting to see the Brownian explanation for the properties of the one-sided, 12 -stable laws.
282
7 Continuous Parameter Martingales
Exercise 7.1.25. An important corollary of Theorem 7.1.16 is the following formula. Working in the setting of that theorem, show that, for any stopping time ζ and t ∈ (0, ∞) and Γ ∈ BRN , h i µ Qµ {ψ : ψ(t) ∈ Γ & ζ(ψ) ≤ t} = EQ µt−ζ Γ − ψ(ζ) , ζ ≤ t , where, as usual, µτ is determined by µ cτ = eτ `µ . As a consequence, h i µ Qµ {ψ : ψ(t) ∈ Γ & ζ(ψ) > t} = µt (Γ) − EQ µt−ζ Γ − ψ(ζ) , ζ ≤ t , which is a quite general, generic statement of what is called Duhamel’s Formula. § 7.2 Brownian Motion and Martingales In this section we will see that continuous martingales and Brownian motion are intimately related concepts. In addition, we will find that martingale theory, and especially Doob’s and Hunt’s Stopping Time Theorems, provides a powerful tool with which to study Brownian paths. § 7.2.1. L´ evy’s Characterization of Brownian Motion. When applied to µ = γ0,I , Theorem 7.1.6 says that a progressively measurable function B : [0, ∞) × Ω −→ RN with B(0, ω) = 0 and B( · , ω) ∈ D(RN ) is a Brownian motion if and only if Z t 1 ∆ϕ B(τ ) dτ, F , P ϕ B(t) − t 2 0
is a martingale for all ϕ ∈ Cc∞ (RN ; R). In this subsection, I, following L´evy,1 will give another martingale characterization of Brownian motion, this time involving many fewer test functions. On the other hand, we will have to assume ahead of time that B( · , ω) ∈ C(RN ) for every ω ∈ Ω. Theorem 7.2.1 (L´ evy). Let B : [0, ∞) × Ω −→ RN be a progressively measurable function satisfying B(0, ω) = 0 and B( · , ω) ∈ C(RN ) for every ω ∈ Ω. Then B(t), Ft , P is a Brownian motion if and only if
ξ, B(t)
RN
2 t|η|2 , Ft , P + η, B(t) RN − 2
is a martingale for every ξ, η ∈ RN . 1
L´ evy’s Theorem is Theorem 11.9 in Chapter VII of Doob’s Stochastic Processes, Wiley (1953). Doob uses a clever but somewhat opaque Central Limit argument. The argument given here is far simpler and is adapted from the one introduced by H. Kunita and S. Watanabe in their article “On square integrable martingales,” Nagoya Math. J. 30 (1967).
§ 7.2 Brownian Motion and Martingales
283
Proof: First suppose that B(t), Ft , P is a Brownian motion. Then, because B(t) − B(s) is independent of Fs and has distribution γ0,I , EP B(t) − B(s) Fs = 0 and EP B(t) ⊗ B(t) − B(s) ⊗ B(s) Fs = (t − s)I. Hence, the necessity is obvious. To prove the sufficiency, Theorem 7.1.3 says that it is enough to prove that i h√ t|ξ|2 P E exp −1 ξ, B(t) RN + 2 , A (*) i h√ s|ξ|2 P = E exp −1 ξ, B(s) RN + 2 , A
for 0 ≤ s < t and A ∈ Fs . The challenge is to learn how to do this by taking full advantage of the assumed continuity. To this end, let ∈ (0, 1] be given, set ζ0 ≡ s, and use induction to define n o ζn = inf t ≥ ζn−1 : B(t) − B ζn−1 ≥ ∧ ζn−1 + ∧ t for n ∈ Z+ . Proceeding by induction, one can easily check that {ζn : n ≥ 0} is a non-decreasing sequence of [s, t]-valued stopping times. Hence, by Theorem 7.1.14 and our assumption, h i h i (**) EP ∆n Fζn−1 = 0 = EP ∆2n − δn Fζn−1 , where ∆n (ω) ≡ ξ, B ζn (ω), ω − B ζn−1 (ω), ω RN 2 δn (ω) ≡ |ξ| ζn (ω) − ζn−1 (ω) . Moreover, because B( · , ω) is continuous, we know that, for each ω ∈ Ω, |∆n (ω)| ≤ |ξ|, δn (ω) ≤ |ξ|2 , and ζn (ω) = t for all but a finite number of n’s. In particular, we can write the difference between the left and the right sides of (*) + P as the sum over n ∈ Z of E Dn Mn , A], where
i h√ Dn ≡ exp −1 ∆n + δ2n − 1 i h√ 2 Mn ≡ exp −1 ξ, B(ζn−1 ) RN + |ξ|2 ζn−1 .
By Taylor’s Theorem, √ −1 ∆n + Dn −
δn 2
−
1 2
√
−1 ∆n +
δn 2
2 1 |ξ|2 2 √ ≤ 6 e −1 ∆n +
3 δn 2 .
284
7 Continuous Parameter Martingales
√ Hence, after rearranging terms, we see that Dn = −1 ∆n − 12 ∆2n − δn + En , where, by our estimates on ∆n and δn , |ξ|2 |ξ|2 δ3 δ2 |En | ≤ 12 |∆n δn | + 8n + 23 e 2 |∆n |3 + 8n ≤ 1 + |ξ|2 e 2 ∆2n + δn ;
and so, after taking (**) into account,we arrive at ∞ ∞ X X EP Dn Mm , A = EP En Mn , A 1
1
≤ 2 1 + |ξ|2 e
|ξ|2 2
∞ X
|ξ|2 EP δn |Mn |, A ≤ 2 1 + |ξ|2 (t − s)e 2 (1+t) .
1
In other words, we have now proved that, for every ∈ (0, 1], the difference |ξ|2
between the two sides of (*) is dominated by 2(1 + |ξ|2 )(t − s)e 2 (1+t) , and so the equality in (*) has been established. As in Theorem 7.1.19, the subtlety here is in the use of the continuity assumption. Indeed, the same example that demonstrated its importance there, does so again here. Namely, if {N (t) : t ≥ 0} is a simple Poisson process and X(t) = N (t) − t, then both X(t), Ft , P and X(t)2 − t, Ft , P are martingales, but X(t), Ft , P is certainly not a Brownian motion. § 7.2.2. Doob–Meyer Decomposition, an Easy Case. The continuous parameter analog of Lemma 5.2.12 is a highly non-trivial result, one that was proved by P.A. Meyer and led him to his profound analysis of stochastic processes. Nonetheless, there is an important case in which Meyer’s result is relatively easy to prove, and that is the case proved in this subsection. However, before getting to that result, there is a rather fussy matter to be dealt with.
Lemma 7.2.2. For each n ∈ N, let Xn : [0, ∞) −→ R be a right-continuous, progressively measurable function with the property that Xn ( · , ω) is continuous for P-almost every ω ∈ Ω. If lim sup kXn ( · , ω) − Xm ( · , ω)k[0,t] = 0 (a.s., P) for each t ∈ (0, ∞),
m→∞ n>m
then there is a right-continuous, progressively measurable X : [0, ∞) −→ R such that X( · , ω) is continuous and Xn ( · , ω) −→ X( · , ω) uniformly on compacts for P-almost every ω ∈ Ω. Proof: Set A = {(t, ω) : limm→∞ supn>m kXn ( · , ω) − Xm ( · , ω)k[0,t] = 0}. Then A is progressively measurable. Next, define ζ(ω) = sup{t ≥ 0 : (t, ω) ∈ A}, note that {ζ < t} ∈ Ft for each t ∈ (0, ∞), and set B = {(t, ω) : ζ(ω) ≥ t}. Then, B is again progressively measurable. To see this, first note that Ω ∈ Ft if t ≤ s {(τ, ω) ∈ [0, t] × Ω : τ ∧ ζ(ω) < s} = {ζ < s} ∈ Ft if t > s,
§ 7.2 Brownian Motion and Martingales
285
and so (τ, ω) τ ∧ζ(ω) and therefore also (τ, ω) τ ∧ζ(ω)−τ are progressively measurable functions. Hence, since B = {(τ, ω) : τ ∧ ζ(ω) − τ ≥ 0}, B is progressively measurable. Now define limn→∞ Xn (t, ω) if (t, ω) ∈ A X(t, ω) = 0 if (t, ω) ∈ B \ A X ζ(ω), ω if (t, ω) ∈ / B. Clearly X( · , ω) is right-continuous. Moreover, because ζ = ∞ (a.s., P), X( · , ω) is continuous and Xn ( · , ω) −→ X( · , ω) uniformly on compacts for P-almost every ω ∈ Ω. Thus, it only remains to check that X is progressively measurable. For this purpose, let Γ ∈ BR be given, and set C = {(t, ω) : X(t, ω) ∈ Γ}. Because A and the Xn ’s are progressively measurable, it is clear that C ∩ A is progressively measurable. Similarly, because B \ A is progressively measurable and C ∩ (B \ A) equals B \ A or ∅ depending on whether 0 ∈ Γ or 0 ∈ / Γ, C ∩ (B \ A), and therefore C ∩ B, are progressively measurable. Hence, we now know that X B is progressively measurable. Finally, we showed earlier that (t, ω) t ∧ ζ(ω) is progressively measurable, and therefore so is (t,ω) ∈ [0, ∞) × Ω 7−→ t ∧ ζ(ω), ω ∈ B. Thus, because X(t, ω) = X t ∧ ζ(ω), ω , we are done. Theorem 7.2.3. Let X(t), Ft , P be an R-valued, square integrable martingale with the property that X( · , ω) is continuous for P-almost every ω ∈ Ω. Then there is a P-almost surely unique progressively measurable function hXi : [0, ∞) × Ω −→ [0, ∞) such that hXi(0, ω) = 0 and hXi( · , ω) is continuous and non-decreasing for P-almost every ω ∈ Ω, and X(t)2 − hXi(t), Ft , P is a martingale. Proof: The uniqueness is an immediate consequence of Corollary 7.1.20. The proof of existence, which is based on a suggestion I got from K. Itˆo, is very much like that of Theorem 7.2.1. Without loss in generality, I will assume that X(0) ≡ 0. I begin by reducing to the case when X is P-almost surely bounded. To this end, suppose that we know the result in this case. Given a general X and n ∈ N, define ζn = inf{t ≥ 0 : |X(t)| ≥ n} and Xn (t) = X(t∧ζn ). Then, |Xn ( · , ω)| ≤ n and, by Doob’s Inequality, ζn (ω) % ∞ for P-almost every ω ∈ Ω. Moreover, by Corollary 7.1.15, Xn (t), Ft , P is a martingale. Thus, by our assumption, for each n, we know hXn i exists. In addition, by Corollary 7.1.15 and uniqueness, we know (cf. Exercise 7.2.10) that, P-almost surely, hXm i(t) = hXn i(t ∧ ζm ) for all m ≤ n and t ≥ 0. Now define hXi so that hXi(t) = hXn i(t) for ζn ≤ t < ζn+1 . Then hXi is progressively measurable and right-continuous, hXi(0) = 0, and, P-almost surely, hXi is continuous and non-decreasing. Furthermore, X(t ∧
286
7 Continuous Parameter Martingales
ζn )2 − hXi(t ∧ ζn ), Ft , P is a martingale for each n ∈ N. Finally, note that, by Doob’s Inequality, EP khXik[0,t] ≤ EP kXk2[0,t] ≤ 4EP |X(t)|2 , and so, as n → ∞, X(t ∧ ζn)2 − hXi(t ∧ ζn ) −→ X(t)2 − hXi(t) in L1 (P; R). Hence, X(t)2 − hXi(t), Ft , P is a martingale. I now assume that |X( · , ω)| ≤ C < ∞ for P-almost every ω ∈ Ω. For each n ∈ N, use induction to define {ζk,n : k ≥ 0} so that ζ0,n = 0, ζk,0 = k, and, for (k, n) ∈ (Z+ )2 , ζk,n is equal to inf{ζ`,n−1 : ζ`,n−1 > ζk−1,n } ∧ inf t ≥ ζk−1,n : (t − ζk−1,n ) ∨ |X(t) − X(ζk−1,n )| ≥ n1 .
Working by induction, one sees that, for each n ∈ N, {ζk,n : k ≥ 0} is a nondecreasing sequence of bounded stopping times. Moreover, because X( · , ω) is P-almost surely continuous, we know that, for each n ∈ N, limk→∞ ζk,n (ω) = ∞ P-almost every ω ∈ Ω. Finally, the sequences {ζk,n : k ≥ 0} are nested in the sense that {ζk,n−1 : k ≥ 0} ⊆ {ζk,n : k ≥ 0} for each n ∈ Z+ . Set Xk,n = X ζk,n ) and, for k ≥ 1, ∆k,n (t) = X t ∧ ζk,n − X t ∧ ζk−1,n . Then X(t)2 = 2Mn (t) + hXin (t), where Mn (t) =
∞ X k=1
Xk−1,n ∆k,n (t) and hXin (t) =
∞ X
∆k,n (t)2 .
k=1
Of course, for P-almost every ω ∈ Ω, all but a finite number of terms in each of these sums vanish. In addition, one should observe that hXin (s) ≤ hXin (t) if s ≥ 0 and t − s > n1 . I now want to show that Mn (t), Ft , P is a P-almost surely continuous martingale for all n ∈ N, and the first step is to show for each (k, n) ∈ Z+ × N, Xk−1,n ∆k,n (t), Ft , P is a P-almost surely continuous martingale. Indeed, if 0 ≤ s < t and A ∈ Fs , then EP Xk−1,n ∆k,n (t), A = EP Xk−1,n ∆k,n (t), A ∩ {ζk−1,n ≤ s} + EP Xk−1,n ∆k,n (t), A ∩ {ζk−1,n > s} .
Next, check that EP Xk−1,n ∆k,n (t), A ∩ {ζk−1,n ≤ s} = EP Xk−1,n X(ζk,n ) − X(ζk−1,n ) , A ∩ {ζk,n ≤ s} h i + EP Xk−1,n X (t ∧ ζk,n ) ∨ s − X(ζk−1,n ) , A ∩ {ζk−1,n ≤ s < ζk,n } = EP Xk−1,n ∆k,n (s), A ∩ {ζk,n ≤ s} + EP Xk−1,n X(s) − X(ζk−1,n ) , A ∩ {ζk−1,n ≤ s < ζk,n } = EP Xk−1,n ∆k,n (s), A ∩ {ζk−1,n ≤ s} ,
§ 7.2 Brownian Motion and Martingales
287
where, in the passage to the second to last equality, I have used the fact that Xk−1,n 1A 1[ζk−1,n ,ζk,n ) (s) is Fs -measurable and applied Theorem 7.1.14. At the same time EP Xk−1,n ∆k,n (t), A ∩ {ζk−1,n > s} = EP Xk−1,n X(t ∧ ζk,n ) − X(t ∧ ζk−1,n ) , A ∩ {s < ζk−1,n ≤ t} = EP Xk−1,n X(t) − X(t) , A ∩ {s < ζk−1,n ≤ t} = 0 = EP Xk−1,n ∆k,n (s), A ∩ {ζk−1,n > s} , where I have used the fact that Xk−1,n 1A 1(s,t] (ζk−1,n ) is Ft∧ζk−1,n -measurable and again applied Theorem 7.1.14 in getting the second to last line. After P combining these, one sees that EP Xk−1,n ∆ (t), A = E Xk−1,n ∆k,n (s), A , k,n which means that Xk−1,n ∆k,n (t), Ft , P is a P-almost surely continuous martingale. Given the preceding, it is clear that, for each n and `, Mn (t ∧ ζ`,n ), Ft , P is a P-almost surely continuous, square integrable martingale. In addition, for k 6= k 0 , Xk−1 ∆k,n (t ∧ ζ`,n ) is orthogonal to Xk0 −1 ∆k0 ,n (t ∧ ζ`,n ) in L2 (P; R). Thus " # P 2 E sup Mn (τ ) ≤ 4EP Mn (t ∧ ζ`,n )2 0≤τ ≤t∧ζ`,n
=4
` X
` X 2 EP Xk−1,n ∆k,n (t ∧ ζ`,n )2 ≤ 4C 2 EP ∆k,n (t ∧ ζ`,n )2
k=1 2 P
k=1 2
= 4C E X(t ∧ ζ`,n )
2
2
≤ 4C E X(t) , P
from which it is easy to see that Mn (t), Ft , P is a P-almost surely continuous, square integrable martingale. I will now show that limm→∞ supn>m kMn − Mm k[0,t] = 0 P-almost surely (m)
(m)
and in L2 (P; R) for each t ∈ [0, ∞). To this end, define Yk−1,n so that Yk−1,n (ω) (m)
= Xk−1,n (ω) − X`−1,m (ω) when ζ`−1,m (ω) ≤ ζk−1,n (ω) < ζ`,m (ω). Then Yk−1,n P∞ (m) (m) 1 (a.s., P), and Mn −Mm = k=1 Yk−1,n ∆k,n . is Fk−1,n -measurable, |Yk−1,n | ≤ m Hence, by the same reasoning as above, ∞ X (m) 4 EP kMn − Mm k2[0,t] ≤ 4 EP (Yk−1,n )2 ∆k,n (t)2 ≤ 2 EP X(t)2 , m k=1
which is more than enough to get the asserted convergence result. We can now apply Lemma 7.2.2 to produce a right-continuous, progressively measurable M : [0, ∞) × Ω −→ R which is P-almost surely continuous and to
288
7 Continuous Parameter Martingales
which {Mn : n ≥ 1} converges uniformly on compacts, both P-almost surely and in L2 (P; R). In particular, M (t), Ft , P is a square integrable martingale. Finally, set hXi = (X 2 − 2M )+ . Obviously, hXi = X 2 − 2M (a.s., P), and hXi is right-continuous, progressively measurable, and P-almost surely continuous. In addition, because, P-almost surely, hXin −→ hXi uniformly on compacts and hXin (s) ≤ hXin (t) when t − s > n1 , it follows that hXi( · , ω) is non-decreasing for P-almost every ω ∈ Ω.
Remark 7.2.4. The reader may be wondering why I chose to complicate the preceding statement and proof by insisting that hXibe progressively measurable with respect to the original family of σ-algebras Ft : t ∈ [0, ∞) . Indeed, Exercise 7.1.22 shows that I could have replaced all the σ-algebras with their completions, and, if I had done so, there would have been no reason not to have taken X( · , ω) to be continuous and hXi( · , ω) to be continuous and nondecreasing for every ω ∈ Ω. However, there is a price to be paid for completing σ-algebras. In the first place, when one does, all statements become dependent on the particular P with which one is dealing. Secondly, because completed σalgebras are nearly never countably generated, certain desirable properties can be lost by introducing them. See, for example, Theorem 9.2.1. By combining Theorem 7.2.3 with Theorem 7.2.1, one can show that, up to time re-parametrization, all continuous martingales are Brownian motions. In order to avoid technical difficulties, I will prove this result only in the simplest case. Corollary 7.2.5. Let X(t), Ft , P be a continuous, square integrable martingale with the properties that, for P-almost every ω ∈ Ω, hXi( · , ω) is strictly increasing and exists a Brownian motion limt→∞ hXi(t, ω) = ∞. Then there B(t), Ft0 , P such that X(t) = X(0) + B hXi(t) , t ∈ [0, ∞) P-almost surely. In particular,
X(t) X(t) = 1 = − lim q lim q t→∞ 2hXi(t) log(2) hXi(t) 2hXi(t) log(2) hXi(t)
t→∞
P-almost surely. Proof: Clearly, given the first part, the last assertion is a trivial application of Exercise 4.3.15. After replacing F and the Ft ’s by their completions and applying Exercise 7.1.22, I may and will assume that X(0, ω) = 0, X( · , ω) is continuous, hXi( · , ω) is continuous and strictly increasing, and limt→∞ hXi(t, ω) = ∞ for every ω ∈ Ω. Next, for each (t, ω) ∈ [0, ∞), set ζt (ω) = hXi−1 (t, ω), where hXi−1 ( · , ω) is the inverse of hXi( · , ω). Clearly, for each ω ∈ Ω, t ζt (ω) is a continuous, strictly increasing function that tends to infinity as t → ∞. Moreover, because hXi is progressively measurable, ζt is a stopping time for each t ∈ [0, ∞). Now set
§ 7.2 Brownian Motion and Martingales
289
B(t) = X(ζt ). Since it is obvious that X(t) = B hXi(t) , all that I have to show is that B(t), Ft0 , P is a Brownian motion for some non-decreasing family {Ft0 : t ≥ 0} of sub-σ-algebras. Trivially, B(0, ω) = 0 and B( · , ω) is continuous for all ω ∈ Ω. In addition, B(t) is Fζt -measurable, and so B is progressively measurable with respect to {Fζt : t ≥ 0}. Thus, by Theorem 7.2.1, I will be done once I show that 2 B(t), Fζt , P and B(t) − t, Fζt , P are martingales. To this end, first observe that " # " # EP
sup X(τ )2 = lim EP τ ∈[0,ζt ]
T →∞
sup
X(τ )2
τ ∈[0,T ∧ζt ]
≤ 4 lim EP X(T ∧ ζt )2 ≤ 4 lim EP hXi(T ∧ ζt ) ≤ 4t. T →∞
T →∞
Thus, limT →∞ X(T ∧ ζt ) −→ B(t) in L2 (P; R). Now let 0 ≤ s < t and A ∈ Fζs be given. Then, for each T > 0, AT ≡ A ∩ {ζs ≤ T } ∈ FT ∧ζs , and so, by Theorem 7.1.14, EP X(T ∧ ζt ), AT = EP X(T ∧ ζs ), AT and EP X(T ∧ ζt )2 − hXi(T ∧ ζt ), AT = EP X(T ∧ ζs )2 − hXi(T ∧ ζs ), AT . Now let T → ∞, and apply the preceding convergence assertion to get the desired conclusion. § 7.2.3. Burkholder’s Inequality Again. In this subsection we will see what Burkholder’s Inequality looks like in the continuous parameter setting, a result whose importance for the theory of stochastic integration is hard to overstate. Theorem 7.2.6 (Burkholder). Let X(t), Ft , P be a P-almost surely continuous, square integrable martingale. Then, for each p ∈ (1, ∞) and t ∈ [0, ∞) (cf. (6.3.2)), p1 (7.2.7) Bp−1 kX(t) − X(0)kLp (P;R) ≤ EP hX(t)i 2 p ≤ Bp kX(t) − X(0)kLp (P;R) .
Proof: After completing the σ-algebras if necessary, I may (cf. Exercise 7.1.22) and will assume that X( · , ω) is continuous and that hXi( · , ω) is continuous and non-decreasing for every ω ∈ Ω. In addition, I may and will assume that X(0) = 0. Finally, I will assume that X is bounded. To justify this last assumption, let ζn = inf{t ≥ 0 : |X(t)| ≥ n}, set Xn (t) = X(t ∧ ζn ), and use Exercise 7.2.10 to see that one can take hXn i = hXi(t ∧ ζn ). Hence, if we know (7.2.7) for bounded martingales, then p1 Bp−1 kX(t ∧ ζn )kLp (P;R) ≤ EP hXi(t ∧ ζn ) 2 p ≤ Bp kX(t ∧ ζn )kLp (P;R)
290
7 Continuous Parameter Martingales
for all n ≥ 1. Since hXi is non-decreasing, we can apply Fatou’s Lemma to the preceding and thereby get
p1 kX(t)kLp (P;R) ≤ lim kX(t ∧ ζn )kLp (P;R) ≤ Bp EP hXi(t) 2 p , n→∞
which is the left-hand side of (7.2.7). To get the right-hand side, note that either kX(t)kLp (P;R) = ∞, in which case there is nothing to do, or kX(t)kLp (P;R) < ∞, in which case, by the second half of Theorem 7.1.9, X(t ∧ ζn ) −→ X(t) in Lp (P; R) and therefore
p1 p1 EP hXi(t) 2 p = lim EP hXi(t ∧ ζn ) 2 p n→∞
≤ Bp lim kX(t ∧ ζn )kLp (P;R) = Bp kX(t)kLp (P;R) . n→∞
Proceeding under the above assumptions and referring to the notation in the proof of Theorem 7.2.3, begin by observing that, for any t ∈ [0, ∞) and n ∈ N, Theorem 7.1.14 shows that X(t ∧ ζk,n ), Ft∧ζk,n , P is a discrete parameter martingale indexed by k ∈ N. In addition, ζk,n = t for all but a finite number of k’s. Hence, by (6.3.7) applied to X(t ∧ ζk,n ), Ft∧ζk,n , P , p1 Bp−1 kX(t)kLp (P;R) ≤ EP hXin (t) 2 p ≤ Bp kX(t)kLp (P;R)
for all n ∈ N.
In particular, this shows that supn≥0 khXin (t)kLp (P;R) < ∞ for every p ∈ (1, ∞), and therefore, since hXin (t) −→ hXi(t) (a.s., P), this is more than enough to p p P P 2 2 verify that E hXin (t) −→ E hXi(t) for every p ∈ (1, ∞).
Exercises for § 7.2 Exercise 7.2.8. Let X(t), Ft , P be a square integrable, continuous martingale. Following the strategy used to prove Theorem 7.2.1, show that Z F X(t) − 0
t 1 2 2 ∂x F
X(τ ) hXi(dτ ), Ft , P
is a martingale for every F ∈ Cb2 (R; C). Hint: Begin by using cutoffs and mollification to reduce to the case when F ∈ Cc∞ (R; R). Next, given s < t and > 0, introduce the stopping times ζ0 = s and ζn = inf{t ≥ ζn−1 : |X(t) − X(ζn−1 )| ≥ } ∧ (ζn−1 + ) ∧ (hXi(ζn−1 ) + ) ∧ t for n ≥ 1. Now proceed as in the proof of Theorem 7.2.1.
Exercises for § 7.2
291
Exercise 7.2.9. Let X(t), Ft , P be a continuous, square integrable martingale with X(0) = 0, and assume that there exists a non-decreasing function A : [0, ∞) −→ [0, ∞) such that hXi(t) ≤ A(t) (a.s., P) for each t ∈ [0, ∞). The goal of this exercise is to show that E(t), Ft , P is a martingale when E(t) = exp X(t) − 12 hXi(t) .
(i) Given R ∈ (0, ∞), set ζR = inf{t ≥ 0 : |X(t)| ≥ R}, and show that ! Z t∧ζR
eX(t∧ζR ) −
1 2
eX(τ ) dhXi, Ft , P
0
is a martingale. Hint: Choose F ∈ Cc∞ (R; R) so that F (x) = ex for x ∈ [−2R, 2R], apply Exercise 7.2.8 to this F , and then use Doob’s Stopping Time Theorem. 1
(ii) Apply Theorem 7.1.17 to the martingale in (i) and e− 2 hXi(t∧ζR ) to show that E(t ∧ ζR ), Ft , P is a martingale.
(iii) By replacing X and R with 2X and 2R in (ii), show that EP E(t ∧ ζR )2 ≤ eA(t) EP e2X(t∧ζR )−2hXi(t∧ζR ) = eA(t) . Conclude that {E(t ∧ ζR ) : R ∈ (0, ∞)} is uniformly P-integrable and therefore that E(t), Ft , P is a martingale. Exercise 7.2.10. If X(t), Ft , P is a P-almost surely continuous, square integrable martingale, ζ is a stopping time, and Y (t) = X(t ∧ ζ), show that hY i(t) = hXi(t ∧ ζ), t ≥ 0, P-almost surely. Exercise 7.2.11. Continuing in the setting of Exercise 7.2.9, first show that, for every λ ∈ R, Eλ (t), Ft , P is a martingale, where Eλ (t) = exp λX(t) −
λ2 2 hXi(t)
.
Next, use Doob’s Inequality to see that, for each λ ≥ 0, ! ! sup X(τ ) ≥ R
P
τ ∈[0,t]
≤P
sup Eλ (τ ) ≥ eλR−
λ2 2
A(t)
≤ e−λR+
λ2 2
A(t)
.
τ ∈[0,t]
Starting from this, conclude that (7.2.12)
R2 P kXk[0,t] ≥ R ≤ 2e− 2A(t) .
Finally, given this estimate, show that the conclusion in Exercise 7.2.8 continues to hold for any F ∈ C 2 (R; C) whose second derivative has at most exponential growth.
292
7 Continuous Parameter Martingales
Exercise 7.2.13. Given a pair continuous martingales of square integrable, hX+Y i−hX−Y i , and show that X(t), Ft , P and Y (t), Ft , P , set hX, Y i = 4 X(t)Y (t) − hX, Y i(t), Ft , P is a martingale. Further, show that hX, Y i is uniquely determined up to a P-null set by this property together with the facts that hX, Y i(0, ω) = 0 and hX, Y i( · , ω) is continuous and has locally bounded variation for P-almost every ω ∈ Ω. Exercise 7.2.14. Let B(t), Ft , P be an RN -valued Brownian motion. Given f, g ∈ Cb1,2 [0, ∞) × RN ; R , set t
Z
X(t) = f t, B(t) −
∂τ + 12 ∆ f τ, B(τ ) dτ,
0
t
Z
∂τ + 12 ∆ g τ, B(τ ) dτ,
Y (t) = g t, B(t) − 0
and show that t
Z
∇f · ∇g τ, B(τ ) dτ.
hX, Y i(t) = 0
Hint: First reduce to the case when f = g. Second, write X(t)2 as f t, B(t)
2
t
Z
∂τ + 12 ∆ f τ, B(τ ) dτ
− 2X(t) 0
Z − 0
t
∂τ + 12 ∆ f τ, B(τ ) dτ
2 ,
and apply Theorem 7.1.17 to the second term. § 7.3 The Reflection Principle Revisited In Exercise 4.3.12 we saw that L´evy’s Reflection Principle (Theorem 1.4.13) has a sharpened version when applied to Brownian motion. In this section I will give another, more powerful way of discussing the reflection principle for Brownian motion. § 7.3.1. Reflecting Symmetric L´ evy Processes. In this subsection, µ will be used to denote a symmetric, infinitely divisible law. Equivalently (cf. Exercise 3.3.11), µ ˆ = e`µ (ξ) , where `µ (ξ) = −
1 ξ, Cξ RN + 2
Z RN
cos ξ, y RN − 1 M (dy)
for some non-negative definite, symmetric C and symmetric L´evy measure M .
§ 7.3 The Reflection Principle Revisited
293
Lemma 7.3.1. Let {Z(t) : t ≥ 0} be a L´evy process for µ, and set Ft = σ {Z(τ ) : τ ∈ [0, t]} . If ζ is a stopping time relative to Ft : t ∈ [0, ∞) and Z(t) if ζ > t ˜ Z(t) ≡ 2Z(t ∧ ζ) − Z(t) = 2Z(ζ) − Z(t) if ζ ≤ t, ˜ : t ≥ 0} is again a L´evy process for µ. then {Z(t) Proof: According to Theorem 7.1.3, all that I have to show is that √ ˜ − t` (ξ) , F , P exp −1 (ξ, Z(t) µ t RN
is a martingale for all ξ ∈ RN . Thus, let 0 ≤ s < t and A ∈ Fs be given. Then, by Theorem 7.1.14 and the fact that `µ (−ξ) = `µ (ξ), i h √ ˜ − t` (ξ) , A ∩ {ζ ≤ s} EP exp −1 (ξ, Z(t) µ RN i h √ √ = EP e2 −1(ξ,Z(s∧ζ))RN exp − −1 (ξ, Z(t) RN − t`µ (ξ) , A ∩ {ζ ≤ s} i h √ √ = EP e2 −1(ξ,Z(s∧ζ))RN exp − −1 ξ, Z(s) RN − s`µ (ξ) , A ∩ {ζ ≤ s} i h √ ˜ − t` (ξ) , A ∩ {ζ ≤ s} . = EP exp −1 ξ, Z(s) µ N R
Similarly, i h √ ˜ − t` (ξ) , A ∩ {ζ > s} EP exp −1 ξ, Z(t) µ RN i h √ √ = EP e2 −1(ξ,Z(t∧ζ))RN exp − −1 (ξ, Z(t) RN − t`µ (ξ) , A ∩ {ζ > s} i h √ = EP exp −1 ξ, Z(t ∧ ζ) RN − (t ∧ ζ)`µ (ξ) , A ∩ {ζ > s} i h √ = EP exp −1 ξ, Z(s ∧ ζ) RN − (s ∧ ζ)`µ (ξ) , A ∩ {ζ > s} i h √ ˜ − s` (ξ) , A ∩ {ζ > s} . = EP exp −1 ξ, Z(s) µ N R
˜ Obviously, the process {Z(t) : t ≥ 0} in Lemma 7.3.1 is the one obtained by reflecting (i.e., reversing the direction of {Z(t) : t ≥ 0}) at time ζ, and the lemma says that the distribution of the resulting process is the same as that of the original one. Most applications of this result are to situations when one knows more or less precisely where the process is at the time when it is reflected. For example, suppose N = 1, a ∈ (0, ∞), and ζa = inf{t ≥ 0 : Z(t) ≥ a}. Noting ˜ = Z(t) for t ≤ ζa and therefore that ζa = inf{t ≥ 0 : Z(t) ˜ ≥ that, because Z(t) a}, we have that P Z(t) ≤ x & ζa ≤ t = P 2Z(ζa ) − Z(t) ≤ x & ζa ≤ t = P Z(t) ≥ 2Z(ζa ) − x & ζa ≤ t .
294
7 Continuous Parameter Martingales
Hence, if x ≤ a, and therefore Z(t) ≥ 2Z(ζa ) − x =⇒ ζa ≤ t when ζa < ∞, then P Z(t) ≤ x & ζa ≤ t = P Z(t) ≥ 2Z(ζa ) − x & ζa < ∞ for x ≤ a. Applying this when x = a and using P ζ ≤ t = P Z(t) ≤ a & ζ ≤ t + a a P Z(t) > a , one gets P ζa ≤ t ≤ 2P Z(t) ≥ a , a conclusion that also could have been reached via Theorem 1.4.13. § 7.3.2. Reflected Brownian Motion. The considerations in the preceding subsection are most interesting when applied to R-valued Brownian motion. Thus, let B(t), Ft , P be an R-valued Brownian motion. To appreciate the improvements that can be made in the calculations just made, again take ζa = inf{t ≥ 0 : B(t) ≥ a} for some a > 0. Then, because Brownian paths are continuous, ζa < ∞ =⇒ B(ζa ) = a and so, since P(ζa < ∞) = 1, we can say that (7.3.2) P B(t) ≤ x & ζa ≤ t = P B(t) ≥ 2a−x for (t, x) ∈ [0, ∞)×(−∞, a]. In particular, by taking x = a and using P B(t) ≥ a = P B(t) ≥ a & ζa ≤ t , we recover the result in Exercise 4.3.12 that P ζa ≤ t = 2P B(t) ≥ a . A more interesting application of Lemma 7.3.1 to Brownian motion is to the case when ζ is the exit time from an interval other than a half-line. Theorem 7.3.3. Let a1 < 0 < a2 be given, define ζ (a1 ,a2 ) = inf{t ≥ 0 : B(t) ∈ / (a1 ,a2 ) (a1 ,a2 ) (a1 , a2 )}, and set Ai (t) = {ζ ≤ t & B(ζ ) = ai } for i ∈ {1, 2}. Then, for Γ ∈ B[a1 ,∞) , 0 ≤ P {B(t) ∈ Γ} ∩ A1 (t) − P {B(t) ∈ 2(a2 − a1 ) + Γ} ∩ A1 (t) = P B(t) ∈ 2a1 − Γ − P B(t) ∈ 2(a2 − a1 ) + Γ and, for Γ ∈ B(−∞,a2 ] , 0 ≤ P {B(t) ∈ Γ} ∩ A2 (t) − P {B(t) ∈ −2(a2 − a1 ) + Γ} ∩ A2 (t) = P B(t) ∈ 2a2 − Γ − P B(t) ∈ −2(a2 − a1 ) + Γ . Hence, for Γ ∈ B[a1 ,∞) , P {B(t) ∈ Γ} ∩ A1 (t) equals ∞ h X i γ0,t Γ − 2a1 + 2(m − 1)(a2 − a1 ) − γ0,t Γ + 2m(a2 − a1 ) m=1
and, for Γ ∈ B(−∞,a2 ] , P {B(t) ∈ Γ} ∩ A2 (t) equals ∞ h X i γ0,t Γ − 2a2 − 2(m − 1)(a2 − a1 ) − γ0,t Γ − 2m(a2 − a1 ) , m=1
where in both cases the convergence is uniform with respect t in compacts and Γ ∈ B(a1 ,a2 ) .
§ 7.3 The Reflection Principle Revisited
295
Proof: Suppose Γ ∈ B[a1 ,∞) . Then, by Lemma 7.3.1, P {B(t) ∈ Γ} ∩ A1 (t) = P {2a1 − B(t) ∈ Γ} ∩ A1 (t) = P B(t) ∈ 2a1 − Γ − P {B(t) ∈ 2a1 − Γ} ∩ A2 (t) , since B(t) ∈ 2a1 − Γ =⇒ B(t) ≤ a1 =⇒ ζ (a1 ,a2 ) ≤ t. Similarly, P {B(t) ∈ Γ} ∩ A2 (t) = P {2a2 − B(t) ∈ Γ} ∩ A1 (t) = P B(t) ∈ 2a2 − Γ − P {B(t) ∈ 2a2 − Γ} ∩ A1 (t) when Γ ∈ B(−∞,a2 ] . Hence, since 2a1 − Γ ⊆ (−∞, a1 ] ⊆ (−∞, a2 ] if Γ ∈ B[a1 ,∞) , P {B(t) ∈ Γ} ∩ A1 (t) = P B(t) ∈ 2a1 − Γ − P B(t) ∈ 2(a2 − a1 ) + Γ + P {B(t) ∈ 2(a2 − a1 ) + Γ} ∩ A1 (t) when Γ ∈ B[a1 ,∞) . Similarly, when Γ ∈ B(−∞,a2 ] , P {B(t) ∈ Γ} ∩ A2 (t) = P B(t) ∈ 2a2 − Γ − P B(t) ∈ −2(a2 − a1 ) + Γ + P {B(t) ∈ −2(a2 − a1 ) + Γ} ∩ A2 (t) . To check that P {B(t) ∈ Γ}∩A1 (t) −P {B(t) ∈ 2(a2 −a1 )+Γ}∩A1 (t) ≥ 0 when Γ ∈ B[a1 ,∞) , first use Theorem 7.1.16 to see that P {B(t) ∈ Γ} ∩ A1 (t) = EP γ0,t−ζ (a1 ,a2 ) (Γ − a1 ), A1 (t) . Second, observe that, because Γ ⊆ [a1 , ∞), γ0,τ 2(a2 − a1 ) + Γ ≤ γ0,τ (Γ) for all τ ≥ 0. The case when Γ ∈ B(−∞,a2 ] and A1 (t) is replaced by A2 (t) is handled in the same way. Given the preceding, one can use induction to check that P {B(t) ∈ Γ}∩A1 (t) equals M h X
i P B(t) ∈ 2a1 − 2(m − 1)(a2 − a1 ) − Γ − P B(t) ∈ 2m(a2 − a1 ) + Γ
m=1
+ P {B(t) ∈ 2M (a2 − a1 ) + Γ} ∩ A1 (t) for all Γ ∈ B[a1 ,∞) . The same line of reasoning applies when Γ ∈ B(−∞,a2 ] and A1 (t) is replaced by A2 (t). Perhaps the most useful consequence of the preceding is the following corollary.
296
7 Continuous Parameter Martingales
Corollary 7.3.4. Given a c ∈ R and an r ∈ (0, ∞), set I = (c − r, c + r) and P I (t, x, Γ) = P {x + B(t) ∈ Γ} ∩ {ζ I > t} , x ∈ I and Γ ∈ BI . Then Z
I
(7.3.5)
P I (t, z, Γ) P I (s, x, dz).
P (s + t, x, Γ) = I
Next, set g˜(t, x) =
X
g(t, x + 4m),
1
x2
where g(t, x) = (2πt)− 2 e− 2t
m∈Z
and p(−1,1) (t, x, y) = g˜(t, y − x) − g˜(t, y + x + 2)
for (t, x, y) ∈ (0, ∞) × [−1, 1]2 .
Then p(−1,1) is a smooth function that is symmetric in (x, y), strictly positive on (0, ∞) × (0, 1)2 , and vanishes when x ∈ {−1, 1}. Finally, if pI (t, x, y) = r−1 p(−1,1) r−2 , r−1 (x − c), r−1 (y − c) ,
(t, x, y) ∈ (0, ∞) × I 2 ,
then I
(7.3.6)
Z
p (s + t, x, y) =
pI (s, x, z)pI (t, z, y) dz
I
and, for (t, x) ∈ (0, ∞) × I, P I (t, x, dy) = pI (t, x, y) dy. Proof: Begin by applying Theorem 7.1.16 to check that P I (s + t, x, Γ) equals W (1) {x + ψ(s) + δs ψ(t) ∈ Γ} ∩ {x + ψ(s) + δs ψ(τ ), τ ∈ [0, t − s]} ∩ {x + ψ(σ) ∈ I, σ ∈ [0, s]} (1) = EW P I t, x + ψ(s), Γ , {x + ψ(σ) ∈ I, σ ∈ [0, s]} Z = P I (t, z, Γ) P I (s, x, dz). I
Next, set a1 = r−1 (c − x) − 1 and a2 = r−1 (x − x) + 1. Then P I (t, x, Γ) = P {B(t) ∈ Γ − x} ∩ {B(τ ) ∈ (ra1 , ra2 ), τ ∈ [0, t]} = P {B(r−2 t) ∈ r−1 (Γ − x)} ∩ {B(r−2 τ ) ∈ (a1 , a2 ), τ ∈ [0, t]} = P B(r−2 t) ∈ r−1 (Γ − x) & ζ (a1 ,a2 ) > r−2 t = P B(r−2 t) ∈ r−1 (Γ − x) − P B(r−2 t) ∈ r−1 (Γ − x) & ζ (a1 ,a2 ) ≤ r−2 t ,
§ 7.3 The Reflection Principle Revisited
297
where, in the passage to the second line, I have used Brownian scaling. Now, use the last part of Theorem 7.3.3, the symmetry of γ0,r−2 t , and elementary rearrangement of terms to arrive first at P I (t, x, Γ) =
Xh
i γr−2 t 4m + r−1 (Γ − x) − γr−2 t 4m + 2 + r−1 (Γ + x − 2c) ,
m∈Z
and then at P I (t, x, dy) = pI (t, x, y) dy. Given this and (7.3.5), (7.3.6) is obvious. Turning to the properties of p(−1,1) (t, x, y), both its symmetry and smoothness are clear. In addition, as the density for P (−1,1) (t, x. · ), it is non-negative, and, because x g˜(t, x) is periodic with period 4, it is easy to see that (−1,1) p (t, ±1, y) = 0. Thus, everything comes down to proving that p(−1,1) (t, x, y) > 0 for (t, x, y) ∈ (0, ∞) × (−1, 1)2 . To this end, first observe that, after rearranging terms, one can write p(−1,1) (t, x, y) as g(t,y − x) − g(t, y + x) + g(t, 2 − x − y) ∞ h X + g(t, y − x + 4m) − g(t, y + x + 2 + 4m) m=1
i + g(t, y − x − 4m) − g(t, y + x − 2 − 4m) . Since each of the terms in the sum over m ∈ Z+ is positive, we have that
2(1−|x|)(1−|y|) t ≥ 1 − 2e g(t, y − x) p(−1,1) (t, x, y) > g(t, y − x) 1 − 2e− if t ≤ 2(1 − |x|)(1 − |y|). Hence, for each θ ∈ (0, 1), p(−1,1) (t, x, y) > 0 for all (t, x, y) ∈ [0, 2θ2 ] × [−1 + θ, 1 − θ]2 . Finally, to handle x, y ∈ [−1 + θ, 1 − θ] and t > 2θ2 , apply (7.3.6) with I = (−1, 1) to see that p
(−1,1)
2
Z
(m + 1)θ , x, y) ≥
p(−1,1) (θ2 , x, z)p(−1,1) (mθ2 , z, y) dz,
|z|≤(1−θ)
and use this and induction to see that p(−1,1) (mθ2 , x, y) > 0 for all m ≥ 1. Thus, if n ∈ Z+ is chosen so that nθ2 < t ≤ (n + 1)θ2 , then another application of (7.3.6) shows that (−1,1)
p
Z (t, x, y) ≥ |z|≤(1−θ)
p(−1,1) (t − nθ2 , x, z)p(−1,1) (nθ2 , z, y) dz > 0.
298
7 Continuous Parameter Martingales Exercises for § 7.3
Exercise 7.3.7. Suppose that G is a non-empty, open subset of RN , define ζxG : C(RN ) −→ [0, ∞] by ζxG (ψ) = inf{t ≥ 0 : x + ψ(t) ∈ / G}, and set P G (t, x, Γ) = W (N ) {ψ : x + ψ(t) ∈ Γ & ζxG (ψ) > t} for (t, x) ∈ (0, ∞) × G and Γ ∈ BG . (i) Show that G
Z
P G (t, z, Γ) P G (s, x, dy).
P (s + t, x, Γ) = G
(ii) As an application of Exercise 7.1.25, show that P G (t, x, Γ) = γ0,tI (Γ − x) − EW
(N )
γ0,(t−ζxG )I Γ − x − ψ(ζxG ) , ζxG ≤ Γ .
. This is the probabilistic version of Duhamel’s Formula, which we will see again in § 10.3.1. (iii) As a consequence of (ii), show that there is a Borel measurable function pG : (0, ∞) × G2 −→ [0, ∞) such that (t, y) pG (t, x, y) is continuous for each x ∈ G and P G (t, x, dy) = pG (t, x, y) dy for each (t, x) ∈ (0, ∞) × G. In particular, use this in conjunction with (i) to conclude that Z G p (s + t, x, y) = pG (t, z, y)pG (s, x, z) dz. G N
Hint: Keep in mind that (τ, ξ) (2πτ )− 2 e− long as ξ stays away from the origin.
|ξ|2 2τ
is smooth and bounded as
(iv) Given c = (c1 , . . . , cN ) ∈ RN and r > 0, let Q(c, r) denote the open cube QN i=1 (ci − r, ci + r), and show that (cf. Corollary 7.3.4) pQ(c,r) (t, x, y) =
N Y
p(ci −r,ci +r) (t, xi , yi )
i=1
for x = (x1 , . . . , xN ), y = (y1 , . . . , yN ) ∈ Q(c, r). In particular, conclude that pQ(c,r) (t, x, y) is uniformly positive on compact subsets of (0, ∞) × Q(c, r)2 . (v) Assume that G is connected, and show that pG (t, x, y) is uniformly positive on compact subsets of (0, ∞) × G2 . Hint: If Q(c, r) ⊆ G, show that pG (t, x, y) ≥ pQ(c,r) (t, x, y) on (0, ∞)×Q(c, r)2 .
Chapter 8 Gaussian Measures on a Banach Space
As I said at the end of § 4.3.2, the distribution of Brownian motion is called Wiener measure because Wiener was the first to construct it. Wiener’s own thinking about his measure had little or nothing in common with the L´evy– Khinchine program. Instead, he looked upon his measure as a Gaussian measure on an infinite dimensional space, and most of what he did with his measure is best understood from that perspective. Thus, in this chapter, we will look at Wiener measure from a strictly Gaussian point of view. More generally, we will be dealing here with measures on a real Banach space E that are centered Gaussian in the sense that, for each x∗ in the dual space E ∗ , x ∈ E 7−→ hx, x∗ i ∈ R is a centered Gaussian random variable. Not surprisingly, such a measure will be said to be a centered Gaussian measure on E . Although the ideas that I will use are already implicit in Wiener’s work, it was I. Segal and his school, especially L. Gross,1 who gave them the form presented here. § 8.1 The Classical Wiener Space In order to motivate what follows, it is helpful to first understand Wiener measure from the point of view which I will be adopting here. § 8.1.1. Classical Wiener Measure. Up until now I have been rather casual about the space from which Brownian paths come. Namely, because Brownian paths are continuous, I have thought of their distribution as being a probability on the space C(RN ) = C [0, ∞); RN . In general, there is no harm done by choosing C(RN ) as the sample space for Brownian paths. However, for my purposes here, I need my sample spaces to be separable Banach spaces, and, although it is a complete, separable metric space, C(RN ) is not a Banach space. With this in mind, define Θ(RN ) to be the space of continuous paths θ : [0, ∞) −→ RN with the properties that θ(0) = 0 and limt→∞ t−1 |θ(t)| = 0. 1
See I.E. Segal’s “Distributions in Hilbert space and canonical systems of operators,” T.A.M.S., 88 (1958) and L. Gross’s “Abstract Wiener spaces,” Proc. 5th Berkeley Symp. on Prob. & Stat., 2 (1965), Univ. of California Press. A good exposition of this topic can be found in H.-H. Kuo’s Gaussian Measures in Banach Spaces, Springer-Verlag, Math. Lec. Notes., # 463 (1975).
299
300
8 Gaussian Measures on a Banach Space
Lemma 8.1.1. The map |ψ(t)| ∈ [0, ∞] t≥0 1 + t is lower semicontinuous, and the pair Θ(RN ), k · kΘ(RN ) is a separable Banach space that is continuously embedded as a Borel measurable subset of C(RN ). In N particular, BΘ(RN ) coincides with BC(RN ) [Θ(R )] = A∩Θ(RN ) : A ∈ BC(RN ) . ∗ Moreover, the dual space Θ(RN ) of Θ(RN ) can be identified with the space of RN -valued, Borel measures λ on [0, ∞) with the properties that λ({0}) = 0 and 1 Z ∗ kλkΘ(RN ) ≡ (1 + t) |λ|(dt) < ∞, ψ ∈ C(RN ) 7−→ kψkΘ(RN ) ≡ sup
[0,∞)
when the duality relation is given by Z hθ, λi =
θ(t) · λ(dt).
[0,∞)
Finally, if (B(t), Ft , P) is an RN -valued Brownian motion, then B ∈ Θ(RN ) P-almost surely and EP kBk2Θ(RN ) ≤ 32N. Proof: It is obvious that the inclusion map taking Θ(RN ) into C(RN ) is continuous. To see that k · kΘ(RN ) is lower semicontinuous on C(RN ) and that Θ(RN ) ∈ BC(RN ) , note that, for any s ∈ [0, ∞) and R ∈ (0, ∞), n o A(s, R) ≡ ψ ∈ C(RN ) : ψ(t) ≤ R(1 + t) for t ≥ s is closed in C(RN ). Hence, since kψkΘ(RN ) ≤ R ⇐⇒ ψ ∈ A(0, R), k · kΘ(RN ) is lower semicontinuous. In addition, since {ψ ∈ C(RN ) : ψ(0) = 0} is also closed, Θ(RN ) =
∞ [ ∞ n o \ ψ ∈ A m, n1 : ψ(0) = 0 ∈ BC(RN ) . n=1 m=1
In order to analyze the space Θ(RN ), k · kΘ(RN ) , define N N N F : Θ(R ) −→ C0 R; R ≡ ψ ∈ C R; R : lim |ψ(s)| = 0 |s|→∞
by θ (es ) , F (θ) (s) = 1 + es 1
s ∈ R.
I use |λ| to denote the variation measure determined by λ.
§ 8.1 The Classical Wiener Space
301
As is well known, C0 R; RN with the uniform norm is a separable Banach space, N N and it is obvious that F is an isometry from Θ(R ) onto C0 R; R . Moreover, by the Riesz Representation Theorem for C0 R; RN , one knows that the dual of C0 R; RN is isometric to the space of totally finite, RN -valued measures on R; BR with the norm given by total variation. Hence, the identification ∗ of Θ(RN ) reduces to the obvious interpretation of the adjoint map F ∗ as a mapping from totally finite RN -valued measures onto the space of RN -valued measures that do not charge 0 and whose variation measure integrates (1 + t). Because of the Strong Law in part (ii) of Exercise 4.3.11, it is clear that almost every Brownian path is in Θ(RN ). In addition, by the Brownian scaling property and Doob’s Inequality (cf. Theorem 7.1.9), P
E
kBk2Θ(RN )
≤ =
∞ X n=0 ∞ X
4 2
−n+1
−n+2
P
E
2
sup |B(t)| 0≤t≤2n
P
E
sup |B(t)|
2
≤ 32EP |B(1)|2 = 32N.
0≤t≤1
n=0
In view of Lemma 8.1.1, we now know that the distribution of RN -valued Brownian motion induces a Borel measure W (N ) on the separable Banach space Θ(RN ), and throughout this chapter I will refer to this measure as the classical Wiener measure. My next goal is to characterize, in terms of Θ(RN ), exactly which measure on Θ(RN ) Wiener’s is, and for this purpose I will use the following simple fact about Borel probability measures on a separable Banach space. Lemma 8.1.2. Let E with norm k · kE be a separable, real Banach space, and use (x, x∗ ) ∈ E × E ∗ 7−→ hx, x∗ i ∈ R to denote the duality relation between E and its dual space E ∗ . Then the Borel field BE coincides with the σ-algebra generated by the maps x ∈ E 7−→ hx, x∗ i as x∗ runs over E ∗ . In particular, if, for µ ∈ M1 (E), one defines its Fourier transform µ ˆ : E ∗ −→ C by Z i h√ µ ˆ(x∗ ) = exp −1 hx, x∗ i µ(dx), x∗ ∈ E ∗ , E
then µ ˆ is a continuous function of weak* convergence on Θ∗ , and µ ˆ uniquely determines µ in the sense that if ν is a second element of M1 (Θ) and µ ˆ = νˆ, then µ = ν. Proof: Since it is clear that each of the maps x ∈ E 7−→ hx, x∗ i ∈ R is continuous and therefore BE -measurable, the first assertion will follow as soon
302
8 Gaussian Measures on a Banach Space
as we show that the norm x kxkE can be expressed as a measurable function of these maps. But, because E is separable, we know (cf. Exercise 5.1.19) that the closed unit ball BE ∗ (0, 1) in E ∗ is separable with respect to the weak* topology and therefore that we can find a sequence {x∗n :, n ≥ 1} ⊆ BE ∗ (0, 1) so that
kxkΘ = sup hx, x∗n i,
x ∈ E.
n∈Z+
Turning to the properties of µ ˆ, note that its continuity with respect to weak* convergence is an immediate consequence of Lebesgue’s Dominated Convergence Theorem. Furthermore, in view of the preceding, we will know that µ ˆ completely determines µ as soon as we show that, for each n ∈ Z+ and X ∗ = x∗1 , . . . , x∗n ∈ n E∗ , µ ˆ determines the marginal distribution µX ∗ ∈ M1 (RN ) of x ∈ E 7−→ hx, x∗1 i, . . . , hx, x∗n i ∈ Rn under µ. But this is clear (cf. Lemma 2.3.3), since ! n X µd ˆ ξm x∗m for ξ = (ξ1 , . . . , ξn ) ∈ Rn . X ∗ (ξ) = µ m=1
I will now compute the Fourier transform of W (N) . To this end, first recall that, for an RN -valued Brownian motion, { ξ, B(t) RN : t ≥ 0 and ξ ∈ RN spans a Gaussian family G(B) in L2 (P; R). Hence, span ξ, θ(t) : t ≥ 0 and ξ ∈ RN is a Gaussian family in L2 (W (N ) ; R). From this, combined with an easy limit argument using Riemann sum approximations, one sees that, ∗ for any λ ∈ Θ(RN ) , θ hθ, λi is a centered Gaussian random variable under W (N ) . Furthermore, because, for 0 ≤ s ≤ t, (N ) (N ) EW ξ, θ(s) RN η, θ(t) RN = EW ξ, θ(s) RN η, θ(s) RN = s ξ, η RN , we can apply Fubini’s Theorem to see that ZZ (N ) EW hθ, λi2 = s ∧ t λ(ds) · λ(dt). [0,∞)2
Therefore, we now know that W (N ) is characterized by its Fourier transform ZZ 1 ∗ \ (N ) (λ) = exp − s ∧ t λ(ds) · λ(dt) , λ ∈ Θ(RN ) . (8.1.3) W 2 [0,∞)2
Equivalently, we have shown that W (N ) is the centered Gaussian measure on ∗ Θ(RN ) with the property that, for each λ ∈ Θ(RNRR ) ,θ hθ, λi is a centered Gaussian random variable with variance equal to s ∧ t λ(ds) · λ(dt). [0,∞)2
§ 8.1 The Classical Wiener Space
303
§ 8.1.2. The Classical Cameron–Martin Space. From the Gaussian standpoint, it is extremely unfortunate that the natural home for Wiener measure is a Banach space rather than a Hilbert space. Indeed, in finite dimensions, every centered, Gaussian measure with non-degenerate covariance can be thought of as the canonical, or standard, Gaussian measure on a Hilbert space. Namely, if γ0,C is the Gaussian measure on RN with mean 0 and non-degenerate covariance C, consider RN as a Hilbert space H with inner product (g, h)H = (g, Ch)RN , and take λH to be the natural Lebesgue measure there: the one that assigns measure 1 to a unit cube in H or, equivalently, the one obtained by pushing the 1 usual Lebesgue measure λRN forward under the linear transformation C 2 . Then we can write khk2 1 − 2H λH (dh) e γ0,C (dh) = N (2π) 2 and 2 − γd 0,C (h) = e
khk H 2
.
As was already pointed out in Exercise 3.1.11, in infinite dimensions there is no precise analog of the preceding canonical representation (cf. Exercise 8.1.7 for further corroboration of this point). Nonetheless, a good deal of insight can be gained by seeing how close one can come. In order to guess on which Hilbert space it is that W (N ) would like to live, I will give R. Feynman’s highly questionable but remarkably powerful way of thinking about such matters. Namely, n given n ∈ Z+ , 0 = t0 < t1 < · · · < tn , and a set A ∈ BRN , we know that W (N ) assigns θ : θ(t1 ), . . . , θ(tn ) ∈ A probability # " Z n X |ym − ym−1 |2 1 dy1 · · · dyn , exp − tm − tm−1 Z(t1 , . . . , tn ) A m=1 N Qn where y0 ≡ 0 and Z(t1 , . . . , tn ) = m=1 2π(tm − tm1 ) 2 . Now rename the variable ym as θ(tm ), and rewrite the preceding as Z(t1 , . . . , tn )−1 times !2 Z n X θ(t ) − θ(t ) t − t m m−1 m m−1 dθ(t1 ) · · · dθ(tn ). exp − tm − tm−1 2 A m=1
Obviously, nothing very significant has happened yet, since nothing very exciting has been done yet. However, if we now close our eyes, suspend our disbelief, and pass to the limit as n tends to infinity and the tk ’s become dense, we arrive at Feynman’s representation 2 of Wiener’s measure: # " Z 2 1 1 (N ) ˙ dt dθ, θ(t) (8.1.4) W dθ) = exp − 2 [0,∞) Z 2
In truth, Feynman himself never dabbled in considerations so mundane as the ones that √ follow. He was interested in the Sch¨ odinger equation, and so he had a factor −1 multiplying the exponent.
304
8 Gaussian Measures on a Banach Space
where θ˙ denotes the velocity (i.e., derivative) of θ. Of course, when we reopen our eyes and take a look at (8.1.4), we see that it is riddled with flaws. Not even one of the ingredients on the right-hand side of (8.1.4) makes sense! In the first place, the constant Z must be 0 (or maybe ∞). Secondly, since the image of the “measure dθ” under n θ ∈ Θ(RN ) 7−→ θ(t1 ) . . . , θ(tn ) ∈ RN is Lebesgue measure for every n ∈ Z+ and 0 < t1 · · · < tn , dθ must be the nonexistent translation invariant measure on the infinite dimensional space Θ(RN ). Finally, the integral in the exponent only makes sense if θ is differentiable in some sense, but almost no Brownian path is. Nonetheless, ridiculous as it is, (8.1.4) is exactly the expression at which one would arrive if one were to make a sufficiently na¨ıve interpretation of the notion that Wiener measure is the standard Gauss measure on the Hilbert space H(RN ) consisting of absolutely continuous h : [0, ∞) −→ RN with h(0) = 0 and ˙ L2 ([0,∞);RN ) < ∞. khkH1 (RN ) = khk Of course, the preceding discussion is entirely heuristic. However, now that we know that H1 (RN ) is the Hilbert space at which to look, it is easy to provide a mathematically rigorous statement of the connection between Θ(RN ), W (N ) , and H1 (RN ). To this end, observe that H(RN ) is continuously embedded in 1 Θ(RN ) as a dense subspace. Indeed, if h ∈ H1 (RN ), then |h(t)| ≤ t 2 khkH1 (RN ) , and so not only is h ∈ Θ(RN ) but also khkΘ(RN ) ≤ 12 khkH1 (RN ) . In addition, since Cc∞ (0, ∞); RN is already dense in Θ(RN ), the density of H1 (RN ) in Θ(RN ) is clear. Knowing this, abstract reasoning (cf. Lemma 8.2.3) guarantees ∗ that Θ(RN ) can be identified as a subspace of H1 (RN ). That is, for each λ ∈ ∗ Θ(RN ) , there is a hλ ∈ H1 (RN ) with the property that h, hλ H1 (RN ) = hh, λi
for all h ∈ H1 (RN ), and in the present setting it is easy to give a concrete ∗ representation of hλ . In fact, if λ ∈ Θ(RN ) , then, for any h ∈ H1 (RN ), Z hh, λi =
Z
Z
h(t) · λ(dt) = (0,∞)
Z = (0,∞)
(0,∞)
! ˙ ) dτ h(τ
(0,t)
˙ ) · λ (τ, ∞) dτ = h, hλ 1 N , h(τ H (R )
where Z hλ (t) = (0,t]
λ (τ, ∞) dτ.
· λ(dt)
§ 8.1 The Classical Wiener Space
305
Moreover, khλ k2H1 (RN ) =
Z
λ (τ, ∞) |2 dτ =
(0,∞)
Z
ZZ
(0,∞)
λ(ds) · λ(dt) dτ
(τ,∞)2
ZZ =
s ∧ t λ(ds) · λ(dt).
(0,∞)2
Hence, by (8.1.3), \ (N ) (λ) = exp − W
(8.1.5)
khλ k2H(RN )
! ,
2
∗
λ ∈ Θ(RN ) .
Although (8.1.5) is far less intuitively appealing than (8.1.4), it provides a mathematically rigorous way in which to think of W (N ) as the standard Gaussian measure on H1 (RN ). Furthermore, there is another way to understand why one should accept (8.1.5) as evidence for this way of thinking about W (N ) . Indeed, ∗ given λ ∈ Θ(RN ) , write Z Z T hθ, λi = lim θ(t) · λ(dt) = − lim θ(t) · dλ (t, ∞) , T →∞
T →∞
[0,T ]
0
where the integral in the last expression is taken in the sense of Riemann– Stieltjes. Next, apply the integration by part formula3 to conclude that t λ (t, ∞) is Riemann–Stieltjes integrable with respect to t θ(t) and that Z T Z T − θ(t) · dλ (t, ∞) = −θ(T ) · λ (T, ∞) + λ (t, ∞) · dθ(t). 0
0
Hence, since |θ(T )| lim |θ(T )||λ|(T, ∞) ≤ lim T →∞ T →∞ 1 + T
Z (8.1.6)
hθ, λi = lim
T →∞
Z (1 + t) |λ|(dt) = 0, (0,∞)
T
h˙ λ (t) · dθ(t),
0
where again the integral is in the sense of Riemann–Stieltjes. Thus, if one ˙ somewhat casually writes dθ(t) = θ(t) dt, one can believe that hθ, λi provides a reasonable interpretation of θ, hλ H(RN ) for all θ ∈ Θ(RN ), not just those that are in H1 (RN ). Because R. Cameron and T. Martin were the first mathematicians to systematically exploit the consequences of this line of reasoning, I will call H1 (RN ) the Cameron–Martin space for classical Wiener measure. 3
See, for example, Theorem 1.2.7 in my A Concise Introduction to the Theory of Integration, Birkh¨ auser (1999).
306
8 Gaussian Measures on a Banach Space Exercises for § 8.1
Exercise 8.1.7. Let H be a separable Hilbert space, and, for each n ∈ Z+ and subset {g1 , . . . , gn } ⊆ H, let A(g1 , . . . , gn ) denote the σ-algebra over H generated by the mapping h ∈ H 7−→ (h, g1 )H , . . . , (h, gn )H ∈ Rn , and check that A=
[
A(g1 , . . . , gn ) : n ∈ Z+ and g1 , . . . , gn ∈ H
is an algebra that generates BH . Show that there always exists a finitely additive WH on A that is uniquely determined by the properties that it is σ-additive on A(g1 , . . . , gn ) for every n ∈ Z+ and {g1 , . . . , gn } ⊆ H and that Z i h√ kgk2H , g ∈ H. exp −1 (h, g)H WH (dh) = exp − 2 H
On the other hand, as we already know, this finitely additive measure admits a countably additive extension to BH if and only if H is finite dimensional. § 8.2 A Structure Theorem for Gaussian Measures Say that a centered Gaussian measure W on a separable Banach space E is non-degenerate if EW hx, x∗ i2 > 0 unless x∗ = 0. (See Exercise 8.2.11.) In this section I will show that any non-degenerate, centered Gaussian measure W on a separable Banach space E shares the same basic structure that W (N ) has on Θ(RN ). In particular, I will show that there is always a Hilbert space H ⊆ E for which W is the standard Gauss measure in the same sense that W (N ) was shown in § 8.1.2 to be the standard Gauss measure for H1 (RN ). § 8.2.1. Fernique’s Theorem. In order to carry out my program, I need a basic integrability result about Banach space–valued, Gaussian random variables. The one that I will use is due to X. Fernique, and his is arguably the most singularly beautiful result in the theory of Gaussian measures on a Banach space. Theorem 8.2.1 (Fernique’s Theorem). Let E be a real, separable Banach space, and suppose that X is an E-valued random variable that is centered and Gaussian in the sense that, for each x∗ ∈ E ∗ , hX, x∗ i is a centered, R-valued Gaussian random variable. If R = inf{r : P(kXkE ≤ r) ≥ 34 )}, then
(8.2.2)
∞ 2n h kXk2E i X 1 e . E e 18R2 ≤ K ≡ e 2 + 3 n=0
(See Corollary 8.4.3 for a sharpened statement.)
§ 8.2 A Structure Theorem for Gaussian Measures
307
Proof: After enlarging the sample space if necessary, I may and will assume that there is an E-valued random variable X 0 that is independent of X and has 1 1 the same distribution as X. Set Y = 2− 2 (X + X 0 ) and Y 0 = 2− 2 (X − X 0 ). Then the pair (Y, Y 0 ) has the same distribution as the pair (X, X 0 ). Indeed, by 2 Lemma 8.1.2, this random variable comes down to showing that the R ∗-valued ∗ 0 ∗ hY, x i, hY , x i has the same distribution as hX, x i, hX 0 , x∗ i , and that is an elementary application of the additivity property of independent Gaussians. Turning to the main assertion, let 0 < s ≤ t be given, and use the preceding to justify P kXkE ≤ s P kXkE ≥ t = P kXkE ≤ s & kX 0 kE ≥ t 1 1 = P kX − X 0 kE ≤ 2 2 s & kX + X 0 kE ≥ 2 2 t 1 1 ≤ P kXkE − kX 0 kE ≤ 2 2 s & kXkE + kX 0 kE ≥ 2 2 t 2 1 1 ≤ P kXkE ∧ kX 0 kE ≥ 2− 2 (t − s) = P kXkE ≥ 2− 2 (t − s) .
Now suppose that P kXk ≤ R ≥ 1 tn = R + 2 2 tn−1 for n ≥ 1. Then
3 4,
and define {tn : n ≥ 0} by t0 = R and
2 P kXkE ≤ R P kXkE ≥ tn ≤ P kXkE ≥ tn−1 and therefore P kXkE ≥ tn ≤ P kXkE ≤ R
P kXkE ≥ tn−1 P kXkE ≤ R
!2
for n ≥ 1. Working by induction, one gets from this that
!2n P kXkE ≥ R P kXkE ≤ R
P kXkE ≥ tn ≤ P kXkE ≤ R
and therefore, since tn = R 2
n+1 2 −1 1 2 2 −1
≤ 32
n+1 2
R, that P kXkE ≥ 32
n+1 2
n R ≤ 3−2 .
Hence, ∞ h kXk2E i X n+1 n n 1 e2 P 32 2 R ≤ kXkE ≤ 32 2 R EP e 18R2 ≤ e 2 P kXkE ≤ 3R + n=0 1
≤ e2 +
∞ X n=0
n e 2
3
= K.
§ 8.2.2. The Basic Structure Theorem. I will now abstract the relationship, proved in § 8.1.2, between Θ(RN ), H1 (RN ), and W (N ) , and for this purpose I will need the following simple lemma.
308
8 Gaussian Measures on a Banach Space
Lemma 8.2.3. Let E be a separable, real Banach space, and suppose that H ⊆ E is a real Hilbert space that is continuously embedded as a dense subspace of E. (i) For each x∗ ∈ E ∗ there is a unique hx∗ ∈ H with the property that h, hx∗ H = hh, x∗ i for all h ∈ H, and the map x∗ ∈ E ∗ 7−→ hx∗ ∈ H is linear, continuous, one-to-one, and onto a dense subspace of H. (ii) If x ∈ E, then x ∈ H if and only if there is a K < ∞ such that |hx, x∗ i| ≤ Kkhx∗ kH for all x∗ ∈ E ∗ . Moreover, for each h ∈ H, khkH = sup{hh, x∗ i : x∗ ∈ E ∗ & kx∗ kE ∗ ≤ 1}. (iii) If L∗ is a weak* dense subspace of E ∗ , then there exists a sequence {x∗n : n ≥ 0} ⊆ L∗ such that {hx∗n : n ≥ 0} is an orthonormal basis for H. Moreover, P∞ if x ∈ E, then x ∈ H if and only if n=0 hx, x∗n i2 < ∞. Finally, h, h0
H
=
∞ X
hh, x∗n ihh0 , x∗n i for all h, h0 ∈ H.
n=0
Proof: Because H is continuously embedded in E, there exists a C < ∞ such that khkE ≤ CkhkH . Thus, if x∗ ∈ E ∗ and f (h) = hh, x∗ i, then f is linear and |f (h)| ≤ khkE kx∗ kE ∗ ≤ Ckx∗ kE ∗ khkH , and so, by the Riesz Representation Theorem for Hilbert spaces, there exists a unique hx∗ ∈ H such that f (h) = h, hx∗ H . In fact, khx∗ kH ≤ Ckx∗ kE ∗ , and uniqueness can be used to check that x∗ hx∗ is linear. To see that x∗ hx∗ is one-to-one, it suffices to show ∗ that x = 0 if hx∗ = 0. But if hx∗ = 0, then hh, x∗ i = 0 for all h ∈ H, and therefore, because H is dense in E, x∗ = 0. Because I will use it later, I will prove slightly more than the density of just {hx∗ : x∗ ∈ E ∗ } in H. Namely, for any weak* dense subset S ∗ of E ∗ , {hx∗ : x∗ ∈ S ∗ } is dense in H. Indeed, if this were not the case, exist an h ∈ H \ {0} with the property that then there would ∗ ∗ hh, x i = h, hx∗ H = 0 for all x ∈ S. But, since S ∗ is weak* dense in E ∗ , this would lead to the contradiction that h = 0. Thus, (i) is now proved. Obviously, if h ∈ H, then |hh, x∗ i| = |(h, hx∗ )H | ≤ khx∗ kH khkH for x∗ ∈ E ∗ . Conversely, if x ∈ E and |hx, x∗ i| ≤ Kkhx∗ kH for some K < ∞ and all x∗ ∈ E ∗ , set f (hx∗ ) = hx, x∗ i for x∗ ∈ E ∗ . Then, because x∗ hx∗ is one-to-one, f ∗ ∗ ∗ is a well-defined, linear functional on {hx : x ∈ E }. Moreover, |f (x∗ )| ≤ Kkhx∗ kH , and therefore, since {hx∗ : x∗ ∈ E ∗ } is dense, f admits a unique extension as a continuous, linear functional on H. Hence, by Riesz’s theorem, there is an h ∈ H such that hx, x∗ i = f (hx∗ ) = h, hx∗
H
= hh, x∗ i,
x∗ ∈ E ∗ ,
which means that x = h ∈ H. In addition, if h ∈ H, then khkH = sup{hh, x∗ i : khx∗ kH ≤ 1} follows from the density of {hx∗ : x∗ ∈ E ∗ }, and this completes the proof of (ii).
§ 8.2 A Structure Theorem for Gaussian Measures
309
Turning to (iii), remember that, by Exercise 5.1.19, the weak* topology on E ∗ is second countable. Hence, the weak* topology on L∗ is also second countable and therefore separable. Thus, we can find a sequence in L∗ that is weak* dense in E ∗ , and then, proceeding as in the hint given for Exercise 5.1.19, extract a subsequence of linearly independent elements whose span S ∗ is weak* dense in E ∗ . Starting with this subsequence, apply the Grahm–Schmidt orthogonalization procedure to produce a sequence {x∗n : n ≥ 0} whose span is S ∗ and for which {hx∗n : n ≥ 0} is orthonormal in H. Moreover, because the span of {hx∗n : n ≥ 0} equals {hx∗ : x∗ ∈ S ∗ }, which, by what we proved earlier, is dense in H, {hx∗n : n ≥ 0} is an orthonormal basis in H. Knowing this, it is immediate that 0
h, h
H
=
∞ X
h, hxn
0
H
h , hxn
n=0
H
=
∞ X
hh, x∗n ihh0 , x∗n i.
n=0
P∞ P∞ 2 ∗ 2 ∗ 2 In particular, n=0 hx, xn i < ∞, P khkH ∗= n=0 hh, xn i . Finally, if x ∈ E and ∗ set g = m=0 hx, xn ihx∗n . Then g ∈ H and hx − g, x i = 0 for all x∗ ∈ S ∗ . Hence, since S ∗ is weak* dense in E ∗ , x = g ∈ H. Given a separable real Hilbert space H, a separable real Banach space E, and a W ∈ M1 (E), I will say that the triple (H, E, W) is an abstract Wiener space if H is continuously embedded as a dense subspace of E and W ∈ M1 (E) has Fourier transform (8.2.4)
c ∗ ) = e− W(x
khx∗ k2 H 2
for all x∗ ∈ E ∗ .
The terminology is justified by the fact, demonstrated at the end of § 8.1.2, that H1 (RN ), Θ(RN ), W (N ) is an abstract Wiener space. The concept of an abstract Wiener space was introduced by Gross, although his description was somewhat different from the one just given (cf. Theorem 8.3.9 for a reconciliation of mine with his definition). Theorem 8.2.5. Suppose that E is a separable, real Banach space and that W ∈ M1 (E) is a centered Gaussian measure that is non-degenerate. Then there exists a unique Hilbert space H such that (H, E, W) is an abstract Wiener space. q Proof: By Fernique’s Theorem, we know that C ≡ EW kxk2E < ∞. To understand the proof of existence, it is best to start with the proof of uniqueness. Thus, suppose that H is a Hilbert space for which (E, H, W) is an abstract Wiener space. Then, for all x∗ , y ∗ ∈ E ∗ , hhx∗ , y ∗ i = (hx∗ , hy∗ )H = hhy∗ , x∗ i. In addition, ∗
hhx∗ , x i =
khx∗ k2H
Z =
hx, x∗ i2 W(dx),
310
8 Gaussian Measures on a Banach Space
and so, by the symmetry just established, Z (*) hhx∗ , y ∗ i = khx∗ k2H = hx, x∗ ihx, y ∗ i W(dx), for all x∗ , y ∗ ∈ E ∗ . Next observe that Z
hx, x∗ i x W(dx) ≤ Ckhx∗ kH , (**) E R and therefore that the integral xhx, x∗ i W(dx) is a well-defined element of E. Moreover, by (*), Z ∗ ∗ ∗ hhx∗ , y i = xhx, x i W(dx), y for all y ∗ ∈ E ∗ , and so Z (***)
hx∗ =
xhx, x∗ i W(dx).
Finally, given h ∈ H, choose {x∗n : n ≥ 1} ⊆ E ∗ so that hx∗n −→ h in H. Then
lim sup h · , x∗n i − h · , x∗m i 2 = lim sup khx∗ − hx∗ kH = 0, m→∞ n>m
L (W;R)
m→∞ n>m
n
m
and so, if Ψ denotes the closure of {h · , x∗ i : x∗ ∈ E ∗ } in L2 (W; R) and F : Ψ −→ E is given by Z F (ψ) = xψ(x) W(dx), ψ ∈ Ψ, then h = F (ψ) for some ψ ∈ Ψ. Conversely, if ψ ∈ Ψ and {x∗n : n ≥ 1} is chosen so that h · , x∗n i −→ ψ in L2 (W; R), then {hx∗n : n ≥ 1} converges in H to some h ∈ H and it converges in E to F (ψ). Hence, F (ψ) = h ∈ H. In other words, H = F (Ψ). The proof of existence is now a matter of checking that if Ψ and F are defined as above and if H = F (Ψ) with kF (ψ)kH = kψkL2 (W;R) , then (H, E, W) is an abstract Wiener space. To this end, observe that Z ∗ hF (ψ), x i = hx, x∗ iψ(x) W(dx) = F (ψ), hx∗ H , and therefore both (*) and (***) hold for this choice of H. Further, given (*), it is clear that khx∗ k2H is the variance of h · , x∗ i and therefore that (8.2.4) holds. At the same time, just as in the derivation of (**), kF (ψ)kE ≤ CkψkL2 (W;R) = CkF (ψ)kH , and so H is continuously embedded inside E. Finally, by the Hahn– Banach Theorem, to show that H is dense in E it suffices to check that the only x∗ ∈ E ∗ such Rthat hF (ψ), x∗ i = 0 for all ψ ∈ Ψ is x∗ = 0. But when ψ = h · , x∗ i, hF (ψ), x∗ i = hx, x∗ i2 W (dx), and therefore, because W is non-degenerate, such an x∗ would have to be 0. § 8.2.3. The Cameron–Marin Space. Given a centered, non-degenerate Gaussian measure W on E, the Hilbert space H for which (H, E, W) is an abstract Wiener space is called its Cameron–Martin space. Here are a couple of important properties of the Cameron–Martin subspace.
§ 8.2 A Structure Theorem for Gaussian Measures
311
Theorem 8.2.6. If (H, E, W) is an abstract Wiener space, then the map x∗ ∈ E ∗ 7−→ hx∗ ∈ H is continuous from the weak* topology on E ∗ into the strong topology on H. In particular, for each R > 0, {hx∗ : x∗ ∈ BE ∗ (0, R)} is a compact subset of H, BH (0, R) is a compact subset of E, and so H ∈ BE . Moreover, when E is infinite dimensional, W(H) = 0. Finally, there is a unique linear, isometric map I : H −→ L2 (W; R) such that I(hx∗ ) = h · , x∗ i for all x∗ ∈ E ∗ , and {I(h) : h ∈ H} is a Gaussian family in L2 (W; R).
c ∗ ) is continuous Proof: To prove the initial assertion, remember that x∗ W(x ∗ ∗ with respect to the weak* topology. Hence, if xk −→ x in the weak* topology, then ! khx∗k − hx∗ k2H c ∗k − x∗ ) −→ 1, = W(x exp − 2
and so hx∗k −→ hx∗ in H. Given the first assertion, the compactness of {hx∗ : x∗ ∈ BE ∗ (0, R)} in H follows from the compactness (cf. Exercise 5.1.19) of BE ∗ (0, R) in the weak* topology. To see that BH (0, R) is compact in E, again apply Exercise 5.1.19 to check that BH (0, R) is compact in the weak topology on H. Therefore, all that we have to show is that the embedding map h ∈ H 7−→ h ∈ E is continuous from the weak topology on H into the strong topology on E. Thus, suppose that hk −→ h weakly in H. Because hx∗ : x∗ ∈ BE ∗ (0, 1) is compact in H, for each > 0 there exist an n ∈ Z+ and a {x∗1 , . . . , x∗n } ⊆ BE ∗ (0, 1) such that ∗
{hx∗ : x ∈ BE ∗ (0, 1)} ⊆
n [
BH (hx∗m , ).
1
Now choose ` so that max1≤m≤n |hhk − h, x∗m i| < for all k ≥ `. Then, for any x∗ ∈ BE ∗ (0, 1) and all k ≥ `, |hhk − h, x∗ i| ≤ + min hk − h, hx∗ − hx∗m H ≤ + 2 sup khk kH . 1≤m≤n
k≥1
Since, by the uniform boundedness principle, supk≥1 khk kH < ∞, this proves that khk − hkE = sup{hhk − h, x∗ i : x∗ ∈ BE ∗ (0, 1)} −→ 0 as k → ∞. S∞ Because H = 1 BH (0, n) and BH (0, n) is a compact subset of E for each n ∈ Z+ , it is clear that H ∈ BE . To see that W(H) = 0 when E is infinite dimensional, choose {x∗n : n ≥ 0} as in the final part of Lemma 8.2.3, and set Xn (x) = hx, x∗n i. Then the Xn ’s are an infinite P∞ sequence of independent, centered, Gaussians with mean value 1, and so n=0 Xn2 = ∞ W-almost surely. Hence, by Lemma 8.2.3, W-almost no x is in H. Turning to the map I, define I(hx∗ ) = h · , x∗ i. Then, for each x∗ , I(hx∗ ) is a centered Gaussian with variance khx∗ k2H , and so I is a linear isometry from
312
8 Gaussian Measures on a Banach Space
{hx∗ : x∗ ∈ E ∗ } into L2 (W; R). Hence, since {hx∗ : x∗ ∈ E ∗ } is dense in H, I admits a unique extension as a linear isometry from H into L2 (W; R). Moreover, as the L2 (W; R)-limit of centered Gaussians, I(h) is a centered Gaussian for each h ∈ H. The map I in Theorem 8.2.6 was introduced for the classical Wiener space by Paley and Wiener, and so I will call it the Paley–Wiener map. To appreciate its importance here, observe that {hx∗ : x∗ ∈ E ∗ } is the subspace of g ∈ H with the property that h ∈ H 7−→ (h, g)H ∈ R admits a continuous extension to E. Even though, when dim(H) = ∞, no such continuous extension exists for general g ∈ H, I(g) can be thought of as an extension of h (h, g)H , albeit one that is defined only up to a W-null set. Of course, one has to be careful when using this interpretation, since, when H is infinite dimensional, I(g)(x) for a given x ∈ E is not well-defined simultaneously of all g ∈ H. Nonetheless, by adopting it, one gets further evidence for the idea that W wants to be the standard Gauss measure on H. Namely, because (8.2.7)
khk2 √ H EW e −1 I(h) = e− 2 ,
h ∈ H,
if W lived on H, then it would certainly be the standard Gauss measure there. Perhaps the most important application of the Paley–Wiener map is the following theorem about the behavior of Gaussian measures under translation. That is, if y ∈ E and τy : E −→ E is given by τy (x) = x + y, we will be looking at the measure (τy )∗ W and its relationship to W. Using the reasoning suggested above, the result is easy to guess. Namely, if W really lived on H and were given by a Feynman-type representation W(dh) =
1 − khk2H e 2 λH (dh), Z
then (τg )∗ W should have the Feynman representation 1 − kh−gk2H 2 λH (dh), e Z
which could be rewritten as (τg )∗ W (dh) = exp h, g H − 12 kgk2H W(dh).
Hence, if we assume that I(g) gives us the correct interpretation of ( · , g)H , we are led to guess that, at least for g ∈ H, (8.2.8) (τg )∗ W(dx) (dh) = Rg (x) W (dx), where Rg = exp I(g) − 12 kgk2H .
That (8.2.8) is correct was proved for the classical Wiener space by Cameron and Martin, and for this reason it is called the Cameron–Martin formula. In fact, one has the following result, the second half of which is due to Segal.
Exercises for § 8.2
313
Theorem 8.2.9. If (H, E, W) is an abstract Wiener space, then, for each g ∈ H, (τg )∗ W W and the Rg in (8.2.8) is the corresponding Radon–Nikodym derivative. Conversely, if (τy )∗ W is not singular with respect to W, then y ∈ H. Proof: Let g ∈ H, and set µ = (τg )∗ W. Then
√ √ ∗ µ ˆ(x∗ ) = EW e −1hx+g,x i = exp −1hg, x∗ i − 12 khx∗ k2H .
(*)
Now define ν by the right-hand side of (8.2.8). Clearly ν ∈ M1 (E). Thus, we will have proved the first part once we show that νˆ is given by the right-hand side of (*). To this end, observe that, for any h1 , h2 ∈ H,
2 ξ1 I(h1 )+ξ2 I(h2 ) ξ22 ξ1 2 2 kh1 kH + ξ1 ξ2 h1 , h2 H + kh2 kH E e = exp 2 2 W
for all ξ1 , ξ2 ∈ C. Indeed, this is obvious when ξ1 and ξ2 are pure imaginary, and, since both sides are entire functions of (ξ1 , ξ2 ) ∈ C2 , it follows in general by analytic continuation. In particular, by taking h1 = g, ξ1 = 1, h2 = hx∗ , and √ ξ2 = −1, it is easy to check that the right-hand side of (*) is equal to νˆ(x∗ ). To prove the second assertion, begin by recalling from Lemma 8.2.3 that if y ∈ E, then y ∈ H if and only if there is a K < ∞ with the property that |hy, x∗ i| ≤ K for all x∗ ∈ E ∗ with khx∗ kH = 1. Now suppose that (τx∗ )∗ W 6⊥ W, and let R be the Radon–Nikodym derivative of its absolutely continuous part. Given x∗ ∈ E ∗ with khx∗ kH = 1, let Fx∗ be the σ-algebra generated by x hx, x∗ i, and check that (τy )∗ W Fx∗ W Fx∗ with Radon–Nikodym derivative hy, x∗ i2 ∗ ∗ . Y (x) = exp hy, x ihx, x i − 2
Hence,
2 1 Y ≥ EW R Fx∗ ≥ EW R 2 Fx∗ , and so (cf. Exercise 8.2.19) hy, x∗ i2 exp − 8
1 1 = EW Y 2 ≥ α ≡ EW R 2 ∈ (0, 1].
Since this means that hy, x∗ i2 ≤ 8 log α1 , the proof is complete.
Exercises for § 8.2 Exercise 8.2.10. Let C ∈ Hom(RN ; RN be a positive definite and symmetric, take E = RN to be the standard Euclidean metric, and let H = RN with the Hilbert inner product (x, y)H = (x, C−1 y)RN . Show that H, E, γ0,C is an abstract Wiener space.
314
8 Gaussian Measures on a Banach Space
Exercise 8.2.11. Let E be a separable Banach space and W a centered Gaussian measure on E, but do not assume that W is non-degenerate. Denote by N the set of x∗ ∈ E ∗ for which EW hx, x∗ i2 = 0, and set ˆ = x ∈ E : hx, x∗ i = 0 for all x∗ ∈ N . E ˆ is closed, that W(E) ˆ = 1, and that W E ˆ is a non-degenerate, Show that E ˆ centered Gaussian measure on E. Hint: Since W {x ∈ E : hx, x∗ i = 6 0} = 0 for each x∗ ∈ N , the only question is ˆ if and only whether one can choose a countable subset C ⊆ N such that x ∈ E ∗ ∗ if hx, x i = 0 for all x ∈ C. For this purpose, recall that, by Exercise 5.1.19, E ∗ with the weak* topology is second countable and therefore that N is separable with respect to the weak* topology. Exercise 8.2.12. Let {xP separable Banach space n : n ≥ 0} be a sequence in the P ∞ ∞ E with the property that n=0 kxn kE < ∞. Show that n=0 |ξn |kxP n k < ∞ for ∞ N γ0,1 -almost every ξ ∈ RN , and define X : RN −→ E so that X(ξ) = n=0 ξn xn P∞ if n=0 |ξn |kxn kE < ∞ and X(ξ) = 0 otherwise. Show that the distribution µ of X is a centered, Gaussian measure on E. In addition, show that µ is non-degenerate if and only if the span of {xn : n ≥ 0} is dense in E. Exercise 8.2.13. Here an application of Fernique’s Theorem to functional analysis. Let E and F be a pair of separable Banach spaces and ψ a Borel measurable, linear map from E to F . Given a centered, Gaussian E-valued random variable X, use Exercise 2.3.21 see that ψ ◦ X is an F -valued, a centered Gaussian random variable, and apply Fernique’s Theorem to conclude that ψ ◦ X is a square integrable and has mean value 0. Next, suppose that ψ is not continuous, and choose {xn : n ≥ 0} ⊆ E and {yn : n ≥ 0} ⊆ F ∗ so that kxn kE = 1 = kyn ∗ kF ∗ and hψ(xn ), yn∗ i ≥ n + 13 . Using Exercise 8.2.12, show that there exist centered, Gaussian F -valued random variables {Xn : n ≥ 0},P {X n : n ≥ 0}, ∞ N −2 and X under γ0,1 such that Xn (ξ) = (n + 1) ξn xn , X(ξ) = n=0 Xn (ξ), and N X n (ξ) = X(ξ) − Xn (ξ) for γ0,1 -almost every ξ ∈ RN . Show that Z
Z
N kψ ◦ ≥ hψ ◦ X(ξ), yn∗ i γ0,1 (dξ) Z N ≥ hψ ◦ Xn (ξ), yn∗ i γ0,1 (dξ) ≥ (n + 1),
X(ξ)k2F
N γ0,1 (dξ)
N and thereby arrive at the contradiction that ψ ◦ X ∈ / L2 (γ0,1 ; F ). Conclude that every Borel measurable, linear map from E to F is continuous. Notice that, as a consequence, we know that the Paley–Wiener integral I(h) of an h in the Cameron–Martin space is equal W-almost everywhere to a Borel measurable, linear function if and only if h = hx∗ for some x∗ ∈ E ∗ .
Exercises for § 8.2
315
Exercise 8.2.14. Let W p bePa centered, Gaussian measure on a separable Ban 2 nach space E, and set σ = m=1 am , where a1 , . . . , an ∈ R. If X1 , . . . , Xn are mutually independent, E-valued random variables with distribution Pn W on some probability space (Ω, F, P), show that the P-distribution of S ≡ m=1 am Xm is the same as the W-distribution of x σx. In particular, EP kSkpE = σ p EW kxkpE for all p ∈ [0, ∞).
Hint: Using Exercise 8.2.11, reduce to the case when W is non-degenerate. For this case, let H be the Cameron–Martin space for W on E, and show that i h √ 2 ∗ σ2 EP e −1hS,x i = e− 2 khx∗ kH for all x∗ ∈ E ∗ .
Exercise 8.2.15. Referring to the setting in Lemma 8.2.3, show that there is a (n) sequence {k · kE : n ≥ 0} of norms on E each of which is commensurate with (N ) k · kE (i.e., Cn−1 k · k ≤ k · kE ≤ Cn k · k for some Cn ∈ [1, ∞)) such that, for each R > 0, (n)
BH (0, R) = {x ∈ E : kxkE ≤ R for all n ≥ 0}. Hint: Choose {x∗m : m ≥ 0} ⊆ E ∗ so that {hx∗m : m ≥ 0} is an orthonormal Pn basis for H, define Pn : E −→ H by Pn x = m=0 hx, x∗m ihx∗m , and set (n) kxkE
=
q
kPn xk2H + kx − Pn xk2E .
Exercise 8.2.16. Referring to the setting in Fernique’s Theorem, observe that all powers of kXkE are integrable, and set σ 2 = E kXk2E . Show that h kXk2E i E e 72σ2 ≤ K.
In particular, for any n ≥ 1, conclude that E kXk2n ≤ (72)n n!Kσ 2n , E which is remarkably close to the equality that holds when E = R. See Corollary 8.4.3 for a sharper statement. Exercise 8.2.17. Again let E be a separable, real Banach space. Suppose that {Xn : n ≥ 1} is a sequence for centered, Gaussian E-valued random variables on some probability space (Ω, F, P) and that Xn −→ X in P-probability. Show that X is again a centered, random variable and that there exists a λ > 0 Gaussian 2 for which supn≥1 EP eλkXn kE < ∞. Conclude, in particular, that Xn −→ X in Lp (P; E) for every p ∈ [1, ∞).
316
8 Gaussian Measures on a Banach Space
Exercise 8.2.18. Given λ ∈ Θ(RN )∗ , I pointed out at the end of § 8.1.2 that the Paley–Wiener integral [I(hλ )](θ) can be interpreted as the Riemann–Stieltjes integral of λ (s, ∞) with respect to θ(s). In this exercise, I will use this observation as the starting point for what is called stochastic integration. (i) Given λ ∈ Θ(RN )∗ and t > 0, set λt (dτ ) = 1[0,t) (τ )λ(dτ ) + δt λ [t, ∞) , and show that for all θ ∈ Θ(RN ) hθ, λt i =
Z
t
λ (τ, ∞) · dθ(τ ),
0
where the integral on the right is taken in the sense of Riemann–Stieltjes. In particular, conclude that t hθ, λt i is continuous for each θ. (ii) Given f ∈ Cc1 [0, ∞); RN , set λf (dτ ) = −f˙ (τ ) dτ , and show that hθ, λtf i =
Z
t
f (τ ) · dθ(τ ), 0
where again the integral on the right is Riemann–Stieltjes. Use this to see that the process Z t f (τ ) · dθ(τ ) : t ≥ 0 0
has the same distribution under W (N ) as (*)
Z t 2 B |f (τ )| dτ : t ≥ 0 , 0
where {B(t) : t ≥ 0} is an R-valued Brownian motion. R t∧τ (iii) Given f ∈ L2loc [0, ∞); RN and t > 0, set htf (τ ) = 0 f (s) ds. Show that the W (N ) -distribution of the process I(htf ) : t ≥ 0 is the same as that of the process in (*). In particular, conclude (cf. part (ii) of Exercise 4.3.16) that there is a continuous modification of the process {I(htf ) : t ≥ 0}. For reasons made clear in (ii), such a continuous modification is denoted by Z
t
f (τ ) · dθ(τ ) : t ≥ 0 .
0
Of course, unless f has bounded variation, the integrals in the preceding are no longer interpretable as Riemann–Stieltjes integrals. In fact, they not even defined θ by θ but only as a stochastic process. For this reason, they are called stochastic integrals.
§ 8.3 From Hilbert to Abstract Wiener Space
317
Exercise 8.2.19. Define Rg as in (8.2.8), and show that p p1 (p − 1)kgk2H W for all p ∈ (0, ∞). E Rg = exp 2 Exercise 8.2.20. Here is another way to think about Segal’s half of Theorem 8.2.9. Using Lemma 8.2.3, choose {x∗n : n ≥ 0} ⊆ E ∗ so that {hx∗n : n ≥ 0} is an orthonormal basis for H. Next, define F : E −→ RN so thatQ F (x)n = hx, x∗n i ∞ N for each n ∈ N, and show that F∗ W = γ0,1 and (F ◦ τy )∗ W = 0 γan ,1 , where Q ∞ N an = hy, x∗n i. Conclude from this that (τy )∗ W ⊥ W if γ0,1 ⊥ 0 γan ,1 . Finally, P∞ use this together with Exercise 5.2.42 to see that (τy )∗ W ⊥ W if 0 a2m = ∞, which, by Lemma 8.2.3, will be the case if y ∈ / H.
§ 8.3 From Hilbert to Abstract Wiener Space Up to this point I have been assuming that we already have at hand a nondegenerate, centered Gaussian measure W on a Banach space E, and, on the basis of this assumption, I produced the associated Cameron–Martin space H. In this section, I will show how one can go in the opposite direction. That is, I will start with a separable, real Hilbert space H and show how to go about finding a separable, real Banach space E for which there exists a W ∈ M1 (E) such that (H, E, W) is an abstract Wiener space. Although I will not adopt his approach, the idea of carrying out such a program is Gross’s. Warning: From now on, unless the contrary is explicitly stated, I will be assuming that the spaces with which I am dealing are all infinite dimensional, separable, and real. § 8.3.1. An Isomorphism Theorem. Because, at an abstract level, all infinite dimensional, separable Hilbert spaces are the same, one should expect that, in a related sense, the set of all abstract Wiener spaces for which one Hilbert space is the Cameron–Martin space is the same as the set of all abstract Wiener spaces for which any other Hilbert space is the Cameron–Martin space. The following simple result verifies this conjecture. Theorem 8.3.1. Let H and H 0 be a pair of Hilbert spaces, and suppose that F is a linear isometry from H onto H 0 . Further, suppose that (H, E, W) is an abstract Wiener space. Then there exists a separable, real Banach space E 0 ⊇ H 0 anda linear isometry F˜ from E onto E 0 such that F˜ H = F and H 0 , E 0 , F˜∗ W is an abstract Wiener space. Proof: Define kh0 kE 0 = kF −1 h0 kE for h0 ∈ H 0 , and let E 0 be the Banach space obtained by completing H 0 with respect to k · kE 0 . Trivially, H 0 is continuously embedded in E 0 as a dense subspace, and F admits a unique extension F˜ as an isometry from E onto E 0 . Moreover, if (x0 )∗ ∈ (E 0 )∗ and F˜ > is the adjoint map from (E 0 )∗ onto E ∗ , then h0 , h0(x0 )∗ H 0 = hh0 , (x0 )∗ i = hF −1 h0 , F˜ > (x0 )∗ i = F −1 h0 , hF˜ > (x0 )∗ H = h0 , F hF˜ > (x0 )∗ H 0 ,
318
8 Gaussian Measures on a Banach Space
and so h0(x0 )∗ = F hF˜ > (x0 )∗ . Hence, i i h √ i h √ h √ 0 ∗ 0 0 ∗ ˜> 0 ∗ ˜ ˜ EF∗ W e −1 hx ,(x ) i = EW e −1 hF x,(x ) i = EW e −1 hx,F (x ) i 1
2
− 1 kF −1 h0
k2
− 1 kh0
k2
(x0 )∗ H 0 , (x0 )∗ H = e 2 = e− 2 khF˜ > (x0 )∗ kH = e 2 which completes the proof that H 0 , E 0 , F˜∗ W is an abstract Wiener space. Theorem 8.3.1 says that there is a one-to-one correspondence between the abstract Wiener spaces associated with one Hilbert space and the abstract Wiener spaces associated with any other. In particular, it allows us to prove the theorem of Gross which states that every Hilbert space is the Cameron–Martin space for some abstract Wiener space.
Corollary 8.3.2. Given a separable, real Hilbert space H, there exists a separable Banach space E and a W ∈ M1 (E) such that (H, E, W) is an abstract Wiener space. Proof: Let F : H 1 (R) −→ H be an isometric isomorphism, and use Theorem 8.3.1 to construct a separable Banach space E and an isometric, isomorphism F˜ : Θ(R) −→ E so that (H, E, W) is an abstract Wiener space when W = F˜∗ W (1) . It is important to recognize that although a non-degenerate, centered Gaussian measure on a Banach space E determines a unique Cameron–Martin space H, a given H will be the Cameron–Martin space for an uncountable number of abstract Wiener spaces. For example, in the classical case when H = H1 (RN ), we could have replaced Θ(RN ) by a subspace which reflected the fact that almost every Brownian path is locally H¨ older continuous of any order less than a half. We will see a definitive, general formulation of this point in Corollary 8.3.10. § 8.3.2. Wiener Series. The proof that I gave of Corollary 8.3.2 is too nonconstructive to reveal much about the relationship between H and the abstract Wiener spaces for which it is the Cameron–Martin space. Thus, in this subsection I will develop another, entirely different way of constructing abstract Wiener spaces for a Hilbert space. The approach here has its origins in one of Wiener’s own constructions of Brownian motion and is based on the following line of reasoning. Given H, choose an orthonormal basis {hn : n ≥ 0}. If there were a standard Gauss measure W on H, then the random variables {Xn : n ≥ 0} given by Xn (h) = h, hn H would be independent, standard normal, R-valued random variables, P∞ and, for each h ∈ H, 0 Xn (h)hn would converge in H to h. Even though W cannot live on H, this line of reasoning suggests that a way to construct an abstract Wiener space is to start with a sequence {Xn : n ≥ 0} of R-valued, independent standard normalPrandom variables on some probability space, find ∞ a Banach space E in which 0 Xn hn converges with probability 1, and take W on E to the distribution of this series.
§ 8.3 From Hilbert to Abstract Wiener Space
319
To convince oneself that this line of reasoning has a chance of leading somewhere, one should observe that L´evy’s construction corresponds to a particular choice of the orthonormal basis {hm : m ≥ 0}.1 To see this, determine {h˙ k,n : (k, n) ∈ N2 } by 1 on k21−n , (2k + 1)2−n n−1 h˙ k,0 = 1[k,k+1) and h˙ k,n = 2 2 −1 on (2k + 1)2−n , (k + 1)21−n 0 elsewhere for n ≥ 1. Clearly, the h˙ k,n ’s are orthonormal in L2 [0, ∞); R . In addition, for each n ∈ N, the span of {h˙ k,n : k ∈ N} equals that of {1[k2−n ,(k+1)2−n ) : k ∈ N}. Perhaps the easiest way to check this is to do so by dimension counting. That is, for a given (`, n) ∈ N2 , note that
{h˙ `,0 } ∪ {h˙ k,m : `2m−1 ≤ k < (` + 1)2m−1 and 1 ≤ m ≤ n} has the same number of elements as {1[k2−n ,(k+1)2−n ) : `2n ≤ k < (` + 1)2n } and that the first set is contained in the span of the second. As a consequence, we know that {h˙ k,n : (k, n) ∈ N2 } is an orthonormal basis in L2 [0, ∞); R , and Rt so, if hk,n (t) = 0 h˙ k,n (τ ) dτ and (e1 , . . . , eN ) is an orthonormal basis in RN , then hk,n,i ≡ hk,n ei : (k, n, i) ∈ N2 × {1, . . . , N } is known as the Haar basis, in H1 (RN ). Finally, if an orthonormal basis, 2 Xk,n,i : (k, n, i) ∈ N ×{1, . . . , N } is a family of independent, N (0, 1)-random PN variables and Xk,n = i=1 Xk,n,i ei , then n X ∞ X N X
Xk,m,i hk,m,i (t) =
m=0 k=0 i=1
n X ∞ X
hk,m (t)Xk,m
m=0 k=0
is precisely the polygonalization that I denoted by Bn (t) in L´evy’s construction (cf. § 4.3.2). The construction by Wiener, alluded to above, was essentially the same, only he chose a different basis for H1 (RN ). Wiener took h˙ k,0 (t) = 1[k,k+1) (t) for 1 k ∈ N and h˙ k,n (t) = 2 2 1[k,k+1) (t) cos πn(t − k) for (k, n) ∈ N × Z+ , which means that he was looking at the series 1 ∞ X X 2 2 sin πn(t − k) Xk,n , (t − k)1[k,k+1) (t)Xk,0 + 1[k,k+1) (t) πn + k=0
1
(k,n)∈N×Z
The observation that L´ evy’s construction (cf. § 4.3.2) can be interpreted in terms of a Wiener series is due to Z. Ciesielski. To be more precise, initially Ciesielski himself was thinking entirely in terms of orthogonal series and did not realize that he was giving a re-interpretation of L´ evy’s construction. Only later did the connection become clear.
320
8 Gaussian Measures on a Banach Space
where again {Xk,n : (k, n) ∈ N2 } is a family of independent, RN -valued, N (0, I)random variables. The reason why L´evy’s choice is easier to handle than Wiener’s is that, in L´evy’s case, for each n ∈ Z+ and t ∈ [0, ∞), hk,n (t) 6= 0 for precisely one k ∈ N. Wiener’s choice has no such property. With these preliminaries, the following theorem should come as no surprise. Theorem 8.3.3. Let H be an infinite dimensional, separable, real Hilbert space and E a Banach space into which H is continuously embedded as a dense subspace. If for some orthonormal basis {hm : m ≥ 0} in H the series ∞ X
(8.3.4)
ξm hm converges in E
m=0 N for γ0,1 -almost every ξ = (ξ0 , . . . , ξm , . . . ) ∈ RN
and if S : RN −→ E is given by P∞ m=0 ξm hm S(ξ) = 0
when the series converges in E otherwise,
N then H, E, W with W = S∗ γ0,1 is an abstract Wiener space. Conversely, if (H, E, W) is an abstract Wiener space and {hm : m ≥ 0} is an orthogonal sequence in H such that, for each m ∈ N, either hm = 0 or khm kH = 1, then " (8.3.5)
E
W
p # n
X
sup I(hm )hm < ∞ for all p ∈ [1, ∞),
n≥0 m=0
E
P∞
and, for W-almost every x ∈ E, m=0 [I(hm )](x)hm converges in E to the W-conditional expectation value of x given σ {I(hm ) : m ≥ 0} . Moreover, ∞ X
[I(hm )](x)hm is W-independent of x −
m=0
∞ X
[I(hm )](x)hm .
m=0
Finally,P if {hm : m ≥ 0} is an orthonormal basis in H, then, for W-almost every ∞ x ∈ E, m=0 [I(hm )](x)hm converges in E to x, and the convergence is also in Lp (W; E) for every p ∈ [1, ∞). Proof: P First assume that (8.3.4) holds for some orthonormal basis, and set n N Sn (ξ) = m=0 ξm hm and W = S∗ γ0,1 . Then, because Sn (ξ) −→ S(ξ) in E for N N γ0,1 -almost every ξ ∈ R , n i h √ Y 2 2 N 1 1 c ∗ ) = lim Eγ0,1 e− 2 (hx∗ ,hm )H = e− 2 khx∗ kH , W(x e −1hSn ,λi = lim n→∞
n→∞
m=0
§ 8.3 From Hilbert to Abstract Wiener Space
321
which proves that (H, E, W) is an abstract Wiener space. Next suppose that (H, E, W) is an abstract Wiener space and that {hm : m ≥ 0} is an orthogonal sequence with khm kH ∈ {0, 1} for each m ≥ 0. By Theorem 8.2.1, x ∈ Lp (W; E) for every p ∈ [1, ∞). Next, for each W∞ n ∈ N, set Fn = σ {I(hm ) : 0 ≤ m ≤ n} . Clearly, Fn ⊆ Fn+1 and F ≡ n=0 Fn is the Pn σ-algebra generated by {I(hm ) : m ≥ 0}. Moreover, if Sn = m=0 I(hm )hm , then, since {I(hm ) : m ≥ 0} is a Gaussian family and hx − Sn (x), x∗ i is perpendicular in L2 (W; R) to I(hm ) for all x∗ ∈ E ∗ and 0 ≤ m ≤ n, x − Sn (x) is W-independent of Fn . Thus Sn = EW [x | Fn ], and so, by Theorem 6.1.12, we know both that (8.3.5) holds and that Sn −→ EW [x | F] W-almost surely. In addition, the W-independence of Sn (x) from x − Sn (x) implies that the limit quantities possess the same independence property. In order to complete the proof at this point, all that I have to do is show that x = EW [x | F] W-almost surely when {hm : m ≥ 0} is an orthonormal basis. W Equivalently, I must check that BE is contained P in the W-completion F of F. n To this end, note that, for each h ∈ H, because m=0 (h, hm )H hm converges in H to h, ! n n X X h, hm H I(hm ) = I h, hm H hm −→ I(h) in L2 (W; R). m=0
m=0
W
Hence, I(h) is F -measurable for every h ∈ H. In particular, this means that W x hx, x∗ i is F -measurable for every x∗ ∈ E ∗ , and so, since BE is generated W by {h · , x∗ i : x∗ ∈ E ∗ }, BE ⊆ F . It is important to acknowledge that the preceding theorem does not give another proof of Wiener’s theorem that Brownian motion exists. Instead, it simply says that, knowing it exists, there are lots of ways in which to construct it. See Exercise 8.3.21 for a more satisfactory proof of the same conclusion in the classical case, one that does not require the a priori existence of W (N ) . The following result shows that, in some sense, a non-degenerate, centered, Gaussian measure W on a Banach space does not fit on a smaller space.
Corollary 8.3.6. If W is a non-degenerate, centered Gaussian measure on a separable Banach space E, then E is the support of W in the sense that W assigns positive probability to every non-empty open subset of E. Proof: Let H be the Cameron–Martin space for W. Since H is dense in E, it suffices to show that W BE (g, r) > 0 for every g ∈ H and r > 0. Moreover, since, by the Cameron–Martin formula (8.2.8) (cf. Exercise 8.2.19) W BE (0, r) = (τ−g )∗ W BE (g, r) = EW R−g , BE (g, r) q kgk2 H W BE (g, r) , ≤e 2
322
8 Gaussian Measures on a Banach Space
I need only show that W BE (0, r) > 0 for all r > 0. To this end, choose an Pn orthonormal basis {hm : m ≥ 0} in H, and set Sn = m=0 I(hm )hm . Then, by Theorem 8.3.3, x Sn (x) is W-independent of x x − Sn (x) and Sn (x) −→ x in E for W-almost every x ∈ E. Hence, W {kx − Sn (x)kE < 2r } ≥ 12 for some n ∈ N, and therefore W BE (0, r) ≥ 12 W kSn kE < 2r . Pn But kSn k2E ≤ CkSn k2H = m=0 I(hm )2 for some C < ∞, and so n+1 r > 0 for any r > 0. BRn+1 0, 2C W kSn kE < 2r ≥ γ0,1
§ 8.3.3. Orthogonal Projections. Associated with any closed, linear subspace L of a Hilbert space H, there is an orthogonal projection map ΠL : H −→ L determined by the property that, for each h ∈ H, h − ΠL h ⊥ L. Equivalently, ΠL h is the element of L that is closest to h. In this subsection I will show that if (H, E, W) is an abstract Wiener space and L is a finite dimensional subspace of H, then ΠL admits a W-almost surely unique extension PL to E. In addition, I will show that PL x −→ x in L2 (W; E) as L % H. Lemma 8.3.7. Let (H, E, W) be an abstract Wiener space P∞ and {hm : m ≥ 0} an orthonormal basis in H. Then, for each h ∈ H, m=0 (h, hm )H I(hm ) converges to I(h) W-almost surely and in Lp (W; R) for every p ∈ [1, ∞). Proof: Define the σ-algebras Fn and F as in the proof P of Theorem 8.3.3. Then, n by the same argument as I used there, one can identify m=0 (h, hm )H I(hm ) as W
EW [I(h) | Fn ]. Thus, since F ⊇ BE , the required convergence statement is an immediate consequence of Corollary 5.2.4.
Theorem 8.3.8. Let (H, E, W) be an abstract Wiener space. For each finite dimensional subspace L of H there is a W-almost surely unique map PL : E −→ H such that, for every h ∈ H and W-almost every x ∈ E, h, PL x H = I(ΠL h)(x), where ΠL denotes orthogonal projection from H onto L. In fact, if {g1 , . . . , gdim(L) } is an orthonormal basis for L, then PL x = Pdim(L) [I(gi )](x)gi , and so PL x ∈ L for W-almost every x ∈ E. In partic1 ular, the distribution of x ∈ E 7−→ PL x ∈ L under W is the same as that Pdim(L) dim(L) of (ξ1 , . . . , ξdim(L) ) ∈ Rdim(L) 7−→ ξi gi ∈ L under γ0,1 . Finally, 1 x PL x is W-independent of x x − PL x. Proof: Set ` = dim(L). It suffices to note that ! ` ` X X I(ΠL h) = I (h, gk )H gk = (h, gk )H I(gk ) = k=1
k=1
` X k=1
! I(gk )gk , h H
for all h ∈ H We now have the preparations needed to prove a result which shows that my definition of an abstract Wiener space is the same as Gross’s. Specifically, Gross’s own definition was based on the property proved in the following.
§ 8.3 From Hilbert to Abstract Wiener Space
323
Theorem 8.3.9. Let (H, E, W) be an abstract Wiener space and {hn : n ≥ 0} an orthonormal basis for H, and set Ln = span {h , . . . , h } n . Then, for all 0 2 W 2 > 0 there exists an n ∈ N such that E kPL xkE ≤ whenever L is a finite dimensional subspace that is perpendicular to Ln . Proof: Without loss in generality, I will assume that k · kE ≤ k · kH . Arguing by contradiction, I will show that if the asserted property does not hold,P then there would exist an orthonormal basis {fn : n ≥ 0} for H such ∞ that 0 I(fn )fn fails to converge in L2 (W; E). Thus, suppose that there exists an > 0 such that for all n ∈ N there exists a finite dimensional L ⊥ Ln with EW kPL xk2E ≥ 2 . Under this assumption, define {nm : m ≥ 0} ⊆ N, {`m : m ≥ 0} ⊆ N, and {f0 , . . . , fnm } : m ≥ 0 ⊆ Lnm inductively by the following prescription. First, take n0 = 0 = `0 and f0 = h0 . Next, knowing nm and {f0, . . . , fnm }, choose a finite dimensional subspace L ⊥ Lnm so that EW kPL xk2E ≥ 2 , set `m = dim(L), and let {gm,1 , . . . , gm,`m } be an orthonormal basis for L. For any δ > 0 there exists an n ≥ nm + `m such that `m X ΠLn gm,i , ΠLn gm,j − δi,j ≤ δ. H i,j=1
In particular, if δ ∈ (0, 1), then the elements of {ΠLn gm,i : 1 ≤ i ≤ `m } are linearly independent and the orthonormal set {˜ gm,i : 1 ≤ i ≤ `m } obtained from them via the Gram–Schmidt orthogonalization procedure satisfies (cf. Exercise 8.3.16) `m X
`m X ΠLn gm,i , ΠLn gm,j − δi,j
k˜ gm,i − gm,i kH ≤ K`m
i=1
i,j=1
for some Km < ∞ which depends only on `m . Moreover, and because L ⊥ Lnm , g˜m,i ⊥ Lnm for all 1 ≤ i ≤ `m. Hence, we can find an nm+1 ≥ nm + `m so that span {hn : nm < n ≤ nm+1 } admits an orthonormal basis {fnm +1 , . . . , fnm+1 } P` with the property that 1m kgm,i − fnm +i kH ≤ 4 . Clearly {fn : n ≥ 0} is an orthonormal basis for H. On the other hand,
2 12
2 12 `m +`m
X
nmX
I(gm,i )gm,i − I(fnm +i )fnm +i EW I(fn )fn ≥ − EW
n=nm +1
1
E
≥−
`m X
2 1 EW I(gm,i )gm,i − I(fnm +i )fnm +i H 2 ,
1
2 1 and so, since EW I(gi,m )gm,i − I(fnm +i )fnm +i H 2 is dominated by
2 1 1 EW I(gm,i ) − I(fnm +i ) gm,i H 2 + EW I(fnm +i )2 2 kgm,i − fnm +i kH
≤ 2kgm,i − fnm +i kH ,
E
324
8 Gaussian Measures on a Banach Space
we have that
2 12 +`m
nmX
EW I(fn )fn ≥
2 n +1 m
for all m ≥ 0,
E
P∞ and this means that 0 I(fn )fn cannot be converging in L2 (W; E). Besides showing that my definition of an abstract Wiener space is the same as Gross’s, Theorem 8.3.9 allows us to prove a very convincing statement, again due to Gross, of just how non-unique is the Banach space for which a given Hilbert space is the Cameron–Martin space. Corollary 8.3.10. If (H, E, W) is an abstract Wiener space, then there exists a separable Banach space E0 that is continuously embedded in E as a measurable subset and has the properties that W(E 0 ) = 1, bounded subsets of E0 are relatively compact in E, and (H, E0 , W E0 is again an abstract Wiener space. Proof: Again I will assume that k · kE ≤ k · kH . Choose {x∗n : n ≥ 0} ⊆ E ∗ so that {hn : n ≥ 0} is an orthonormal basis in H when hn = hx∗n , and set Ln = span {h0 , . . . , hn } . Next, using Theorem 8.3.9, choose an increasing sequence {nm : m ≥ 0} so that n0 = 0 and 1 EW kPL xk2E 2 ≤ 2−m for m ≥ 1 and finite dimensional L ⊥ Lnm , and define Q` for ` ≥ 0 on E into H so that
Q0 x =
hx, x∗0 ih0
and Q` x =
n` X
hx, x∗n ihn
when ` ≥ 1.
n=n`−1 +1
Finally, set Sm = PLnm = that kxkE0 ≡ kQ0 xkE +
Pm
`=0
∞ X
Q` , and define E0 to be the set of x ∈ E such
`2 Q` xkE < ∞
and kSm x − xkE −→ 0.
`=1
To show that k · kE0 is a norm on E0 and that E0 with norm k · kE0 is a Banach space, first note that if x ∈ E0 , then kxkE = lim kSm xkE ≤ kQ0 xkE + lim m→∞
m X
kQ` xkE ≤ kxkE0 ,
m→∞ `=1
and therefore k · kE0 is certainly a norm on E0 . Next, suppose that the sequence {xk : k ≥ 1} ⊆ E0 is a Cauchy sequence with respect to k · kE0 . By the preceding, we know that {xk : k ≥ 1} is also Cauchy convergent with respect to
§ 8.3 From Hilbert to Abstract Wiener Space
325
k · kE , and so there exists an x ∈ E such that xk −→ x in E. We need to show that x ∈ E0 and that kxk − xkE0 −→ 0. Because {xk : k ≥ 1} is bounded in E0 , it is clear that kxkE0 < ∞. In addition, for any m ≥ 0 and k ≥ 1, kx − Sm xkE = lim kx` − Sm x` kE ≤ lim kx` − Sm x` kE0 `→∞
= lim
`→∞
X
2
n kQn x` kE ≤
`→∞ n>m
X
n2 kQn xk k + sup kx` − xk kE0 .
n>m
`>k
Thus, by choosing k for a given > 0 so that sup`>k kx` − xk kE0 < , we conclude that limm→∞ kx − Sm xkE < and therefore that Sm x −→ x in E. Hence, x ∈ E0 . Finally, to see that xk −→ x in E0 , simply note that ∞ X
kx − xk kE0 = kQ0 (x − xk )kE +
m2 kQm (x − xk )kE
m=1
kQ0 (x` − xk )kE +
≤ lim `→∞
∞ X
! 2
m kQm (x` − xk )kE
≤ sup kx` − xk kE0 , `>k
m=1
which tends to 0 as k → ∞. To show that bounded subsets of E0 are relatively compact in E, it suffices to show that if {x` : ` ≥ 1} ⊆ BE0 (0, R), then there is an x ∈ E to which a subsequence converges in E. For this purpose, observe that, for each m ≥ 0, there is a subsequence {x`k : k ≥ 1} along which Sm x`k converges in Lnm . Hence, by a diagonalization argument, {x`k : k ≥ 1} can be chosen so that {Sm x`k : k ≥ 1} converges in Lnm for all m ≥ 0. Since, for 1 ≤ j < k, X kx`k − x`j kE ≤ kSm x`k − Sm x`j kE + kQn (x`k − x`j )kE n>m
≤ kSm x`k − Sm x`j kE + 2R
X 1 , n2 n>m
it follows that {x`k : k ≥ 1} is Cauchy convergent in E and therefore that it converges in E. I must still show that E0 ∈ BE and that (H, E0 , W0 ) is an abstract Wiener space when W0 = W E0 . To see the first of these, observe that x ∈ E 7−→ kxkE0 ∈ [0, ∞] is lower semicontinuous and that {x : kSm x − xkE −→ 0} ∈ BE . In addition, because, by Theorem 8.3.3, kSm x − xkE −→ 0 for W-almostevery x ∈ E, we will know that W(E0 ) = 1 once I show that W kxkE0 < ∞ = 1, which follows immediately from ∞ X EW kxkE0 = EW kQ0 xkE + m2 EW kQm xkE 1
≤ EW kQ0 xkE +
∞ X 1
1 m2 EW kQm xk2E 2 < ∞.
326
8 Gaussian Measures on a Banach Space
The next step is to check that H is continuously embedded in E0 . Certainly h ∈ H =⇒ kSm h − hkE ≤ kSm h − hkH −→ 0. Next suppose that h ∈ H \ {0} and that h ⊥ Lnm , and let L be the line spanned by h. Then PL x = khk−2 H [I(h)](x)h, and so, because L ⊥ Lnm ,
1 khkE 1 W 2 2 khkE . = ≥ E I(h) khkH khk2H 2m
Hence, we now know that h ⊥ Lnm =⇒ khkE ≤ 2−m khkH . In particular, kQm+1 hkE ≤ 2−m kQm+1 hkH ≤ 2−m khkH for all m ≥ 0 and h ∈ H, and so ! ∞ ∞ X X m2 2 khkH = 25khkH . khkE0 = kQ0 hkE + m kQm hkE ≤ 1 + 2 m 2 m=1 m=1
To complete the proof, I must show that H is dense in E0 and that, for each c0 (y ∗ ) = e− 12 khy∗ k2H , where W0 = W E0 and hy∗ ∈ H is determined y ∗ ∈ E0∗ , W by h, hy∗ H = hh, y ∗ i for h ∈ H. Both these facts rely on the observation that X kx − Sm xkE0 = n2 kQn xkE −→ 0 for all x ∈ E0 . n>m
Knowing this, the density of H in E0 is obvious. Finally, if y ∗ ∈ E0∗ , then, by the preceding and Lemma 8.3.7, hx, y ∗ i = lim hSm x, y ∗ i = lim m→∞
= lim
m→∞
m→∞
nm X
hy∗ , hn
H
nm X
hx, x∗n ihhn , y ∗ i
n=0
I(hn ) (x) = I(hy∗ ) (x)
n=0
for W0 -almost every x ∈ E0 . Hence h · , y ∗ i under W0 is a centered Gaussian with variance khy∗ k2H . § 8.3.4. Pinned Brownian Motion. Theorem 8.3.8 has a particularly inter esting application to the classical abstract Wiener space H1 (RN ), Θ(RN ), W (N ) . Namely, suppose that 0 = t0 < t1 < · · · < tn , and let L be the span of htm e : 1 ≤ m ≤ n and e ∈ SN −1 , where ht (τ ) ≡ t ∧ τ . In this case, PL θ =
n X htm − htm−1 θ(tm ) − θ(tm−1 ) , t − tm−1 m=1 m
and so θ(t1 ,... ,tn ) (t) ≡ [θ − PL θ](t) ( t−tm−1 if t ∈ [tm−1 , tm ] θ(t) − θ(tm−1 ) − tm (8.3.11) −tm−1 θ(tm ) − θ(tm−1 ) = θ(t) − θ(tn ) if t ∈ [tn , ∞).
§ 8.3 From Hilbert to Abstract Wiener Space
327
Thus, if (θ, ~y) ∈ Θ(RN ) × (RN )n 7−→ θ(t1 ,... ,tn ),~y ∈ Θ(RN ) is given by θ(t1 ,... ,tn ),~y = θ(t1 ,... ,tn ) +
n X htm − htm−1 (ym − ym−1 ), t − tm−1 m=1 m
where ~y = (y1 , . . . , yn ) and y0 ≡ 0, then, for any Borel measurable F : Θ(RN )× (RN )n −→ [0, ∞), Z F θ, θ(t1 ), . . . , θ(tn ) W (N ) (dθ) Θ(RN )
(8.3.12)
Z
Z
F θ(t1 ,... ,tn ),~y , ~y W
= (RN )n
(N )
(dθ) γ0,C(t1 ,... ,tn ) (d~y),
Θ(RN )
where C(t1 , . . . , tn )(m,i),(m0 i0 ) = tm ∧tm0 δi,i0 for 1 ≤ m, m0 ≤ n and 1 ≤ i, i0 ≤ N is the covariance of θ (θ(t1 ), . . . , θ(tn )) under W (N ) . Equivalently, if θˇ(t1 ,... ,tn ),~y = θ(t1 ,... ,tn ) +
n X htm − htm−1 ym , t − tm−1 m=1 m
then Z
F θ, θ(t1 ) − θ(t0 ), . . . , θ(tn ) − θ(tn−1 ) W (N ) (dθ)
Θ(RN )
(8.3.13)
Z
Z
= (RN )n
(N ) ˇ ~ F θ(t1 ,... ,tn ),~y , y W (dθ) γ0,D(t1 ,... ,tn ) (d~y),
Θ(RN )
where D(t1 , . . . , tn )(m,i),(m0 ,i0 ) = (tm − tm−1 )δm,m0 δi,i0 for 1 ≤ m, m0 ≤ n and 1 ≤ i, i0 ≤ N is the covariance matrix for θ(t1 ) − θ(t0 ), . . . , θ(tn ) − θ(tn−1 ) under W (N ) . There are several comments that should be made about these conclusions. In the first place, it is clear from (8.3.11) that t θ(t1 ,... ,tn ) (t) returns to the origin at each of the times {tm : 1 ≤ m ≤ n}. In addition, the excursions θ(t1 ,... ,tn ) [tm−1 , tm ], 1 ≤ m ≤ n, are independent of each other and of θ(t1 ,... ,tn ) [tn , ∞). (N )
Secondly, if W(t1 ,... ,tn ),~y denotes the W (N ) -distribution of θ (8.3.12) says that (N ) θ W(t1 ,... ,tn ),(θ(t1 ),... ,θ(tn ))
θ(t1 ,... ,tn ),~y , then
is a regular conditional probability distribution (cf. § 9.2) of W (N ) given the σalgebra generated by {θ(t1 ), . . . , θ(tn )}. Expressed in more colloquial terms, the process θ(t1 ,... ,tn ),~y (t) : t ≥ 0 is Brownian motion pinned to the points {ym : 1 ≤ m ≤ n} at times {tm : 1 ≤ m ≤ n}.
328
8 Gaussian Measures on a Banach Space
§ 8.3.5. Orthogonal Invariance. Consider the standard Gauss distribution γ0,I on RN . Obviously, γ0,I is rotation invariant. That is, if O is an orthogonal transformation on RN , then γ0,I is invariant under the transformation TO : RN −→ RN given by TO x = Ox. On the other hand, none of these transformations can be ergodic, since any radial function on RN is invariant under TO for every O. Now think about the analogous situation when RN is replaced by an infinite dimensional Hilbert space H and (H, E, W) is an associated abstract Wiener space. As I am about to show, W still enjoys rotation invariance with respect to orthogonal transformations on H. On the other hand, because kxkH = ∞ for W-almost every x ∈ E, there are no non-trivial radial functions now, a fact that leaves open the possibility that some orthogonal transformation of H give rise to ergodic transformations for W. The purpose of this subsection is to investigate these matters, and I begin with the following formulation of the rotation invariance of W. Theorem 8.3.14. Let (H, E, W) be an abstract Wiener space and O an orthogonal transformation on H. Then there is a W-almost surely unique, Borel measurable map TO : E −→ E such that I(h) ◦ TO = I(O> h) W-almost surely for each h ∈ H. Moreover, W = (TO )∗ W. Proof: To prove uniqueness, note that if T and T 0 both satisfy the defining property for TO , then, for each x∗ ∈ E ∗ , hT x, x∗ i = I(hx∗ )(T x) = I(O> hx∗ ) = I(hx∗ )(T 0 x) = hT 0 x, x∗ i for W-almost every x ∈ E. Hence, since E ∗ is separable in the weak* topology, T x = T 0 x for W-almost every x ∈ E. To prove existence, choose an orthonormal basis {hm : m ≥ for H, and let P∞ P0} ∞ C be the set of x ∈ E for which both m=0 [I(hm )](x)hm and m=0 [I(hm )](x)Ohm converge in E. By Theorem 8.3.3, we know that W(C) = 1 and that P∞ m=0 [I(hm )](x)Ohm if x ∈ C x TO x ≡ 0 if x ∈ /C has distribution W. Hence, all that remains is to check that I(h)◦TO = I(O> h) W-almost surely for each h ∈ H. To this end, let x∗ ∈ E ∗ , and observe that ∗
[I(hx∗ )](TO x) = hTO x, x i =
∞ X
hx∗ , Ohm
H
[I(hm )](x)
m=0
=
∞ X
O> hx∗ , hm
H
[I(hm )](x)
m=0
for W-almost every x ∈ E. Thus, since, by Lemma 8.3.7, the last of these series convergences W-almost surely to I(O> hx∗ ), we have that I(hx∗ ) ◦ TO =
§ 8.3 From Hilbert to Abstract Wiener Space
329
I(O> hx∗ ) W-almost surely. To handle general h ∈ H, simply note that both h ∈ H 7−→ I(h) ◦ TO ∈ L2 (W; R) and h ∈ H 7−→ I(O> h) ∈ L2 (W; R) are isometric, and remember that {hx∗ : x∗ ∈ E ∗ } is dense in H. I next want to discuss the possibility of TO being ergodic for some orthogonal transformations O. First notice that TO cannot be ergodic if O has a non-trivial, finite dimensional invariant Pn subspace L, since if {h1 , . . . , hn } were an orthonormal basis for L, then m=1 I(hm )2 would be a non-constant, TO invariant function. Thus, the only candidates for ergodicity are O’s that have no non-trivial, finite dimensional, invariant subspaces. In a more general and highly abstract context, I. Segal2 showed that the existence of a non-trivial, finite dimensional subspace for O is the only obstruction to TO being ergodic. Here I will show less. Theorem 8.3.15. Let (H, E, W) be an abstract Wiener space. If O is an orthogonal transformation on H with the property that, for every g, h ∈ H, limn→∞ On g, h H = 0, then TO is ergodic.
Proof: What I have to show is that any TO -invariant element Φ ∈ L2 (W; R) is W-almost surely constant, and for this purpose it suffices to check that (*)
lim EW (Φ ◦ TOn )Φ = 0 n→∞
for all Φ ∈ L2 (W; R) with mean value 0. In fact, if {hm : m ≥ 1} is an orthonormal basis for H, then it suffices to check (*) when Φ(x) = F [I(h1 )](x), . . . , [I(hN )](x) for some N ∈ Z+ and bounded, Borel measurable F : RN −→ R. The reason why it is sufficient to check it for such Φ’s is that, because TO is W-measure preserving, the set of Φ’s for which (*) holds is closed in L2 (W; R). Hence, if we start with any Φ ∈ L2 (W; R) with mean value 0, we can first approximate it in L2 (W; R) by bounded functions with mean value 0 and then condition these bounded approximates with respect to σ {I(h1 ), . . . , I(hN )} to give them the required form. Now suppose that Φ = F I(h1 ), . . . , I(hN ) for some N and bounded, measurable F . Then ZZ EW Φ ◦ TOn Φ = F (ξ)F (η) γ0,Cn (dξ × dη), RN ×RN 2
See I.E. Segal’s “Ergodic subsgroups of the orthogonal group on a real Hilbert Space,” Annals of Math. 66 # 2, pp. 297–303 (1957). For a treatment in the setting here, see my article “Some thoughts about Segals ergodic theorem,” Colloq. Math. 118 # 1, pp. 89-105 (2010).
330
8 Gaussian Measures on a Banach Space
where Cn =
I B> n
Bn I
with Bn =
hk , On h`
H
1≤k,`≤N
and the block structure corresponds to RN × RN . Finally, by our hypothesis about O, we can find a subsequence {nm : m ≥ 0} such that limm→∞ Bnm = 0, from which it is clear that γ0,Cnm tends to γ0,I × γ0,I in variation and therefore lim EW (Φ ◦ TOnm )Φ = EW [Φ]2 = 0.
m→∞
Perhaps the best tests for whether an orthogonal transformation satisfies the hypothesis in Theorem 8.3.15 come from spectral theory. To be more precise, if Hc and Oc are the space and operator obtained by complexifying H and O, the Spectral Theorem for normal operators allows one to write Z Oc =
2π
√
e
−1α
dEα ,
0
where {Eα : α ∈ [0, 2π)} is a resolution of the identity in Hc by orthogonal projection operators. The spectrum of Oc is said to be absolutely continuous if, for each h ∈ Hc , the non-decreasing function α Eα h, h Hc is absolutely continuous, which, by polarization, means that α Eα h, h0 Hc is absolutely continuous for all h, h0 ∈ Hc . The reason for introducing this concept here is that, by combining the Riemann–Lebesgue Lemma with Theorem 8.3.15, one can prove that TO is ergodic if the spectrum of Oc is absolutely continuous.3 Indeed, given h, h0 ∈ H, let f be the Radon–Nikodym derivative of α Eα h, h0 H , c and apply the Riemann–Lebesgue Lemma to see that n
0
O h, h
2π
Z
H
=
√
e
−1nα
f (α) dα −→ 0 as n → ∞.
0
See Exercises 8.3.24, 8.3.25, and 8.5.15 for a more concrete examples. Exercises for § 8.3 Exercise 8.3.16. The purpose of this exercise is to provide the linear algebraic facts that I used in the proof of Theorem 8.3.9. Namely, I want to show that if a set {h1 , . . . , hn } ⊆ H is approximately orthonormal, then the vectors hi differ by very little from their Gram–Schmidt orthogonalization. 3
This conclusion highlights the poverty of the result here in comparison to Segal’s result, which says that TO is ergodic as soon as the spectrum of Oc is continuous.
Exercises for § 8.3
331
(i) Suppose that A = aij 1≤i,j≤n ∈ Rn ⊗Rn is a lower triangular matrix whose diagonal entries are non-negative. Show that there is a Cn < ∞, depending only on n, such that kIRn − Akop ≤ Cn kIRn − AA> kop . Hint: Show that it suffices to treat the case when AA> ≤ 2IRn , and set ∆ = IRn − AA> . Assuming that AA> ≤ 2IRn , work by induction on n, at each step using the lower triangularity of A, to see that 12 ` X 1 a2` j if 1 ≤ ` < n |a` ` an ` | ≤ |∆n ` | + (AA> )n2 n j=1
n−1 X 1 − a2n n ≤ |∆n n | + a2n ` . `=1
(ii) Let {h1 , . . . , hn } ⊆ H, set B = (hi , hj )H 1≤i,j≤n , and assume that kIRn − Bkop < 1. Show that the hi ’s are linearly independent. (iii) Continuing part (ii), let {f1 , . . . , fn } be the orthonormal set obtained from the hi ’s by the Gram–Schmidt orthogonalization procedure, and let A be the matrix whose (i, j)th entry is (hi , fj )H . Show that A is lower triangular and that its diagonal entries are non-negative. In addition, show that AA> = B. (iv) By combining (i) and (iii), show that there is a Kn < ∞, depending only on n, such that n X
khi − fi kH ≤ Kn
i=1
Hint: Note that hi = khi −
n X δi,j − (hi , hj )H . i,j=1
Pn
j=1
fi k2H
aij fj and therefore that =
n X
IRn − A
2 ij
≤ nkIRn − Ak2op .
j=1
Exercise 8.3.17. Given a Hilbert space H, the problem of determining for which Banach spaces H arises as the Cameron–Martin space is an extremely delicate one. For example, one might hope that H will be the Cameron–Martin space for E if H is dense in E and its closed unit ball BH (0, 1) is compact in E. However, this is not the case. For example,qtake H = `2 (N; R) and let E be the P∞ ξn2 completion of H with respect to kξkE ≡ n=0 n+1 . Show that BH (0, 1) is compact as a subset of E but that there is no W ∈ M1 (E) for which (H, E, W) is an abstract Wiener space. Hint: The first part is an easy application of the standard diagonalization ar2 P ξn 1 kξk`2 (N;R) . To ≤ m+1 gument combined with the obvious fact that n≥m n+1 prove the second part, note that in order for W to exist it would be necessary P∞ ξn2 N to be γ0,1 -almost surely convergent. for n=0 n+1
332
8 Gaussian Measures on a Banach Space
Exercise 8.3.18. Let (H, E, W) be an abstract Wiener space, and assume that H is infinite dimensional. As was pointed out, {hx∗ : x∗ ∈ E ∗ } is the subspace of g ∈ H for which there exists a C < ∞ with the property that |(h, g)H | ≤ CkhkE for all h ∈ H. Show that for each g ∈ H there is separable Banach space Eg that is continuously embedded as a Borel subset of E such that W(Eg ) = 1, (H, Eg , W Eg ) is an abstract Wiener space, and |(h, g)H | ≤ khkEg for all h ∈ H. Hint: Refer to the notation used in the proof of Corollary 8.3.10. Choose nm % 1 ∞ so that n0 = 0 and, for m ≥ 1, kΠL⊥ gkH ≤ 2−m and EW kPL k2E 2 ≤ 2−m nm for finite dimensional L ⊥ Lnm . Next, define Eg to be the space of x ∈ E with the properties that PLnm x −→ x in E and
kxkEg ≡
X
kQ` xkE + Q` x, g H < ∞,
`=0
Pn` ∗ ∗ where Q0 x = hx, x∗0 ihx∗0 and Q` x = n=n`−1 +1 hx, xn ihxn for ` ≥ 1. Using the reasoning in the proof of Corollary 8.3.10, show that Eg has the required properties. Exercise 8.3.19. Let N = 1. Using Theorem 8.3.3, take Wiener’s choice of orthonormal basis and check that there are independent, standard normal random variables {Xm : m ≥ 1} under W (1) such that, for W (1) -almost almost every θ, 1
θ(t) = tX0 (θ) + 2 2
∞ X
Xm (θ)
m=1
sin(πmt) , mπ
t ∈ [0, 1],
where the convergence is uniform. From this, show that, W (1) -almost surely, 1
Z
√ ∞ 1 X Xm (θ)2 + 8X0 (θ)Xm (θ) X0 (θ)2 , + 2 θ(t) dt = m2 π m=1 3 2
0
where the convergence of the series is absolute. Using the preceding, conclude that, for any α ∈ (0, ∞),
EW
(1)
Z −α 0
1
#− 12 #− 12 " "Y ∞ ∞ X 1 2α . 1 + 4α θ(t)2 dt = 1+ 2 2 m2 π 2 + 2α m π m=1 m=1
Finally, recall Euler’s product formula ∞ Y sinh z = 1+ m=1
z2 m2 π 2
,
z ∈ C,
Exercises for § 8.3
333
and arrive first at W (1)
E
Z exp −α
1
√ − 1 θ(t) dt = cosh 2α 2 , 2
0
and then, after an application of Brownian rescaling, at " !# Z T √ − 1 W (1) 2 E exp −α θ(t) dt = cosh 2α T 2 . 0
This is a famous calculation that can be made using many different methods. We will return to it in § 10.1.3. See, in addition, Exercise 8.4.7. Hint: Use Euler’s product formula to see that ∞ X 1 sinh t d = 2t log 2 2 n π + t2 t dt n=1
for t ∈ R.
Exercise 8.3.20. Related to the preceding exercise, but easier, is finding the Laplace transform of the variance !2 Z Z 1 T 1 T 2 θ(t) dt θ(t) dt − VT (θ) ≡ T 0 T 0
of a Brownian path over the interval [0, T ]. To do this calculation, first use Brownian scaling to show that (1) (1) EW e−αVT = EW e−αT V1 . Next, use elementary Fourier series to show that (cf. part (iii) of Exercise 8.2.18) R 2 1 2 X ∞ Z 1 ∞ f (t) dθ(t) X k 0 , V1 (θ) = 2 θ(t) cos(kπt) dt = k2 π2 0 k=1
k=1
1 2
where fk (t) = 2 sin(kπt) for k ≥ 1. Since the fk ’s are orthonormal as elements of L2 [0, ∞); R , this leads to − 12 ∞ Y 2α W (1) −αV1 . E e = 1+ 2 2 k π k=1
Now apply Euler’s formula to arrive at s W
E e−αVT =
√
2αT √ . sinh( 2αT )
Finally, using Wiener’s choice of basis, show that θ V1 (θ) has the same dis2 R1 (1) tribution as θ θ(t) − tθ(1) dt under W , a fact for which I would like 0 but do not have any conceptual explanation.
334
8 Gaussian Measures on a Banach Space
Exercise 8.3.21. The purpose of this exercise is to show that, without knowing ahead of time that W (N ) lives on Θ(RN ), for the Hilbert space H1 (RN ) one N -almost surely in Θ(RN ). can give a proof that any Wiener series converges γ0,1 N Thus, let {hm : m ≥ 0} be an orthonormal basis Pn in H(R ) and, for n ∈ N N and ω = (ω0 , . . . , ωm , . . . ) ∈ R , set Sn (t, ω) = m=0 ωm hm (t). The goal is to N show that {Sn ( · , ω) : n ≥ 0} converges in Θ(RN ) for γ0,1 -almost every ω ∈ RN . (i) For ξ ∈ RN , set ht,ξ (τ ) = t∧τ ξ, check that ξ, Sn (t) RN = ht,ξ , Sn (t) H1 (RN ) , N and apply Theorem 1.4.2 to show that limn→∞ ξ, Sn (t) RN exists both γ0,1 2 N N almost surely and in L (γ0,1 ; R) for each (t, ξ) ∈ [0, ∞) × R . Conclude from N this that, for each t ∈ [0, ∞), limn→∞ Sn (t) exists both γ0,1 -almost surely and 2 N N in L (γ0,1 ; R ). (ii) On the basis of part (i), show that we will be done once we know that, N for γ0,1 -almost every x ∈ RN , {Sn ( · , x) : n ≥ 0} is equicontinuous on finite intervals and that supn≥0 t−1 |Sn (t, x)| −→ 0 as t → ∞. Show that both these will follow from the existence of a C < ∞ such that " # Sn (t) − Sn (s) N 3 γ0,1 ≤ CT 8 for all T ∈ (0, ∞). (*) E sup sup 1 (t − s) 8 0≤s
(iii) As an application of Theorem 4.3.2, show that (*) will follow once one checks that N γ0,1 4 E sup |Sn (t) − Sn (s)| ≤ B(t − s)2 , 0 ≤ s < t, n≥0
for some B < ∞. Next, apply (6.1.14) to see that 4 N N 4 sup Eγ0,1 |Sn (t) − Sn (s)|4 . Eγ0,1 sup |Sn (t) − Sn (s)|4 ≤ 3 n≥0 n≥0
In addition, because Sn (t) − Sn (s) is a centered Gaussian, argue that 2 N N Eγ0,1 |Sn (t) − Sn (s)|4 ≤ 3Eγ0,1 |Sn (t) − Sn (s)|2 . Finally, repeat the sort of reasoning used in (i) to check that N Eγ0,1 |Sn (t) − Sn (s)|2 ≤ N (t − s) for 0 ≤ s < t. Exercise 8.3.22. In this exercise we discuss some properties of pinned Brownian motion. Given T > 0, set θT (t) = θ(t) − t∧T T θ(T ). As I pointed out at the end of § 8.3.2, the W (N ) -distribution of θT is that of a Brownian motion conditioned to be back at 0 at time T . Next take ΘT (RN ) to be the space of (N ) continuous paths θ : [0, T ] −→ RN satisfying θ(0) = 0 = θ(T ), and let WT (N ) N N denote the W -distribution of θ ∈ Θ(R ) 7−→ θT ∈ ΘT (R ).
Exercises for § 8.3
335
(i) Show that the W (N ) -distribution of {θT (t) : t ≥ 0} is the same as that of 1 {T 2 θ1 (T −1 t) : t ≥ 0}.
(ii) Set H1T (RN ) = {h [0, T ] : h ∈ H1 (RN ) & h(T ) = 0}, and define ˙ L2 ([0,T ];RN ) . Show that the triple H1 (RN ), ΘT (RN ), W (N ) khkH1T (RN ) = khk T T (N )
is an abstract Wiener space. In addition, show that WT is invariant under time reversal. That is, {θ(t) : t ∈ [0, T ]} and {θ(T − t) : t ∈ [0, T ]} have the (N ) same distribution under WT . Hint: Begin by identifying ΘT (RN )∗ as the space of finite, RN -valued Borel measures λ on [0, T ] such that λ({0}) = 0 = λ({T }). Exercise 8.3.23. Say that D ⊆ E ∗ is determining if x = y whenever hx, x∗ i = hy, x∗ i for all x∗ ∈ D. Next, referring to Theorem 8.3.14, suppose that O is an orthogonal transformation on H and that F : E 7−→ E has the properties that F H = O and that x hF (x), x∗ i is continuous for all x∗ ’s from a determining set D. Show that TO x = F (x) for W-almost every x ∈ E. Exercise 8.3.24. Consider H1 (RN ), Θ(RN ), W (N ) , the classical Wiener space. Given α ∈ (0, ∞), define Oα : H1 (RN ) −→ H(RN ) by [Oα h](t) = 1 α− 2 h(αt), show that Oα is an orthogonal transformation, and apply Exercise 8.3.23 to see that TOα is the Brownian scaling map Sα given by Sα θ(t) = 1 α− 2 θ(αt) discussed in part (iii) of Exercise 4.3.10. The main goal of this exercise is to apply Theorem 8.3.15 to show that TOα is ergodic for every α ∈ (0, ∞)\{1}. (i) Given an orthogonal transformation O on H1 (RN ), show that On h, h0 H1 (RN ) tends to 0 for all h, h0 ∈ H1 (RN ) if limn→∞ On h, h0 H1 (RN ) = 0 for all h, h0 ∈ ˙ h˙ 0 ∈ C ∞ (0, ∞); RN . H(RN ) with h, c
(ii) Complete the program by showing that Oαn h, h0 H1 (RN ) tends to 0 for all ˙ h˙ 0 ∈ C ∞ (0, ∞); RN . α ∈ (0, ∞) \ {1} and h, h0 ∈ H1 (RN ) with h, c (iii) There is another way to think about the operator Oα . Namely, let λRN be Lebesgue measure on R, define U : H(RN ) −→ L2 (λRN ; RN ) by U h(x) = x ˙ x ), and show that U is an isometry from H1 (RN ) onto L2 (λRN ; RN ). Fure 2 h(e ther, show that U ◦ Oα = τlog α ◦ U , where τα : L2 (λRN ; RN ) −→ L2 (λRN ; RN ) is the translation map τα f (x) = f (x + α). Conclude from this that
Oαn h, h0
H1 (RN )
= (2π)−1
Z R
e−
√
−1nξ log α
d Uch(ξ), U h0
CN
dξ,
and use this, together with the Riemann–Lebesgue Lemma, to give a second proof that Oαn h, h0 H1 (RN ) tends to 0 as n → ∞ when α 6= 1.
336
8 Gaussian Measures on a Banach Space
(iv) As a consequence of the above and Theorem 6.2.7, show that for each α ∈ (0, ∞) \ {1}, q ∈ [1, ∞), and F ∈ Lq (W (N ) ; C), n−1 (N ) 1 X F Sαn θ = EW [F ] W (N ) -almost surely and in Lq (W (N ) ; C). n→∞ n m=0
lim
Next, replace Theorem 6.2.7 by Theorem 6.2.12 to show that Z t (N ) 1 τ −1 F Sτ θ dτ = EW [F ] lim t→∞ log t 1
W (N ) -almost surely and in Lq (W (N ) ; C). In particular, use this to show that, for n ∈ N, ( Qn Z t 2 n 1 m=1 (2m − 1) if n is even τ − 2 −1 θ(τ )n dτ = lim t→∞ log t 1 0 if n is odd.
Exercise 8.3.25. Here is a second reasonably explicit example to which Theorem 8.3.15 applies. Again consider the classical case when H = H1 (RN ), and assume that N ∈ Z+ is even. Choose a skew-symmetric A ∈ Hom(RN ; RN ) whose kernel is {0}. That is, A> = −A and Ax = 0 =⇒ x = 0. (i) Define OA on H1 (RN ) by Z
t
˙ ) dτ, eτ A h(τ
OA h(t) = 0
and show that OA is an orthogonal transformation that satisfies the hypotheses in Theorem 8.3.15. Hint: Using elementary spectral theory, show that there exist non-zero, real numbers α1 , . . . , α N and an orthonormal basis (e1 , . . . , eN ) in RN such that 2 Ae2m−1 = αm e2m and Ae2m = −αm e2m−1 for 1 ≤ m ≤ N2 . Thus, if Lm is the space spanned by e2m−1 and e2m , then Lm is invariant under A and the action of eτ A on Lm in terms of this basis is given by cos(αm τ ) − sin(αm τ ) . sin(αm τ ) cos(αm τ ) n Finally, observe that OA = OnA , and apply the Riemann–Lebesgue Lemma.
(ii) With the help of Exercise 8.3.23, show that Z t TOA θ(t) = eτ A dθ(τ ), 0
where the integral is taken in the sense of Riemann–Stieltjes.
§ 8.4 A Large Deviations Result and Strassen’s Theorem
337
§ 8.4 A Large Deviations Result and Strassen’s Theorem In this section I will prove the analog of Corollary 1.3.13 for non-degenerate, centered Gaussian measures on a Banach space. Once we have that result, I will apply it to prove Strassen’s Theorem, which is the law of the iterated logarithm for such measures. § 8.4.1. Large Deviations for Abstract Wiener Space. The goal of this subsection is to derive the following result. Theorem 8.4.1. Let (H, E, W) be an abstract Wiener space, and, for > 0, 1 denote by W the W-distribution of x 2 x. Then, for each Γ ∈ BE ,
− inf◦ h∈Γ
(8.4.2)
khk2H ≤ lim log W (Γ) 2 &0
khk2H . 2 h∈Γ
≤ lim log W (Γ) ≤ − inf &0
The original version of Theorem 8.4.1 was proved by M. Schilder for the classical Wiener measure using a method that does not extend easily to the general case. The statement that I have given is due to Donsker and S.R.S. Varadhan, and my proof derives from an approach (which very much resembles the arguments given in § 1.3 to prove Cram´er’s Theorem) that was introduced into this context by Varadhan. The lower bound is an easy application of the Cameron–Martin formula. Indeed, all that I have to do is show that if h ∈ H and r > 0, then
khk2H . lim log W BE (h, r) ≥ − 2 &0
(*)
To this end, note that, for any x∗ ∈ E ∗ and δ > 0,
1 1 W BE (hx∗ , δ) = W BE (− 2 hx∗ , − 2 δ) i h −1 2 ∗ 1 1 = EW e− 2 hx,x i− 2 khx∗ kH , BE (0, − 2 δ) 2 −1 ∗ 1 1 ≥ e−δ kx kE∗ − 2 khx∗ kH W BE (0, − 2 δ) ,
which means that
khx∗ k2H , BE (hx∗ , δ) ⊆ BE (h, r) =⇒ lim log W BE (hx∗ , r) ≥ −δkx∗ kE ∗ − &0 2
and therefore, after letting δ & 0 and remembering that {hx∗ : x ∈ E ∗ } is dense in H, that (*) holds.
338
8 Gaussian Measures on a Banach Space
The proof of the upper bound in (8.4.2) is a little more involved. The first step is to show that it suffices to treat the case when Γ is relatively compact. To this end, refer to Corollary 8.3.10, and set CR equal to the closure in E of BE0 (0, R). 2 By Fernique’s Theorem applied to W on E0 , we know that EW eαkxkE0 ≤ K < ∞ for some α > 0. Hence W E \ CR = W E \ C
1
− 2 R
≤ Ke−α
R2
,
and so, for any Γ ∈ BE and R > 0, R2 W Γ ≤ 2W(Γ ∩ CR ) ∨ Ke−α .
Thus, if we can prove the upper bound for relatively compact Γ’s, then, because Γ ∩ CR is relatively compact, we will know that, for all R > 0,
khk2H 2 h∈Γ
lim log W (Γ) ≤ −
inf
&0
∧ αR2
,
from which the general result is immediate. To prove the upper bound when Γ is relatively compact, I will show that, for any y ∈ E, ( kyk2 − 2 H if y ∈ H lim lim log W BE (y, r) ≤ (**) r&0 &0 −∞ if y ∈ / H.
To see that (**) is enough, assume that it is true and let Γ ∈ BE \{∅} be relatively compact. Given β ∈ (0, 1), for each y ∈ Γ choose r(y) > 0 and (y) > 0 so that
(
W BE (y, r(y)) ≤
e−
e
(1−β) 2 2 kykH
1 − β
if y ∈ H
if y ∈ /H
for all 0 < ≤ (y). Because Γ is relatively compact, we can find N ∈ Z+ and SN {y1 , . . . , yN } ⊆ Γ such that Γ ⊆ 1 BE (yn , rn ), where rn = r(yn ). Then, for sufficiently small > 0, 1 1−β 2 , inf khkH ∧ W (Γ) ≤ N exp − β 2 h∈Γ
and so
lim log W (Γ) ≤ −
&0
Now let β & 0.
1−β inf khk2H 2 h∈Γ
1 . ∧ β
§ 8.4 A Large Deviations Result and Strassen’s Theorem
339
Finally, to prove (**), observe that
i h −1 −1 ∗ ∗ W BE (y, r) = W BE ( √y , √r ) = EW e− 2 hx,x i e 2 hx,x i , BE ( √y , √r ) khx∗ k2 −1 ∗ −1 ∗ ∗ −1 ∗ ∗ H ≤ e− (hy,x i−rkx kE∗ ) EW e 2 hx,x i = e− hy,x i− 2 −rkx kE∗ ,
for all x∗ ∈ E. Hence,
lim lim log W BE (y, r) ≤ − sup hy, x∗ i − 12 khx∗ k2H .
r&0 &0
x∗ ∈E ∗
Finally, note that the preceding supremum is the same as half the supremum kyk2 of hy, x∗ i over x∗ with khx∗ kH = 1, which, by Lemma 8.2.3, is equal to 2 H if y ∈ H and to ∞ if y ∈ / H. An interesting corollary of Theorem 8.4.1 is the following sharpening, due to Donsker and Varadhan, of Fernique’s Theorem.
Corollary 8.4.3. Let W be a non-degenerate, centered, Gaussian measure on the separable Banach space E, let H be the associated Cameron–Martin space, and determine Σ > 0 by Σ−1 = inf{khkH : khkE = 1}. Then 1 lim R−2 log W kxkE ≥ R = − 2 . 2Σ
R→∞
α2 2 In particular, EW e 2 kxkE is finite if α < Σ−1 and infinite if α ≥ Σ−1 .
Proof: Set f (r) = inf{khkH : khkE ≥ r}. Clearly f (r) = rf (1) and f (1) = Σ−1 . Thus, by the upper bound in (8.4.2), we know that
Σ−2 f (1)2 . = lim R−2 log W kxkE ≥ R = lim R−2 log WR−2 kxkE ≥ 1 ≤ − R→∞ R→∞ 2 2
Similarly, by the lower bound in (8.4.2), for any δ ∈ (0, 1),
lim R−2 log W kxkE ≥ R ≥ lim R−2 log W kxkE > R R→∞
R→∞
≥ − inf
khk2H : khkE > R 2
≥−
1 f (1 + δ)2 = −(1 + δ)2 2 , 2Σ 2
and so we have now proved the first assertion. α2 kxk2E is finite when α < Given the first assertion, it is obvious that EW e 2 Σ−1 and infinite when α > Σ−1 . The case when α = Σ−1 is more delicate. To handle it, I first show that Σ = sup{khx∗ kH : kx∗ kE ∗ = 1}. Indeed, if x∗ ∈ E ∗ and kx∗ kE ∗ = 1, set g = khhxx∗∗kE , note that kgkE = 1, and check that
340
8 Gaussian Measures on a Banach Space
1 ≥ hg, x∗ i = g, hx∗ H = kgkH khx∗ kH . Hence khx∗ kH ≤ kgk−1 H ≤ Σ. Next, suppose that h ∈ H with khkE = 1. Then, by the Hahn–Banach Theorem, there exists a x∗ ∈ E ∗ with kxkE ∗ = 1 and hh, x∗ i = 1. In particular, khkH khx∗ kH ≥ h, hx∗ H = hh, x∗ i = 1, and therefore khk−1 H ≤ khx∗ kH , which, together with the preceding, completes the verification. The next step is to show that there exists an x∗ ∈ E ∗ with kx∗ kE ∗ = 1 such that khx∗ kH = Σ. To this end, choose {x∗k : k ≥ 1} ⊆ E ∗ with kx∗k kE ∗ = 1 so that khx∗k kH −→ Σ. Because BE ∗ (0, 1) is compact in the weak* topology and, by Theorem 8.2.6, x∗ ∈ E ∗ 7−→ hx∗ ∈ H is continuous from the weak* topology into the strong topology, we can assume that {x∗k : k ≥ 1} is weak* convergent to some x∗ ∈ BE ∗ (0, 1) and that khx∗ kH = Σ, which is possible only if kx∗ kE ∗ = 1. Finally, knowing that this x∗ exists, note that h · , x∗ i is a centered Gaussian under W with variance Σ2 . Hence, since kxkE ≥ |hx, x∗ i|, h kxk2E i Z ξ2 e 2Σ2 γ0,Σ2 (dξ) = ∞. EW e 2Σ2 ≥ R
§ 8.4.2. Strassen’s Law of the Iterated Logarithm. Just as in § 1.5 we were able to prove a law of the iterated logarithm on the basis of the large deviation estimates in § 1.3, so here the estimates in the preceding subsection will allow us to prove a law of the iterated for centered Gaussian random variables on a Banach space. Specifically, I will prove the following theorem, whose statement is modeled on V. Strassen’s famous law of the iterated for Brownian motion (cf. § 8.6.3). q Sn , where Recall from § 1.5 the notation Λn = 2n log(2) (n ∨ 3) and S˜n = Λ n Pn Sn = 1 Xm .
Theorem 8.4.4. Suppose that W is a non-degenerate, centered, Gaussian measure on the Banach space E, and let H be its Cameron–Martin space. If {Xn : n ≥ 1} is a sequence of independent, E-valued, W-distributed random variables on some probability space (Ω, F, P), then, P-almost surely, the sequence {S˜n : n ≥ 1} is relatively compact in E and the closed unit ball BH (0, 1) in H coincides with its set of limit points. Equivalently, P-almost surely, limn→∞ kS˜n − BH (0, 1)kE = 0 and, for each h ∈ BH (0, 1), limn→∞ kS˜n − hkE = 0.
Because, by Theorem 8.2.6, BH (0, 1) is compact in E, the equivalence of the two formulations is obvious, and so I will concentrate on the second formulation. I begin by showing that limn→∞ kS˜n − BH (0, 1)kE = 0 P-almost surely, and the fact that underlies my proof is the estimate that, for each open subset G of E and α < inf{khkH : h ∈ / G}, there is an M ∈ (0, ∞) with the property that √ α2 Λ2 Sn for all n ∈ Z+ and Λ ≥ M n. ∈ / G ≤ exp − (*) P 2n Λ
§ 8.4 A Large Deviations Result and Strassen’s Theorem
341
To check (*), first note (cf. Exercise 8.2.14) that the distribution of Sn under 1 ˜ /G = P is the same as that of x n 2 x under W and therefore that P SΛn ∈
W n2 (G{). Hence, (*) is really just an application of the upper bound in (8.4.2). Λ Given (*), I proceed in very much the same way as I did at the analogous place in § 1.5. Namely, for any β ∈ (1, 2),
lim kS˜n − BH (0, 1)kE ≤ lim
max
m→∞ β m−1 ≤n≤β m
n→∞
kS˜n − BH (0, 1)kE
kSn − BH (0, Λ[β m−1 ] )kE m→∞ β Λn ≤n≤β m
Sn
− BH (0, 1) ≤ lim max m
. m→∞ 1≤n≤β Λ m−1
≤ lim
max m−1
[β
]
E
At this point in § 1.5 (cf. the proof of Lemma 1.5.3), I applied L´evy’s reflection principle to get rid of the “max.” However, L´evy’s argument works only for R-valued random variables, and so here I will replace his estimate by one based on the idea in Exercise 1.4.25. Lemma 8.4.5. Let {YmP: m ≥ 1} be mutually independent, E-valued random n variables, and set Sn = m=1 Ym for n ≥ 1. Then, for any closed F ⊆ E and δ > 0, P(kSn − F kE ≥ δ) . P max kSm − F kE ≥ 2δ ≤ 1≤m≤n 1 − max1≤m≤n P(kSn − Sm kE ≥ δ)
Proof: Set Am = {kSm − F kE ≥ 2δ and kSk − F kE < 2δ for 1 ≤ k < m}. Following the hint for Exercise 1.4.25, observe that P max kSm − F kE ≥ 2δ min P(kSn − Sm kE < δ) 1≤m≤n
≤
n X m=1
1≤m≤n
n X P Am ∩ {kSn − Sm kE < δ} ≤ P Am ∩ {kSn − F kE ≥ δ} , m=1
which, because the Am ’s are disjoint, is dominated by P kSn − F kE ≥ δ . Applying the preceding to the situation at hand, we see that !
Sn − BH (0, 1) P max
≥ 2δ 1≤n≤β m Λ[β m−1 ] E
S[βm ] − BH (0, 1) ≥ δ P Λ[β m−1 ] E . ≤ 1 − max1≤n≤β m P kSn kE ≥ δΛ[β m−1 ]
342
8 Gaussian Measures on a Banach Space
After combining this with the estimate in (*), it is an easy matter to show that, for each δ > 0, there is a β ∈ (1, 2) such that !
∞ X
Sn
− BH (0, 1) P max
≥ 2δ < ∞,
β m−1 ≤n≤β m Λ[β m−1 ] E m=1
from which it should be clear why limn→∞ kS˜n − BH (0, 1)kE = 0 P-almost surely. The proof that, P-almost surely, limn→∞ kS˜n − hkE = 0 for all h ∈ BH (0, 1) differs in no substantive way from the proof of the analogous assertion in the second part of Theorem 1.5.9. Namely, because BH (0, 1) is separable, it suffices to work with one h ∈ BH (0, 1) at a time. Furthermore, just as I did there, I can reduce the problem to showing that, for each k ≥ 2, > 0, and h with khkH < 1, ∞ X
P S˜km −km−1 − h E < = ∞. m=1
But, if khkH < α < 1, then (8.4.2) says that
2 m m−1 ) P S˜km −km−1 − h E < = W km −km−1 BE (h, ) ≥ e−α log(2) (k −k Λ2 km −km−1
for all large enough m’s. Exercises for § 8.4 Exercise 8.4.6. Let (H, E, W) be an abstract Wiener space, and assume that dim(H) = ∞. If W is defined for > 0 as in Theorem 8.4.1, show that W1 ⊥ W2 if 2 6= 1 . Hint: Choose {x∗m : m ≥ 0} ⊆ E ∗ so that {hx∗m : m ≥ 0} is an orthonormal basis in H, and show that n−1 1 X hx, x∗m i2 = W -almost surely. n→∞ n m=0
lim
Exercise 8.4.7. Show that the Σ in Corollary 8.4.3 is 12 in the case of the classical abstract Wiener space H1 (RN ), Θ(RN ), W (N ) and therefore that
lim R−2 log W (N ) kθkΘ(RN ) ≥ R = −2.
R→∞
Next, show that ! lim R
R→∞
−2
log W
(N )
sup |θ(τ )| ≥ R τ ∈[0,t]
=−
1 2t
§ 8.5 Euclidean Free Fields
343
and that ! 2 sup |θ(τ )| ≥ R θ(t) = 0 = − . t
lim R−2 log W (N )
R→∞
τ ∈[0,t]
Finally, show that lim R
R→∞
−1
log W
(N )
Z
t 2
|θ(τ )| dτ ≥ R 0
=−
π2 8t2
and that lim R
−1
R→∞
log W
(N )
Z 0
t
π2 |θ(τ )| dτ ≥ R θ(t) = 0 = − 2 . 2t 2
Hint: In each case after the first, Brownian scaling can be used to reduce the problem to the case when t = 1, and the challenge is to find the optimal constant C for which khkE ≤ CkhkH , h ∈ H for the appropriate abstract Wiener space N (E, H, W). In the second case E = C [0, 1] : R ≡ θ [0, 1] : θ ∈ Θ(RN ) 0 and H = η [0, 1] : η ∈ H1 (RN ) , in the third (cf. part (ii) of Exercise 8.3.22) N E = Θ1 (RN ) and H = H11 (RN ) , in the fourth E = L2 [0, 1]; {η R ) and H = 1 N 2 N 1 N [0, 1] : η ∈ H (R )}, and in the fifth E = L [0, 1]; R and H = H (R ). 1 The optimization problems when E = Θ(RN ) or C0 [0, 1]; RN are rather easy 1 consequences of |η(t)| ≤ t 2 kηkH1 (RN ) . When E = Θ1 (RN ), one should start with ˙ L1 ([0,1];RN ) ≤ kηkH11 (RN ) . the observation that if η ∈ H11 (RN ), then 2kηku ≤ kηk In the final two cases, one can either use elementary variational calculus or one can make use of, respectively, the orthonormal bases
1
2 2 sin n +
1 2
1 πτ : n ≥ 0 and 2 2 sin nπτ : n ≥ 1 in L2 [0, 1]; R).
Exercise 8.4.8. Suppose that f ∈ C E; R , and show, as a consequence of Theorem 8.4.4, that
lim f S˜n = min{f (h) : khkH ≤ 1} and lim f S˜n = max{f (h) : khkH ≤ 1} n→∞
n→∞
W N -almost surely. § 8.5 Euclidean Free Fields In this section I will give a very cursory introduction to a family of abstract Wiener spaces they played an important role in the attempt to give a mathematically rigorous construction of quantum fields. From the physical standpoint, the fields treated here are “trivial” in the sense that they model “free” (i.e., non-interacting) fields. Nonetheless, they are interesting from a mathematical
344
8 Gaussian Measures on a Banach Space
standpoint and, if nothing else, show how profoundly properties of a process are effected by the dimension of its parameter set. I begin with the case when the parameter set is one dimensional and the resulting process can be seen as a minor variant of Brownian motion. As we will see, the intractability of the higher dimensional analogs increases with the number of dimensions. § 8.5.1. The Ornstein–Uhlenbeck Process. Given x ∈ RN and θ ∈ Θ(RN ), consider the integral equation Z 1 t U(τ, x, θ) dτ, t ≥ 0. (8.5.1) U(t, x, θ) = x + θ(t) − 2 0
A completely elementary argument (e.g., via Gronwall’s Inequality) shows that, for each x and θ, there is at most one solution. Furthermore, integration by parts allows one to check that if Z t τ − 2t e 2 dθ(τ ), U(t, 0, θ) = e 0
where the integral is taken in the sense of Riemann-Stieltjes, then t
U(t, x, θ) = e− 2 x + U(t, 0, θ)
is one, and therefore the one and only, solution. The stochastic process {U(t, x) : t ≥ 0} under W (N ) was introduced by L. Ornstein and G. Uhlenbeck1 and is known as the Ornstein–Uhlenbeck process starting from x. From our immediate point of view, its importance is that it leads to a completely tractable example of a free field. Intuitively, U(t, 0, θ) is a Brownian motion that has been subjected to a linear restoring force. Thus, locally it should behave very much like a Brownian motion. However, over long time intervals it should feel the effect of the restoring force, which is always pushing it back toward the origin. To see how these intuitive ideas are reflected in the distribution of {U(t, 0, θ) : t ≥ 0}, I begin by using t Exercise 8.2.18 to identify e, U(t, 0) RN as e− 2 I(hte ) for each e ∈ SN −1 , where t∧τ hte (τ ) = 2 e 2 − 1 e. Hence, the span of ξ, U(t, 0) RN : t ≥ 0 & ξ ∈ RN is a Gaussian family in L2 (W (N ) ; R), and
EW
(N )
|t−s| s+t U(s, 0) ⊗ U(t, 0) = e− 2 − e− 2 I.
The key to understanding the process {U(t, 0) : t t ≥ 0} is the observation that it has the same distribution as the process e− 2 B et − 1 : t ≥ 0 , where 1
In their article “On the theory of Brownian motion,” Phys. Reviews 36 # 3, pp. 823-841 (1930), L. Ornstein and G. Uhlenbeck introduced this process in an attempt to reconcile some of the more disturbing properties of Wiener paths with physical reality.
§ 8.5 Euclidean Free Fields
345
{B(t) : t ≥ 0} is a Brownian motion, a fact that follows immediately from the observation that they are Gaussian families with the same covariance structure. In particular, by combining this with the Law of the Iterated Logarithm proved in Exercise 4.3.15, we see that, for each e ∈ SN −1 , e, U(t, x) RN e, U(t, x) RN √ √ = 1 = − lim lim (8.5.2) t→∞ 2 log t 2 log t t→∞
W (N ) -almost surely, which confirms the suspicion that the restoring force dampens the Brownian excursions out toward infinity. A second indication that U( · , x) tends to spend more time than Brownian paths do near the origin is that its distribution at time t will be γe− 2t x,(1−e−t )I , and so, as distinguished from Brownian motion itself, its distribution as time t tends to a limit, namely γ0,I . This observation suggests that it might be interesting to look at an ancient Ornstein–Uhlenbeck process, one that already has been running for an infinite amount of time. To be more precise, since the distribution of an ancient Ornstein–Uhlenbeck at time 0 would be γ0,I , what we should look at is the process that we get by making the x in U( · , x, θ) a standard normal random variable. Thus, I will say that a stochastic process {UA (t) : t ≥ 0} is an ancient Ornstein–Uhlenbeck process if its distribution is that of {U(t, x, θ) : t ≥ 0} under γ0,I × W (N ) . If {U process, then it is clear A (t) : t ≥ 0} is an ancient Ornstein–Uhlenbeck that ξ, UA (t) RN : t ≥ 0 & ξ ∈ RN spans a Gaussian family with covariance
|t−s| EP UA (s) ⊗ UA (t) = e− 2 I.
As we see that if {B(t) : t ≥ 0} is a Brownian motion, then −at consequence, e 2 B et : t ≥ 0 is an ancient Ornstein–Uhlenbeck process. In addition, as we suspected, the ancient Ornstein–Uhlenbeck process is a stationary process in the sense that, for each T > 0, the distribution of {UA (t + T ) : t ≥ 0} is the same as that of {UA (t) : t ≥ 0}, which can be checked either by using the preceding representation in terms of Brownian motion or by observing that its covariance is a function of t − s. In fact, even more is true: it is time reversible in the sense that, for each T > 0, {UA (t) : t ∈ [0, T ]} has the same distribution as {UA (T − t) : t ∈ [0, T ]}. This observation suggests that we can give the ancient Ornstein–Uhlenbeck its past by running it backwards. That is, define UR : [0, ∞) × RN × Θ(RN )2 −→ RN by U(t, x, θ+ ) if t ≥ 0 UR (t, x, θ+ , θ− ) = U(−t, x, θ− ) if t < 0,
and consider the process {UR (t, x, θ+ , θ− ) : t ∈ R} under γ0,I × W (N ) × W (N ) . This process also spans a Gaussian family, and it is still true that |t−s| (N ) (N ) (8.5.3) Eγ0,I ×W ×W UR (s) ⊗ UR (t) = u(s, t)I, where u(s, t) ≡ e− 2 ,
346
8 Gaussian Measures on a Banach Space
only now for all s, t ∈ R. One advantage of having added the past is that the statement of reversibility takes a more appealing form. Namely, {UR (t) : t ∈ R} is reversible in the sense that its distribution is the same whether one runs it forward or backward in time. That is, {UR (−t) : t ∈ R} has the same distribution as {UR (t) : t ∈ R}. For this reason, I will say that {UR (t) : t ≥ 0} is a reversible Ornstein–Uhlenbeck process if its distribution is the same as that of {UR (t, x, θ+ , θ− ) : t ≥ 0} under γ0,I × W (N ) × W (N ) . An alternative way to realize a reversible Ornstein–Uhlenbeck process is to start with an RN -valued Brownian motion {B(t) : t ≥ 0} and consider the t t process {e− 2 B(et ) : t ∈ R}. Clearly ξ, e− 2 B(et ) RN : (t, ξ) ∈ R × RN is a Gaussian family with covariance given by (8.5.3). It is amusing to observe that, when one uses this realization, the reversibility of the Ornstein–Uhlenbeck process is equivalent to the time inversion invariance (cf. Exercise 4.3.11) of the original Brownian motion. § 8.5.2. Ornstein–Uhlenbeck as an Abstract Wiener Space. So far, my treatment of the Ornstein–Uhlenbeck process has been based on its relationship to Brownian motion. Here I will look at it as an abstract Wiener space. Begin with the one-sided process 0) : t ≥ 0}. Seeing as this process − t {U(t, t 2 has the same distribution as e B e − 1 : t ≥ 0}, it is reasonably clear that the Hilbert space associated with this process should be the space HU (RN ) t of functions hU (t) = e− 2 h et − 1), h ∈ H1 (RN ). Thus, define the map F U : H1 (RN ) −→ HU (RN ) accordingly, and introduce the Hilbert norm k · kHU (RN ) on HU (RN ) that makes F U into an isometry. Equivalently, Z h d i2 1 U 2 ds (1 + s) 2 hU log(1 + s) kh kHU (RN ) = [0,∞) ds 1 ˙U U 2 khU k2 2 = kh˙ U k2 2 N . N + h ,h N + L ([0,∞);R )
L ([0,∞);R )
4
L ([0,∞);R )
Note that h˙ U , hU
L2 ([0,∞);RN )
=
1 2
Z [0,∞)
d U |h (t)|2 dt = dt
1 lim |hU (t)|2 2 t→∞
= 0. 1
To check the final equality, observe that it is equivalent to limt→∞ t− 2 |h(t)| = 0 1 1 for h ∈ H(RN ). Hence, since supt>0 t− 2 |h(t)| ≤ khkH1 (RN ) and limt→∞ t− 2 |h(t)| = 0 if h˙ has compact support, the same result is true for all h ∈ H1 (RN ). In particular, q khU kHU (RN ) = kh˙ U k2L2 ([0,∞);RN ) + 14 khU k2L2 ([0,∞);RN ) .
If we were to follow the prescription in Theorem 8.3.1, we would next complete t HU (RN ) with respect to the norm supt≥0 e− 2 |hU (t)|. However, we already know
§ 8.5 Euclidean Free Fields
347
from (8.5.2) that {U(t, 0) : t ≥ 0} lives on ΘU (RN ), the space of θ ∈ Θ(RN ) such that limt→∞ (log t)−1 |θ(t)| = 0 with Banach norm −1 kθk ≡ sup log(e + t) |θ(t)|, t≥0
and so we will adopt ΘU (RN ) as the Banach space for HU (RN ). Clearly, the dual space ΘU (RN )∗ of ΘU (RN ) can be identified with the space of RN -valued Borel measures λ on [0, ∞) that give 0 mass to {0} and satisfy kλkΛU (RN ) ≡ R log(e + t) |λ|(dt) < ∞. [0,∞) (N ) Theorem 8.5.4. Let U0 ∈ M1 ΘU (RN ) be the distribution of {U(t, 0) : (N ) t ≥ 0} under W (N ) . Then HU (RN ), ΘU (RN ), U0 is an abstract Wiener space. Proof: Since Cc∞ (0, ∞); RN is contained in HU (RN ) and is dense in ΘU (RN ), we know that HU (RN ) is dense in ΘU (RN ). In addition, because η U (t) = t e− 2 η(et − 1), where η ∈ H1 (RN ), and kη U kHU (RN ) = kηkH1 (RN ) , kη U ku ≤ 1 kη U kHU (RN ) follows from |η(t)| ≤ t 2 kηkH1 (RN ) . Hence, HU (RN ) is continuously embedded in ΘU (RN ). To complete the proof, remember our earlier calculation of the covariance of {U(t; 0) : t ≥ 0}, and use it to check that (N )
EU0
hθ, λi2 =
ZZ u0 (s, t) λ(ds) · λ(dt),
where u0 (s, t) ≡ e−
|s−t| 2
− e−
s+t 2
.
[0,∞)2
U N Hence, what I need to show is that if λ ∈ ΘU (RN )∗ −→ hU λ ∈ H (R ) is the U U U map determined by hh , λi = h , hλ HU (RN ) , then
(8.5.5)
2 khU λ kHU (RN ) =
ZZ u0 (s, t) λ(ds) · λ(dt).
[0,∞)2
In order to do this, we must first know how hU λ is constructed from λ. But if (8.5.5) is going to hold, then, by polarization, U e, hU λ (τ ) RN = hhλ , δτ ei =
ZZ u0 (s, t) δτ (ds) e, λ(dt)
[0,∞)2
Z =
e,
! u0 (τ, t) λ(dt)
[0,∞)
. RN
RN
348
8 Gaussian Measures on a Banach Space
R Thus, one should guess that hU λ (τ ) = [0,∞) u0 (τ, t) λ(dt) and must check that, U U N U U N with this choice, h λ ∈ H (R ), (8.5.5) holds, and, for all h ∈ H (R ), U U U hh , λi = h , hλ HU (RN ) . The key to proving all these is the equality Z Z hU (τ )u0 (τ, t) dτ = hU (t), (*) h˙ U (τ )∂τ u0 (τ, t) dτ + 14 [0,∞)
[0,∞)
which is an elementary application of integration by parts. Applying (*) with N = 1 to hU = u0 ( · , s), we see that Z ∂τ u0 (s, τ )∂τ u0 (t, τ ) dτ = u0 (s, t), [0,∞) U N from which it follows easily both that hU λ ∈ H (R ) and that (8.5.5) holds. U U N U U In addition, if h ∈ H (R ), then hh , λi = h , hU λ HU (RN ) follows from (*) after one integrates both sides of the preceding with respect to λ(dt). I turn next to the reversible case. By the considerations in § 8.4.1, we know (N ) that the distribution UR of {UR (t) : t ≥ 0} under γ0,1 × W (N ) × W (N ) is a Borel measure on the space Banach space ΘU (R; RN ) of continuous θ : R −→ RN such that lim|t|→∞ (log t)−1 |θ(t)| = 0 with norm
−1 kθkΘU (R;RN ) ≡ sup log(e + |t|) |θ(t)| < ∞. t∈R
Furthermore, it should be clear that one can identify ΘU (R; RN )∗ with the space of RN -valued Borel measures λ on R satisfying Z kλkΛU (R;RN ) ≡ log(e + |t|) |λ|(dt) < ∞. R
Theorem 8.5.6. Take H1 (R; RN ) to be the separable Hilbert space of absolutely continuous h : R −→ RN satisfying khkH1 (R;RN ) ≡
q
˙ 2 2 N + 1 khk2 2 N < ∞. khk 4 L (R:R ) L (R:R ) (N )
Then H1 (R; RN ), ΘU (R; RN ), UR
is an abstract Wiener space.
|s−t| − 2
, and let λ ∈ ΛU (R; RN ). By the same reasoning Proof: Set u(s, t) ≡ e as I used in the preceding proof, hh, λi = h, hλ
H1 (R;RN )
§ 8.5 Euclidean Free Fields and khλ k2H1 (R;RN ) =
349
ZZ u(s, t) λ(ds) · λ(dt) R×R
u(τ, t) λ(dt). Hence, since ξ, θ(t) RN : t ≥ 0 & ξ ∈ RN (N ) (N ) spans a Gaussian family in L2 UR ; R and u(s, t)I = EUR θ(s) ⊗ θ(t) , the proof is complete. when hλ (τ ) =
R
R
§ 8.5.3. Higher Dimensional Free Fields. Thinking a la Feynman, Theorem (N ) 8.5.6 is saying that UR wants to be the measure on H 1 (R; R) given by 1
√
( 2π)dim(H1 (R;RN ))
Z 1 2 2 1 ˙ |h(t)| + 4 |h(t)| dt λH1 (R;RN ) (dh), exp − 2 R
where λH1 (R;RN ) is the Lebesgue measure on H1 (R; RN ). I am now going to look at the analogous situation when N = 1 but the parameter set R is replaced by Rν for some ν ≥ 2. That is, I want to look at the measure that Feynman would have written as 1
√ ( 2π)dim(H 1 (Rν ;R))
Z 1 2 2 1 |∇h(x))| + 4 |h(x)| dx λH 1 (Rν ;R) (dh), exp − 2 Rν
where H 1 (Rν ; R) is the separable Hilbert space obtained by completing the Schwartz test function space S (Rν ; R) with respect to the Hilbert norm khkH 1 (Rν ;R) ≡
q
k∇hk2L2 (Rν ;R) + 14 khk2L2 (Rν ;R) .
When ν = 1 this is exactly the Hilbert space H 1 (R; R) described in Theorem 8.5.6 for N = 1. When ν ≥ 2, generic elements of H 1 (Rν ; R) are better than generic elements of L2 (Rν ; R) but are not enough better to be continuous. In fact, they are not even well-defined pointwise, and matters get worse as ν gets larger. Thus, although Feynman’s representation is already questionable when ν = 1, its interpretation when ν ≥ 2 is even more fraught with difficulties. As we will see, these difficulties are reflected mathematically by the fact that, in order to construct an abstract Wiener space for H 1 (Rν ; R) when ν ≥ 2, we will have to resort to Banach spaces whose elements are generalized functions (i.e., distributions in the sense of L. Schwartz).2 2
The need to deal with generalized functions is the primary source of the difficulties that mathematicians have when they attempt to construct non-trivial quantum fields. Without going into any details, suffice it to say that in order to construct interacting (i.e., non-Gaussian) fields, one has to take non-linear functions of a Gaussian field. However, if the Gaussian field is distribution valued, it is not at all clear how to apply a non-linear function to it.
350
8 Gaussian Measures on a Banach Space
The approach that I will adopt is based on the following subterfuge. The space H 1 (Rν ; R) is one of a continuously graded family of spaces known as Sobolev spaces. Sobolev spaces are graded according to the number of derivatives “better or worse” than L2 (Rν ; R) their elements are. To be more precise, for each s ∈ R, define the Bessel operator B s on S (Rν ; C) so that s s ϕ(ξ) = 1 + |ξ|2 − 2 ϕ(ξ). d ˆ B 4 m When s = −2m, it is clear that B s = 14 −∆ , and so, in general, it is reasonable to think of B s as an operator that, depending on whether s ≤ 0 or s ≥ 0, involves taking or restoring derivatives of order |s|. In particular, kϕkH 1 (Rν ;R) = kB −1 ϕkL2 (Rν ;R) for ϕ ∈ S (Rν ; R). More generally, define the Sobolev space H s (Rν ; R) to be the separable Hilbert space obtained by completing S (Rν ; R) with respect to s Z s 1 −s 1 2 dξ. ˆ + |ξ|2 |h(ξ)| khkH s (Rν ;R) ≡ kB hkL2 (Rν ;R) = 4 ν (2π) Rν
Obviously, H 0 (Rν ; R) is just L2 (Rν ; R). When s > 0, H s (Rν ; R) is a subspace of L2 (Rν ; R), and the quality of its elements will improve as s gets larger. However, when s < 0, some elements of H s (Rν ; R) will be strictly worse than elements of L2 (Rν ; R), and their quality will deteriorate as s becomes more negative. Nonetheless, for every s ∈ R, H s (Rν ; R) ⊆ S 0 (Rν ; R), where S 0 (Rν ; R), whose elements are called real-valued tempered distributions, is the dual space of S (Rν ; R). In fact, with a little effort, one can check that an alternative description of H s (Rν ; R) is as the subspace of u ∈ S 0 (Rν ; R) with the property that B −s u ∈ L2 (Rν ; R). Equivalently, H s (Rν ; R) is the isometric image in S (Rν ; R) of L2 (Rν ; R) under the map B s , and, more generally, H s2 (Rν ; R) is the isometric image of H s1 (Rν ; R) under B s2 −s1 . Thus, by Theorem 8.3.1, once we understand the abstract Wiener spaces for any one of the spaces H s (Rν ; R), understanding the abstract Wiener spaces for any of the others comes down to understanding the action of the Bessel operators, a task that, depending on what one wants to know, can be highly non-trivial. ν+1
Lemma 8.5.7. The space H 2 (Rν ; R) is continuously embedded as a dense subspace of the separable Banach space C0 (Rν ; R) whose elements are continuous functions that tend to 0 at infinity and whose norm is the uniform norm. Moreover, given a totally finite, signed Borel measure λ on Rν , the function Z 1−ν |x−y| π 2 − 2 , λ(dy), with Kν ≡ hλ (x) ≡ Kν e Γ ν+1 Rν 2
is an element of H
ν+1 2
khλ k
(Rν ; R),
ZZ
2 H
ν+1 2
(Rν ;R)
= Kν Rν ×Rν
e−
|x−y| 2
λ(dx)λ(dy),
§ 8.5 Euclidean Free Fields
351
and hh, λi = h, hλ
H
ν+1 2
for each h ∈ H
(Rν ;R)
ν+1 2
(Rν ; R).
Proof: To prove the initial assertion, use the Fourier inversion formula to write Z √ −ν ˆ dξ h(x) = (2π) e− −1(x,ξ)Rν h(ξ) Rν
for h ∈ S (R ; R), and derive from this the estimate 12 Z ν+1 ν 2 − 2 1 khk dξ + |ξ| khku ≤ (2π)− 2 4 ν
H
Rν
Hence, since H norm k · k ν+1 H
2
ν+1 2
ν+1 2
(Rν ;R)
.
(Rν ; R) is the completion of S (Rν ; R) with respect to the ν+1 , it is clear that H 2 (Rν ; R) is continuously embedded in
(Rν ;R)
ν+1
C0 (R ; R). In addition, since S (Rν ; R) is dense in C0 (Rν ; R), H 2 (Rν ; R) is also. To carry out the next step, let λ be given, and observe that the Fourier − ν+1 2 ˆ λ(ξ) and therefore that transform of B ν+1 λ is 14 + |ξ|2 √ Z ˆ e− −1(x,ξ)Rν λ(ξ) 1 ν+1 dξ B λ(x) = ν+1 2 (2π)ν Rν 1 2 + |ξ| 4 √ Z Z −1(y−x,ξ)Rν e 1 dξ λ(dy). = ν+1 (2π)ν Rν 2 Rν 1 + |ξ|2 4 ν
Now use (3.3.19) (with N = ν and t = 12 ) to see that √ Z |y−x| e −1(y−x,ξ)Rν 1 − 2 , dξ = K e ν ν+1 (2π)ν Rν 1 + |ξ|2 2
and thereby arrive at hλ = B khλ k2
H
ν+1 2
(Rν ;R)
4 ν+1
λ. In particular, this shows that Z 2 ˆ |λ(ξ)| 1 dξ < ∞. = (2π)ν Rν 1 + |ξ|2 ν+1 2 4
Now let h ∈ S (Rν ; R), and use the preceding to justify ν+1 ν+1 hh, λi = hB − 2 h, B − 2 B ν+1 λi = h, hλ ν+1 H
2
(Rν ;R)
.
ν+1
Since both sides are continuous with respect to convergence in H 2 (Rν ; R), we ν+1 for all h ∈ H 2 (Rν ; R). In have now proved that hh, λi = h, hλ ν+1 ν H
2
(R ;R)
particular, khλ k
ZZ
2 H
ν+1 2
(Rν ;R)
= hhλ , λi = Kν Rν ×Rν
e−
|y−x| 2
λ(dx)λ(dy).
352
8 Gaussian Measures on a Banach Space ν+1
Theorem 8.5.8. Let Θ 2 (Rν ; R) be the space of continuous θ : Rν −→ R sat−1 ν+1 isfying lim|x|→∞ log(e+|x|) |θ(x)| = 0, and turn Θ 2 (Rν ; R) into a separable −1 = supx∈RN log(e + |x|) |θ(x)|. Then Banach space with norm kθk ν+1 ν Θ
ν+1 2
2
(R ;R)
ν
(R ; R) is continuously embedded as a dense subspace of Θ H ν+1 there is a W ν+1 ν ∈ M1 Θ 2 (Rν ; R) such that H
2
ν+1 2
(Rν ; R), and
(R ;R)
H
ν+1 2
(Rν ; R), Θ
ν+1 2
(Rν ; R), W
H
ν+1 2
(Rν ;R)
is an abstract Wiener space. Moreover, for each α ∈ 0, 12 , W
H
every θ is H¨ older continuous of order α and, for each α > 12 , W
H
ν+1 2
(Rν ;R)
ν+1 2
(Rν ;R)
-almost
-almost
no θ is anywhere H¨ older continuous of order α. Proof: The initial part of the first assertion follows from the first part of Lemma 8.5.7 plus the essentially trivial fact that C0 (Rν ; R) is continuously emν+1 bedded as a dense subspace of Θ 2 (Rν ; R). Further, by the second part of that same lemma combined with Theorem 8.3.3, we will have proved the second part of the first assertion here once we show that, when {hm : m ≥ 0} is P∞ ν+1 an orthonormal basis in H 2 (Rν ; R), the Wiener series m=0 ωm hm converges ν+1 N -almost every ω = (ω0 , . . . , ωm , . . . ) ∈ RN . Thus, set in Θ 2 (Rν ; R) for γ0,1 Pn Sn (ω) = m=0 ωm hm for n ≥ 1. More or less mimicking the steps outlined in Exercise 8.3.21, I will begin by showing that, for each α ∈ 0, 12 and R ∈ [1, ∞),
(*)
|Sn (y) − Sn (x)| < ∞, |y − x|α n≥0 x,y∈Q(z,R)
N sup Eγ0,1 sup
z∈Rν
sup
x6=y
where Q(z, R) = z + [−R, R)ν . Indeed, by the argument given in that exercise combined with the higher dimensional analog of Kolmogorov’s continuity criterion in Exercise 4.3.18, (*) will follow once we show that N Eγ0,1 |Sn (y) − Sn (x)|2 ≤ C|y − x|,
x, y ∈ Rν ,
for some C < ∞. To this end, set λ = δy − δx , and apply Lemma 8.5.7 to check E
N γ0,1
n X 2 2 |Sn (y) − Sn (x)| = hm , hλ
H
ν+1 2
(Rν ;R)
m=0
≤ khλ k2
H
ν+1 2
(Rν ;R)
= 2Kν 1 − e−
|y−x| 2
.
Knowing (*), it becomes an easy matter to see that there exists a measurable S : Rν × RN −→ R such that x S(x, ω) is continuous of each ω and
§ 8.5 Euclidean Free Fields
353
N Sn ( · , ω) −→ S( · , ω) uniformly on compacts for γ0,1 -almost every ω ∈ RN . In N fact, because of (*), it suffices to check that limn→∞ Sn (x) exists γ0,1 -almost ν surely for each x ∈ R , and this follows immediately from Theorem 1.4.2 plus ∞ X
Var ωm hm (x) =
m=0
∞ X
hm , hδx
2 H
ν+1 2
= khδx k2
(Rν ;R)
H
m=0
ν+1 2
(Rν ;R)
= Kν .
N -almost every ω, x S(x, ω) Furthermore, again from (*), we know that, γ0,1 1 is α-H¨ older continuous so long as α ∈ 0, 2 . N I must still check that, γ0,1 -almost surely, the convergence of Sn ( · , ω) to ν+1 S( · , ω) is taking place in Θ 2 (Rν ; R), and, in view of the fact that we already N know that, γ0,1 -almost surely, it is taking place uniformly on compacts, this reduces to showing that −1 N lim log(e + |x| sup |Sn (x)| −→ 0 γ0,1 -almost surely.
|x|→∞
n≥0
For this purpose, observe that (*) says that N γ0,1 sup E sup kSn ku,Q(z,1) < ∞, z∈Rν
n≥0
where k · ku,C denotes the uniform norm over a set C ⊆ Rν . At this point, I would like to apply Fernique’s Theorem (Theorem 8.2.1) to the Banach space `∞ N; Cb (Q(z, 1); R) and thereby conclude that there exists an α > 0 such that N (**) B ≡ sup Eγ0,1 exp α sup kSn k2u,Q(z,1) < ∞. z∈Rν
n≥0
∞
However, ` N; Cb (Q(z, 1); R) is not separable. Nonetheless, there are two ways to get around this technicality. The first is to observe that the only place separability was used in the proof of Fernique’s Theorem was at the beginning, where I used it to guarantee that BE is generated by the maps x hx, x∗ i as ∗ ∗ x runs over E and therefore that the distribution of X is determined by the distribution of {hX, x∗ i : x∗ ∈ E ∗ }. But, even though `∞ N; Cb (Q(z, 1); R) is not separable, one can easily check that it nevertheless possesses this property. The second way to deal with the problem is to apply his theorem to `∞ {0, . . . , N }; Cb (Q(z, 1); R) , which is separable, and to note that the resulting estimate can be made uniform in N ∈ N. Either way, p one arrives at (**). 2 Now set ψ(t) = eαt − 1 for t ≥ 0. Then ψ −1 (s) = α−1 log(1 + s), and ν sup kSn ku,Q(0,M ) = max sup kSn ku,Q(m,1) : m ∈ Q(0, M ) ∩ Z n≥0
n≥0
≤ ψ −1
X
m∈Q(0,M )∩Zν
ψ sup kSn ku,Q(m,1) .
n≥0
354
8 Gaussian Measures on a Banach Space
Thus, because ψ −1 is concave, Jensen’s Inequality applies and yields N Eγ0,1 sup kSn ku,Q(0,M ) ≤ ψ −1 (2M )ν B , n≥0
and therefore
#
"
Sn (x) ≤ log(e + |x|) |x|≥R n≥0
N γ0,1
sup sup
E
X
≤
1 m≥(log R) 4
X
h i N Eγ0,1 supn≥0 kSn ku,Q(0,em4 )
log(e + e(m−1)4 )
1 m≥(log R) 4
p log(1 + 2ν eν(m+1)4 B) √ −→ 0 α log(e + e(m−1)4 )
as R → ∞.
To complete the proof, I must show that, for any α > 12 , W
H
ν+1 2
(Rν ;R)
-almost
no θ is anywhere H¨older continuous of order α, and for this purpose I will proceed as in the proof of Theorem 4.3.4. Because the {θ(x + y) : x ∈ Rν } has the same W ν+1 ν -distribution for all y, it suffices for me to show that, W ν+1 ν H
2
(R ;R)
H
2
(R ;R)
almost surely, there is no x ∈ Q(0, 1) at which θ is H¨older continuous of order α > 12 . Now suppose that α ∈ 12 , 1 , and observe that, for any L ∈ Z+ and e ∈ Sν−1 , the set H(α) of θ’s that are α-H¨older continuous at some x ∈ Q(0, 1) is contained in ∞ \ ∞ [
[
L n \ θ : θ
m+`e n
−θ
m+(`−1)e n
≤
M nα
o .
M =1 n=1 m∈Q(0,n)∩Zν `=1
Hence, again using translation invariance, we see that we need only show that there is an L ∈ Z+ such that, for each M ∈ Z+ , (`−1)e M , 1 ≤ ` ≤ L ≤ − θ θ : θ `e nν W ν+1 ν α n n n H
2
(R ;R)
−1
tends to 0 as n → ∞. To this end, set U (t, θ) = Kν 2 θ(te), and observe that the W ν+1 ν -distribution of {U (t) : t ≥ 0} is that of an R-valued ancient H
2
(R ;R)
Ornstein–Uhlenbeck process. Thus, what I have to estimate is ` `−1 `−1 ` P e− 2n B e n − e− 2n B e n ≤ nMα , 1 ≤ ` ≤ L ,
where B(t), Ft , P is an R-valued Brownian motion. But clearly this probability is dominated by the sum of ` `−1 ` P B e n − B e n ≤ M2ne 2n α , 1 ≤ ` ≤ L
Exercises for § 8.5
355
and P ∃1 ≤ ` ≤ L
`−1 1 1 − e− 2n B e n ≥
`
M e 2n 2nα
.
M 2 n2(1−α)
8 , which, since α < 1, The second of these is easily dominated by 2Le− means that it causes no problems. As for the first, one can use the independence of Brownian increments and Brownian scaling to dominate it by the Lth power of 1 P B(1)−B e− n ≤ M (2nα )−1 . Hence, I can take any L such that α− 12 L > ν. As a consequence of the preceding and Theorem 8.3.1, we have the following corollary.
Corollary 8.5.9. Given s ∈ R, set ν+1 ν+1 Θs (Rν ; R) = B s− 2 θ : θ ∈ Θ 2 (Rν ; R) ,
kθkΘs (Rν ;R) = kB
ν+1 2 −s
θk
Θ
and WH s (Rν ;R) = (B s−
ν+1 2
)∗ W
ν+1 2
H
(Rν ;R)
ν+1 2
,
(Rν ;R) s
.
Then Θs (Rν ; R) is a separable Banach space in which H (Rν ; R) is continuously embedded as a dense subspace, and H s (Rν ; R), Θs (Rν ; R), WH s (Rν ;R) is an abstract Wiener space. Exercises for § 8.5 Exercise 8.5.10. In this exercise we will show how to use the Ornstein–Uhlenbeck process to prove Poincar´ e’s Inequality Varγ0,1 (ϕ) = kϕ − hϕ, γ0,1 ik2L2 (γ0,1 ;R) ≤ kϕ0 k2L2 (γ0,1 ;R)
(8.5.11)
for the standard Gaussian distribution on R. I will outline the proof of (8.5.11) for ϕ ∈ S (R; R), but the estimate immediately extends to any ϕ ∈ L2 (γ0,1 ; R) whose (distributional) first derivative is again in L2 (γ0,1 ; R). (i) For ϕ ∈ S (R; R), set uϕ (t, x) = EW
(1)
ϕ U (t, x) ,
where {U (t, x) : t ≥ 0} is the one-sided, R-valued Ornstein–Uhlenbeck process t starting at x. Show that u0ϕ (t, x) = e− 2 uϕ0 (t, x) and that
lim uϕ (t, · ) = ϕ and
t&0
lim uϕ (t, · ) = hϕ, γ0,1 i
t→∞
in L2 (γ0,1 ; R).
Show that another expression for uϕ is ! Z t 1 (y − e− 2 x)2 −t − 2 dy. ϕ(y) exp − uϕ (t, x) = 2π(1 − e ) 2(1 − e−t ) R
Using this second expression, show that uϕ (t, · ) ∈ S (R; R) and that t ∈ [0, ∞) 7−→ uϕ (t, · ) ∈ S (R; R) is continuous. In addition, show that u˙ ϕ (t, x) = 1 00 0 2 uϕ (t, x) − xuϕ (t, x) .
356
8 Gaussian Measures on a Banach Space
(ii) For ϕ1 , ϕ2 ∈ C 2 (R; R) whose second derivative are tempered, show that ϕ1 , ϕ002 − xϕ2
L2 (γ0,1 ;R)
= − ϕ01 , ϕ02
L2 (γ0,1 ;R)
,
and use this together with (i) to show that, for any ϕ ∈ S (R; R), huϕ (t, · ), γ0,1 i = hϕ, γ0,1 i and
d kuϕ (t, · )k2L2 (γ0,1 ;R) = −e−t kuϕ0 (t, · )k2L2 (γ0,1 ;R) . dt
Conclude that kuϕ (t, · )kL2 (γ0,1 ;R) ≤ kϕkL2 (γ0,1 ;R) and d kuϕ (t, · )k2L2 (γ0,1 ;R) ≥ −e−t kϕ0 k2L2 (γ0,1 ;R) . dt
Finally, integrate the preceding inequality to arrive at (8.5.11). Exercise 8.5.12. In this exercise I will outline how the ideas in Exercise 8.5.10 can be used to give another derivation of the logarithmic Sobolev Inequality (2.4.42). Again, I restrict my attention to ϕ ∈ S (R; R), since the general case can be easily obtained from this by taking limits. (i) Begin by showing that (2.4.42) for ϕ ∈ S (R; R) once one knows that (*)
ϕ log ϕ
γ0,1
1 ≤ 2
(ϕ0 )2 ϕ
γ0,1
for uniformly positive ϕ ∈ R ⊕ S (R; R). (ii) Given a uniformly positive ϕ ∈ R ⊕ S (R; R), use the results in Exercise 8.5.10 to show that e−t uϕ0 (t, · )2 d
. uϕ (t, · ) log uϕ (t, · ) γ0,1 = − uϕ (t, · ) γ0,1 2 dt
(iii) Continuing (ii), apply Schwarz’s inequality to check that uϕ0 (t, x)2 ≤ u (ϕ0 )2 (t, x), uϕ (t, x) ϕ
and combine this with (ii) to get
e−t d
uϕ (t, · ) log uϕ (t, · ) γ0,1 ≥ − 2 dt
Finally, integrate this to arrive at (*).
(ϕ0 )2 ϕ
. γ0,1
Exercises for § 8.5
357
Exercise 8.5.13. Although it should be clear that the arguments given in Exercises 8.5.10 and 8.5.12 work equally well in RN and yield (8.5.11) and (2.4.42) with γ0,1 replaced by γ0,I and (ϕ0 )2 replaced by |∇ϕ|2 , it is significant that each of these inequalities for R implies its RN analog. Indeed, show that Fubini’s Theorem is all that one needs to pass to the higher dimensional results. The reason why this remark is significant is that it allows one to prove infinite dimensional versions of both Poincar´e’s Inequality and the logarithmic Sobolev Inequality, and both of these play a crucial role in infinite dimensional analysis. In fact, Nelson’s interest in hypercontractive estimates sprung from his brilliant insight that hypercontractive estimates would allow him to construct a non-trivial (i.e., non-Gaussian), translation invariant quantum field for R2 . Exercise 8.5.14. It is interesting to see what happens if one changes the sign of the second term on the right-hand side of (8.5.1), thereby converting the centripetal force into a centrifugal one. (i) Show that, for each θ ∈ Θ(RN ), the unique solution to V(t, θ) = θ(t) +
1 2
t
Z
V(τ, θ) dτ,
t ≥ 0,
0
is
Z
t
t
V(t, θ) = e 2
τ
e− 2 dθ(τ ),
0
where the integral is taken in the sense of Riemann–Stieltjes. (ii) Show that ξ, V(t, · ) RN : (t, ξ) ∈ [0, ∞) × RN under W (N ) is a Gaussian family with covariance v(s, t) = e
s+t 2
−e
|t−s| 2
.
(iii) Let {B(t) : t ≥ 0} be an RN -valued Brownian motion, and show that the distribution of t e 2 B 1 − e−t : t ≥ 0
is the W (N ) -distribution of {V(t) : t ≥ 0}. Next, let ΘV (RN ) be the space of continuous θ : [0, ∞) −→ RN with the properties that θ(0) = 0 = lim e−t |θ(t)|, t→∞
and set kθkΘV (RN ) ≡ supt≥0 e−t |θ(t)|. Show that ΘV (RN ); k · kΘV (RN ) is a separable Banach space and that there exists a unique V (N ) ∈ M1 ΘV (RN ) such that the distribution of {θ(t) : t ≥ 0} under V (N ) is the same as the distribution of {V(t) : t ≥ 0} under W (N ) .
358
8 Gaussian Measures on a Banach Space
(iv) Let HV (RN ) be the space of absolutely continuous h : [0, ∞) −→ RN with the properties that h(0) = 0 and h˙ − 12 h ∈ L2 [0, ∞); RN . Show that HV (RN ) with norm
khkHV (RN ) ≡ h˙ − 12 h L2 ([0,∞);RN ) V N is a separable Hilbert space that is continuously embedded in Θ (R ) as a dense V N V N (N ) subspace. Finally, show that H (R ), Θ (R ), V is an abstract Wiener space.
(v) There is a subtlety here that is worth mentioning. Namely, show that HU (RN ) is isometrically embedded in HV (RN ). On the other hand, as distinguished from elements of HU (RN ), it is not true that kη˙ − 12 ηk2L2 (R;RN ) = ˙ 2L2 (R;RN ) + 41 kηk2L2 (R;RN ) , the point being that whereas the elements h of kηk HV (RN ) with h˙ ∈ Cc (0, ∞); RN are dense in HU (RN ), they are not dense in
HV (RN ). Exercise 8.5.15. Given x ∈ Rν and a slowly increasing ϕ ∈ C(Rν ; R), define τx ϕ ∈ C(Rν ; R) so that τx ϕ(y) = ϕ(x + y) for y ∈ Rν . Next, extend τx to S 0 (Rν ; R) so that hϕ, τx ui = hτ−x ϕ, ui for ϕ ∈ S (Rν ; R), and check that this is a legitimate extension in the sense that it is consistent with the original definition when applied to u’s that are slowly increasing, continuous functions. Finally, given s ∈ R, define Ox : H s (Rν ; R) −→ H s (Rν ; R) by Ox h = τx h. (i) Show that B s ◦ τx = τx ◦ B s for all s ∈ R and x ∈ Rν . (ii) Given s ∈ R, define Ox = τx H s (Rν ; R), and show that Ox is an orthogonal transformation. (iii) Referring to Theorem 8.3.14 and Corollary 8.5.9, show that the measure preserving transformation TOx that Ox determines on Θs (Rν ; R), WH s (Rν ;R) is the restriction of τx to Θs (Rν ; R). (iv) If x 6= 0, show that TOx is ergodic on Θs (Rν ; R), WH s (Rν ;R) . § 8.6 Brownian Motion on a Banach Space In this concluding section I will discuss Brownian motion on a Banach space. More precisely, given a non-degenerate, centered, Gaussian measure W on a separable Banach space E, we will see that there exists an E-valued stochastic process {B(t) : t ≥ 0} with the properties that B(0) = 0, t B(t) is continuous, and, for all 0 ≤ s < t, B(t) − B(s) is independent of σ {B(τ ) : τ ∈ [0, s]} and has distribution (cf. the notation in § 8.4) Wt−s . § 8.6.1. Abstract Wiener Formulation. Let W on E be as above, use H to denote its Cameron–Martin space, and take H 1 (H) to be the Hilbert space of absolutely continuous h : [0, ∞) −→ H such that h(0) = 0 and khkH 1 (H) = ˙ L2 ([0,∞);H) < ∞. Finally, let Θ(E) be the space of continuous θ : [0, ∞) −→ khk
§ 8.6 Brownian Motion on a Banach Space
359
E = 0, and turn Θ(E) into a Banach space with norm E satisfying limt→∞ kθ(t)k t −1 kθkΘ(E) = supt≥0 (1 + t) kθ(t)kE . By exactly the same line of reasoning as I used when E = RN , one can show that Θ(E) is a separable Banach space in which H 1 (E) is continuously embedded as a dense subspace. My goal is to prove the following statement.
Theorem 8.6.1. With H 1 (H) and Θ(E) as above, there is a unique W (E) ∈ M1 Θ(E) such that H 1 (H), Θ(E), W (E) is an abstract Wiener space. 1 Choose an orthonormal basis {h1m : m ≥ 0} in H (R), and, for n ≥ 0, t ≥ 0, P n N 1 and x = (x0 , . . . , xm , . . . ) ∈ E , set Sn (t, x) = m=0 hm (t)xm . I will show N that, W -almost surely, {Sn ( · , x) : n ≥ 0} converges in Θ(E), and, for the most part, the proof follows the same basic line of reasoning as that suggested in Exercise 8.3.21 when E = RN . However, there is a problem here that we did not encounter there. Namely, unless E is finite dimensional, bounded subsets will not necessarily be relatively compact in E. Hence, local uniform equicontinuity plus local boundedness is not sufficient to guarantee that a collection of E-valued paths is relatively compact in C [0, ∞); E , and that is the reason why we have to work a little harder here.
Lemma 8.6.2. For W N -almost every x ∈ E N , {Sn ( · , x) : n ≥ 0} is relatively compact in Θ(E). Proof: Choose E0 ⊆ E, as in Corollary 8.3.10, so that bounded subsets of E0 are relatively compact in E and H, E0 , W E0 is again an abstract Wiener space. Without loss in generality, I will assume that k · kE ≤ k · kE0 , and, by Fernique’s Theorem, we know that C ≡ EW0 kxk4E0 < ∞. Pn Since (cf. Exercise 8.2.14) Sn (t, x) − Sn (s, x) = m=0 h1t − h1s , h1m H 1 (R) xm , where h1τ = · ∧ τ , the W0N -distribution of Sn (t) − Sn (s) is Wn , where 2n = Pn 1 1 1 2 WN kSn (t) − Sn (s)k4E0 ≤ C(t − s)2 . 0 ht − hs , hm H 1 (R) ≤ t − s. Hence, E In addition, {kSn (t) − Sn (s)kE0 : n ≥ 1} is a submartingale, and so, by Doob’s Inequality plus Kolmogorov’s Continuity Criterion, there exists a K < ∞ such that, for each T > 0, (*)
EW
N
sup
sup
n≥0 0≤s
kSn (t) − Sn (s)kE0
(t − s)
1 8
3
≤ KT 4 .
From (*) and Sn (0) = 0, we know that, W N -almost surely, {Sn ( · , x) : n ≥ 0} is uniformly k · kE0 -bounded and uniformly k · kE0 -equicontinuous on each interval [0, T ]. Since this means that, for every T > 0, {Sn (t, x) : n ≥ 0 & t ∈ [0, T ]} is relatively compact in E and {Sn ( · , x) [0, T ] : n ≥ 0} is uniformly k · kE equicontinuous W N -almost surely, the Ascoli–Arzela Theorem guarantees that, W N -almost surely, {Sn ( · , x) : n ≥ 0} is relatively compact in C [0, ∞); E with
360
8 Gaussian Measures on a Banach Space
the topology of uniform convergence on compacts. Thus, in order to complete the proof, all that I have to show is that, W N -almost surely, lim sup sup
T →∞ n≥0 t≥T
kSn (t, x)kE = 0. t
But, sup t≥2k
X 7` X kSn (t, x)kE kSn (t, x)kE ≤ 2− 8 ≤ sup t t ` `+1 2 ≤t≤2 `≥k
`≥k
and therefore, by (*), " EW
N
kSn (t, x)kE sup sup t n≥0 t≥2k
#
sup 0≤t≤2`+1
kSn (t, x)kE 1
t8
,
3
≤
24 K 1 8
2 −1
k
2− 8 .
Now that we have the requisite compactness of {Sn : n ≥ 0}, convergence comes to checking a criterion of the sort given in the following simple lemma. Lemma 8.6.3. Suppose that {θn : n ≥ 0} is a relatively compact sequence in Θ(E). If limn→∞ hθn (t), x∗ i exists for each t in a dense subset of [0, ∞) and x∗ in a weak* dense subset of E ∗ , then {θn : n ≥ 0} converges in Θ(E). Proof: For a relatively compact sequence to be convergent, it is necessary and sufficient that every convergent subsequence have the same limit. Thus, suppose that θ and θ0 are limit points of {θn : n ≥ 0}. Then, by hypothesis, hθ(t), x∗ i = hθ0 (t), x∗ i for t in a dense subset of [0, ∞) and x∗ in a weak* dense subset of E ∗ . But this means that the same equality holds for all (t, x∗ ) ∈ [0, ∞) × E ∗ and therefore that θ = θ0 . Proof of Theorem 8.6.1: In view of Lemmas 8.6.2 and 8.6.3 and the separability of E ∗ in the weak* topology, we will know that {Sn ( · , x) : n ≥ 0} converges in Θ(E) for W N -almost every x ∈ E N once we show that, for each (t, x∗ ) ∈ [0, ∞) × E ∗ , {hSn (t, x), x∗ i : n ≥ 0} converges for W N -almost Pn in R N ∗ ∗ ∗ ∗ 1 every x ∈ E . But if x ∈ E , then hSn (t, x), x i = 0 hxm , x ihm (t), the random variables x hxm , x∗ ih1m (t) are P independent, centered Gaussians under ∞ N W with variance khx∗ k2H h1m (t)2 , and 0 h1m (t)2 = kht k2H 1 (R) = t. Thus, by Theorem 1.4.2, we have the required convergence. Next, define B : [0, ∞) × E N −→ E so that limn→∞ Sn (t, x) if {Sn ( · , x) : n ≥ 0} converges in Θ(E) B(t, x) = 0 otherwise. Given λ ∈ Θ(E)∗ , determine hλ ∈ H 1 (H) by h, hλ H 1 (H) = hh, λi for all h ∈ H 1 (H). I want to show that, under W N , x
hB( · , x), λi is a centered Gaussian
§ 8.6 Brownian Motion on a Banach Space
361
with variance khλ k2H 1 (H) . To this end, define x∗m ∈ E ∗ so that1 hx, x∗m i = hh1m x, λi for x ∈ E. Then, n X hB( · , x), λi = lim hSn ( · , x), λi = lim hxm , x∗m i n→∞
n→∞
W N -almost surely.
0
Hence, hB( · , x), λi is certainly a centered Gaussian under W N , and, because we are dealing with Gaussian random variables, almost sure convergence implies L2 convergence. To compute its variance, choose an orthonormal basis {hk : k ≥ 0} for H, and note that, for each m ≥ 0, WN
E
∞ X ∗ 2 2 hxm , xm i = khx∗m kH = hh1m hk , λi2 . k=0
Thus, since {h1m hk : (m, k) ∈ N2 } is an orthonormal basis in H 1 (H), WN
E
∞ ∞ X X 2 2 1 2 hB( · ), λi = hhm hk , λi = h1m hk , hλ H 1 (H) = khλ k2H 1 (H) . m,k=0
m,k=0
Finally, to complete the proof, all that remains is to take W (E) to be the W N -distribution of x B( · , x). § 8.6.2. Brownian Formulation. Let (H, E, W) be an abstract Wiener space. Given a probability space (Ω, F, P), a non-decreasing family of sub-σ-algebras {Ft : t ≥ 0}, and a measurable map B : [0, ∞) × Ω −→ E, say that the triple B(t), Ft , P is a W-Brownian motion if (1) B is {Ft : t ≥ 0}-progressively measurable, (2) B(0, ω) = 0 and B( · , ω) ∈ C [0, ∞); E for P-almost every ω, (3) B(1) has distribution W, and, for all 0 ≤ s < t, B(t)−B(s) is independent 1 of Fs and has the same distribution as (t − s) 2 B(1).
Lemma 8.6.4. Suppose that {B(t) : t ≥ 0} satisfies conditions (1) and (2). Then B(t), Ft , P is a W-Brownian motion if and only if hB(t), x∗ i, Ft , P is an R-valued Brownian motion for each x∗ ∈ E ∗ with khx∗ kH = 1. In addition, if B(t), Ft , P is a W-Brownian motion, then the span G(B) of {hB(t), x∗ i : (t, x∗ ) ∈ [0, ∞) × E ∗ } is a Gaussian family in L2 (P; R) and (8.6.5) EP hB(t1 ), x∗1 ihB(t2 ), x∗2 i = (t1 ∧ t2 ) hx∗1 , hx∗2 H . Conversely, if G(B) is a Gaussian family in L2 (P; R) and (8.6.5) holds, then B(t), Ft , P is a W-Brownian motion when Ft = σ {B(τ ) : τ ∈ [0, t]} . 1
Given h1 ∈ H 1 (R) and x ∈ E, I use h1 x to denote the element θ of Θ(E) determined by θ(t) = h1 (t)x.
362
8 Gaussian Measures on a Banach Space
Proof: If B(t), Ft , P is a W-Brownian motion and x∗ ∈ E ∗ with khx∗ kH = 1, then hB(t), x∗ i − hB(s), x∗ i = hB(t) − B(s), x∗ i is independent of Fs and is a centered Gaussian with variance (t − s). Thus, hB(t), x∗ i, Ft , P is an R-valued Brownian motion. Next assume that hB(t), x∗ i, Ft , P is an R-valued Brownian motion for every x∗ with khx∗ kH = 1. Then hB(t) − B(s), x∗ i is independent of Fs for every x∗ ∈ E ∗ , and so, since BE is generated by {h · , x∗ i : x∗ ∈ E ∗ }, B(t) − B(s) is independent of Fs . In addition, hB(t) − B(s), x∗ i is a centered Gaussian with variance (t − s)khx∗ k2H , and therefore B(1) has distribution W and B(t) − B(s) 1 has the same distribution as (t − s) 2 B(1). Thus, B(t), Ft , P is a W-Brownian motion. Again assume that B(t), Ft , P is a W-Brownian motion. To prove that G(B) is a Gaussian family for which (8.6.5) holds, it suffices to show that, for all 0 ≤ t1 ≤ t2 and x∗1 , x∗2 ∈ E ∗ , hB(t1 ), x∗1 i + hB(t2 ), x∗2 i is a centered Gaussian with covariance t1 khx1 ∗ + hx∗2 k2H + (t2 − t1 )khx∗2 k2H . Indeed, we would then know not only that G(B) is a Gaussian family but also that the variance of hB(t1 ), x∗1 i ± hB(t2 ), x∗2 i is t1 khx1 ∗ ± hx∗2 k2H + (t2 − t1 )khx∗2 k2H , from which (8.6.5) is immediate. But
hB(t1 ), x∗1 i + hB(t2 ), x∗2 i = hB(t1 ), x∗1 + x∗2 i + hB(t2 ) − B(t1 ), x∗2 i, and the terms on the right are independent, centered Gaussians, the first with variance t1 khx∗1 + hx∗2 k2H and the second with variance (t2 − t1 )khx∗2 k2H . Finally, take Ft = σ {B(τ ) : τ ∈ [0, t]} , and assume that G(B) is a Gaussian family satisfying (8.6.5). Given x∗ with khx∗ kH = 1 and 0 ≤ s < t, we know that hB(t) − B(s), x∗ i = hB(t), x∗ i − hB(s), x∗ i is orthogonal in L2 (P; R) to hB(τ ), y ∗ i for every τ ∈ [0, s] and y ∗ ∈ E ∗ . Hence, since Fs is generated by {hB(τ ), y ∗ i : (τ, y ∗ ) ∈ [0, s]×E ∗ }, we know that hB(t)−B(s), x∗ i is independent of Fs . In addition, hB(t) − B(s), x∗ i is a centered Gaussian with variance t − s, and so we have proved that hB(t), x∗ i, Ft , P is an R-valued Brownian motion. Now apply the first part of the lemma to conclude that B(t), Ft , P is a WBrownian motion. Theorem 8.6.6. Refer to the notation in Theorem 8.6.1. When Ω = Θ(E), F = BE , and Ft = σ {θ(τ ) : τ ∈ [0, t]} , θ(t), Ft , W (E) is a W-Brownian motion. Conversely, if B(t), Ft , P is any W-Brownian motion, then B( · , ω) ∈ Θ(E) P-almost surely and W (E) is the P-distribution of ω B( · , ω). Proof: To prove the first assertion, let t1 , t2 ∈ [0, ∞) and x∗1 , x∗2 ∈ E ∗ be given, and define λi ∈ Θ(E)∗ so that hθ, λi i = hθ(ti ), x∗i i for i ∈ {1, 2}. Then (cf. the notation in the proof of Theorem 8.6.1) hλi = h1ti hx∗i , and so EW
(E)
hθ(t1 ), x∗1 ihθ(t2 ), x∗2 i = hλ1 hλ2 H 1 (H) = (t1 ∧ t2 ) hx∗1 , hx∗2 H .
§ 8.6 Brownian Motion on a Banach Space
363
Starting from this, it is an easy matter to check that the span of {hθ(t), x∗ i : (t, x∗ ) ∈ [0, ∞) × E ∗ } is a Gaussian family in L2 (W (E) ; R) that satisfies (8.6.5). To prove the converse, begin by observing that, because G(B) is a Gaussian family satisfying (8.6.5), the distribution of ω ∈ Ω 7−→ B( · , ω) ∈ C [0, ∞); E under P is the same as that of θ ∈ Θ(E) 7−→ θ( · ) ∈ C [0, ∞); E under W (E) . Hence kθ(t)kE kB(t)kE (E) = 0 = 1, lim =0 =W P lim t→∞ t→∞ t t
and so B( · , ω) ∈ Θ(E) P-almost surely and the distribution of ω Θ(E) is W (E) .
B( · , ω) on
§ 8.6.3. Strassen’s Theorem Revisited. What I called Strassen’s Theorem in § 8.4.2 is not the form in which Strassen himself presented it. Instead, his formulation was in terms of rescaled R-valued Brownian motion, not partial sums of independent random variables. The true statement of Strassen’s Theorem is the following in the present setting. Theorem 8.6.7 (Strassen). Given θ ∈ Θ(E), define θ˜n (t) = θ(nt) Λn for n ≥ 1 q and t ∈ [0, ∞), where Λn = 2n log(2) (n ∨ 3). Then, for W (E) -almost every θ, the sequence {θ˜n : n ≥ 0} is relatively compact in Θ(E) and BH 1 (H) (0, 1) is its set of limit points. Equivalently, for W (E) -almost every θ,
lim kθ˜n − BH 1 (H) (0, 1)kΘ(E) = 0
n→∞
and, for each h ∈ BH 1 (H) (0, 1), limn→∞ kθ˜n − hkΘ(E) = 0.
Not surprisingly, the proof differs only slightly from that of Theorem 8.4.4. In proving the W (E) -almost sure convergence of {θ˜n : n ≥ 1} to BH 1 (H) (0, 1), there are two new ingredients here. The first is the use of the Brownian scaling invariance property (cf. Exercise 8.6.8), which says that the W (E) is invariant 1 under the scaling maps Sα : Θ(E) −→ Θ(E) given by Sα θ = α− 2 θ(α · ) for α > 0 and is easily proved as a consequence of the fact that these maps are isometric from H 1 (H) onto itself. The second new ingredient is the observation that, for any R > 0, r ∈ (0, 1], and θ ∈ Θ(E), kθ(r · ) − BH 1 (H) (0, R)kΘ(E) ≤ kθ − BH 1 (H) (0, R)kΘ(E) . To see this, let h ∈ BH 1 (H) (0, R) be given, and check that h(r · ) is again in BH (0, R) and that kθ(r · ) − h(r · )kΘ(E) ≤ kθ − hkΘ(E) .
364
8 Gaussian Measures on a Banach Space
Taking these into account and applying (8.4.2), one can now justify
W (E) m−1max m ˜ θn − BH 1 (H) (0, 1) Θ(E) ≥ δ β ≤n≤β !
m
β 2 θ(nβ −m · ) (E)
≥δ − BH 1 (H) (0, 1) =W max
Λn β m−1 ≤n≤β m Θ(E)
m−1 Λ δ [β ]
≥ m ≤ W (E) m−1max m θ β −m n · − BH 1 (H) 0,
m
β ≤n≤β β 2 Λ[β m−1 ] β2 Θ(E)
Λ[β m−1 ] δ
≥ m ≤ W (E) θ − BH 1 (H) 0,
m
β 2 Λ[β m−1 ] β2 Θ(E) m
1 B (0, 1) ≥ δ θ − = W (E) β 2 Λ−1 m−1 H (H) [β ] Θ(E) R2 [β m−1 ] (E) m−1 log(2) [β ] = Wβ m Λ−2 kθ − BH 1 (H) (0, 1)kΘ(E) ≥ δ ≤ exp − βm [β m−1 ]
for all β ∈ (1, 2), R < inf{khkH 1 (H) : khkΘ(E) ≥ δ}, and sufficiently large m ≥ 1. Armed with this information, one can simply repeat the argument given at the analogous place in the proof of Theorem 8.4.4. The proof that, W (E) -almost surely, θ˜n approaches every h ∈ C infinitely often also requires only minor modification. To begin, one remarks that if A ⊆ Θ(E) is relatively compact, then kθ(t)kE = 0. T →∞ θ∈A t∈[T / −1 ,T ] 1 + t lim sup
sup
Thus, since, by the preceding, for W (E) -almost every θ, the union of {θn : n ≥ 1} and BH 1 (H) (0, 1) is relatively compact in Θ(E), it suffices to prove that
θ˜n (t) − θ˜n (k −1 ) − h(t) − h(k −1 ) kE = 0 W (E) -almost surely lim sup 1+t n→∞ t∈[k−1 ,k]
for each h ∈ BH 1 (H) (0, 1) and k ≥ 2. Because, for a fixed k ≥ 2, the random variables θ˜k2m − θ˜k2m (k −1 ) [k −1 , k], m ≥ 1, are W (E) -independent random variables, we can use the Borel–Cantelli Lemma as in § 8.4.2 and thereby reduce the problem to showing that, if θˇkm (t) = θ˜km (t + k −1 ) − θ˜km (k −1 ), then ∞ X
W (E) kθˇk2m − hkΘ(E) ≤ δ = ∞
m=1
for each δ > 0, k ≥ 2, and h ∈ BH 1 (H) (0, 1). Finally, since W (E) km Λ−1 is the k2m W (E) distribution of θ θˇk2m , the rest of the argument is the same as the one given in § 8.4.2.
Exercises for § 8.6
365
Exercises for § 8.6 Exercise 8.6.8. Let H 1 (H), Θ(E), W (E) be as in Theorem 8.6.1. 1
(i) Given α > 0, define Sα : Θ(E) −→ Θ(E) so that Sα θ(t) = α− 2 θ(αt), t ∈ [0, ∞), and show that (Sα )∗ W (E) = W (E) . Again, this property is called Brownian scaling invariance. (ii) Define I : Θ(E) −→ C [0, ∞); E so the Iθ(0) = 0 and Iθ(t) = tθ(t−1 ) for t > 0. Show that I is an isometry from Θ(E) onto itself and that I H 1 (H) is an isometry on H onto itself. Finally, use this to prove the Brownian time inversion invariance property: I∗ W (E) = W (E) .
Exercise 8.6.9. Let H U (H) be the Hilbert space of absolutely continuous hU : R −→ H with the property that q khkH U (H) = kh˙ U k2L2 (R;H) + 14 khU k2L2 (R;H) < ∞,
and take ΘU (E) to be the Banach space of continuous θU : R −→ E satisfying U kθ U (t) . If F : Θ(E) −→ = 0 with norm kθU kΘU (E) = supt∈R log(e+|t|) lim|t|→∞ kθlog(t)k t t
C(R; E) is given by [F (θ)](t) = e− 2 θ(et ), show that F takes Θ(E) continuously into ΘU (E) and that H U (H), ΘU (E), U (E) is an abstract Wiener space when (E) (E) UR = F∗ W (E) . Of course, one should recognize the measure UR as the distribution of an E-valued, reversible, Ornstein–Uhlenbeck process.
Exercise 8.6.10. A particularly interesting case of the construction in Exercise 8.6.9 is when H = H 1 (RN ) and E = Θ(RN ). Working in that setting, define B : R × [0, ∞) × ΘU Θ(E) −→ RN by B (s, t), θ = [θ(s)](t), and show that, Θ(RN ) for each s ∈ R, B(s, t), F(s,t) , UR is an RN -valued Brownian motion when F(s,t) = σ {B(s, τ ) : τ ∈ [0, t]} . Next, for each t ∈ [0, ∞), show that the √ Θ(E) UR -distribution of θ B( · , t) is that of t times a reversible, RN -valued Ornstein–Uhlenbeck process.
Exercise 8.6.11. Continuing in the same setting as in the preceding, set σ 2 = (E) EW kθk2Θ(E) , and combine the result in Exercise 8.2.16 with Brownian scaling invariance to show that ! R2 (E) , W sup kθ(t)kE ≥ R ≤ K exp − 72σ 2 t τ ∈[0,t]
where K is the constant in Fernique’s Theorem. Next, use this together with Theorem 8.4.4 and the reasoning in Exercise 4.3.16 to show that
kθ(t)kE kθ(t)kE = L = lim q lim q t&0 2t log(2) t 2t log(2) where L = sup khkE : h ∈ BH (0, 1) . t→∞
W (E) -almost surely, 1 t
366
8 Gaussian Measures on a Banach Space
Exercise 8.6.12. It should be recognized that Theorem 8.4.4 is an immediate corollary of Theorem 8.6.7. To see this, check that {θ(n) : n ≥ 1} has the same distribution under W (E) as {Sn : n ≥ 1} has under W N and that BH (0, 1) = {h(1) : h ∈ BH 1 (H) }, and use these to show that Theorem 8.4.4 follows from Theorem 8.6.7.
Exercise 8.6.13. For θ ∈ Θ(E) and n ∈ Z+ , define θ˘n ∈ Θ(E) so that θ˘n (t) =
s
n θ log(2) (n ∨ 3)
t , n
t ∈ [0, ∞),
and show that, W (E) -almost surely, {θ˘n : n ≥ 1} is relatively compact in Θ(E) and that BH 1 (H) (0, 1) is the set of its limit points.
Hint: Referring to (ii) in Exercise 8.6.8, show that it suffices to prove these properties for the sequence {(Iθ)˘n : n ≥ 1}. Next check that
(Iθ)˘n − Ih = ˜ θn − h Θ(E) Θ(E)
for h ∈ H 1 (H),
and use Theorem 8.6.7 and the fact that I is an isometry of H 1 (H) onto itself.
Chapter 9 Convergence of Measures on a Polish Space
In Chapters 2 and 3, I introduced a notion of convergence on M1 (RN ) that is appropriate when discussing either Central Limit phenomena or the sort of limits that arose in connection with infinitely divisible laws. In this chapter, I will give a systematic treatment of this sort of convergence and show how it extends to probability measures on any Polish space, that is, any complete, separable, metric space. Unfortunately, this extension will entail an excursion into territory that borders on abstract nonsense, although I hope to avoid crossing that border. In any case, just as Banach’s great achievement was the ingenious use for infinite dimensional vector spaces of completeness to replace local compactness, so here we will have to learn how to substitute compactness by completeness in measure theoretic arguments. § 9.1 Prohorov–Varadarajan Theory The goal in this section is to generalize results like Lemma 2.1.7 and Theorem 3.1.1 to a very abstract setting. § 9.1.1. Some Background. When discussing the convergence of probability measures on a measurable space (E, B), one always has at least two senses in which the convergence may take place, and (depending on additional structure that the space may possess) one may have more. To be more precise, let B(E; R) ≡ B (E, B); R be the space of bounded, R-valued, B-measurable functions on E, use M1 (E) ≡ M1 (E, B) to denote the space of all probability measures on (E, B), and define the duality relation Z hϕ, µi = ϕ dµ for ϕ ∈ B(E; R) and µ ∈ M1 (E). E
Next, again use kϕku ≡ supx∈E |ϕ(x)| to denote the uniform norm of ϕ ∈ B(E; R), and consider the neighborhood basis at µ ∈ M1 (E) determined by the sets U (µ, r) = ν ∈ M1 (E) : hϕ, νi − hϕ, µi < r for ϕ ∈ B(E, R) with kϕku ≤ 1 as r runs over (0, ∞). For obvious reasons, the topology defined by these neighborhoods U is called the uniform topology on M1 (E). In order to develop some feeling for the uniform topology, I will begin by examining a few of its elementary properties. 367
368
9 Convergence of Measures on a Polish Space
Lemma 9.1.1. M1 (E) by
Define the variation distance between elements µ and ν of
n o kν − µkvar = sup hϕ, µi − hϕ, νi : ϕ ∈ B(E; R) with kϕku ≤ 1 . Then (µ, ν) ∈ M1 (E)2 7−→ kµ − νkvar is a metric on M1 (E) that is compatible with the uniform topology. Moreover, if µ, ν ∈ M1 (E) are two elements of M1 (E) and λ is any element of M1 (E) with respect to which both µ and ν are absolutely continuous (e.g., µ+ν 2 ), then
(9.1.2)
kµ − νkvar = kg − f kL1 (λ;R) ,
where f =
∂ν dµ . and g = ∂λ dλ
In particular, kµ − νkvar ≤ 2, and equality holds precisely when ν ⊥ µ (i.e., they are singular). Finally, the metric (µ, ν) ∈ M1 (E)2 7−→ kµ − νkvar is complete. Proof: The first assertion needing comment is the one in (9.1.2). But, for every ϕ ∈ B(E; R) with kϕku ≤ 1, Z hϕ, νi − hϕ, µi = ϕ(g − f ) dλ ≤ kg − f kL1 (λ;R) , E
and equality holds when ϕ = sgn ◦ (g − f ). To prove the assertion that follows (9.1.2), note that kg − f kL1 (λ;R) ≤ kf kL1 (λ;R) + kgkL1 (λ;R) = 2 and that the inequality is strict if and only if f g > 0 on a set of strictly positive λ-measure or, equivalently, if and only if µ 6⊥ ν. Thus, all that remains is to check the completeness assertion. To this end, let {µn : n ≥ 1} ⊆ M1 (E) satisfying lim sup kµn − µm kvar = 0 m→∞ n≥m
P∞ be given, and set λ = n=1 2−n µn . Clearly, λ is an element of M1 (E) with n respect to which each µn is absolutely continuous. Moreover, if fn = dµ dλ , then, 1 by (9.1.2), {fn : n ≥ 1} is a Cauchy convergent sequence in L (λ; R). Hence, since L1 (λ; R) is complete, there is an f ∈ L1 (λ; R) to which the fn ’s converge in L1 (λ; R). Obviously, we may choose f to be non-negative, and certainly it has λ-integral 1. Thus, the measure µ given by dµ = f dλ is an element of M1 (E), and, by (9.1.2), kµn − µkvar −→ 0. As a consequence of Lemma 9.1.1, we see that the uniform topology on M1 (E) admits a complete metric and that convergence in this topology is intimately related to L1 -convergence in the L1 -space of an appropriate element of M1 (E).
§ 9.1 Prohorov–Varadarajan Theory
369
In fact, M1 (E) looks in the uniform topology like a galaxy that is broken into many constellations, each constellation consisting of measures that are all absolutely continuous with respect to some fixed measure. In particular, there will usually be too many constellations for M1 (E) in the uniform topology to be separable. To wit, if E is uncountable and {x} ∈ B for every x ∈ E, then the point masses δx , x ∈ E, (i.e., δx (Γ) = 1Γ (x)) form an uncountable subset of M1 (E) and kδy − δx kvar = 2 for y 6= x. Hence, in this case, M1 (E) cannot be covered by a countable collection of open k · kvar -balls of radius 1. As I said at the beginning of this section, the uniform topology is not the only one available. Indeed, for many purposes and, in particular, for probability theory, it is too rigid a topology to be useful. For this reason, it is often convenient to consider a more lenient topology on M1 (E). The first one that comes to mind is the one that results from eliminating the uniformity in the uniform topology. That is, given a µ ∈ M1 (E), define o n (9.1.3) S µ, δ; ϕ1 , . . . , ϕn ≡ ν ∈ M1 (E) : max hϕk , νi − hϕk , µi < δ 1≤k≤n
for n ∈ Z+ , ϕ1 , . . . , ϕn ∈ B(E; R), and δ > 0. Clearly these sets S determine a Hausdorff topology on M1 (E) in which the net {µα : α ∈ A} converges to µ if and only if limα hϕ, µα i = hϕ, µi for every ϕ ∈ B(E; R). For historical reasons, in spite of the fact that it is obviously weaker than the uniform topology, this topology on M1 (E) is sometimes called the strong topology, although, in some of the statistics literature, it is also known as the τ -topology. A good understanding of the relationship between the strong and uniform topologies is most easily gained through functional analytic considerations that will not be particularly important for what follows. Nonetheless, it will be useful to recognize that, except in very special circumstances, the strong topology is strictly weaker than the uniform topology. For example, take E = [0, 1] withits Borel field, and consider the probability measures µn (dt) = 1 + sin(2nπt) dt for n ∈ Z+ . Noting that, since | sin(2nπt) − sin(2mπt)| ≤ 2 and therefore Z 1 | sin(2nπt) − sin(2mπt)| 1 dt 2 kµn − µm kvar = 2 0 Z 2 1 1 1 sin(2nπt) − sin(2mπt) dt = ≥ 4 4 0
for m 6= n, one sees that {µn : n ≥ 1} not only fails to converge in the uniform topology, it does 1not even have any limit points as n →2 ∞. On the other hand, because 2 2 sin(2nπt) : n ≥ 1 is orthonormal in L λ[0,1] ; R , Bessel’s Inequality says that !2 Z ∞ X 2 ϕ(t) sin(2nπt) dt ≤ kϕk2L2 (λ[0,1] ) ≤ kϕk2u < ∞ n=1
[0,1]
370
9 Convergence of Measures on a Polish Space
and therefore hϕ, µn i −→ hϕ, λ[0,1] i for every ϕ ∈ B [0, 1]; R . In other words, {µn : n ≥ 1} converges to λ[0,1] in the strong topology, but it converges to nothing at all in the uniform topology. § 9.1.2. The Weak Topology. Although the strong topology is weaker than the uniform and can be effectively used in various applications, it is still not weak enough for most probabilistic applications. Indeed, even when E possesses a good topological structure and B = BE is the Borel field over E, the strong topology on M1 (E) shows no respect for the topology on E. For example, suppose that E is a metric space and, for each x ∈ E, consider the point mass δx on BE . Then, no matter how close x ∈ E \ {x} gets to y in the sense of the topology on E, δx is not getting close to δy in the strong topology on M1 (E). More generally (cf. Exercise 9.1.15), measures cannot be close in the strong topology unless their sets of small measure are essentially the same. Thus, for example, the convergence that is occurring in The Central Limit Theorem (cf. Theorem 2.1.8) cannot, in general, be taking place in the strong topology; and since The Central Limit Theorem is an archetypal example of the sort of convergence result at which probabilists look, it is only sensible for us to take a hint from the result that we got there. Thus, let E be a metric space, set B = BE , and consider the neighborhood basis at µ ∈ M1 (E) given by the sets S(µ, δ; ϕ1 , . . . , ϕn ) in (9.1.3) when the ϕk ’s are restricted to be elements of Cb (E; R). The topology that results is much weaker than the strong topology, and is therefore justifiably called the weak topology on M1 (E). (The reader who is familiar with the language of functional analysis will, with considerable justice, complain about this terminology. Indeed, if one thinks of Cb (E; R) as a Banach space and of M1 (E) as a subspace of its dual space Cb (E; R)∗ , then the topology that I am calling the weak topology is what a functional analyst would call the weak∗ topology. However, because it is the most commonly accepted choice of probabilists, I will continue to use the term weak instead of the more correct term weak∗ .) In particular, the weak topology respects the topology on E: δy tends to δx in the weak topology on M1 (E) if and only if y −→ x in E. Lemma 2.3.3 provides further evidence that the weak topology is well adapted to the sort of analysis encountered in probability theory, since,