High Dimensional Nonlinear Diffusion Stochastic Processes

Series on Advances in Mathematics for Applied Sciences - Vo!. 56

MBH-BiMENSMNAL NONUNEAR MFFUSMN STGCHASHC PROCESSES Modeling for Engineering Appiications

Yevgeny Mamontov Magnus WiHander

Worid Scientific

H!GH-0!MENSiONAL NONUNEAR OiFFMStON STOCHASTiC PROCESSES Modeling for Engineering Appiications

Series on Advances in Mathematics for Apptied Sciences - Vo). 56

H!GH-0!MENSiONAL NONUNEAR R!FFUS!0N STOCHASHC PROCESSES ModeMing for Engineering Appiications

Yevgeny Mamontov Magnus Wiiiander Department of Physics, M C 2 Gothenburg University and Chaimers University of Technoiogy Sweden

A W o r i d Scientific In

S/ngapore * MewJersey * London * Hong Kong

PtiMisneo' oy World Scientific Publishing Co. Pte. Ltd. P O Box 128, Farrer Road, Singapore 912805 M R S ofice.' Suite IB, 1060 Main Street, River Edge, NJ 07661 M f oj9!ce.' 57 Shelton Street, Covent Garden, London W C 2 H 9 H E

Library of Congress Cataloging-in-Publication Data Mamontov, Yevgeny, 1955High-dimensional nonlinear diffusion stochastic processes : modelling for engineering applications / Yevgeny Mamontov and Magnus Willander. p. cm. — (Series on advances in mathematics for applied sciences ; vol. 56) Includes bibliographical references and index. ISBN 9810243855 (alk. paper) 1. Engineering — Mathematical models. 2. Stochastic processes. 3. Diffusion processes. 4. Differential equations, Nonlinear. 1. Willander, M . II. Title. III. Series. T A 3 4 2 .M35 2001 620'.001'5118-dc21

00-053437

British Library Catatoguing-in-Publication Data A catalogue record for this book is available from the British Library.

Copyright O 2001 by World Scientific Publishing Co. Pte. Ltd. A H n'gAfj rej^rveo'. 77H.S ocoA, or parM fA^r^p^ may woy ^ reproa'Mcea' ;'n any/br/n or &y any meanj, e/^cfroni'c or mecAafHca/, :'nc/M^;ngpAoMcopy;'ng, recording or any ;'n/ormafi'on yforage an^ r^fri'eva/ jy^em now Anown or fo oe /nvenMa', w^Aouy tvr/fMn perm/Mion/rom (ne PM^/i'yn^r.

For photocopying of material in this volume, please pay a copying fee through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, M A 01923, U S A . In this case permission to photocopy is not required from the publisher.

Printed in Singapore by World Scientific Printers

To oMr /a??M^es

Preface

Almost every book concerning diffusion stochastic processes (DSPs) is devoted to the problems arising wstde stochastic theory in the course of its logical development. The corresponding results serve inherent needs of m a thematics. This w a y of research is related to what is usually called pure mathematics. Those of its achievements which w prmctpJe allow quantitative treatments can also be applied to the stochastic problems arising oM^stde pure mathematics. Such applications always assume practical implementation. They are addressed to scientists or engineers, both mathematicians and nonmathematicians, w h o are not necessarily specialists in stochastic theory. In so doing, the forms of the recipes granted by pure mathematics are not discussed since the main attention is paid to the corresponding numerical techniques. However, the advantage to allow quantitative analysis in principle does not always m e a n to allow it in reality. In reality one has to deal with: (1) people w h o work in dissimilar sciences or branches of engineering and perhaps do not consider pure mathematics as the main field of their activities; some of the above recipes m a y be recognized by these people as not very transparent; (2) tools such as computer systems or software environments that, alas, are not of unlimited capabilities; indeed, great efforts are in

Pre/bee

VH1

m a n y cases m a d e to cram the above recipes (not always aimed at any practical applications) into tight frames of the existing tools. These circumstances constitute the problems which are out of a customary stream of pure mathematics and, in view of this, form a n e w and very stimulating field of research. The present book is intended (see Section 1.13) to facilitate solution of these two problems in the field of nonlinear diffusion stochastic processes (DSPs) of a large number of variables. More precisely, the book considers D S P s in Euclidean space of high dimension (much greater than a few units) with the coefficients nonlinearly dependent on the space coordinates. M a n y complex stochastic systems in science and engineering lead to such DSPs. A s is well-known, D S P s are closely related to Ito's stochastic differential equations (ISDEs). This fact explains w h y D S P theory is usually interpreted from the viewpoint of these eqautions. However, such treatment is not only unnecessary but also presumes reader's expertise in ISDEs. To eliminate this complication means to contribute to solution of the above problem (1). Accordingly, the present book derives (see Chapters 2-4) analysis of high-dimensional D S P s only from the well-known expressions for drift vector and diffusion matrix of the processes and from Kolmogorov's forward equations. N o techniques directly related to I S D E theory are involved. The corresponding prerequisites for reading (Section 1.1) are merely awareness of some basic facts associated with probability theory. The outcome of the purely D S P treatment is two-fold: *

compared with the ISDE-based alternative, the developed treatment is simpler, compact and uniform; it can be more willingly accepted by nonspecialists in stochastic integral and differential calculuses, in particular, by nonmathematicians;

*

the derived representations have not been revealed in stochastic theory yet and are also of purely mathematical interest; they provide a deeper insight into behavior of the basic characteristics of D S P s that can contribute to a greater success of the subsequent analyses.

Regarding the above problem (2), i.e. to better adapt the DSP-theory results to limited power of majority of computing systems, the present book concentrates on the analytical part of a combined, analytical-numerical approach. The main advantage of this approach is that it presents a reasonable compromise between accuracy of the applied models and complexity of

Pre/ace

IX

their numerical analysis. The above analytical part is constructed (Chapters 2-4) with the help of the purely D S P treatment. It grants the benefit to derive such analytical approximations which allow for the nonlinearities of the D S P coefficients and, in conjunction with proper, parallel simulation techniques, can lead to realistic computing expenses even if the process is of a high dimension. These features enable one to solve m u c h wider family of high-dimensional stochastic problems by m e a n s of numerical method and m a k e this analysis more accessible for scientific and engineering community. The present book concerns m a n y aspects to better elucidate the involved notions and techniques. In particular, it includes a separate chapter (Chapter 3) devoted to nonstationary invariant processes. This topic is of importance in both qualitative and quantitative analyses. Nevertheless, it is regrettably missed out in most of the works on applications of DSPs. Another special feature of the present book is the chapter (Chapter 5) on modelling non-Markov phenomena with various ISDEs and their connection to high-dimensional DSPs. This connection is provided by m e a n s of the well-known stochastic adaptive interpolation (SAI) method. The chapter equips this method with bases of function Banach spaces and shows h o w it can involve the proposed analytical-numerical treatment to approximately solve Ito's stochastic partial (integro-)differential equations. The specific examples concerning the above chapter are considered in Chapter 6. It is devoted to Ito's stochastic partial differential equations of semiconductor theory which are based on the fluid-dynamics model. The first part of the chapter emphasizes the capabilities of c o m m o n , nonrandom semiconductor equations to describe the electron (or hole) fluid not only in macroscale but also in mesoscale and microscale domains. This topic is considered in connection with a certain limit case of a proper random walk. The second part of the chapter discusses Ito's stochastic generalization of these equations for macroscale and weakly mesoscale domains and s u m m a rizes some of the corresponding results on noise in semiconductors. This also includes related topics for future development. Chapter 7 focuses on the distinguishing features of engineering applications. A combined, analytical-numerical approach, its advantages and disadvantages as well as the connection to c o m m o n analytical D S P results are discussed in Chapter 8. For reader's convenience, the book includes the introductory chapter (Chapter 1) that lists some basic facts and methods of D S P theory and some

X

Pre/ace

of the related practical problems. This chapter leads to the detailed formulation (Section 1.13) of the purpose of the book. The book includes m a n y appendixes. They present the proofs of the lemmata and theorems, descriptions of the examples and the technical details which are needed in the course of the consideration, can contribute to better understand some stochastic features of the nonrandom models or can be helpful in practical implementation of the described methods. The theorems are numbered with the two-position numbers in the form (K.L) where L is the cardinal number of the theorem in Chapter K. The definitions, lemmata, corollaries and remarks are numbered similarly. The formulas in the book are numbered with the two- or three-position numbers in the forms (K.L) or (K.M.L) respectively. In so doing, position L is the cardinal number of the formula in Chapter K or, if the chapter is divided into sections, in Section K.M, i.e. the M t h section of Chapter K. The end of Index includes the list of all the definitions, lemmata, theorems, corollaries and remarks. The application field which the book aims the developed models and m e thods at can include different subjects but can be formulated very briefly. This field is complex systems, no matter which specific science or engineering they are associated with. The book considers various aspects related to applied research. For instance, it: * * *

* *

stresses the mathematical reading and practical meaning of the signal-to-noise ratio (Appendix B); explains h o w a common, nonrandom equation for fluid concentration includes not very simple stochastic phenomena (Appendix F); presents a /M^Jy ?:77te-c?07Mam, approximate analytical model to evaluate the particle-velocity covariance for a fairly general class of fluids (Section 4.10 and Appendix C); shows h o w to describe the long, non-exponential "tails" of this covariance by means of the I S D E with a fMwJwear friction (Section 4.10.3); derives the nonrandom model which suggests an explanation of stochastic resonance and other s^ocAas^cs-wc^Mced effects in the m e a n responses of the systems governed by nonJmear D S P s (Section 2.4).

These examples demonstrate not only the capabilities of the developed techniques but also emphasize the usefulness of the complex-system-related approaches to solve some problems which have not been solved with the traditional, statistical-physics methods yet. F r o m this veiwpoint, the book

Pre/ace

xi

can be regarded as a kind of complement to such books as "ZMfrodMcf:o7t fo fAe PAys:cs o/Complex Sys^ews. TAe Afesoseoptc ApproacA ^o F^MC^Mot^ows, NonJmear^y a%d <Sey-Orga7Mzafton" (Serra, Andretta, Compiani and Zanarini; 1986), "S^ocAas^:c Dy^awtcctZ -Sys^ews. Con.cep^s, NMwertcaJ Afe^Aoc^s, Da^a AMaJysts" (Honerkamp, 1994), "S^a^s^caJ PAysMS.' A n Atft^anced Approach w ^ A App^ca?t07ts" (Honerkamp, 1998) which deal with physics of complex systems, some of the corresponding analysis methods and an innovative, stochastics-based vision of theoretical physics. W h o should read this book? W e believe that any reader whose work or study concerns nonlinear high-dimensional stochastic systems will find in it something that could really help and m a k e the painstaking but interesting job more enjoyable and fruitful. To be specific in this issue, w e would point out the following groups of readers: * * * * *

nonmathematicians (e.g.,theoretical physicists, engineers in industry, specialists in models for finance or biology, computing scientists); mathematicians; undergraduate and postgraduate students of the corresponding specialties; managers in applied sciences and engineering dealing with the advancements in the related fields; any specialists w h o use diffusion stochastic processes to model highdimensional (or large-scale) nonlinear stochastic systems.

and other communities w h o are interested in the topic of the book. To what extent is this book suitable and useful for you personally? The best w a y to get the answer is to look through the "Contents" and Section 1.1 "Prerequisites for Reading". They present the information that helps you to m a k e your decision.

Contents

Preface

vii

C h a p t e r 1 Introductory Chapter 1.1 Prerequisites for Reading 1.2 R a n d o m Variable. Stochastic Process. R a n d o m Field. High-Dimensional Process. One-Point Process 1.3 Two-Point Process. Expectation. Markov Process. Example of Non-Markov Process Associated with Multidimensional Markov Process 1.4 Preceding, Subsequent and Transition Probability Densities. The Chapman-Kolmogorov Equation. Initial Condition for Markov Process 1.4.1 The Chapman-Kolmogorov equation 1.4.2 Initial condition for Markov process 1.5 Homogeneous Markov Process. Example of Markov Process: The Wiener Process 1.6 Expectation, Variance and Standard Deviations of Markov Process 1.7 Invariant and Stationary Markov Processes. Covariance. Spectral Densities 1.8 Diffusion Process

xiii

1 1 3

10

16 19 20 23 26 30 37

xiv

1.9 1.10 1.11 1.12

1.13

Con^enfs

Example of Diffusion Processes: Solutions of Ito's Stochastic Ordinary Differential Equation The Kolmogorov Backward Equation Figures of Merit. Diffusion Modelling of High-Dimensional Systems C o m m o n Analytical Techniques to Determine Probability Densities of Diffusion Processes. The Kolmogorov Forward Equation 1.12.1 Probability density 1.12.2 Invariant probability density 1.12.3 Stationary probability density The Purpose and Content of This Book

Chapter 2 Diffusion Processes 2.1 Introduction 2.2 Time-Derivatives of Expectation and Variance 2.3 Ordinary Differential Equation Systems for Expectation 2.3.1 The first-order system 2.3.2 The second-order system 2.3.3 Systems of the higher orders 2.4 Models for Noise-Induced Phenomena in Expectation 2.4.1 The case of stochastic resonance 2.4.2 Practically efficient implementation of the second-order system 2.5 Ordinary Differential Equation System for Variance 2.5.1 Damping matrix 2.5.2 The uncorrelated-matrixes approximation 2.5.3 Nonlinearity of the drift function 2.5.4 Fundamental limitation of the state-spaceindependent approximations for the diffusion 2.6

and damping matrixes The Steady-State Approximation for The Probability Density

Chapter 3 Invariant Diffusion Processes 3.1 Introduction 3.2 Preliminary Remarks 3.3 Expectation. The Finite-Equation Method

40 46 48

51 51 54 57 60

. .

63 63 64 66 66 68 70 71 71 73 76 76 77 80

81 82 85 85 85 86

Con^enfs

3.4 3.5

3.6

3.7

Explicit Expression for Variance The Simplified Detailed-Balance Approximation for Invariant Probability Density 3.5.1 Partial differential equation for logarithm of the density 3.5.2 Truncated equation for the logarithm and the detailed-balance equation 3.5.3 Case of the detailed balance 3.5.4 The detailed-balance approximation 3.5.5 The simplified detailed-balance approximation. Theorem on the approximating density Analytical-Numerical Approach to Non-Invariant and Invariant Diffusion Processes 3.6.1 Choice of the bounded domain of the integration 3.6.2 Evaluation of the multifold integrals. The Monte Carlo technique 3.6.3 Summary of the approach Discussion

Chapter 4 Stationary Diffusion Processes 4.1 Introduction 4.2 Previous Results Related to Covariance and Spectral-Density Matrixes 4.3 Time-Separation Derivative of Covariance in the Limit Case of Zero Time Separation 4.4 Flicker Effect 4.5 Time-Separation Derivative of Covariance in the General Case 4.6 Case of the Uncorrelated Matrixes 4.7 Representations for Spectral Density in the Uncorrelated-Matrixes Case 4.8 Example: Comparison of the Dampings for a Particle Near the Minimum of Its Potential Energy 4.9 The Deterministic-Transition Approximation 4.10 Example: Non-exponential Covariance of Velocity of a Particle in a Fluid 4.10.1 Covariance in the general case

xv

88 90 90 91 93 95 96

..

99 100 102 104 105 107 107 108 109 Ill 112 114 117 118 123 126 126

xvi

Confers

4.10.2

4.11

4.12

Covariance and the quatities related to it in a simple fluid 4.10.3 Case of the hard-sphere fluid 4.10.4 Summary of the procedure in the general case . . Analytical-Numerical Approach to Stationary Process .... 4.11.1 Practical issues 4.11.2 Summary of the approach Discussion

128 130 133 134 134 136 137

Chapter 5

5.1 5.2 5.3

5.4 5.5

Ito's Stochastic Partial Differential 141 Equations as Non-Markov Models Leading to High-Dimensional Diffusion Processes Introduction 141 Various Types of Ito's Stochastic Differential Equations . . . 142 Method to Reduce ISPIDE to System of ISODEs 144 5.3.1 Projection approach 148 5.3.2 Stochastic collocation method 151 5.3.3 Stochastic-adaptive-interpolation method 153 Related Computational Issues 159 Discussion 160

Chapter 6

163

6.1 6.2

163

6.3

Ito's Stochastic Partial Differential Equations for Electron Fluids in Semiconductors Introduction Microscopic Phenomena in Macroscopic Models of Multiparticle Systems 6.2.1 Microscopic random walks in deterministic macroscopic models of multiparticle systems 6.2.2 Macroscale, mesoscale and microscale domains in terms of the wave-diffusion equation 6.2.3 Stochastic generalization of the deterministic macroscopic models of multiparticle systems The ISPDE System for Electron Fluid in M-Type Semiconductor 6.3.1 Deterministic model for electron fluid in semiconductor

165 165 171 176 177 177

Con?en?s

6.3.2

6.4

Mesoscopic wave-diffusion equations in the deterministic fluid-dynamic model 6.3.3 Stochastic generalization of the deterministic fluid-dynamic model. The semiconductor-fluid ISPDE system Semiconductor Noises and the SF-ISPDE System: Discussion and Future Development 6.4.1 The SF-ISPDE system in connection with semiconductor noises 6.4.2 Some directions for future development

xvii

Chapter 7 7.1 7.2 7.3 7.4

Distinguishing Features of Engineering Applications

High-Dimensional Diffusion Processes Efficient Multiple Analysis Reasonable Amount of the Main Computer Memory Real-Time Signal Transformation by Diffusion Process

Chapter 8

8.1 8.2

Analytical-Numerical Approach to Engineering Problems and C o m m o n Analytical Techniques Analytical-Numerical Approach to Engineering Problems Severe Practical Limitations of C o m m o n Analytical Techniques in the High-Dimensional Case. Possible Alternatives

180

187 192 192 195 197 197 198 198 199 201

201

203

Appendix A

Example of Markov Processes: Solutions of the Cauchy Problems for Ordinary Differential Equation System

205

Appendix B

Signal-to-Noise Ratio

209

Appendix C

Example of Application of Corollary 1.2: Nonlinear Friction and Unbounded Stationary Probability Density of the Particle Velocity in Uniform Fluid Description of the Model

213

C.l

213

xviii

C.2 C.3 C.4

Co?:?enfs

Energy-Independent Momentum-Relaxation Time. Equilibrium Probability Density Energy-Dependent Momentum-Relaxation Time: General Case Energy-Dependent Momentum-Relaxation Time: Case of Simple Fluid

220 222 227

Appendix D

Proofs of the Theorems in Chapter 2 and Other Details Proof of Theorem 2.1 Proof of Theorem 2.2 Green's Formula for the Differential Operator of Kolmogorov's Backward Equation Proof of Theorem 2.3 Quasi-Neutral Equilibrium Point Proof of Theorem 2.4

231

D.l D.2 D.3

231 232 235 237 239 241

Appendix E Proofs of the Theorems in Chapter 4 E.l Proof of Lemma 4.1 E.2 Proof of Theorem 4.2 E.3 Proof of Theorem 4.3 E.4 Proof of Theorem 4.4

243 243 243 244 245

Appendix F

Hidden Randomness in Nonrandom Equation for the Particle Concentration of Uniform Fluid and Chemical-Reaction /Generation-Recombination Noise

247

Appendix G

Example: Eigenvalues and Eigenfunctions of the Linear Differential Operator Associated with a Bounded Domain in Three-Dimensional Space

255

Appendix H

Resources for Engineering Parallel Computing under Windows 95

261

D.4 D.5 D.6

Bibliography

265

Index

281

Chapter 1

Introductory Chapter

1.1

Prerequisites for R e a d i n g

This book is addressed to all scientists and engineers w h o would like to be well-equipped for the case if they need to deal with diffusion stochastic processes in practice. This community comprises not only mathematicians but also a great number of nonmathematicians acting in the natural and social sciences and technologies. Therefore, the prerequisites for reading this book include only such elements of mathematics and probability theory which are commonly used in applications, more specifically: ^-dimensional Euclidean space R^; vectors and matrixes; ordinary and partial differential equations; the Lebesgue integrals; elementary event, space of elementary events, random event; probability of random event; stochastically independent random events; finite-dimensional random variable; stochastically independent random variables; probability density of random variable; conditional and marginal probability densities; expectation, variance, covariance of random variables. This list does not include purely theoretical topics which are not often (if at all) involved in solving applied problems. For instance, the book does 1

2

7n&*of%MC?ory C7:apfer

not involve the measure-theoretical treatment of probability theory and stochastic processes. This treatment is not very familiar to nonmathematicians. In spite of that, it is applied in almost every book to stochastic processes and stochastic differential equations. This fact is perhaps the reason that the mentioned field is still separated not only from m a n y nonmathematical applications but also from some important formalisms, e.g. classical theoretical physics and statistical mechanics (e.g., Balescu, 1975; Klimontovich, 1982; Resibois and D e Leener, 1977). Indeed, engineers or statisticians willing to acquire more knowledge in stochastic processes do have to squeeze through the measure-theoretical presentation not very transparent to them. Understandably, this road in m a n y cases remains unpassed. Another curious phenomenon is that statistical mechanics had in its own, nonmathematical w a y to obtain some results which were rigorously derived in mathematics long before and are well-known in stochastic theory. Resibois and D e Leener (1977) includes m a n y examples of this kind. The present book not involving the measure-theoretical treatment describes stochastic processes in terms of multidimensional random variables which are familiar to people working in applications. To the authors' knowledge, such approach has never been attempted before. Nevertheless, the authors hope that both nonmathematicians and mathematicians will not feel too m u c h inconvenience with it. The readers interested in the details of the rigorous, measure-theoretical presentation of probability theory and stochastic processes are referred to m a n y books on the subject, for instance: Arnold (1974), Bellomo e? a/. (1992), Feller (1968, 1971), Friedman (1975, 1976), Gard (1988), G i k h m a n and Skorokhod (1969), G i h m a n and Skorohod (1972), Gnedenko (1982), Has'minskh (1980), Kloeden and Platen (1995), Kolmogorov (1956), Ochi (1990), Prohorov and Rozanov (1969), Skorokhod (1965), W a n g and Hajek (1985). Another feature of the present book following from the above prerequisites is that it describes random variables by means of the corresponding probability densities (or, more precisely, the densities of the probability distributions of random variables). These densities exist not for every random variable. Luckily, the two following facts are in favor of the density-based approach. Ftrs?, the basic models in both classical statistical mechanics (e.g., Balescu, 1975; Klimontovich, 1982; Resibois and D e Leener, 1977) and mathematical kinetic theory (e.g., Bellomo, Palczewski and Toscani, 1988; Bellomo e? aL, 1991; Bellomo, 1995; Bellomo and Lo Schiavo, 2000) are inherently

R a n d o m VartaoZe, Process and F:eM. HtgA-D:mens:o?:a^ Process

3

associated with the probability densities. The well-known multidecade experience in application of these models (discussed, for example, by Bellomo, 1995; Bellomo and Lo Schiavo, 1997) demonstrates that the density-based treatments are sound and productive, both theoretically and practically. -Second, probability distributions present probabilities of random events as functions of the so-called Borel sets in Euclidean space. This feature is, however, impractical to describe random variables in the computing-oriented algorithms. Other related quantities should be used for this purpose. The most practically reasonable and sharp w a y is to apply the corresponding probability densities. They present not only the basic tool for the theoretical modeling (see above) but also the focus of practical statistical data processing (e.g., Cramer, 1946) in various fields of science and engineering. Potential readers of the present book include the following groups: *

*

*

*

mathematicians w h o wish to be aware of the n e w analytical results for high-dimensional diffusion stochastic processes with nonlinear coefficients; nonmathematicians (e.g., theoretical physicists, engineers in industry, specialists in models for finance or biology) w h o apply diffusion stochastic process theory to model complex, high-dimensional stochastic systems; managers in applied sciences and engineering w h o are responsible for the advanced level of the system-modelling/product-design activities of the technical personnel headed by them; undergraduate and postgraduate students of the corresponding specialties.

Such books as Papoulis (1991) or Yates and G o o d m a n (1999) can help the readers of the second, third and fourth groups to recollect the probability-related prerequisites listed at the very beginning of this section. It is hoped that the present book will be a useful and not very difficult reading for a wide audience.

1.2

R a n d o m Variable. Stochastic Process. R a n d o m Field. High-Dimensional Process. One-Point Process

Chapters 1-4 include a more systematic and detailed presentation of the results by Mamontov, Willander and Lewin (1999), M a m o n t o v and Willan-

4

7?:&*odMCfory CAapfer

der (1999) (and correct a few misprints occurred therein). S o m e applications of these results are described and discussed in Chapters 5—8. To begin with, w e consider notion of random variable. Let 6f be an integer positive number and R = (-oo,oo). R a n d o m variable in 6?-dimensional Euclidean space R*^ is real function % of elementary event ^ . This m e a n s that

* = X(Q

(1.2.1)

= (^,...,*JT(E]

(1.2.2)

where

^

^7

f Xi(Q = (X^),...,x,,(QfeR',

X(Q =

foralHeE,

(1.2.3)

x.(Qj ^ and x^, ^ = 1,...,^, are entries of the corresponding vectors, E is the space of elementary events. Every elementary event ^ can be interpreted as a specific outcome in a series of the trials in the experiment associated with random variable x Let g , be the sigma-algebra of Borel sets in R'' generated by the ^-dimensional rectangular parallelepipeds in R^ (e.g., Section 1.1 of Arnold, 1974). Non-Borel sets in R'' exist. However, it is not easy to exhibit specific examples of them. The necessary and sufficient conditions for a set to be a Borel one are k n o w n for the so-called analytic sets. They are formulated in two different forms, namely, as Suslin's criterion and Luzin's criterion. However, such criteria are a topic of set theory and are beyond the present book. In practical applications, any set in g^ can be viewed as a countable (i.e. either finite or infinitely countable) union of mutually non-intersecting ^-dimensional rectangular parallelepipeds in R^. (This interpretation follows the idea of Luzin's criterion (e.g., Luzin, 1972).) Overwhelming majority of the sets in R^ (intervals of the n u m b e r axis, plane figures, volume bodies, and m a n y others) employed in the real-world problems belongs to family g^. Note that g . by definition contains both empty set 0 and space R^, i.e.

Random Var:aMe, Process otnoJ f:eM. #:g/:-D:meHs:ofM/ Process

5

8 ^ 0 and 8
(1.2.4)

In particular, ^(0) = 0 and F ( E ) = 1. Note, however, that a random event of zero probability need not be 0. O n e shall tell that proposition regarding elementary events ^ holds almost surely ("almost certainly" or "with probability 1") if and only if it is true for every ^ E E with the possible exception of some ^ which constitute a random event of zero probability. For brevity, any proposition regarding elementary events ^ which is valid is in what follows assumed to be valid almost surely if the other is not stated. Quantity (1.2.4), as a function of set E E g ^ at fixed %; completely describes random variable x and is called probability distribution of x. This distribution is defined on g^. Probability distribution of x is called absolutely continuous if and only if, for any 2 E g^ such that the Lebesgue integral J^ 0,

forallxER',

fp.(Ar)^=l,

P{^eE:x($)6B}=J*P,.(*)^, E

(1.2.5) (1.2.6)

forallEE^,

(1.2.7)

6

/KfroGfHctory Chapter

is called density of the probability distribution of random variable % or, briefly, probability density of X- It is well-known (e.g., pp. 9 and 215 in W o n g and Hajek, 1985) that if the probability distribution is absolutely continuous, then density p. is Lebesgue integrable. In this case, random variable % is sometimes called continuous. Note that probability-density function need not be continuous or bounded. A n example of the applied problem where J = 3 and the probability density tends to infinity as ] x ], norm of vector x, tends to zero is presented in Appendix C. M o r e generally, probability density can be a generalized function defined on R^. T h e theory of generalized functions is well-developed (e.g., Gel'fand and Shilov, 1964,1968,1967; Gel'fand and Vilenkin, 1964; Gel'fand, Graev and Vilenkin, 1966) and extensively applied (e.g., Vladimirov, 1984). O n e of the most widely used generalized functions is the ^ -variable Dirac deltafunction 8 determined with equation
8(x) = n s ( * , )

(1-2.8)

where 8 the single-variable (J = l) Dirac delta-function. Function 8 is defined as the w e a k limit of the function sequence (e.g., Section 5.1 of Vladimirov, 1984)). This property in particular underlies approximations for the delta-function. For instance, the well-known approximation (e.g., pp. 35-36 of Gel'fand and Shilov (1964)) is n^]gl/(g^+jc^) for all ^ g S R such that g # 0. This function of scalars jc at fixed g and its derivatives of all orders approximate 8 and its derivatives of the same orders respectively and, besides, coincide with the approximated quantities in the limit case as g-*0. Generalized functions have a lot of interesting and useful properties (e.g., Chapter II of Vladimirov, 1984). To mention a few, w e point out the following: * *

*

the space of generalized functions defined on R*^ is a function Banach space (e.g., Section 5.4 of Vladimirov, 1984); a linear combination of generalized functions defined on R^ with infinitely differentiable on R^ coefficients is a generalized function defined on R*' (e.g., Section 5.10 of Vladimirov, 1984); any generalized function defined on R^ is infinitely differentiable and the result of the differentiation is independent of the order of the differentiation (e.g., p. 95 in Vladimirov, 1984);

R a n d o m Var:'aMe, Process and P:eM. R^/t-Dtmensiona/ Process

*

7

the convolution of any generalized function G defined on R'' with delta-function 8 is equal to C (e.g., p. 123 in Vladimirov, 1984).

T h e above interpretation of probability densities as generalized functions provides a unified treatment of the probability-distribution families which include not only absolutely continuous distributions but the ones of other types [e.g., see pp. 8-9 of Chapter 1 of W o n g and Hajek (1985) for the details]. For instance, delta-function 8 enables one to use probability densities (in particular, to describe probability (1.2.4) by means of familiar integral formula (1.2.7)) even if random variable % is discrete. W e consider this topic below. R a n d o m variable x is called discrete (e.g., p. 8 of W o n g and Hajek, 1985) if and only if there exists at most countable set .Y<=R^ of $-independent (i.e. nonrandom) vectors such that ^ P ( { ^ E E : x ( 6 W } ) = l.

(1.2.9)

The corresponding probability density p. can be determined with the help of 8 as follows p(jc)=^F({$€E:x(Q=^})8(^-^),

forall^SR^.

(1.2.10)

Substituting (1.2.10) into (1.2.7), one obtains equation P { ^ e E : x ( Q e E } = E-^ypn-F({$eE: x(Q=.x}) (that is the same as (3.7) in W o n g and Hajek, 1985). Note that if a random variable is discrete, then its probability distribution is singular. However, the inverse statement is in general not true. The above properties of generalized functions point out that application of delta-function 8 in the expressions analogous to (1.2.10) can extend the probability-density description to combined, continuous-discrete random variables. This m a y be a helpful option in m a n y applied problems. If set A' owns only one vector, say, x, i.e. -Y={x}, then x($)=x for all ^eE,Eq.(1.2.9)is simplified to P ( { ^ € E : x(^)=^}) = l and random variable X becomes nonrandom, deterministic. In this case, one has p(.x) = 8(x-.x),

forallxeR'',

(1.2.11)

instead of (1.2.10). R e m a r k 1.1 A n y deterministic vector in R^, say, the above vector .x can be considered as random varaible x with probability density (1.2.11).

8

/nfrodMcfory CAapfer

Quantiy p (x), as a function of entries x,, / = 1, ,<%, of vector x (see (1.2.2)), is called joint probability density of entries x?, ? = 1,...,^, of random variable x (see (1.2.3)). R a n d o m variables are called (mutually) stochastically independent if and only if their joint probability density presents the product of the probability densities which completely describe respective random variables. For instance, all the above entries are mutually stochastically independent if and only if p,(.x., ,x.) = n;^p.,(j);,) where probability density p., completely describes entry X; of random variable xFor the reasons discussed in Section 1.1, w e note the following. R e m a r k 1.2 The present book assumes that probability distribution of any random variable under consideration has the density which can be a c o m m o n or generalized function. In so doing, the integrals of the integrands which involve the densities are understood in an appropriate sense. R a n d o m variables m a y depend on scalar or vector parameters. R a n d o m variable x dependent on one scalar parameter, say, time f is called stochastic process. For stochastic process x defined for all rE 7 where /cR is an open interval (i.e. coinciding with its interior, J=Int 7), quantity X ( Q in the above equation is replaced with x ( ^ ) - In particular, Eq. (1.2.1) denoting the values of function x at time point ; is written as * = X(^)-

(1-2.12)

Whereas for process x function x('^) at every fixed time point ; E / is a random variable, quantity x(^') &t every fixed elementary event ^ S E presents a function of f and is called sample function (or sample trajectory or sample path) of the process (corresponding to the chosen ^ ) . Note that random variables x(';f) at different values of f are generally stochastically dependent. Stochastic process x on / is called stochastic process with almost surely continuous sample functions if and only if P { ^ e E : x ( ^ ) e C ° ( 7 ) } = l. The details on this property can be found elsewhere (e.g., pp. 63-64 of Soize, 1994; Section 4 of Chapter 2 of W o n g and Hajek, 1985). In m a n y cases, the answer to the question if one or another process can be considered as the process with almost surely continuous sample functions is prompted by the nature of the specific phenomenon modelled with it. R a n d o m variable x dependent on two or more scalar parameters is called random field. For instance, random variable x dependent on both time f and vector z € R**, 1, in non-empty set QcR*^ is a random field.

Random VartaMe, Process and PteM. HtgA-DtmenstonaZ Process

9

(Note that dimension t? need not be equal to dimension ;?.) In applied problems, set Q is usually a domain, i.e. open in R^ and connected set, with piecewise smooth boundary 3Q which is formed by at most countable union of smooth (and open in R*^ if 2) "surfaces" in R^. For the random field, quantities % ( Q or x(^.?) in the above equations are replaced with x(S,f,z). In particular, Eq. (1.2.12) denoting the values of function % is written as *=X(M-

(1-2.13)

At every fixed (f,z) E /x Q where Q is closure of Q , i.e. Q = Q U 3 Q , function %(' ,f,z) is a random variable. At every fixed elementary event ^ e E , quantity x ( ^ v ) presents a function of (f,z) and is called sample function (or surface) of the random field (corresponding to the chosen ^ ). R a n d o m variables x(')f)Z) at different values of f orz are generally stochastically dependent. W e return to random fields in Chapters 5 and 6. For stochastic process %, Euclidean space R^ is the state space. Its dimension is tf and determines the number of entries of the process vector (see (1.2.3)). The higher the dimension is, the more difficult it is to analyze the process in practice. Process % is called multidimensional w h e n 6? > 2. Multidimensional process x is called high-dimensional (or large-scale) w h e n d is not too low, more specifically, number J is m u c h greater than a few units.

(1.2.14)

This feature is discussed in Section 7.1. In practice, high-dimensional stochastic processes can usually be treated only by means of the well-known purely numerical techniques (e.g., Kloeden and Platen, 1995). The present book describes and discusses a version of the analytical basis for the combined, analytical-numerical approach (see Section 8.1 for the discussion on it). There are m a n y different types and kinds of stochastic processes. The corresponding information is surveyed, for instance, by Kovalenko, Kuznetsov and Shurenkov (1996). A n important aspect in classification of stochastic processes is h o w complex the stochastic dependence between the random-variable values of the process at different time points is. In connection with this, w e first point out the most simple processes.

10

/nfrodHcfory CAap^er

Defintion 1.1 Stochastic process x on interval 7 is called one-point process if and only if it is completely described by the probability distribution of random variable x('0 at every fEJ. The term "one-point" in this definition is due to the fact that the probability distribution is associated with any one time point. Definition 1.1 in particular means that, for one-point process %, random variable x('^) at every fixed f E 7 is stochastically independent of random variable x( ^.) for all such f.E 7 that f.#f. It follows from Definition 1.1 and R e m a r k 1.2 that one-point process is completely described by the probability density of rand o m variable x( J) at every f E / , i.e. by the ; -dependent version of the above function p.. In so doing, Eqs. (1.2.5), (1.2.6) are written as p.(;,.x) > 0,

for all (f,x)E /X R',

fp.(^jc)Jjc=l,

for all f E / .

(1.2.15) (1.2.16)

One-point stochastic processes can be examplified with the random-sign process .$($,;) which is defined for all f E R and determined in the following way: at every fixedtER,random variable ^(,f) takes anyofvalues land -1 with probability 1/2. This process is a part of the random walk (e.g., pp. 345-346 of Papoulis, 1991) and the random telegraph signal (e.g., pp. 292, 643, 644 of Papoulis, 1991). One-point processes are, however, not very popular in applications. The reason is that the above independence is too strong idealization of the interrelations observed in the real-worlds problems. If one applies the weaker independence requirement, the less idealized model can be obtained. It is discussed in the next section.

1.3

Two-Point Process. Expectation. M a r k o v Process. E x a m p l e of N o n - M a r k o v Process Associated with Multidimensional M a r k o v Process

Before considering Markov processes (Definition 1.6), w e need the notions formulated in Definitions 1.2-1.5 below. Definition 1.2 Stochastic process x on interval 7 is called two-point process if and only if it is completely described by the probability distributi-

Expectation. Afar&ou Process. Rxamp/e o/*AToK-Afar^ou Process

11

on of random variable

X(',fJ

= (x(',;J-x(-.;Jf

d.3.1)

at every f ,^ e / where, without loss of generality, one assumes that f_<^.

(1.3.2)

T i m e points f_ and ^^ are called preceding and subsequent respectively. The term "two-point" in this definition is due to the fact that the probability distribution is associated with any two time points. Definition 1.2 in particular m e a n s that, for two-point process x, random variable (1.3.1) at every fixed f_,^6/ is stochasticallyindependent of random variable x('^.) for all f.E/ such that f.?^f_ and f.?^. The terms "preceding" and "subsequent" in Definition 1.2 emphasize the following aspect. M o m e n t f in (1.3.2) is regarded as the preceding time point with respect to all the values of time which are not less than f . M o m e n t ^ in (1.3.2) is regarded as the subsequent time point with respect to all the values of time which are not greater than f^. R e m a r k 1.3 In contrast to notion of one-point process, notion of twopoint process does allow stochastic dependence of random variables x('-f_) and x('
(1.3.3)

*,=x(^J.

(1.3.4)

12

TnfrodMcfory C7:ap?er

In so doing, Eqs. (1.2.15), (1.2.16) are rewritten as

for all x_,j^eR^, ;_,^E/: f_<^, f pj.(f_,.x_,f„.xj
forallf_,^e7:^<^.

(1.3.5) (1.3.6)

R^xR'*

Since random variables %('
E[((;_,x(-,;_),f„x(-.fj)]= / ((;_,.x_,^,*Jp.(;_,*_,^,j;J
for all ;_,^E/: ;_<;,,

(1.3.7)

is called expectation (or m e a n ) of random variable ((f_,x(',f_),f,,x(' -^)) with respect to processxExpectation of function ( m a y exist or not exist. In what follows, w e need certain technical freedom in evaluation of the integral in (1.3.7). For brevity, w e describe any function ( granting this freedom with the help of Definitions 1.5 preceded by Definition 1.4. Definition 1.4 Let stochastic process x on interval / be two-point with joint probability density p.. Process x is called non-anticipative two-point process if and only if integral f p.(f ,^ ,f ,jc ) J ^ is independent R"

of f^ forallx_eR'', ;_,^e/: f_<^.

(1.3.8)

T h e term "non-anticipative" in this definition is due to the independence of all the subsequent m o m e n t s ^ of time (see the inequality in (1.3.8)).

Expectation. MarAou Process. Example o/ATon-MarAoM Process

13

D e f i n i t i o n 1.5 L e t stochastic process x o n interval 7 b e non-anticipative two-point. A n y real function ^ of (^,x_,^,.xj defined o n J x R ^ X / X R ' ' is called regular w i t h respect to process / o n interval / if a n d only if equalities

^ t ;

R'' X R'*

R'' R**

for all f , r e / : f
(1.3.9)

f f((^_,JC.,^,^)p(^_,A:_,^,ArJ^^ ^ _ = f f((f_,%_,^,^Jp(f_,^_,^,A:JJj;_ ^X,,

forallf,^e7:^<^,

(1.3.10)

)Lm f f((;_,jt_,^,xjp.(f_,;t:_,^,%j6b;_ J^=f((^_,j;_,^,jcJp_(^_,^_)J^_, <.^< forallf.e/.

(1.3.11)

hold w h e r e quantity p _(?_,*_) denotes the integral in (1.3.8), i.e.

for all x_ER^, r_,^e/: ;_<;^.

(1.3.12)

This definition allows one to evaluate expectation (1.3.7) for function ( regular with respect to process % on interval 7 by m e a n s of the integrals on the left- or right-hand sides of (1.3.10). In so doing, the limit value of (1.3.7) as ^J,f_ is determined according to (1.3.11) (see also (1.3.9)) as E[((r_,x(^-),^0,x(',f.+ 0))]=limE[^,x(',^),^x('^J)]

= f((;_,.x_,; ,x_)p (; ,.x_)&c for all ^_e 7.

(1.3.13)

R e m a r k 1.4 If quantity ((f ,%_,^,.xjp.(;,%,^,;tj, as a function of (.x ,%J at every fixed f ,^6E/ such that f_
14

Zn^rodMcfory Chapter

(1.3.10) hold. More precisely, if the above integrability is the case, then the Lebesgue integrals

R'*

R''

as c o m m o n (not generalized) functions of x_ and x^ respectively, exist almost everywhere in R'' (e.g., p. 19 in Vladimirov, 1984), are integrable and Eqs. (1.3.9), (1.3.10) are valid at f_<^. (The term "almost everywhere in R^" m e a n s up to any set E S g ^ such that the Lebesgue integral J^^Jt is equal to zero.) Let function p be defined with equality p(f_,x_,^,xJ = Py(f_,.x_,^,.iJ/p_(;_,x_), foralljc,^ER^, f_,f,€7: f_
(1.3.14)

This quantity and Defintion 1.5 enable one to introduce notion of Markov process in the following way. Definition 1.6 Non-anticipative two-point stochastic process x on interval J is called Markov stochastic process on interval J if and only if its sample functions are almost surely continuous, function ((; ,.x ,^,xj = 1 is regular on 7 with respect to x and equation

R"" [ R**

= f f P-(f-,X-)P(f-*.X-'f.'X.)P(f.,.y.'?*'^)^.x. J ^ for all ^ € R ^ , f_,^.,^e/: f_<^.<^,

(1.3.15)

holds where X. denotes the values of process x at time point f., i.e. ^.= X(^^.).

(1-3.16)

R e m a r k 1.5 The Fubini theorem (see R e m a r k 1.4) can be mentioned again. Indeed, if the integrand in the double integrals in (1.3.15), as a function of (x_,x.) ateveryfixed^SR'' andf_,f.,^e/suchthatf_
E^pec^ct^ton. MarAou Process. E^amp^e o/No^-AfarAo:; Process

15

Lebesgue integrals f p (^ ,j; )p(f_,^_,f,,^,)p(^,,%.,^,%JJA:,,

f p (^ ,j; )p(^_,^_^.,%.)p(^.,%.,^,^J^_, R''

as c o m m o n (not generalized) functions of (.x_,xj and (x.,.xj respectively, exist almost everywhere in R'', are integrable and Eq. (1.3.14) is valid at

Note that, for ((f_,*_,^,%J = l, Eqs. (1.3.6), (1.3.9), (1.3.10) can be combined into equation
for all ; ,r e / : ^
(1.3.17)

and (1.3.11) is written as

R^R^

J

R**

forallr.e/.

(1.3.18)

Equations (1.3.14), (1.3.15), (1.3.17), (1.3.18) are discussed in Section 1.4. R e m a r k 1.6 below notes one of the remarkable features of multidimensional Markov processes (e.g., Section (2.3.3) of Arnold, 1974). R e m a r k 1.6 If Markov process x is multidimensional, then its entries (1.2.3) are the processes which are generally neither Markov nor two-point. Indeed, since the process is multidimensional, then (see the text above (1.2.14)) J > 2 , so it has at least two entries. If w e assume that any one of them, say, process x% (^ = 1, ,^) is two-point, then, according to Definition 1.2, it should be completely described with probability distribution of rand o m variable (X^('^-)'X^('^))^' However, the latter fact need not be the case since (see R e m a r k 1.3) the mentioned random variable in general stochastically depends on other entries of random variable (1.3.1). Thus, process x^ is generally not two-point and hence (see Definitions 1.4 and 1.6) non-Markov.

16

ZnfrodMc^ory (%ap(er

This remark points out that non-Markov processes can be modelled as entries or subvectors (i.e. groups of the entries) of Markov processes. The obvious advantage is that the non-Markov phenomena can then be analyzed by m e a n s of the well-developed mathematical theory of Markov processes (e.g., Dynkin, 1965). This useful option seems to be underestimated in applied sciences. For instance, treatments in theoretical physics sometimes describe the nonMarkov effects too straightforwardly. A good example is the Mori stochastic model (see Mori (1965) and (3.10)-(3.12) therein) with its so-called m e m o r y function (see also Chapter 9 of Hansen and McDonald, 1986). O n e of the resulting difficulties in this direction is that it leads to the models (e.g., M o ri's one) significantly more complex than the possible alternative descriptions based on Markov processes. Moreover, proper mathematical treatments for such models are as a rule far beyond available capabilities of theoretical physics. These circumstances do not contribute too m u c h to timely and efficient solutions of applied problems. Unlike this, Chapter 5 of the present book discusses h o w a scalar nonMarkov process can be described with entries of a high-dimensional Markov process in connection with Ito's stochastic partial (integro-)differential equations.

1.4

Preceding, Subsequent a n d Transition Probability Densities. T h e C h a p m a n - K o l m o g o r o v Equation. Initial Condition for M a r k o v Process

This section continues to discuss Markov processes and Eqs. (1.3.14), (1.3.15), (1.3.17), (1.3.18). W e first point out equations p_(;_,*_)>0, forall(f,jcje/xR^,

(1.4.1)

fp_(f_,.x_)Jjc_ = l,

(1.4.2)

forallf.E/,

which follow from (1.3.12), (1.3.5) and (1.3.12), (1.3.17), (1.3.18) respectively. They show that function p introduced with (1.3.12) is a probability density. Because of (1.3.12), quantity p_(f_,x_), as a function of x_ at every fixed f_E 7, presents marginal probability density of random variable %(''?-)

Preceding, SMAseguenf a^d TYans^toyt Proha&tZtty De?ts:%es

17

at preceding time point ; . In terms of Markov process %, density p can be called preceding probability density of %. Let function p^ be defined as follows p^(f_,^,xj= f p.(f ,x ,;^,.xJJ.x_,

for all ^ER'', f_,^E/: f_<^.

(1.4.3)

Relations (1.4.3), (1.3.5), (1.3.17), (1.3.18) and (1.4.2) show that p^;_,^,ArJ>0,

forall.^ER'',;_,^E./:;_<^,

fpXf_,^,*J
forall^,^E7:f_<^,

l i m f p ^ . , ^ , ^ ) ^ = l,

for all ^_E/,

(1.4.4)

(1.4.5)

(1.4.6)

i.e. p^ is a probability density. Because of (1.4.3)-(1.4.5), quantity p (t ,^ ,jc ), as afunctionof x^ at every fixed ^ , ^ E / undercondition(1.3.2), is marginal probability density of random variable x('<^) at subsequent time point ^. In terms of Markov process %, density p ^ can be called subsequent probability density of /. R e m a r k 1.7 Since preceding and subsequent probability densities p and p^ correspond to the preceding and subsequent time points, they should coincide with each other if the points coincide. This m e a n s that

for all ^ E R ^ , f_,;^E/: ;_<^.

(1.4.7)

P^-,f-+0,.yJ = lim p^(f_,^,xj = p_(;_,.*,), for all jt^ER'', f_E/.

(1.4.8)

Equations (1.4.7) and (1.4.8) also point out that quantity p^(; ,f^,xj, as a function of f^ at any fixed ; E 7, is right-continuous at f^= f_ uniformly with respect to x:^E R^. In terms of Markov process %, the latter m e a n s that the process is right-continuous in the probability distribution. Relations (1.3.14), (1.3.5), (1.4.1) and (1.3.12) show that

18

/n&*OG?Mcfory CAap^er

for all^.^SR'', ;_,^EV: ^_<^,

(1.4.9)

) p(f ,x ,f ,x )Jjc =1,

for all ^_eR^, ^ , ^ e / : f_
(1.4.10)

i.e. p is a probability density. Because of (1.3.14) and (1.3.12), quantity p(f ,x ,^,.xj, as a function of ^ at every fixed x E R ^ and f_,^E/ under condition (1.3.2), is conditional probability density of random variable X(',fJ under the condition that equality x($^-)=*- holds. Definition 1.7 (e.g., Section (2.2.4) of Arnold, 1974) Probability density p described above as the conditional probability density for Markov process X is called transition probability density of x T h e term "transition" in this definition is due to the fact that density p provides the transition from preceding probability density p_ to subsequent one p ^. This is explicitly shown by equation

p^(f_,^,xj= f p (; ,x )p(; ,.x_,^,.xj6?x_, R'

for all x^eR', f_,^E/: ;_<;^,

(1.4.11)

for all jc_,^eR^, ;_,;,e7: ;_<;^.

(1.4.12)

that follows from (1.4.3) and

The latter is merely an equivalent form of (1.3.14). Equations (1.4.11) and (1.4.8) leads to the property that can be expressed as follows (e.g., (9.4.7) in Arnold, 1974) Hm

p(f-;-K-'^''Xt) = 3(it'-*'-)'

for all x_,^E R^, ;_E 7,

(1.4.13)

where 6 is determined with (1.2.8). Property (1.4.13) specifies h o w to interpret Eqs. (1.4.9)-(1.4.12) at f^=f_ or, more precisely, in the limit case as t ^ . In particular, w e shall apply the extended version

Preceding, <SM&se<7Menf and rrans^toTt Pro6atA:H^y Dens;^:es

foralljc_,^e]R'', ^ e / ,

19

(1.4.14)

of (1.4.13) instead of (1.4.13).

Transition probability density p of M a r k o v process x verifies the Chapman-Kolmogorov equation (e.g., (2.2.4) of Arnold, 1974). It is derived below by m e a n s of the preceding and subsequent probability densities. T o do that, w e consider the two following versions of (1.4.11): one corresponds to variable (f.,x.) (see (1.3.16)) instead of (^,.t J (see (1.3.4)) w h e reas another corresponds to variable (f.,x.) instead of (f_,*_) (see (1.3.3)). These versions are p^,f.,.x,) = f p (f ,x_)p(;_,jc_,f.,jr,)Jx_,

for all x.ER^, f_,;.eJ: ;_<;.,

(1.4.15)

P,(A'^'*t)= f P-^.'-XjP^.'^.'^'^J^-X.' for all ^ G R ^ , ^.,f,e7: f.<^.

(1.4.16)

Equations (1.4.11) and (1.4.16) express the s a m e quantity, namely, the subsequent probability density of process % at time point ^, i.e.

forallj^eR^, ;_,;.,^eJ: ;_<;.<^.

(1.4.17)

In view of (1.4.7) at (f„.xj replaced with (^.,j<;.) and (1.4.15), Eq. (1.4.16) is equivalent to

p,(;.-;„*j={ j^ p^_,^_)p(^_,^_,^.,x.)^%_ p(f.,.x,,^,xj
20

ZH&*odMcfory C/:ap^er

< p(f_,jc_,^,jcj-f p(f_,x_,f.,.x.)p(Y.,x.,^,xJJ.x. P^(^-!^-)^,= 0, for all ^_,^SR^, ?_,f.,^E./: ^_
(1.4.18)

Since functions p and p_ are independent ofeach other and p_ has properties (1.4.1) and (1.4.2), Eq. (1.4.18) is equivalent to p(f ,.x_,f+,JtJ= fp(f ,x ,f.,;t,)p(f,,.x.,^,A:JJ.x., for all ^ ^ S R ' ' , ;_,;.,f,€/: ^
(1.4.19)

Equation (1.4.19) is called the Chapman-Kolmogorov equation for transition probability density p of Markov process x. i.4.2

ZfMftaZ cow^t^ow ^?r AfmrAov process

Markov process x is completely described with its joint probability density p.. A s shown above, this density can be presented with (1.4.12) in terms of preceding and transition probability densities p_ and p . In so doing, both p and p as well as subsequent density p^ are expressed by means of p. (see (1.3.12), (1.3.14), (1.4.3)). However, Eq. (1.4.12) also enables one to construct p. with the help of p_ and p of properties (1.4.1), (1.4.2) and (1.4.9), (1.4.10), (1.4.14) respectively, not involving (1.3.12) and (1.3.14). Equation (1.4.3) is not needed either since p^ is described with (1.4.11) in terms of p_ and p as well. Thus, Markov process x can be constructed on the basis of p_ and p. This is the w a y which is usually used. There are infinitely m a n y Markov processes with the same transition probability density p . At every fixed f_€= 7, they differ from each other only with respective versions of density p_. Thus, to specify the process, one should specify the corresponding p . The latter is usually done as described below. Let f S / be arbitrarily fixed. M o m e n t f is regarded as a specific value of preceding time point f_. Let also x be a random variable with probability density p^, i.e. p„(*J>0, / P c ( ^ o = l.

for all ^ S R ^ ,

(1.4.20) d-4.21)

Preceding, .SMAse
21

where ^ denotes values of x„ < ^=X.(Q-

d.4.22)

Density p is used as the version of subsequent density p (f,-), i.e. P-Co-*.) = P.(*„) -

for ^ 1 *cE R''.

(1.4.23)

In terms of process x and random variable Xo &s well as their values (1.2.12) and (1.4.22), this m e a n s that X(',0 = X.('),

(1-4.24)

*],.,„=*c-

(1.4.25)

R a n d o m variable x„ can in particular be continuous or discrete or of a more complex type, for instance, continuous-discrete (see the text below (1.2.7) d o w n to R e m a r k 1.1). Density p^ enables one to uniquely prescribe the process at any m o m e n t f > f that can be regarded as the subsequent time point f^. In so doing, variable (f,x) (see (1.2.12)) is applied instead of (^,jcj (see (1.3.4)). If process x is specified with (1.4.23), w e apply notation p, to subsequent density function p , i.e.

p,(^,;,*) = pX;.,f,*),

for all xeR'', ;„,;e/: ;^<; if (1.4.23) holds.

(1.4.26)

In this case, Eqs. (1.4.4), (1.4.5), (1.4.7), (1.4.8), (1.4.11) become p,(f„,f,.x)>0,

forallxSR'', ^,;e/:f,
J*p,((„,;,*)<^ = l,

(1.4.27)

;„,fE/: ;,
(1.4.28)

forallxER'', f_,;e/:f„
(1.4.29)

p-(f.*) = p.(f.'f'*).

P.(fo.'.+0'*) = MmPA'''.x) = P.(j'). for all * E R*', ;„E/.

(14.30)

22

Zn^rodMcfory CAqpfer

forallxER'', ^,K=/:^<;.

(1.4.31)

The corresponding versions of Eqs. (1.4.9), (1.4.10), (1.4.14), (1.4.12) are:

for all ^,^ER^, ^,rE7: ^
(1.4.32)

j*p(^,^,^Ar)Jx=l,

for all ^ER'', f„,fE7: ^<^,

(1.4.33)

p(^^o^o+0^) = * ^ P ( ^ ^ . ^ ^ ) = S(^-^), for all ^,^E R^, f^E/,

(1.4.34)

p. (^, ^, 4 ^) = P (f,, *., ^, ^ p , ( ^ ) , for all ^,jcER^, ^ J E / : ;„
(1.4.35)

Equations (1.4.23)-(1.4.25) present different forms of the s a m e initial condition imposed on %. Definition 1.8 T i m e point ^, r a n d o m variable x„ and probability density p^ for which the properties pointed out in Definition 1.6 (see Section 1.3) are valid under conditions (1.4.23) and (1.4.35) are called initial time point, initial value (or initial r a n d o m variable) and initial probability density respectively of M a r k o v process % specified with t h e m and described with (1.4.35), (1.4.31). R e m a r k 1.8 If values (1.4.22) are reduced to a single, deterministic vector, say, e^, i.e. equality X.(Q=ec

foralHEE,

(1.4.36)

holds or, that is the s a m e (see R e m a r k 1.1 in Section 1.2), Po(*o) = 3(*.-'o).

for all ^ E R " ,

(1.4.37)

then (cf., (1.2.11)) r a n d o m variable Xo is in fact nonrandom, deterministic. In this case, Eq. (1.4.31) becomes

23

#o??:ogeHeoMS AfarAou Process. T/te IVte/ter Process

P.(fo-f-*)=P(^o-^). foralljcSR'', f.^e/: f„
(1.4.38)

and, thus, density p is a particular case of transition density p . Generally, any random variable and any probability density can serve as the initial ones for a Markov process. In some cases, they are, however, subject to additional conditions. This fact is considered in Section 1.6 in connection with invaraint Markov processes important in applications. Section 1.5 deals with the examples of Markov processes.

1.5

H o m o g e n e o u s M a r k o v Process. E x a m p l e of M a r k o v Process: T h e W i e n e r Process

W e point out two examples of Markov processes % described as shown in Section 1.4.2. O n e of them is related to ordinary differential equations (ODEs). It is discussed in Appendix A. The other one is the topic of this section. It presents the stochastic process introduced by Wiener (1923). The Wiener process plays a fundamental role in the theory of stochastic processes and, in particular, of Ito's stochastic differential equations (ISDEs) (see Section 1.9, Appendix C, Chapters 5 and 6). To consider the Wiener process, w e need the following definition. Definition 1.9 (e.g., (2.2.8) of Arnold, 1974) Markov process x on interval V is called homogeneous Markov process if and only if dependence of its transition probability density on f and f^ is presented as a function of A = f,-f_e [0,A),

for all ;_,^e/: ;_<;^,

(1.5.1)

more precisely, P(?...X-.f,..xJ= p(f_,x_,f_+A,xJ= p(;^-A,.x_,^,.xJ= p,,(.x_,A,.rJ, forallx_,^6R'', f_,^E7: f_<^, AE[0,A),

(1.5.2)

where A E(0,oo) is the length of interval 7. Quantity (1.5.1) is called time separation. For homogeneous Markov processes in most of applied problems, transition probability densities, as functions of time separation A , are defined for

24

/H^rodMcfory CAop^er

all A > 0 , i.e. A = 00. In this case, Eqs. (1.4.9M1.4.12), (1.4.14) are written as p,_(x_,A,.iJ>0,

foralljc.^GR'', A > 0 , (1.5.3)

f p ^ _ , A , j c J ^ = l,

foralljcGR^, A > 0 ,

(1.5.4)

forall^ER'', ^ e / , A > 0 ,

(1.5.5)

p^.(r_,^_,^_+ A,j;J = p (;_,x_)p,,(.r_,A,xJ, foralljc_,^eR^, ^ _ € 7 A > 0 ,

(1.5.6)

p^_,0,jrj = lim p ^ _ , A,j;J = 6(^-%_), for all ^ ^ e R ' ' .

(1.5.7)

For homogeneous Markov process %, the Chapman-Kolmogorov equation (1.4.19) is written as P^-.^1 + ^ 2 ^ J = /PA(^-^1^.)PA(^.^2-^)^.' foralljc^^SR'', A ^ A ^ ^ O . Definition 1.10 Homogeneous Markov process % with transition probability density p^ and specified with initial probability density p^ (as described in Section 1.4.2) is called the J-dimensional Wiener stochastic process on interval [f ,°°) where f E R is arbitrarily fixed if and only if equations P.(*.) = 8(xJ,

for all ^ S R ^ ,

(1.5.8)

(x,-x )T(x^-;<: )

p,,(.x_,A,jfJ = (2nA) ^exp

2A

forallA;_,^eR^, A = ^ - ^ > 0 , hold.

(1.5.9)

RomogeneoMS AfarAou Process. TAe Wtener Process

25

W e in what follows denote the ^-dimensional Wiener stochastic process on [^,°o) with vector W^,;) = (^(^,;),..,W^,f))T. Function 6 in (1.5.8) is determined according to (1.2.8). It follows from (1.5.6), (1.5.8), (1.5.9) and (1.2.8) that joint probability density p. for the 6?-dimensional Wiener process is "/
P,(*..A.*)=n

—=exp

(*
S(^)

foralljc,jceR'', A = r-^>0. In view of Definition 1.10 (see also the text between Remarks 1.1 and 1.2 in Section 1.2), this means that every entry of the 6?-dimensional Wiener process is the one-dimensional Wiener process and these entries are mutually stochastically independent. The Wiener process can also be regarded as the well-known limit case of the random walk (e.g., pp. 346-347 of Papoulis, 1991). R e m a r k 1.9 It is well-known (e.g., p. 48 of Arnold, 1974) that almost all sample functions of the Wiener process are continuous. However, these functions are nowhere differentiable and hence derivative JW(i;,;)/<& does not exist as a c o m m o n stochastic process. This derivative presents a generalized stochastic process and is also k n o w n as (Gaussian) white noise (e.g., Section 3.2 of Arnold, 1974). A n y equation which includes t^($,;)/;A has to be treated in the sense of generalized processes. In terms of R e m a r k 1.8, vector ^ for density (1.5.8) is zero because of (1.4.37). Then Eq. (1.4.38) for the Wiener process is written as (see also (1.5.2)) p,(^,^+A,^) = p^(0,A,jc) that, by virtue of (1.5.9), is

p,(f.,f„+A,*) = (2nA) 2,^exp

forall.xe]r,A = ;-;>0. 2A

This equation demonstrates that random variable W ( ,f + A ) is Gaussian with zero expectations and variance matrix A / where 7 is the identical t% X J-matrix. The corresponding standard-deviation matrix is /A/. The latter also points out that the physical dimension of the Wiener process is the square root of time.

26 1.6

7n?rodMcfory CAap^er

Expectation, Variance a n d Standard Deviations of M a r k o v Process

T h e most important deterministic characteristics of Markov process % are its expectation vector and variance matrix. They are considered below. Quantity(1.3.7)forfunctions ((f_,x ,f^,.xjs.x and ((f_,.x_,;,,.x,) = .x^ are called expectations of process x at preceding m o m e n t f_ and subsequent m o m e n t ^ respectively. W e denote them with e(f ) and e(f J, i.e. e(fJ=E[x(-,f_)],

forallf_,^E7:;_<^,

(1.6.1)

e(;J=E[x(',?J],

forallf_,^E7:f_<^.

(1.6.2)

Quantity (1.3.7) for functions ((f_,x_,f,,-xj = [x_-e(f_)][x_-e(f_)]^ and ((f_,.x ,^,;tj = [^-e(rj] [jt^- e(^)]^ are called variance matrixes of process x at preceding m o m e n t ^ and subsequent m o m e n t ^ respectively. W e denote them with F(f_) and H(fJ, i.e. ^.)=E{[x(^.)-6(r_)][x(,f.)-^.)]T}, for all ;_,^EV: ^_<^,

(1.6.3)

^)=E{[x(',fJ-e(fJ][x('-fJ-^)]^}forallf_,^e7:^_<^.

(1.6.4)

Obviously, these matrixes are symmetric and non-negative definite. For process x specified with initial random variable Xo (see Section 1.4.2), Eqs. (1.4.24) and (1.6.1), (1.6.3) at ?_=;„ show that

<;.W„

d.6.5)

H M = Ki

d-6.6)

where e and t^ are expectation vector and variance matrix of %-, i.e.

<^[X„(-)]=j*P.(^.

(16.7)

R"

Entry e.(^), A; = l,...,^, of vector e(r) is called expectation of entry x^ of

E^pec^a^ion, Variance a^d Standard Deuta^tons of Afar%ou Process

27

process x at time f . Entry 1^.(;)>0, A; = l,...,^, of matrix F ( ^ is called variance of process x^ at time f. Scalar Jt^(f) is called standard deviation, or root-mean-square ( R M S ) value, of x% at time f. Entry t^(f)=I^(f), ^'^ = 1,...,J, / # ^ , of matrix V(;) is called cross variance of entries x& and X/ of process x at time f . R a n d o m variables X^('^) and X;('^) are called uncorrelated at time ; if and only if ^(f)=0 (or, equivalently, !^(f)=0). If random variables are stochastically independent, then they are uncorrelated. The reverse statem e n t is generally not true. Quantity ^(^)/J^(^)^;(^) under condition K.(f)K,(f)#0 is called correlation coefficient of random variables X^(^) and X;('-^) at time f. L e m m a 1.1 Let x be a Markov process on interval 7 with preceding and subsequent probability densities p_(f_,x_) and p^(f_,^,.xj (see (1.4.11), (1.4.12)). Let also Z be any real function of (f,x) defined on J X R ^ such that both functions ((f_,x_,^,xJ = Z(f ,.x_) and ((f_,^_,^,^J = Z(f,%) for all f , ^ E / under condition (1.3.2) are regular with respect to x (see Definition 1.5 in Section 1.3). T h e n relations ( Z(f ,jt )p (f ,x )p(f ,x ,^,xj6?.x 6?x^ = (Z(; ,jt )p^(f ,f ,x )Jx , R^XR'*

R*

for all ;„,;_,;,€/: ^ < ^ _ < ^ , ) Z ( ^ ,jc )p (^ ,% )p(f ,;c ,^ ,jc )
= j* Z(^,^) j*p_(^,^)p(^,^,^_,^_)^ R^xR''

P(^)^.!^'^t)^-^t

[R''

= fZ(^,^)p^,^,jcJJ^ for all f,,f_,^eJ: ^ ^ ^ ^ ^ <

E[Z(^x(-f)]=j'Z(r^)pX^^^)^, R"

for all f,,fE/: ^ ^^,

(1.6.9)

28

/H^rodMcfory CAapfer

E [ Z ( ^ 0 , x ( , ^ 0 ) ) ] = limE[Z(f,x('-f))]=

Z(^^)p(^^)^,

for all ^ 6 7 ,

(1.6.10)

hold. Proo/^ The assertion follows from the hypothesis, Definition 1.5, (1.4.12), (1.3.13), (1.4.7), (1.4.11), Definition 1.6 and (1.3.15). L e m m a 1.1 enables one to express expectation of function Z(f,x(*,?)) for Markov process % in the terms which, in contrast to (1.3.7) and (1.4.12), do not include transition probability density p . W e consider a few applications of L e m m a 1.1 below. L e m m a 1.2 Let hypothesis of L e m m a 1.1 where process x is specified with initial condition (1.4.24) (see also (1.4.23)) and Z(f,.x) = x hold. Then expectation function e (see (1.6.1), (1.6.2)) is described with equations <') = ^ [X('*')]=/* P.(f.-^)<^.

forall;„,;E7:f,<;,
for all ^ S 7 .

(1.6.11) (1.6.12)

.Proo/ The assertion follows from the hypothesis, (1.6.1), (1.6.2), (1.4.24), (1.4.23), (1.4.26), (1.4.30) and (1.6.7). R e m a r k 1.10 L e m m a 1.2 demonstrates that expectation of the process is generally affected by its randomness presented with the subsequent probability density of the process. Curiously, this feature is not always realized in applications. The latter can partly be explained with the fact that, in the linear stochastic models (e.g., Chapter 8 of Arnold, 1974) which are transparent enough to be analytically treated by nonmathematicians, the expectation is always independent of stochasticity of the system. L e m m a 1.3 Let hypothesis of L e m m a 1.2 hold and, for process x therein, hypothesis of L e m m a 1.1 at Z(f,jc) = j;ji:^ also hold. Then variance function M (see (1.6.3), (1.6.4)) is described with equa-

Expee^a^on, Variance anc! ^a^dmrd Deu:a^:o?:s o/*AfarAou Process

29

tions

forall^,^e/:^
^

+ 0 ) = l i m ^ ) = !^,

for all ^ S 7 .

(1.6.13)

(1.6.14)

F*roo/ is similar to that of L e m m a 1.2 and also invloves the fact that [A:-e(f)][^-e(t)]T=jcA:^-[e(f)]^-e(r)jcT+e(f)[e(f)]^and Eqs. (1.6.3), (1.6.4), (1.6.8) instead of (1.6.1), (1.6.2), (1.6.7) respectively. In most of applied fields, the following properties hold: expectation-vector function (1.6.11) is uniformly bounded for all ?^?,'

(1.6.15)

variance-matrix function (1.6.13) is uniformly bounded for all f ^f„.

(1.6.16)

These properties at f=f^ m e a n s that ll
(1.6.17)

IIK,II<°°.

(1.6.18)

where H* H is n o r m of vector or matrix. If the variance matrix is finite, then the corresponding expectation vector is finite (e.g., see (11.3.3) of Arnold, 1974, for details). T h e reverse statement is generally not true. Variance matrix F(f) is finite for all ^ , f S / such that f^
for all ?,,?€/: f,^?,

(1.6.19)

where tr() is trace of matrix (i.e. tr[^)]=X^^^(f)). Vector (1.6.1) (or (1.6.11)) and matrix (1.6.3) (or (1.6.13)) lead to other parameters crucial in engineering applications. O n e of them, signal-tonoise ratio, is discussed Appendix B. In some problems, for example, modelling time-development of capital gain which is generally a stochastic variable, feature (1.6.15) is merely irre-

30

Zn&*odMC%ory CAap^er

levant. Indeed, the capital gain, the key quantity in modern economy, can be described with expectation (1.6.11). So, this expectation should, of course, tends to infinity as ?-* oo. Interestingly, variance (1.6.13) m a y also tend to infinity as ;-* oo and in such a w a y that the corresponding SNR(f) (see (B.3)) decreases. Then, in view of (B.4), the reliability of gaining the capital can become negligible mathematically (e.g., see the text between (B.4) and (B.5) on the 10-dB detectability limit) and hence inacceptable by the investors (provided that they are aware of the low-SNR effect). If so, economics will likely have to find the remedy to keep the above S N R at sufficiently high level. T o what extent this remedy will agree with the market principles is the question beyond mathematical modelling. Expectation e(f) of Markov process % is the deterministic representative of % - If the process is nonrandom (see Appendix A for the example), then it coincides with e(f). This follows from the fact that Eq. (1.6.11) holds even if density p, is replaced with its deterministic version

pA-^)=6[*-<;)], forall.xER'', ;„,;e7:f>f„.

(16.20)

In this case, variance matrix I^(f) is, of course, zero (see (1.6.13) under condition (1.6.20)). Equation (1.6.20) presents the simplest expression for density p,(^,f,.x).

1.7

Invariant a n d Stationary M a r k o v Processes. Covariance. Spectral Densities

Definition 1.8 (see Section 1.4.2) and Eqs. (1.6.17), (1.6.18) provides m u c h freedom in choice of the initial probability densities. There is also a very important family of the so-called invariant Markov processes which prescribes the initial densities in some special way. Since in engineering such processes are usually defined on entire real axis R rather than on its interval 7, the present section is devoted only to the ones defined for all f . Definition 1.11 Markov process % with transition probability density p(f , -,^, -) defined for all f_,f^E R such that f _ ^ is called invariant Markov process if and only if there exists scalar real function p such that p,(;,x)>0,

forall(t,Jc)eR^^,

(1.7.1)

7/n'atrtan^ an J .S^at^tonary AfarAov Processes

fp.(^,x)^=l,

forallrER,

31

(1.7.2)

p,(^,xj = fp,(;_,.x_)p(;_,;E_,;„*j6?*_, R^

for all ^ E R ^ , f_,^€ R: r_<^,

(1.7.3)

and the properties pointed out in Definition 1.6 (see Section 1.3) are valid under condition p_(;,*) = p,(;,.x),

forall(f,x)eR^.

(1.7.4)

For the invariant process, function p,. is called invariant probability density. Equation (1.4.12) for the invariant process becomes p,(f_,x ,^,xj = p,(f ,x )p(f ,x_,^,.Kj, for all *_,j^eR'', f_,^E R: ;_<;^.

(1.7.5)

Depending on p , integral equation (1.7.3) for p, can have no solutions like in case of transition density (1.5.9) (p. 72 of Has'minskii, 1980) or exactly one solution as in case of the Ornstein-Uhlenbeck processes (e.g., Section 8.3 of Arnold, 1974) or more than one solution. T h e present book assumes that (1.7.3) has the unique solution. It follows from (1.7.3), (1.7.4) and (1.4.11) that P.(f-,f.Jc) = P,(?,*), forallxeR'', f_,;ER:^
(1.7.6)

This enables one to use the following versions of L e m m a t a 1.2 and 1.3 for invariant M a r k o v process %. L e m m a 1.4 Let hypothesis of L e m m a 1.1 where process % is invariant with invariant probability density p. and Z(f,x) = .x hold. Then e(f)=fxp,(;,x)
for all ^ S R .

(1.7.7)

R"

.Proo/ T h e assertion follows from the hypothesis and allowing for expression (1.7.6) in (1.6.9).

32

7n&*oc%Mcfory CActpter

L e m m a 1.5 Let hypothesis of L e m m a 1.4 hold and, for process % therein, hypothesis of L e m m a 1.1 at Z(f,jt) = x x ^ also hold. Then F(f)=J*[*-e(f)][*-e(;)]Tp,(;,*),^,

Proo/^

for all r e R .

(1.7.8)

is similar to that of L e m m a 1.4.

Equation (1.7.6) shows that the invariant process can be regarded as the process with the subsequent density independent of the preceding time point. This independence in particular emphasizes that the interval which the invariant process is defined on is not limited from below by any preceding m o m e n t and hence is entire axis R. Thus, the versions of (1.6.15), (1.6.16) corresponding to (1.7.7), (1.7.8) are expectation-vector function (1.7.7) is uniformly bounded for all f € R,

(1.7.9)

variance-matrix function (1.7.8) is uniformly bounded for all re R.

(1.7.10)

Comparison of (1.7.4) and (1.7.6) shows that, for invariant Markov process, both marginal densities, preceding p_(',') and subsequent p^(f , , ) ones, are equal to each other and are presented by m e a n s of the s a m e density p . ( , ) . In other words, invariant process can be described with the help of a single marginal probability density which is invariant density p,.. The above facts in particular enables one to characterize invariant Markov processes by m e a n s of the classification of time dependences which is c o m m o n for deterministic functions defined for all m o m e n t s of time. For instance, one can apply the following definition. Definition 1.12 Markov process is called stationary or periodic (quasiperiodic, almost-periodic) if and only if it is invariant and its invariant probability density p(f,x) is stationary (i.e., time-independent), P;(f,%)=p^c),

for all ^ E R ^ ,

(1.7.11)

or periodic (quasi-periodic, almost-periodic respectively) in f uniformly with respect to x S R^. For stationary Markov process, density (1.7.11) is called

/nuaWa?:? and .Sta?:onary Afar^ot^ Processes

33

its stationary probability density. If the equilibrium state of the modelled system coincides with its stationary state, then stationary process corresponds to the equilibrium modes. In this particular case, notion of stationary Markov process can be regarded as a stochastic generalization of notion of equilibrium point of a deterministic O D E system. Note that (e.g., p. 71 of Has'minskii, 1980) transition probability density of stationary Markov process is homogeneous, i.e. Eq. (1.5.2) holds at A = oo. In this case, Eqs. (1.7.1)-(1.7.6) become p/%)>0,

for all ^ S R ^ ,

fp,(*)J*=l,

(1.7.12) (1.7.13)

P,(^)=/ P,(*Jp„(-r-,A,.xJ^_, for allien'', A > 0 ,

(1.7.14)

P-(f.*) = P,(f-.f.*) = P;(-*), forall.xER^,f_,fe]R:;_
(1.7.15)

Py(f---r.,f-+A,xJ = p,(.x_)p^(;r_,A,*J, forall^_,^6R^, r 6 R , A > 0 .

(1.7.16)

T h e corresponding versions of L e m m a t a 1.4 and 1.5 are formulated below. L e m m a 1.6 Let hypothesis of L e m m a 1.4 where invariant process x is stationary with stationary probability density p^ hold. Then e(f) = e,=j*p,(;r)dx,

(1.7.17)

R"

where e^. is the expectation vector of the process. i*roo/ T h e assertion follows from the hypothesis and substitution of (1.7.11) into (1.7.7). L e m m a 1.7

Let hypothesis of L e m m a 1.5 where process / is the s a m e

34

7nfro6?Hcfory C^ap^er

as in L e m m a 1.6 hold. Then ^)^=,f(*-2,)(*-'yp,M^

(1.7.18)

where K is the variance matrix of the process. JProo/^ The assertion follows from the hypothesis and substitution of (1.7.11) into (1.7.8). More details on stationary and periodic Markov processes can be found in Section 2 of Chapter III of Has'minskii (1980) [see also Section 4 of Chapter VI of Soize (1994) on the stationary processes]. Notion of invariant Markov process as the process defined for all fE R m a y be associated with the stochastic generalization of the steady-state solution of deterministic O D E system pointed out in Appendix A. This topic is discussed in Section 1.12.2. Very important characteristics of stationary Markov process x are its covariance matrix (e.g., (8.2.6.b) in Arnold, 1974; (55) on p. 70 of Soize, 1994) C(A)=E{[x(;,')-eJ[x(f+A,<)-e,]T}, C(A) = [C(-A)f,

forallA>0, forallA<0,

(1.7.19) (1.7.20)

and spectral-density matrix (e.g., p. 24 of Arnold, 1974; (86) on p. 76 of Soize, 1994) ^(/) = 2fC(A)exp(-t27i:/A)^A,

for all/ER,

(1.7.21)

where t is the imaginary unit (i.e. ^ = -1) and / is the conventional frequency coupled to the angular frequency h) as follows M = 2n/.

(1.7.22)

It follows from (1.6.3), (1.7.17)-(1.7.20) that C(0) = M,.

(1.7.23)

Importantly, Eq. (1.7.21) can be applied only if (e.g., p. 24 of Arnold, 1974; R e m a r k 6 on p. 76 of Soize, 1994)

TnuarMtyt? and R^tonary Mar^ou Processes

f!lC(A)]}JA
35

(1.7.24)

Equation (1.7.21) shows that spectral-density matrix 3(/) presents the inverse Fourier transform of covariance matrix C(A). Thus, the latter one can be expressed as the Fourier transform of the former one, i.e.

C(A)=If,S(/)exp(t2n.fA)6;/, forallAER. (1.7.25)

R e m a r k 1.11 Note that representation (1.7.25) is valid only if function C depends on time separation A rather than on both A and time f (e.g., p. 24 of Arnold, 1974). Thus, if the latter is the case, i.e. C depends on both A and f, Eq. (1.7.25) does not hold, the spectral-density matrix is not defined and Eq. (1.7.21) is meaningless. Matrix (1.7.19) can be expressed in terms of joint probability density (1.7.16) with the help of the following lemma. L e m m a 1.8 Let hypotheses of L e m m a 1.7 hold, transition probability density of process x is p^ and function ((f,x ,;+A,*J = jc_.x, for all fER, A > 0 , be regular with respect to process % (see Definition 1.5 in Section 1.3). Then C(A) =}(*.- ej (x^- ejTp,(x_)p^(x ,A,*JJ*_<^, R"

for all A > 0 , limC(A)=^,

(1.7.26) (1.7.27)

where the integral in (1.7.26) can be evaluated by means of the corresponding versions of the integrals in (1.3.10) and function C is continuous at A = 0, i.e. limC(A)=limC(A) = C(0) = F,. A;o

.Proo/

(1.7.28)

AfO

The assertion stems from the hypothesis and the fact that (x - e )

36

ZnfrodMc^ory CAap?er

X(j;-e^)^=^ j;^ -e^ jt - e ^ +e^e^ . W e note that Eq. (1.7.27) follows from (1.7.26), (1.3.11) and Eq. (1.7.28) follows from (1.7.27), (1.7.20), symmetricity of variance matrix F_ and (1.7.23). Spectral density (1.7.21) can also be presented in terms of its real and imaginary parts. In so doing, it is assumed that (1.7.24) holds and hence the density exists. In view of (1.7.20), spectral density (1.7.21) is written as

3(/) = 2f{[C(A)]Texp(t27t/A) + C(A)exp(-t27i/A)}JA, o for all/ER.

(1.7.29)

It follows from (1.7.29) that matrix ,$(/) is Hermitian, i.e. [^(/)]*=^(/) for all / S R where [-$(/)]* is the complex conjugate matrix of .$(/). This property in particular means that matrixes Re^(/) and Im-S^/) are symmetric and skewsymmetric respectively, every eigenvalue of matrix Re,S(/) is real and every eigenvalue of matrix Im-S(/) is imaginary. Expressions R e S(/) = 4 f ^(A)l +C(A) ^ s ^ n / A ) J A , o ^

for all/ER,

(1.7.30)

Im,S(/) = 4f ^(A)J ^(A) sin(2x.fA)jA' o ^

^all/SR,

(1.7.31)

are derived directly from (1.7.29). Matrixes Re.S(/) and Im^(/) are obviously even and odd functions of /. M a n y applied problems deal with spectral densities of such processes which are linear functions of stationary Markov process. The details on this topic can be found, for example, in Mamontov and Willander (1997b). They demonstrate that real part (1.7.30) is usually of a more practical importance than imaginary part (1.7.31). Entry C^(A), ^ = 1, ,J, of matrix C(A) is called covariance of entry x% of process / at time separation A . Entry 3^.(/), % = 1,...,J, of matrix ^(/) is called spectral density (or "power spectrum") of entry x% of process % at frequency/. Entry ^;(/) = [^(/)]*, A:,/ = l,...,
Dt/^MSton Process

37

quency/. Quantity J & . ( / ) , A; = l,...,^, is sometimes called spot noise (e.g., (5.17), (5.18) in Fish, 1993) of entry /^ of process x - This notion is usually used to interpret measurement data on noise in engineering systems [e.g., see Chapter 5 of Fish (1993) for the details]. In spite of the fact that the rigorous mathematical results mentioned in R e m a r k 1.11 are well-known, some works in engineering applications ignore them. They try to apply spectral densities to the stochastic processes with covariances depending not only on time separation A but also on time f. For instance, this can be found in Eq. (6) in Demir e? ctJ. (1996). Curiously, the same authors in their next paper (Demir and Sangiovanni-Vincentelli, 1996) do recognize (see p. 455 therein) the above mathematical results. Nevertheless, they ("for practical purposes") still use notion of spectral density for the process (see (20), (13), (9), (8), (4) in Demir and Sangiovanni-Vincentelli, 1996) with the (A,f)-dependent covariance (see p. 455 therein). A similar treatment is employed in Roychowdhury ef aJ. (1997). A meaning of these exercises is not very clear.

1.8

Diffusion Process

The above treatment concerns the internal structure of Markov stochastic processes. In particular, it demonstrates that the key characteristics of these processes are their transition probability densities. So far, however, w e did not discuss any equations which would provide connection of such densities to, say, the external world, more precisely, to the process parameters inherently associated with a specific applied field. The definition below resolves this problem. Definition 1.13 (e.g., (2.5.1), (2.5.2) in Arnold, 1974) Markov stochastic process x on interval / for which there is number ae > 0 such that lim fllx^-.x_ll^*p(f_,.i ,^,.xj6f.x^=0, <,.K ^"? „ forall(;_,*_)€JxR^,

(1.8.1)

is called diffusion stochastic process (DSP) if and only if there exist real vector g(f,x) and square real matrix JY(f,.x) defined on 7 x R ^ such that

38

ZTt^roduc^ory CAapfer

!itn

f(-Xt"* )p(^-<^-.^'^*)^t=^(^-'^-)<

forall(;_,x_)eJxR'', !im

(1.8.2)

f(^^^_)(^^-jc_)^p(^,^_,^,j);JJji;^=^f(f ,^ ),

forall(f_,*_)e/xR''.

(1.8.3)

If vector g(^,jc) and matrix H(f,.x) exist, they are called drift and diffusion coefficients, or drift vector and diffusion matrix of the above process %. Accordingly, functions g and 77 are called drift function and diffusion function. Since quantity /V(f,x)/2 is sometimes (i.e., in theoretical physics) called diffusion coefficient, w e shall call it diffusion parameter to distinguish it from #(f,x). Note that matrix 77(;,x) is symmetric non-negative definite for all (f,x) e /xR''

(1.8.4)

because of (1.8.3) and (1.4.9). This matrix can sometimes be zero. For instance, for solution n(^,.x^,f) of the Cauchy problem (A.l), (1.4.25), Eq. (1.8.3) leads to /7(f,Jt)sO because of deterministic transition (A.4) and continuous differentiability of the solution with respect to f. Diffusion processes under condition (1.2.14) are the stochastic processes which the present book is concentrated on. Theory of D S P s w a s developed by Kolmogorov (1931). It is inherently associated with linear parabolic partial differential equations (PDEs). More details on D S P theory can be found, for example, in Arnold (1974), Feller (1968, 1971), Friedman (1975, 1976), Has'minskii (1980), G i k h m a n and Skorokhod (1969), Gnedenko (1982), Prohorov and Rozanov (1969). Application of D S P s to cooperative phenomena in a broad family of specific problems in the natural and social sciences is surveyed by H a k e n (1975). This work w a s historically the source of m a n y ideas which formed the basis of synergetics (Haken, 1977). Theory of D S P s is also one of the key tools used in physics of complex systems (e.g., Serra, Andretta, Compiani and Zanarini, 1986; see also Honerkamp, 1994). T h e meaning of the drift vector and diffusion matrix (e.g., see (2.5.3) in

39

D:/^Hs:on Process

Arnold, 1974, for more detail) is emphasized with the following fact. The integrals on the left-hand sides of (1.8.2) and (1.8.3) presents the so-called conditional expectations of random variables X(<^)*X('^-) and [X(''fJ"X(''f-)][X(''^)"X(''^-)]^ respectively under the condition that X(^,f_)=^_- Equations (1.8.2) and (1.8.3) point out that the above expectations are proportional to time interval ^- f_ in the limit case as its length tends to zero. These equations also show that the physical dimensions of vector g(?,.x) and matrix //(f.jc) are the physical dimension of x divided by time and the squared physical dimension of .x divided by time respectively. There are infinitely m a n y D S P s with the same transition probability density p. A n individual process is specified with an initial probability density as described in Section 1.4.2. Definition 1.13 is nothing but a particular definition of D S P . The point is that it applies condition (1.8.1). The more general definition (e.g., (2.5.1) in Arnold, 1974) does not include it. However (e.g., (2.5.2) in Arnold, 1974; L e m m a 4.1 on p. 114 of Friedman, 1975; p. 36 of Kloeden and Platen, 1995), Eq. (1.8.1) presents the condition which enables one to use the whole Euclidean space R^ as the integration domain in (1.8.2) and (1.8.3). This allows to read the integrals therein as the conditional expectations (see above). Also note that condition (1.8.1) is usually provided by solutions of ISDEs (see Section 1.9), important examples of DSPs. In m a n y engineering fields (e.g., Arnold and Lefever, 1981) these equations and D S P s describe the phenomena crucial for the overall performance of the systems. Applications of D S P theory to various real-world problems can also be found, for example, in Barucci and Landi, (1996), E r m a k and M c C a m m o n (1978), Gardiner (1994), Krylov (1980), Ricciardi (1977), V a n K a m p e n (1981), M a m o n tov and Willander (1997b), M a o (1997), M a r u y a m a , (1977); Wilmott, D e w ynne and Howison (1993). If diffusion process % with drift function g and diffusion TV is homogeneous (see Definition 1.9 in Section 1.5), then (e.g., (2.6.8) in Arnold, 1974) these functions are independent of time, i.e. g(f,x) s g(x) and //(;, x) =#(x). In this case, relations (1.8.1)-(1.8.4) are reduced to the following ones: lim - f li^-jc i l ^ p ^ , A , x J ^ = 0 , A*° A ^

for all .x ER'',

(1.8.5)

lim — f(x^-x )p^(x ,A,xJ<^=g(.x ),

for all X E R^,

(1.8.6)

7n?rodMcfo?y Chapter

40

lim — f(Xt*.x )(j;^-^ )^p.(^ ,A,JcJ<^^=7/(^ ), A^O A -*^ R"

foralljc.GR^,

(1.8.7)

matrix 7f(jc) is symmetric non-negative definite for all * e R^.

(1.8.8)

A simple example of homogeneous diffusion process is the Wiener process (see Definition 1.10 in Section 1.5). This follows from (1.5.9) and (1.8.5) -(1.8.7). T h e corresponding drift vector is zero and diffusion matrix is 7. The more general examples are considered in the next section. 1.9

E x a m p l e of Diffusion Processes: Solutions of Ito's Stochastic Ordinary Differential Equation

W e first note the following fact. Transition probability density (A.4) where (f ,.x) and (f,jc) are replaced with (f_,x_) and (f^,*J respectively verifies Eqs. (1.8.1)-(1.8.3) where N(f_,x_) s= 0. This m e a n s that D S P s with zero diffusion matrix can be associated with the deterministic transition (see the text below (A.4)) and granted by solutions of the initial-value problem for O D E system (A.l). This naturally poses the question: can D S P s with nonzero diffusion matrix be described as solutions of an initial-value problem for the equation system which, of course, differs from Eq. (A.l) but is in some respects similar to it? The answer is affirmative and is provided by the equations discussed below. Let us consider equation < < x^i^+^f.x^f+f^f,*)^^),

forall;e/:f>f„,

(1.9.1)

with initial condition (1.4.25). Vector jc in (1.4.25) denotes (see (1.4.22)) values of random variable Y described in Section 1.4.2. Real vector function g and real matrix function A in (1.9.1) are assumed to be defined on / X R ^ where, as in the previous sections, 7 c R is an open interval. The n u m b e r of columns of matrix A(;,.x) in (1.9.1) is equal to the number of entries of the Wiener-process vector W(i;,f) which can be one-dimensional or

Example o/*D:/^Ms:oM Processes.' So/u^oy:s o/'7^o's E q u a ^ o n

41

multidimensional. Solutions of initial-value problem (1.9.1), (1.4.25) are seeked in the form of (1.2.12) where stochastic process % satisfies initial condition (1.4.24). Equation (1.9.1) is called Ito's stochastic differential equation (ISDE) system (due to the pioneering works by mathematician Kiyosi Ito). Since it presents the stochastic generalization of deterministic O D E system (see (A.l)), w e also term it Ito's stochastic ordinary differential equation (ISODE) system. This helps one to distinguish Eq. (1.9.1) from the stochastic generalizations (in the Ito sense) of deterministic P D E s and partial integro-differential equations (PIDEs) (discussed Chapter 5) which are also well-known. Equation (1.9.1) is usually written in the following symbolic, more compact form J.x=g(f,.x)<&+/:(;,*);7W(^,;),

forall;<E7:;>;,.

(1.9.2)

Note that this symbolic equation is understood mathematically only as Eq. (1.9.1). T h e autonomous version of (1.9.2) is (see also (6.3.5) of Arnold, 1974) Jx=g(x)<&+/:(.x)JW(^;), forallfe/:f>f,.

(1.9.3)

S o m e researchers and engineers apply equation

— =g(f,x)+ /:(;,*) ^ * ^ ) ,

forallfe/:;>;.

(1.9.4)

instead of (1.9.2). Model (1.9.4) at J = 2, (^^-independent function A and ; -independent and linear in x function g w a s first used by Paul Langevin (1908). It w a s the first stochastic differential equation in the literature. However, in view of R e m a r k 1.9 (see Section 1.5), Eq. (1.9.4) should be studied in terms of the generalized processes. This is also emphasized by the well-known fact (e.g., (7.2.6) Arnold, 1974) that solutions of Eq. (1.9.2) are as a rule nondifferentiable in f so, usually, the left-hand side of (1.9.4) cannot be regarded as a c o m m o n stochastic process. The mentioned features significantly complicate analysis of (1.9.4) compared to Eq. (1.9.2) which, in contrast to (1.9.4), can be treated in terms of c o m m o n stochastic processes. T h e proper, generalized-process treatment of (1.9.4) is especially complex in the most practically interesting case as functions g, A are nonlinear in x (one of the discussion on this topic can be found, for instance, in Sections 2.3 and 3 of Streit, 1996). Luckily, multiplication of Eq. (1.9.4) by d; trans-

42

/n&YMfKc^ory CAapfer

forms it into (1.9.2) thereby enabling one to avoid the unnecessarily complications. R e m a r k 1.12 (e.g., pp. 55-56 of Arnold, 1974) White noise 6?W(5,;)/d; is associated with the stationary D S P with drift vector g(x) = (-l/-c)x and diffusion matrix 7/(.x) = (1 /-r)7 where scalar T > 0 is independent of (f, * ) . Its covariance and spectral-density matrixes are [exp(-IAI/T)/(2i;)]Z and 2[1+(TM)2] i/ respectively. Generalized process JW(i;,f)/^f presents the limit case of the above D S P as T J, 0. W e also note that precisely the same covariance and spectral-density matrixes correspond to the random telegraph signal (e.g., p. 644 of Papoulis, 1991) with the signal levels 1//21 and -l//2r of probability 1/2 each and expectation -r/2 of the time interval between the level switching. The above results at sufficiently small T > 0 can be used to construct approximate sample functions of white noise. System (1.9.2) (or (1.9.3)) is the stochastic model widely used in engineering and applied sciences. This model, especially in high-dimensional state space (see (1.2.14)), is well-known since long ago in computational chemistry (e.g, Ermak and M c C a m m o n , 1978; Espanol and Warren, 1995; Groot and Warren, 1997). The authors of the present book apply (1.9.2) (and (1.9.3)) to model noise in semiconductor systems (e.g., Mamontov and Willander, 1995b, 1997a, 1997b; Willander ef a?., 1996). A n example of the research software tools based on (1.9.2) are the packages for transient and steady-state modelling of noise in nonlinear semiconductor circuits described by Casinovi and H o (1996) and Schein and Denk (1998). These tools are, however, semiheuristic since they apply such expressions for function A which are not consistently derived from the basic physics of stochastic phenomena in semiconductor devices. The commercial software tools which employ model (1.9.2) can be examplified with HyperChem 6 (see U R L : http:// www.hyper.com/products/description/hyper6.htm) by Hypercube, Inc. (Gainesville, FL, USA), a molecular modelling package for chemical engineering, research and education (available for the Windows 95/98/NT operating systems). Another commercial software for computational chemistry also based on (1.9.2)^ is the C^*DPD package (see U R L : http://www.msi.com /materials/cerius2/DPD.html; see also Espanol and Warren, 1995; Groot and Warren, 1997) by Molecular Simulations, Inc. (San Diego, CA, U S A ) available only for the SGI and I B M workstations. All the above software tools are based on the direct numerical simulation of the stochastic solutions (in

Example o/'Dt/^HSMn Processes.* No^M^to^s o/'/M's Equafton

43

the vein of, for example, Kloeden and Platen, 1995), so the corresponding computing-time expense are normally very high (the latter aspect is discussed in Section 1.11). R e m a r k 1.13 O n e of the most c o m m o n physical interpretation of I S O D E system (1.9.2) concerns particles of a uniform fluid (e.g., E r m a k and M c C a m m o n , 1978; Lowen, Hansen and Roux, 1991; Klimontovich, 1994; Espanol and Warren, 1995; Mamontov and Willander, 1995b, 1995c, 1997d; Willander and Mamontov, 1999) and is as follows. Vector x is read as the particle-momentum vector, term g(f,.x) J; is associated with the differential of the particle scattering (or friction) force and other forces which do not explicitly depend on chance ^, term A(;,;c)-iW(^,f) is due to the differential of the forces acting on a particle by its surrounding in the fluid. W e only add to this picture that the latter forces are associated with the multiparticle nature of the fluid and propagation of sound waves in the fluid, so term A(f,jc) is proportional to the velocity of these waves. The corresponding specific examples are also included in the present book. They are considered in Appendix C, Sections 4.8 and 4.10. W e also note that, if in the above picture drift vector g(f, x) is neglected and A(;, x) = i/2D / where scalar D > 0 is independent of (f, x), then solution of I S O D E system (1.9.2) with initial condition (1.4.36) (see also (1.4.37)) is k n o w n as Brownian motion. The reason for this term is that such solution describes extremely irregular trajectories of a particle suspended in a liquid, i.e. the phenomenon first systematically investigated by botanist Robert Brown (1828). R e m a r k 1.13 not only includes the specific example but also stresses the following general issue. R e m a r k 1.14 Equation (1.9.2) without the second term on the righthand side coincides with O D E (A.l) which is usually used in studies of the real-world systems to model their macroscopic variables (including line, surface or volume densities of the variables distributed in the physical space). The mentioned term in (1.9.2), due to the explicit presence of the stochastic process (namely, W(i;,;)), stands for the related microscopic phenomena not involved in (A.l). Thus, I S O D E model (1.9.2) can be regarded as a combined, macroscopic-microscopic or, more briefly, mesoscopic description [e.g., see the discussions in Lindenberg and Seshadri (1981), Serra, Andretta, Compiani and Zanarini (1986), Bocquet (1998)]. It enriches the macroscopic

44

/nfrodMcfory Chapter

modelling with at least some microscopic effects being, in so doing, m u c h simpler than the corresponding microscopic modelling. In other words, the mesoscopic approach presents a compromise between a relatively inaccurate and simple purely macroscopic treatment and a more accurate but m u c h more complex purely microscopic treatment. The above suffix "scopic" pertains to the level of detail of the modelledphenomena description. A lot of topics in I S O D E s are well-developed. They include the corresponding qualitative theory (e.g., Arnold, 1974; Friedman, 1975,1976; Has'minskii, 1980; M a o , 1997; G i h m a n and Skorohod, 1972; Skorokhod, 1965) and quantitative techniques (e.g., Kloeden and Platen, 1995). In particular, it is well-known (e.g., (9.3.1) in Arnold, 1974; Theorem 4.2 on p. 115 of Friedman, 1975; Theorem 4.6.1 in Section 4.6 of Kloeden and Platen, 1995) that, under rather mild conditions, Eq. (1.8.1) (or (1.8.5)) holds for solutions of system (1.9.2) (or (1.9.3) respectively) and these solutions are D S P s with drift function g and diffusion function 77 where matrix 77(f,%) in (1.8.3) is coupled to matrix A(f,.x) in (1.9.2) as follows 77(f,*) = /;(;,.x)[A(;,.x)]T,

forall^e./xR''.

(1.9.5)

The version of (1.9.5) corresponding to (1.9.3) is 7f(jc) = A(jc)[A(%)]T,

for all ^ E R ^ .

(1.9.6)

Thus, in case of (1.9.2), (1.9.5) (or (1.9.3), (1.9.6)), D S P s can be studied by m e a n s of methods of both D S P and I S O D E theories (e.g., Arnold, 1974; Friedman, 1975, 1976; Kloeden and Platen, 1995). Moreover, m a n y works on the qualitative aspects (e.g., Ikeda and Watanabe, 1989; Ito and M c K e an, 1974; Kryiov, 1995; Strook and Varadhan, 1979) analyze D S P s solely by means of the I S O D E treatment. However, in some applications, for instance, described in Mamontov and Willander (1997b) (and related works Willander e? a^., 1996; Mamontov and Willander, 1997a) diffusion function 7/ is k n o w n whereas function A is not known. Determination of A from Eq. (1.9.5) (or (1.9.6)) is not always simple and practically efficient (e.g., Freidlin, 1968; Section 1 in Chapter 6 of Friedman, 1975). O n e of the main difficulties is to construct such solution A of (1.9.5) (or (1.9.6)) that would satisfy the well-known conditions (e.g., (6.2.2) and Section 6.3 of Arnold, 1974) for existence and uniqueness of solutions of (1.9.2). This is a problem inherent in the I S O D E description (1.9.2), (1.9.5). Unlike this, D S P treatment

Examp/e o/'Dt/^MSton Processes; RofUptons o/'AS's EtyMa?:on

45

does not force to solve (1.9.5) for matrix A(f,.x) that is a valuable advantage. The latter also explains w h y the present book proceeds with drift function g and, diffusion function /f, does not solve Eq. (1.9.5) (or (1.9.6)) for matrix A(f,.x) (or A(.x)) and does not apply the I S O D E theory to analyze DSPs.

(1.9.7)

A n additional benefit of our purely D S P approach is that it presents a more uniform and compact treatment than the combined, D S P - I S O D E description. This fact m a y facilitate acceptance of the proposed techniques by nonspecialists in I S O D E theory, in particular, by nonmathematicians. Since D S P s are in m a n y cases granted by solutions of Eq. (1.9.2), w e point out the following fact of both theoretical and practical importance. R e m a r k 1.15 O n e of the sufficient conditions for existence and uniqueness of solutions of initial-value problem (1.9.2), (1.4.25) is that there is a n u m b e r <$>0 independent of (f,x) such that llg(^jc)f+ tr{A(f,A;)[A(f^)]T}< <^(l+l^f).

(1.9.8)

This inequality presents the restriction on growth of functions g and /: and can be regarded as the linear-growth condition. If it does not hold, then the so-called explosion of the solutions can take place (e.g., pp. 112-113 of Arnold, 1974). Specialists working in applied fields often ignore this warning. For instance, they include the term proportional to the third power of entries of vector jc in drift vector g(f,Jt) thereby violating condition (1.9.8). The first thing which should be undertaken in this situation is to m a k e sure that deterministic version (A.l) of (1.9.2) is free from the explosions. If this is not the case, then a good idea is to correct functions g and A in a complete accordance with (1.9.8). There is a family of the stochastic differential equations (SDEs) (e.g., (1.8) on p. 46 of Skorokhod, 1965) which are more general than I S O D E (1.9.2) and still related to Markov processes. A n S D E of this kind compared to (1.9.2) includes one more term on the right-hand side. This term serves to model discontinuous and hence nondiffusion stochastic processes. Because of this, such S D E s are beyond the scope of the present book devoted

46

ZnfrotfMCfory C/ntpier

to diffusion processes. W e only mention that theory of the above SDEs (e.g., Chapter 3 of Skorokhod, 1965) is more complex than ISODE theory and offers less options for the corresponding practical treatments. Diffusion processes can be examplified in various ways. Solutions of I S O D E systems present a very important still particular option. The next section continues the topic of Section 1.8.

1.10

T h e Kolmogorov Backward Equation

Drift vector g(;,jt) and diffsuion matrix 77(;,^) of diffusion stochastic process X on interval 7 (see Definition 1.13 in Section 1.8) together with diffusion relations (1.8.1)-(1.8.3) enables one to determine the corresponding probability densities. This can be done in different ways. The present section considers one of them. Theorem 1.1 (pp. 104-105 of Gihman and Skorohod, 1972) Let x be a D S P on interval 7 with transition probability density p(f_,.x_,^,xj and drift and diffusion functions g, 77 (see (1.8.2), (1.8.3)). W e assume that the assumptions below hold. (1) Quantity p(^ ,A; ,f ,j: ) is continuous and bounded in (f ,jc ,f J for all f ,;^E7 such that f-^ >se, at arbitrary fixed n u m b e r ae.>0. (2) For any fixed f^€ 7 and all f € 7 such that f < ^, functions 3p/3f_ , V p and d^p/dx .9.x_,, A:,/ = l,2,...,<%, exist where column V, = (3/d^,3/3*2,...,3/3.xjT

(1.10.1)

is the Hamilton operator. W e also denote X ( ^ ) = [g(^)]^+(l/2)tr[R(^)^], forall(^,^)e7xR^.

(1.10.2)

Then transition probability density p(f_,x_,^,xj, as a function of (?_,x_) S/xR'' where f_<^ at any fixed (^,xje7xR^, is the solution of the Kolmogorov backward equation (KBE) (see (125) in Kolmogorov, 1931)

7%e JMmogorou BacAt^ctrd E^Ma^to?:

47

^+X(f_,;c_)p = 0, 3f_ for all (^_,^_) e /x R': ;_< ;^,

(1.10.3)

under final condition lim p(^_,jc ,f^,jcj = 8(jc^ ^ _ ) , forall^_eR^,

(1.10.4)

i.e. the fundamental solution of the Cauchy problems for K B E (1.10.3). The above term "backward" is due to the fact that Eq. (1.10.3) describes development of function p in the course of decrease in preceding time point f_< ^ at fixed ;^. Final condition (1.10.4) is the reverse version of initial condition (1.4.13). Note that the Cauchy problems for Eq. (1.10.3) are formulated on the whole space R*', not on its bounded or unbounded domain in it. If process % in Theorem 1.1 is homogeneous, then result (1.10.3), (1.10.4) are reformulated in terms of homogeneous transition probability density p^ (see (1.5.2)) as follows. Corollary 1.1 Let hypothesis of Theorem 1.1 be valid, process % is homogeneous, and its transition probability density is p. described with (1.5.2) at A =oo. In this case, drift and diffusion functions g and /7 are independent of f (see the text above (1.8.5)), so differential operation (1.10.2) also becomes time-independent, i.e. ^ M = [gM]^^+(^2)tr[77(^)^],

for all ^ S R ^ .

(1.10.5)

W e assume that the assumptions below hold. (1) Quantity p^(x_,A,.xJ is continuous and bounded in (x_,A) for all A > ^ at arbitrary fixed number se. > 0. (2) For all A > 0 , functions 3p^/3A, V, p^ and d2p/3x_^3x_,, %;,; = i,2,..,d, exist. Then transition probability density p,,(jc_,A,.xJ, as a function of x € R*' and A > 0 at any fixed x^€ R^, is the solution of autonomous K B E

--^+X(.x)p,, = 0, 3A

forall^,^eR^, A > 0 ,

(1.10.6)

under condition (1.5.7), i.e. the fundamental solution of the Cauchy pro-

48

7n?rodHcfory CAapfer

blems for K B E (1.10.6). It follows from Corollary 1.1 that the versions of autonomous K B E (1.10.6) for the Wiener process (see Definition 1.10 in Section 1.5) and Brownian motion (see Remark 1.13 in Section 1.9) are

^H^-p*.

^r^-p,

d.io.7)

respectively. The latter one is the standard diffusion P D E where scalar D is diffusion parameter. Both equations are the simplest examples of (1.10.6). The above considerations demonstrate h o w drift and diffusion functions of a D S P can be used to determine its transition probability density. Another Kolmogorov's equation describing development of p in the course of increase in subsequent time point f,^f_ is discussed in Section 1.12.

1.11

Figures of Merit. Diffusion Modelling of High-Dimensional Systems

A s is mentioned in Section 1.9, if a D S P is described as a solution of I S O D E system, it is often constructed in the form of its sample functions. However, individual sample functions do not give m u c h to researchers or designers of the modelled systems. The point is that, as a rule, it is neither possible nor even necessary to formulate the key features of the system in terms of the details of extremely irregular behavior of the sample functions. Stochastic engineering system is usually designed in terms of, and characterized with, a comparatively small number of the deterministic parameters which are crucial for the system operating, performance and quality. These parameters are called figures of merit of the system. If the system is described with stochastic processes, then the figures of merit comprise deterministic quantities related to the processes. These quantities include the figures of merit which this book deals with. The above sample functions are obtained numerically, with the help of computer software, random-number-generating programs and the Monte Carlo algorithms. The focus on the figures of merit necessitates to aggregate a huge amount of data granted with the sample functions in a relatively small number of the core parameters. This is usually done by means

49

Figures o^Mer:?. H:g/:-Dtmensto?M/ Sys^e/Hs

of statistical processing of the sample functions that, along with their generation, is also purely numerical. The above purely numerical approach is typical w h e n the D S P s are granted as solutions of I S O D E systems (e.g., Casinovi and Ho, 1996; Schein and Denk, 1998). Let us consider h o w well such approach is suitable for research and engineering design associated with high-dimensional D S P s , i.e. under condition (1.2.14). To do that, w e point out the following two issues. First, the most important figures of merit of Markov process % usually include the quantities below: (1) the expectation vector ((1.6.11), (1.7.7) or (1.7.17)) and the variance matrix ((1.6.13), (1.7.8) or (1.7.18)); they also present the data to evaluate other important parameters, for instance, the corresponding S N R s (see Appendix B ) ; (2) the covariance matrix ((1.7.26), (1.7.20), (1.7.28)) and the spectraldensity matrix ((1.7.30), (1.7.31)) if a D S P is stationary; they also present the data to evaluate other parameters, for instance, the corresponding coherence functions (see the text below (1.7.31)). Second, high-dimensional DSPs, i.e. under condition (1.2.14), are used in various fields [e.g., E r m a k and M c C a m m o n (1978) and the references therein, Espanol and Warren (1995), Mamontov and Willander (1997b)]. However, the mentioned condition is not the only one typical in engineering and applied research. The other requirements are as follows: multiple analysis of the quantities in the above Points (1) and (2) should be efficient on the workstations of a low cost of purchase and ownership,

(1.11.1)

the main computer m e m o r y necessary for analysis of the quantities in the above Points (1) and (2) under condition (1.2.14) must be limited by the amount proportional to J^,

(1.11.2)

inequality T,„,.^-r^, holds,

(1.11.3)

where T.„, and T^, are the characteristic times of the internal changes in the system and the external driving signals respectively. Features (1.11.1) —(1.11.3) are discussed in Sections 7.2—7.4 respectively. Condition (1.11.3)

50

/nfrodMc^ory CAapfer

can help to overcome the difficulties arising from (1.2.14), (1.11.1) and (1.11.2). W e employ (1.11.3) in Section 3.5.2. To evaluate the figures of merit in Points (1) and (2) with an acceptable accuracy of the statistical processing once, one usually needs a few thousands or tens of thousands of the sample functions. This task definitely seems to be time-consuming if the D S P is high-dimensional, i.e. if condition (1.2.14) holds. Multiple analysis (1.11.1) prescribes to determine the figures of merit m a n y times. The system researcher or designer m a y want to evaluate them, say, a few tens or hundreds of times within the system optimization cycle. This drastically increases the total computing time, especially if the c o m m o n workstations (see (1.11.1)) are applied. Condition (1.11.2) imposes the additional requirements on efficiency of the computational algorithms and even more complicates the matter. Thus, the above purely numerical approach turns out to be not very well suited for engineering applications. A possible alternative m a y be incorporation of an appropriate analytical results into a purely numerical treatment or, in other words, to solve some parts of the computational problem analytically thereby eliminating their rather time-consuming numerical solving. The analytical-numerical approach (see Section 8.1) can in some applied fields turn out the only reasonable option even if its overall accuracy is worse than that of the numericalonly alternative. O n the w a y to the combined treatment, there are a few natural questions. *

W h a t are the advantages of the analytical-numerical approach to engineering problems? * Which analytical results of c o m m o n D S P theory might in principle be used as the analytical basis for the above approach? * H o w well do they meet requirements (1.2.14), (1.11.1) and (1.11.2)? * W h a t can be done if the answer to the latter question is not reassuring and h o w this is related to the purpose of the present book? The above four topics are discussed in Sections 8.1, 1.12, 8.2 and 1.13 respectively. So, before reading Sections 1.12 and 1.13, one is suggested to read Sections 8.1 and 8.2 respectively.

Ana^yftcaJ Tec^ntqMes. TAe Xb^mogorou Forward EqMafton

1.12

51

C o m m o n Analytical Techniques to Determine Probability Densities of Diffusion Processes. T h e K o l m o g o r o v F o r w a r d Equation

The figures of merit in Points (1) and (2) in Section 1.11 apply the following probability densities: p,, p,, p^ and p^. They can in principle be determined with the help of the c o m m o n analytical results discussed in the present section. According to Theorem 1.1 and Corollary 1.1 (see Section 1.10), K B E s (1.10.3) and (1.10.6) provide the w a y to determine the transition densities p and p^. However, they can also be obtained by means of the corresponding Kolmogorov forward equations (KFEs). Moreover, the latter describe other densities such as p„ (see (1.4.31), (1.4.30)), p. (see Definition 1.11 in Section 1.7) and p^ (see Definition 1.12 in Section 1.7). The above equations are of a key importance since they grant the formulas for the figures of m e rit (see Points (1) and (2) in Section 1.11). It is thus not surprising that K F E s are in the core of the c o m m o n analytical techniques for D S P probability densities. The present section discusses some basic results on K F E s .

J.i2.J

f r o & a M H f y denstfy

Denisty p, is determined by means of transition probability density p by means of (1.4.31). It can also be described in such a w a y that does not involve p , namely, according to the following theorem. T h e o r e m 1.2 (e.g., p. 377 of G i k h m a n and Skorokhod, 1969); (2.6.9) in Arnold, 1974). Let x be a D S P on interval / with transition probability density p(f ,x ,;^,.xj and drift and diffusion functions g, # (see (1.8.2), (1.8.3)). Let the assumptions below be valid. (1) Limit relation (1.8.1)-(1.8.3) hold uniformly with respect to (^ ,;c )

e/xR^. (2) For every fixed (; ,.x_)e7xR'', quantities 3[g^,xJp(;_,.x_,^,.xJ]/d.x^, 3^[#R,(f,,xJp(f_,x_,^,.xJ]/dx^8x^, %c,/ = l,2,...,J, 3p(f_,.x_,^,.xJ/d^ are continuous in (^,xj€/xR^ where f^>f_. Let differential operator [A^(f,x)]* be the formal adjoint of operator (1.10.2). This, in terms of density p, means that [^,^)]*p(^_,^,^,jcj=-^[^(^,j!:jp(^,x_,^,jcj],

(1.12.1)

52

7n^roduc^o7y C7:ap?er

where S(^,j;Jp(f_,^,^,^) = [S^^,^)p(r_,j; ,^,A;J,...,S^(^,^)p(^,^_,^,^)]T,

for all A; = l,2,...,^, (r^xJeVxR^: f^>;_.

(1.12.2)

Then the following assertions are valid. (1) Quantity p(f_,x ,^,jcj, as a function of (f+,%J, is the solution of K F E (see (133) in Kolmogorov, 1931)

-^ = [^*,)]*P, for all (^.jcjE/XR'': ^ > f ,

(1.12.3)

under initial condition (1.4.13), i.e. the fundamental solution of the Cauchy problems for Eq. (1.12.3) under additional conditions (1.4.9), (1.4.10). (2) Quantity p,(f^,f,x) (see (1.4.31)), as a function of (f,.x), is the solution

of KFE 3p. - ^ = [X(;,*)]*p„ for all (r,x)eyxR^: ;>;^,

(1.12.4)

under initial condition (1.4.30) and additional conditions (1.4.27), (1.4.28). The smoothness of density p, is the same as that of transition density p in assumption (2). T h e above term "forward" is due to the fact that Eq. (1.12.3) describes development of function p in the course of increase in subsequent time point f^f_ at fixed f . Note that the Cauchy problems for Eq. (1.12.3) are formulated on the whole space R'' (rather than on a bounded or unbounded domain in it). In so doing, condition (1.4.9) points out one of the features of the function family where the solutions are seeked. Condition (1.4.10) can be regarded as the corresponding boundary condition (at the boundary that, for the whole space, is in infinity).

Ana^y^caJ TecAnMyMes. TAe 7&^mogorou Forwarc! Efyua^on

53

The Kolmogorov forward equation (1.12.3) is also known as the Fokker -Planck equation (due to physicists Fokker (1914) and Planck (1917) where the former investigated Brownian motion in a radiation field and the latter attempted to build a complete theory of fluctuations based on it). Application of K F E to various problems in research and engineering are described in m a n y books, for instance, by Risken (1989), Soize (1994), Grasman and V a n Herwaarden (1999). The readers inclined to look at K F E from the statistical-mechanics viewpoint will find the corresponding derivation of it in Section 11.8 of Balescu (1975), §21 of Lifshitz and Pitaevskii (1993) or Section 4 of Chapter IX of Resibois and D e Leener (1977). Vector (1.12.2) is called probability flux (or probability current) corresponding to probability density p . Notion of probability flux was introduced by W . Feller (see (10.3) and Section 10 of Feller, 1954). The term "probability current" is used, for instance, in Soize (1994) (e.g., see p. 190 therein). Assertion (2) of Theorem 1.2 presents the general recipe for evaluation of density p„. Note that K F E (1.12.4) is formulated in terms of the space derivatives ofthe products of functions g^,^,, A:,/= 1,2, ,6?, p rather than in terms of the space derivatives of these functions themselves. So, generally speaking, this equation is not P D E . To simplify the matter, i.e. to m a k e it P D E , assumption (2) of Theorem 1.2 is usually strengthened to the corresponding smoothness of each of functions # % , % ? , p (e.g., Theorem 1 on p. 102 of G i h m a n and Skorohod, 1972; Theorem 1 on p. 1205 of Mamontov, Willander and Lewin, 1999). There are, however, applied problems where it is essential not to involve these simplifications. O n e such example is mentioned below Remark 1.18 in Section 1.12.3 and is considered in Appendix C. If the above strengthening can be used, its advantage is that it enables one to transform the structure of [X(;,.x)]* (see (1.12.1)) into a more convenient form. The following remark considers one of the corresponding examples. R e m a r k 1.16 If hypothesis of Theorem 1.2 is valid and, besides, functions ^7 and p are differentiable with respect to jc, then the probability flux can be presented as follows

3(f...xJp(^,.i-^*J = C(^,A:Jp(f_,x_,;,,.yJ, (^,jcje7xR^: ^>f_, where

(1.12.5)

54

/nfroducfory CTtapfer

G(^,A;) = F(^^)-(l/2)77(^%)^,

(;,x)e/xR^,

(1.12.6)

F(^) = (^(r^),^(^),...,F,(^)f, F^^)=g,(^)-i^ ^ M , 2 ;=j

for all ^=1,2, .,J, (f,Jc)e/xR''.

OAT.

(1.12.7)

Vector (1.12.7) is sometimes called the Fichera drift vector (e.g., (7) on p. 191 of Soize, 1994). In practice, density p, is determined according to assertion (2) of Theor e m 1.2, i.e. as the solution of K F E (1.12.4) with initial condition (1.4.30) and probability-densities conditions (1.4.27), (1.4.28). In so doing, operator [X(f,jt)]* in (1.12.4) can sometimes be described as pointed out in R e m a r k 1.16.

Curiously, there is very little literature on invariant probability densities of invariant, generally nonstationary D S P s . T h e rare works on this topic, however, include important results. The brief s u m m a r y is presented in R e m a r k 2 on pp. 140-141 of Has'minskii (1980) and concerns Theorem 5 in Il'in and Has'minskii (1965). It is discussed below. In terms of invariant D S P % with the properties described in Definition 1.11 (see Section 1.7) and with drift and diffusion functions g and 7f, the main results of the above H'in-Has'minskii theorem are as follows. (1) Under rather mild conditions, invariant probability density p of % is determined according to the limit relation p.(;,x)=limp,(f„,f,.x),

forall^eR^,

(1.12.8)

where p,(f^,f,x) is the solution of the Cauchy problem (1.12.3), (1.4.30) under additional conditions (1.4.27), (1.4.28) and the limit is independent of initial probability desnity p in initial condition (1.4.30). (2) In so doing, density p, is the unique solution of K F E (1.12.3), i.e. -L
forall^jtjeR^,

(1.12.9)

A?m/y?:caJ recTtntgMes. TAe Xo/mogoroM Forward EgMaiton

55

of the following property: solution p. is bounded in a certain sense (see below) and satisfies conditions (1.7.1), (1.7.2) where the convergence of the integral in (1.7.2) is uniform in f . (1.12.10) F r o m practical viewpoint, an important advantage of K F E (1.12.9) compared to relation (1.12.8) is that it does not involve evaluation of the limit on the right-hand side of (1.12.8). Point (1) rigorously formulates the property which seems to be typical in m a n y engineering systems. Namely, in the course of moving initial point ^ from f, density p,(f„,f, *) is "forgetting" its initial state p^( -) and converges to state p (f, -) which, thus, can be considered as the steady one. This bevavior is qualitatively the same as the one pointed out by the deterministic versions (A.7),(A.12)of p., p. respectively and deterministic relation (A.ll) (see the text below (A. 12) d o w n to the end of Appendix A). T h e difference is that (1.12.8) presents the stochastic generalization of the deterministic property (A.ll). Features (1.7.9) and (1.7.10) can also be regarded as the limit cases of (1.6.15) and (1.6.16) respectively as f -*-oo. T h e above discussion points out that invariant D S P s present the steady-state modes of the systems modelled with DSPs. In so doing, invariant probability densities are the steady-state probability densities. The boundedness in (1.12.10) is treated in the Il'in—Has'minskii theorem as the boundedness uniform with respect to (f.j^ER^. T h e authors of the theorem emphasize the importance of the uniform convergence in (1.12.10). T h e remark below is devoted to this topic. R e m a r k 1.17 (Remark on p. 260 of Il'in and Has'minskii, 1965) G e n e rally speaking, K F E (1.12.9) can describe more than one invariant probability density, i.e. can have more than one solution satisfying conditions (1.7.1), (1.7.2). For instance, for one-dimensional (^ = 1) homogeneous D S P s with drift and diffusion scalars g(x) = -;t and #(.i) = 2, K F E (1.12.9) is written as (see (1.12.1))

3^3(^)6^ 3f 3* 3*2' This equation has the following two solutions

^ ^

, ^

(H2.ll)

56

Z?t?rodMC?ory C/tapfer

P,.i(^^) = ^ = exp - — , y2-rc \ ^7

P;.2^^) = ^ = exp - -^ y2-!t \

—-^- , ^ 7

forall(^jc)eR^.

(1.12.12)

Each of them is uniformly bounded in (;,jc) and is a probability density since meets requirements (1.7.1) and (1.7.2). However, density p.. has property (1.12.10) whereas density p ^ does not. Thus, condition (1.7.2) alone does not determine the unique bounded solution of K F E (1.12.11). This is the key point. T h e unifrom convergence in (1.12.10) is needed to provide the uniqueness. Regarding Point (2) and R e m a r k 1.17, w e note the following. Theorem 1.2 on density p,(f^,f,.x) in (1.12.8) does not point out that this density must be uniformly bounded with respect to (^,jc)ER^^. Moreover, a probability density generally need not be bounded in this sense. Another kind of boundedness is likely more natural for it. A s is noted above, Eq. (1.12.8) is the probabilistic version of the deterministic property (A.ll). In view of this and (A. 12), the probabilistic versions of the boundedness in (A.10) are feature (1.7.9) and similar to it feature (1.7.10). Thus, it is more relevant to consider boundedness of p. in the sense of properties (1.7.9) and (1.7.10) of quantities (1.7.7) and (1.7.8). Interestingly, expectation (1.7.7) for the above density p ^ (see (1.12.12)) which satisfies (1.12.10) does have property (1.7.9) whereas the expectation for density p,. ^ which does not satisfy (1.12.10) does not have property (1.7.9) either. It is not unprobable that the Il'in—Has'minskii theorem can be modified in such a w a y that the uniform boundedness of desnity p. would in principle be weakened to (1.7.9) and (1.7.10), i.e. to uniform boundedness of the expectation (1.7.7) and variance (1.7.8) corresponding to the density. This can be the topic to be investigated rigorously in a separate research. In what follows, w e do not consider p,. to be uniformly bounded in ( f ^ E R ^ . Since vector S(f,x)p,(f,.x) is probability flux (see Section 1.12.1), equality S(;,*)p,(f,*)=0,

forall(f,jc)eR^,

(1.12.13)

is called the detailed-balance (DB) equation. T h e term "detailed" is due to the fact that every entry of the probability-flux vector is zero. This condition is employed in Chapter 3. In the particular case as the D S P is stationary, the corresponding version of the D B equation is Eq. (1.12.18). It is used,

57

Ana/yftcct/ Tec/m^Mes. TAe Xb^mogorou Forward EaMa?:o7:

for instance, on page 191 of Soize (1994). Invariant probability density of stationary D S P is considered in the next section.

Stationary D S P is characterized with its transition and stationary probability densities (see (1.7.12)-(1.7.16)). They are involved in (1.7.26) and hence are needed to determine the figures of merit in Point (2) in Section 1.11. If a D S P is stationary, its transition density is homogeneous (see the text above (1.7.12)) and drift and diffusion functions are independent of time (see the text above (1.8.5)). In this case, the following particular version of Theorem 1.2 on the transition density can be used. T h e o r e m 1.3 (e.g., (2.6.9) in Arnold, 1974; pp. 114-118 of Soize, 1994). Let x be a homogeneous D S P on R with transition probability density p^(x ,A,xJ and drift and diffusion functions g, 7? (see (1.8.6), (1.8.7)). W e assume that the assumptions below hold: (1) Limit relations (1.8.5)-(1.8.7) hold uniformly with respect to x € R^. (2) Quantities d[g^(xJp,,(.x_,A,xJ]/ax^, 3^[7^.;(A:Jp^(x_,A,^)]/3^3^;, ^,/ = 1,2, ,6?, 3p,,(x_,A,xJ/3A are continuous in j^€R^ and A > 0 at arbitrarily fixed x e R^. Let [X{.x)]* be the f -independent version of operator [X{f,x)]*. This, in terms of density p., means that [^J]*p^_,A,^) = -V^[S(^)p,(A:_,A,A:J],

(1.12.14)

where

S(^)PA(^-A,^) = [S^Jp^_,A,^), ,^(^)p^,A,^)f, , , , 1 A 3[/4,(.x )p,,(* ,A,jt )1 3t(*JP^,A,xJ=g^Jp,,(*_,A,^)-i^ "^ 1 J * -forallA; = l,2, ^ ^ E R ' , A > 0 .

(1.12.15)

Then quantity p^(jc ,A,jcJ, as a functions of x G R*^ and A > 0, is the solution of autonomous K F E

— = [Ar(jtJ]*p,,, forall^GR'', A > 0 , 3A

(1.12.16)

58

/HfrodMCtory C&tp?er

under initial condition (1.5.7), i.e. the fundamental solution of the Cauchy problems for Eq. (1.12.16) under additional conditions (1.5.3), (1.5.4). Simple examples of K F E (1.12.16) are its versions for the Wiener process (see Definition 1.10 in Section 1.5) and Brownian motion (see R e m a r k 1.13 in Section 1.9). These versions are precisely the same as K B E s (1.10.7). T h e question on the stationary density is solved by the corollary below. Corollary 1.2 (e.g., pp. 121-123 of Soize, 1994) Let x be a homogeneous D S P on R with drift and diffusion functions g, 77 (see (1.8.6), (1.8.7)) and p^ be a probability density (see (1.7.12), (1.7.13)). W e assume that the assumptions below hold: (1) Limit relations (1.8.5)-(1.8.7) hold uniformly with respect to jr E R^. (2) Quantities 3[g^(x)p/x)]/3.x^, 9^[7^.;(^)p/^)]/d^d^, %,7 = 1,2,...,
forallxeR'.

(1.12.17)

(2) T h e inverse statement is also true, namely, if function p^ is the solution of stationary K F E (1.12.17) under probability-density conditions (1.7.12) and (1.7.13) and D S P x is specified with p^ as the initial density (i.e. at po = p^ in (1.4.23)), then process x is stationary and its stationary probability density is p^. Corollary 1.2 presents the general recipe to evaluate stationary density p . M o r e details on this topic can be found, for instance, in Chapter W of Has'minskii (1980). R e m a r k 1.18 Itfollowsfromrepresentation(1.12.14)thatstationaryKFE (1.12.17) holds if the stationary version of the D B condition (1.12.13) holds, i.e. equation S(jc)p/^)=0,

foralljcER^,

(1.12.18)

is valid where the left-hand side presents the version of (1.12.15) corresponding to time-independent functions g, 7V and the stationary density p^. Clearly, assertion (2) of Corollary 1.2 is also true if (1.12.17) is replaced

A?m^y?tca/ reeAntques. TTte Xo/mo^orou Foru^ctrc! EgMafton

59

with (1.12.18). Appendix C presents an example of the problem associated with the physical picture mentioned in R e m a r k 1.13 (see Section 1.9). It is shown that in this example the continuity in assumption (2) of Corollary 1.2 is the case even if stationary density p is unbounded and hence discontinuous. Another application of this corollary demonstrated in Appendix C is the explicit influence of nonlinearities of drift and diffusion functions g, 7/ upon the shape of density p . The results of Appendix C are also applied to the time-domain derivation of the so-called long, non-exponential "tails" of the particle-velocity covariance for the hard-sphere fluid in Section 4.10.3 and are used in Chapter 6 in connection with modelling noise in semiconductor systems. T h e present section reminds about the c o m m o n analytical recipes which can be applied to evaluate the figures of merit in Points (1), (2) in Section 1.11. Let us briefly summarize them. (1) Density p,(f^,f,.x) used in (1.6.11) and (1.6.13) can be determined as the solution of the Cauchy problem (1.12.4), (1.4.30) under the probability-density conditions (1.4.27), (1.4.28). (2) Density p.(f,.x) used in (1.7.7) and (1.7.8) can be determined as the solution of K F E (1.12.9) under conditions (1.12.10), (1.7.9), (1.7.10). (In so doing, the boundedness (1.12.10) can in some problems be associated with features (1.7.9) or (1.7.10) (see R e m a r k 1.17 in Section 1.12.2).) (3) Density p/x) used in (1.7.17), (1.7.18) and (1.7.26) can be determined as the solution of K F E (1.12.17) or the D B equation (1.12.18) under the probability-density conditions (1.7.12), (1.7.13). (4) Density p^(x ,A,xJ used in (1.7.26) can be determined as the fundamental solution of the Cauchy problem (1.12.16), (1.5.7) under the probability-density conditions (1.5.3), (1.5.4). The covariance and spectral-density matrix of a stationary D S P are determined as described in Point (2) in Section 1.11. The question is h o w well the recipes in Points (l)-(4) are suited for the practical applications with requirements (1.2.14), (1.11.1), (1.11.2). This topic is discussed in Section 8.2.

60 1.13

Z?:?rodHcfory CAapfer

T h e P u r p o s e a n d Content of This B o o k

Section 8.2 shows that the c o m m o n analytical recipes in Points (l)-(4) in Section 1.12.3 are not at all suitable for practical applications characterized with (1.2.14), (1.11.1), (1.11.2). In other words, they cannot serve as the analytical basis of the analytical-numerical treatment of high-dimensional DSPs. Points (1) and (2) in Section 8.2 suggest the possible alternatives which can help to resolve the above problem. They focus on the topics summarized below. Briefly, the purpose of the book is to develop the analytical approximations for the figures of merit in Points (1) and (2) in Section 1.11 which can form the analytical basis of the analytical-numerical treatment of high-dimensional D S P s . T h e developed analytical recipes should enable one to computationally implement them in accordance with features (1.2.14), (1.11.1), (1.11.2). In so doing, no limitations on the form of the time- or space-dependences of the corresponding drift and diffusion functions can be involved. T h e obtained results are not intended for one or another specific application field. They are thought to be equally suitable for a wide range of engineering fields related to various applied sciences such as biology, chemistry, economics, mechanics, medicine, physics, sociology. Chapter 2 deals with general nonstationary DSPs. It derives the exact analytical expressions for the time derivatives of expectation vector (1.6.11) and variance matrix (1.6.13) in terms of drift and diffusion functions g, 77 directly from the basic Eqs. (1.8.2), (1.8.3). The O D E systems of different orders for the expectation vector are also obtained. The crucial advantage of these systems (of the order 2 or greater) is that they explicitly include influence of diffusion function TV upon the expectation. Owing to this, they present the general mathematical tools to model various noise-induced phen o m e n a in the expectations such as stochastic resonance, stochastic linearization, stochastic self-oscillations, stochastic phase transitions, stochastic chaos and others. Chapter 3 deals with invariant (nonstationary) DSPs. Involving condition (1.11.3), it develops the efficient (in the sense of (1.2.14), (1.11.1), (1.11.2)) analytical approximation for the invariant probability density. This not only gives a practical meaning to (1.7.7), (1.7.8) (and, in the stationary case, to (1.7.17), (1.7.18)) but also enables one to approximately transform the expression for the time derivative of the variance matrix (1.6.13) de-

7%e Purpose and CoTt^en^ o/° TAts BooA

61

rived in Chapter 2 to the first-order O D E system for this matrix. The latter completes development of the uniform, ODE-based modelling (cf., Point (2) in Section 8.2) for both the expectation and variance. The corresponding practical method is also formulated and discussed. In so doing, its relevance in connection with (1.2.14), (1.11.1), (1.11.2) is stressed. Section 3.6.3 is the s u m m a r y of the proposed method and can be read independently as a guide for the readers interested merely in applying the method. Chapter 4 deals with stationary DSPs. It derives the approximate analytical recipe for the covariance matrix (1.7.26) which (together with (1.7.20), (1.7.28)) can also be used to evaluate spectral-density matrix (1.7.30), (1.7.31). This recipe meets requirements (1.2.14), (1.11.1), (1.11.2). In so doing, the proper attention is paid to the effects of nonlinearities of the drift and diffusion functions. A few physics-related aspects are discussed. T h e development is examplified with the derivation of the so-called long, nonexponential "tails" of the covariance of a particle velocity in the hard-sphere fluid. This derivation is completely time-domain and, to our knowledge, is the first one of this kind. The corresponding recipe for the general case of arbitrary fluid is presented in Section 4.10.4. Chapter 4 also proposed the stationary-DSP version of the method in Chapter 3. Section 4.11.2 presents its s u m m a r y which, similarly to Section 3.6.3, can be read independently as a guide for the readers interested merely in applying the method. Chapter 5 presents more details on the topic pointed out in R e m a r k 1.6 and in the text below it in Section 1.3. Chapter 5 sharpens the idea that Ito's stochastic partial integro-differential equations (ISPIDEs) is a meaningful and flexible tool to model non-Markov behavior in applied problems. In connection with these equations, importance of the stochastic-adaptiveinterpolation (SAI) technique by Professor Bellomo and his co-workers is emphasized and some related improvements are discussed. In particular, the SAI technique enables one to reduce ISPIDE (or a system of these equations) to high-dimensional D S P s considered in Chapters 2^4. Thus, these D S P s are naturally appear in connection with applications of ISPIDEs. O n e specific application of this kind is discussed in Chapter 6. It deals with Ito's stochastic partial differential equation (ISPDE) system which describes an inviscid isothermal isotropic uniform fluid and is applicable to various engineering problems. The capabilities of this system to model stochastic phenomena in both macroscale and mesoscale domains are emphasized. The corresponding limit cases are pointed out. Chapter 6 also show h o w the above I S P D E system can be used to describe noise in various semi-

62

/nfrodMcfory CAapfer

conductor devices. The obtained analytical results are in agreement with the previous results by independent authors derived by means of other theories. Chapters 7 and 8: *

include the discussion on various features of engineering applications of high-dimensional DSPs; * stress the advantages of the analytical-numerical treatment in applied problems; * point out the practical difficulties concerning the c o m m o n analytical results of D S P theory and arising in connection with requirements (1.2.14), (1.11.1), (1.11.2); * suggest possible ways to overcome the above difficulties; these suggestions were taken into account in formulation of the purpose of the book and the directions of the development described in Chapters 2-6. The appendixes in the book include proofs of the lemmata and theorems, descriptions of the examples and the technical details which are needed in the course of the consideration (Appendixes D.3 and D.5), can contribute to better understand some stochastic features of the nonrandom models ( Appendix F) or can be helpful in practical implementation of the described methods (Appendix H ) .

Chapter 2

Diffusion Processes

2.1

Introduction

This chapter focuses on the topic pointed out in the corresponding paragraph of Section 1.13. A s discussed in Section 8.2, probability density p, involved in (1.6.11) and (1.6.13) is associated with unacceptable computational expenses if conditions (1.2.14), (1.11.1), (1.11.2) hold. The chapter derives such representations for expectation and variance functions e and M which, unlike (1.6.11) and (1.6.13), do not apply density p, and thereby rem o v e the above problem. Section 2.2 presents Theorems 2.1 and 2.2 on the time-derivatives of e and M. R e m a r k 2.1 The innovative feature of Theorems 2.1 and 2.2 is that they derive expressions for time-derivatives of expectation (1.6.11) and variance (1.6.13) directly from diffusion relations (1.8.2) and (1.8.3), not involving I S O D E (1.9.2) or any results inherently related to SDEs. This derivation fully agrees with condition (1.9.7) and is new. Theorems 2.1 and 2.2 constitute a necessary stage in our purely D S P approach. If one ignores condition (1.9.7) and thereby returns to the I S O D E treatment (leaving the scope of the present book), then one m a y try to reformulate Theorems 2.1 and 2.2 in terms of the corresponding I S O D E techniques. The models for e and M developed in Section 2.2 are not closed: they still include density p„. The hierarchy of the closures of the model for e is developed in Section 2.3. The resulting, closed descriptions are the O D E 63

64

Dt/yMSMM Processes

systems of various orders and different levels of accuracy and complexity. Application of these results to noise-induced phenomena in the expectation, more specifically, to the influence of diffusion function TV upon e are considered in Section 2.4. It presents the unified model for such phenomena as stochastic resonance, stochastic linearization, stochastic self-oscillations stochastic phase transitions, stochastic chaos. The closure of the model for M from Section 2.2 is the topic of Sections 2.5 and 2.6. Section 2.5 derives the linear O D E for matrix F(f) with no linearizing assumptions. Section 2.6 discusses the closure of this O D E by means of replacement of probability density p, with invariant probability density p. which can approximately be evaluated on the basis of drift and diffusion functions g and 7/. Such approximation is constructed in Chapter 3 which also includes the version of the present treatment for invariant DSPs.

2.2

Time-Derivatives of Expectation a n d Variance

The results of this section are formulated in Theorems 2.1 and 2.2 below. T h e o r e m 2.1 Let the following assumptions be valid. (1) Hypothesis of L e m m a 1.2 (see Section 1.6) under condition [f ,oc)c/ holds where f E / is arbitrarily fixed. (2) Limitrelation(1.8.2)holdsuniformlywithrespectto (? ,x ) e [f ,oo)xR^ (cf., assumption (1) of Theorem 1.2 in Section 1.12.1). (3) Expectation (1.6.11) exists for all ?^f„. (4) Integral J ^ g(f, x) p ,(f„, f, x) ^x converges for any f ^ f„. (5) g(-,.x)EC°([;,,oo)), p,(;„,-,.x)€C°([f,,oo)) for almost all x E R ^ where the continuity at f=f is understood in the sense of the limit as ;J, f (e.g., see (1.4.30)) and convergence of integral Jg«g(f,x)p,(^,;,.x)J.x is uniform with respect to f ^ f . Then e€C*([f ,oo)) and equalities <^(;+0) . ^^=JgMPo(*)^-

(2.2.1)

R**

^=Jg(;,*)p,(;.,f,*)
hold.

for all f > ^ , (2.2.2)

?ltnte-Dertua?tue o/'Expec^cttton o ^ d Var:atnce

7*roo/^

65

is presented in Appendix D.l.

T h e o r e m 2.2 Let the following assumptions be valid: (1) Hypothesis of Theorem 2.1 holds and hypothesis of L e m m a 1.3 (see Section 1.6) under condition [L,°°)c^ also holds. (2) Limit relation (1.8.3) holds uniformly with respect to (r, jc_) e [; ,oo) x R*^ (cf., assumption (1) of Theorem 1.2 in Section 1.12.1). (3) Variance (1.6.13) exists for all f > f„. (4) Integral L,
(5) 7/(',j;)GC°([^,oo))foralmostall^eR''wherethecontinuityatr=r is understood in the sense of the limit as f J, ^ and the integral in notation ^(f)=f77(;,x)p,(;,,f,*)<^,

for all r > ^ ,

(2.2.3)

R"

converges uniformly in f ^ f„. W e denote MM=j*^-e(f)][^^)-g(f,e(y))]Tp,(^^^)^,

for all f>f„. (2.2.4)

R'

Then the following assertions are valid. (1) 7J,,Af<=C°([f„,°°)),and

"A)=J"MPc(*)<^.

(2.2.5)

R"

^ . ) =/(* * ^)[^..^) - ^ . ^ . ) ] ^ P . M ^ -

(2-2.6)

R"

(2) !^e C\[^,oo)), and equalities

- ^ - ^ = [M(^)]T+M(fJ+^J.

^ ^ = [M(^)]T+Af(r) +N,(r), are valid.

(22.7)

for all f > f„,

(2.2.8)

66

Di/jfHSM)M Processes

Proo/^

is presented in Appendix D.2.

Theorems 2.1 and 2.2 derive (2.2.2) and (2.2.8) directly from (1.8.2) and (1.8.3). This is the main result of the present section. Equations (2.2.2) and (2.2.8) enable one to obtain approximate descriptions for vector e(f) and matrix V(?) in the form of O D E systems in Sections 2.3 and 2.5 respectively.

2.3

Ordinary Differential Equation Systems for Expectation

This section considers the approximate descriptions for e(f) in the form of the first- and higher-order O D E s .

2.3.i

7%e /trsf-or<%er sys^e/n

If hypothesis of Theorem 2.1 holds, then, substituting the simplest approximation (1.6.20) for p, into (2.2.2), w e get

^Lg(f,e(f)),

foral!f>f.

(2.3.1)

Thus, result (2.2.2) overcomes the difficulties mentioned in Point (1) in Section 8.2 (see also Section 1.13). Equations (2.3.1) and (1.6.12) where ^ is described with (1.6.7) m e a n that expectation e(f) can approximately be determined as the solution of initial-value problem (A.l), (A.6). It can be solved numerically by means of various method. Perhaps, the simplest and most stable one is the so-called semi-explicit Euler method (e.g., Hall and Watt, 1976) f*='*-l^* ,

, ^=^-1+^

3g(^-i'<;*-i)

%-i, Z,. ,

^>0,

'o = '..

(2.3.2)

-l

g(?A.
^ = 1,2,

^0 = ^ '

(2.3.3)

where ^ is the %:th time step and e^ is the approximate value of e(^). This method is both ^4 -stable and L -stable. The principal term of its local truncation error at the &:th step can be evaluated according to the following simple formula - (y^/2) [g(^,ej *g(^-i.^-i)] which can be used to automatically choose the stepsize 3..

Ordmary Dt//eren?ta/ E^Ma^ons /or E^pec^a^ton

67

R e m a r k 2.2 In computational practice, partial derivatives of nonlinear functions of m a n y variables (for instance, the derivatives in (2.3.3)) are usually calculated with the help of the corresponding finite-difference (FD) formulas or the automatic-differentiation techniques (e.g., Griewank and Corliss, 1991; Griewank, 2000; see also U R L : http://www-unix.mcs.anl.gov /autodiff/ADIFOR/ on the A D I F O R software available for both the Unix and W i n d o w s 95/NT workstations). The automatic-differentiation approach m a y be more accurate and efficient than the F D one. However, the latter treatm e n t is generally more preferable. W e note some of the reasons for that. *

T h e finite-difference techniques do not presume the knowledge of the analytical expression for the function (e.g., this is typically the case w h e n the function is evaluated by m e a n s of the software developed by independent authors); if the expression is not available, automatic differention can do nothing at all.

*

If the above analytical dependnece is available but its implementation is associated with cumbersome computations (e.g., if the functions is described with the integral dependence on parameters which are the variables of the functions), then the numerical, F D differentiation can provide more computationally efficient algorithms and more simple and maintanable software structures than the automatic-differention approach.

Thus, estimations of the computational expenses for evaluation of the partial derivatives can in m a n y cases be based on the numerical-differentiation scenario. Let us examine w h e n representation (2.3.1) for (2.2.2) is exact. If L e m m a 1.1 (see Section 1.6) holds at Z(f,x)sg(f,x), then the right-hand side of (2.2.2) is expectation of random variable g(^,x('^))< i-e.

^[g(f.x('';))]=Jg(;-*)p.(fo-'-*M*.

f°r aii ^^.

O n the other hand, one has g(f,e(f)) =g[f,E(x(',;))],

for all f > ;„.

These equalities point out that (2.3.1) is equivalent to (2.2.2) if and only if

68

D:/^ston Processes

g[f,E(x(-,f))]=E[g(;,x(-,?))],

for all r > ^ .

(2.3.4)

This holds if, for example, function g is linear with respect to x. If (2.3.4) does not hold, then (2.3.1) m a y be considered only as an approximation for (2.2.2). In applications of high-dimensional D S P s granted as solutions of I S O D E system (1.9.2) (see also (1.9.5)), initial-value problem (A.l), (A.6), as the m o del for e(f), is usually assumed, not derived (see (35) in Demir ef a?., 1996). In so doing, initial probability density p is sometimes subject to the restrictions not related to the essence of this model. For instance, Demir e? c^. (1996) assumes p^ to be Gaussian and with the specific variance (see Section V.E and (61) therein). In contrast to treatments of this kind, the present approach not only derives (2.2.2) and approximation (2.3.1) but also points out condition (2.3.4) of equivalence of (2.2.2) and (2.3.1). Another fact showing an approximate nature of O D E (2.3.1) is that it does not include diffusion matrix W(f,x) and thereby ignores the influence of function 77 upon expectation function e (cf., R e m a r k 1.10 in Section 1.6). So, w e need to consider expressions for e(f) which would be more informative than Eq. (2.3.1) but still more simple than (2.2.2). This is the topic of Section 2.3.2.

2.3.2

T A e seco?M%-or<%er sys^ewt

This section is based on the following theorem. T h e o r e m 2.3 (1) (2) (3) (4)

Let the assumptions below be valid.

Hypothesis of Theorem 2.1 is fulfilled. Hypothesis of Thoerem 1.2 holds at [^,oo)c/. g(f,-) (= C2(R'') for any fixed f > ^. Equalities H m {&('.*)[S(',*)PA-'.*)] + (l/2)p,(?.,f,*)/V(f,*)V^(;,*)} = 0, t*)-"=° for all & = 1,2, ,;„, (2.3.5)

where the action of S(f,x) on the probability density is described with (1.12.2) and the limits are uniform with respect to f>:^ hold. (5) Function 3g/3t exists for all f > ^ and almost all x € R^. (6) 3g/3^(,x)eC°([^,oo)), 3p,(f„,-,x)/dfeC°([;.,oo)) for almost all xSR*' where the continuity at f=f is understood in the sense of the limit as

OrcHnary D://erenf:a^ EqMa(:o/:s /or Expec^a^toK

69

fj,^ and integrals J^[3g(^^)/3^]p.(^,^jc)^, Jg^g(^,^)[3p,(^,^^)/3^]Jjc converge uniformly with respect to f ^ f . W e denote A(f,x) = (^(f,*),/^;,*), ...,^(f, j;))^, for all (^,^)e [;„,oo)xR'',

(2.3.6)

^ ( f ^ ) = (l/2)tr[7/(f,A:)d^(4^)/3jc'], for all 4r = l,2,...,
(2.3.7)

T h e n the following assertions are valid. (1)
for all ^ > r ,

(2.3.8)

holds. Proo/ D.4.

applies the results of Appendix D.3 and is presented in Appendix

Assumption (4) ofthis theorem m e a n s that S(f,;t)p,(^,f,.x) and p,(^,f,x) tend to zero sufficiently fast as [xll-*oo. If hypothesis of Theorem 2.3 holds, then, substituting the simplest, deterministic approximation (1.6.20) for density p, into (2.3.8), one gets

^Lg(;,
for all f > ^,

(2.3.9)

where

g(f,x) = ^ M ^ + M ^ g ( ; , x ) + M,,.x), 3? a*

for all ^ S E ^ .

(2.3.10)

W e note that O D E system (2.3.9) cannot be deduced directly from (2.3.1). Equation (2.3.9) also m e a n s that expectation e(f) of process % can approximately be determined as the solution of the second-order O D E

70

Di/^MSton Processes

^=g(^^)

(2.3.11)

with initial conditions (A. 6) and

^ - ^ ) -

(2.3.12)

'=<.. Condition (2.3.12) follows from (A.l) and (A.6). R e m a r k 2.3 Initial-value problems (A.1), (A.6) and (2.3.11), (A.6), (2.3.12) are equivalent if and only if A(;,.x) = 0. If this relation does not hold, the latter problem cannot be obtained from the former. Thus, vector A(f,.x) (see (2.3.6), (2.3.7)) in (2.3.10) is the n e w factor granted by the second-order O D E (2.3.11). Initial-value problem (2.3.11), (A.6), (2.3.12) is a more accurate description of expectation function e than initial-value problem (A.l), (A.6). The advantage of (2.3.11) compared to (A.l) is that it, by means of (2.3.6) and (2.3.7), includes dependence of the solutions on diffusion function R. The disadvantage is that (2.3.11) is more complex O D E than (A.l). Section 2.4.2 considers this difficulty in detail and points out the w a y to remove it.

2.3.3

SysfeFMso//Ae%MgAer orders

Theorem 2.3 and its proof in Appendix D.4 point out a general recipe to construct more sophisticated approximations for e(f). Expression (2.3.9) is derived from (2.2.2) by m e a n s of differentiating it with respect to f, accounting (1.12.4) as well as the corresponding version (2.3.5) of (D.3.9), and application of (1.6.20) to (2.3.8). A similar technique m a y then be applied to (2.3.8). A s a result, one will get the expression for <^e(f)/;?;3. Subsequent application of (1.6.20) will give the O D E of the third order for e(?). T h e n one can consider initial-value problem for the latter O D E where the initial conditions are (A.6), (2.3.12) and

= g(f.-e.). J;2 See (2.3.10) for g. If this description is still not sufficiently accurate, the

AfocMs /br Nb:se-/7t<^Mced PAewomenat w Expec(ct(:oK

71

above manipulations can be applied to the expression for J^e(f)/^^, and so on. In so doing, the key point at each step is validity of the corresponding version of condition (D.3.8). Moreover, the above procedure requires functions g and R to be sufficiently smooth. Complexity of the resulting models rapidly increases with increase in the order of the time derivative of e.

2.4

M o d e l s for Noise-Induced P h e n o m e n a in Expectation

The advances of the last decade demonstrates a keen interest of m a n y researchers and engineers in various noise-induced phenomena in expectations of stochastic processes (and random fields): stochastic resonance, stochastic linearization, stochastic self-oscillations, stochastic phase transitions, stochastic chaos. The first part of these terms, the word "stochastic", points out that the listed effects are due to the influence of noise upon the expectations. This influence for Markov processes has already been noted in R e m a r k 1.10 (see Section 1.6). There is a vast literature on the topic. To mention a few works, w e point out Bulsara ef ct%. (1993), Lin and Y i m (1996), Gammaitoni ef ct^. (1998), Luchinsky, McClintock and D y k m a n (1998) (see also "Stochastic Resonance H o m e Page in Perugia (Italy)" at U R L : http://umbrars.com/sr/index.htm). Stochastic resonance, linearization, self-oscillations, phase transitions, chaos are related to different aspects of stochastic systems. However, they have something in common. Namely, they are associated with noise and nonlinearities of the system. These are the features which are taken into account with expression (2.3.7) for vector (2.3.6) by means of (2.3.10) in O D E (2.3.11) (or (2.3.9)) (see also Remark 2.3 in Section 2.3.2). Indeed, the nonlinearities in Eq. (2.3.7) are presented by the second derivatives of entries of drift function g whereas the noise is taken into account by diffusion function # . A s an example, w e analyze the relation of O D E (2.3.11) only to stochastic resonance. This topic was discussed in connection with microelectronics applications by Willander and Mamontov (2000) and is considered in Sections 2.4.1 and 2.4.2 below.

2.4.1

7%e c
Stochastic resonance is usually associated with the following features.

72

D:/^MSton Processes

*

The stochastic system under consideration is nonlinear (no stochastic resonance was found so far in linear stochastic systems). * The above system is driven by a deterministic periodic input signal. * The nonlinearities and the periodic input cause stochastic resonance in the periodic output signal. * Stochastic resonance in the output presents (e.g., Section 4.1 of Luchinsky, McClintock and Dykman, 1998) an increase in the expectation of the periodic output signal or even the stronger phenomenon that S N R of the periodic output signal achieves a local maximum value at the nonzero R M S of the signal (see Appendix B on notion of SNR). The improvement of the output-signal amplitude or S N R is the most practically useful phenomenon which stochastic resonance causes. According to the above features, we shall consider the case when g(f,x)=g(u,x),

#(;,*) = H(v,x),

v = ^+v(f)

(2.4.1)

where v is a vector independent of (f,x) and u(f) is a small periodic signal. In so doing, we analyze the periodic solution of the corresponding equation. This equation is O D E (2.3.11) under condition (2.4.1) (see also (2.3.10)), i.e.

j ^ 3g(u +v(;),*) Ju(;) ^ 3g(u + u(;),*) g ^ ^ ) , ^ ^ ^ ) . 6/;2

gu

J;

(2.4.2)

gx

The periodic solutions can be studied by means of the well-known techniques. The corresponding theory can be found, for example, in Demidovic, (1967), Hale (1980), Nayfeh and Balachandran (1995), Pliss (1966), Pontryagin (1962). More practice-oriented analytical treatments are also available, for instance, the finite-equation method developed by the authors (Mamontov, 1989; Mamontov and Willander, 1997c). However, the general theory does not always explain the resonance in a compact form. To fill this gap, w e simplify O D E (2.4.2). W e assume,that: *

signal u(f) is sufficiently small to enable one to replace Eq. (2.4.2) with its simplified representation ^

3g(u,x) Ju(;) + M ^ ) g ( ^

^

^.4.3)

Mode/s /or Nb:se-7H
73

and the autonomous version

^ = 3 g M ^' 3x ^

+ ^ '

^ ^ '

of Eq. (2.4.3) has the unique asymptotically stable quasi-neutral equilibrium point x=x (see Appendix D.5 for notion of this point). In this case, Eq. (2.4.4) can be re-written similarly to Eq. (D.5.11), i.e.

so the corresponding form of Eq. (2.4.3) is asymptotically stable linear O D E J2(x-x) Jf^

3g(v,x) ^u(^) ^ 3g(u,x) ^f(x-x) ^ 3A(u,x) , 3u ^^ 3x Jf 3x

-.

for all x close to x.

(2.4.5)

A s is well-known, if signal v(f) is a harmonic, then linear O D E (2.4.5) has the unique periodic solution which is also a harmonic and its frequency is identical to that of signal v(f). The fairly c o m m o n derivation which is mainly of a technical nature shows that the last term on the right-hand side of (2.4.5), i.e. the term related to function A, generally causes the resonance in the periodic solution x-x even in the one-dimensional case. Expressions (2.3.6), (2.3.7) for A stress that this resonance is due to both the nonlinearity of drift function g and the randomness-related diffusion function //. A s is mentioned at the beginning of the present section, such resonance is stochastic. Stochastic linearization, self-oscillations, phase transitions, chaos can also be analyzed on the basis of the corresponding treatments of the secondorder O D E system (2.3.11). In so doing, the effects associated with vector A(f,x) in (2.3.10) are of a key importance.

2.4.2

J^racftcaHy e/jf!c%enf %ntpZeFne?tfaftom of f/te second-order sysfewt

Initial-value problem (2.3.11), (A.6), (2.3.12) can numerically be solved by m e a n s of the s a m e technique as that discussed in Section 2.3.1 for initial-

74

Dt/^Mston Processes

value problem (A.l), (A.6), i.e. the semi-explicit Euler method (see (2.3.2), (2.3.3)). T h e corresponding overall efficiency is, as follows from R e m a r k 2.2 (see Section 2.3.1), proportional to the computing expense required to evaluate the right-hand side of the O D E system (for instance, vectors g(f, jc) in (A.l) or g(f,.x) in (2.3.11)) once. In case of system (2.3.11), the above expense is mainly due to calculation of matrix 3g(f,Jt)/3x and vector (2.3.6) in (2.3.10). This procedure involves not only J nonlinear scalar functions gpg^, ,g^ but also all their ^-derivatives of the first and second orders and all the entries of symmetric matrix #(?,x). T h e total n u m b e r N of the nonlinear scalar functions of 6? scalar variables is J(J^+4J+l)/2 or N - - J3/2,

for high J (see (1.2.14)), (2.4.6)

that disagrees with strategy (1.11.2). This problem can be removed if one applies the technique below to simultaneously evaluate matrix 3g(;,x)/3x and vector (2.3.6) in (2.3.10). Since matrix W;,.x) is non-negative definite symmetric (see (1.8.4)), then there exists square matrix y4(f,.x) such that /f(f,*)=,4(f,A:)[,4(;,.*)]T, for all (f,*)E [^,oo)xR^.

(2.4.7)

W e denote the / th column of matrix ^4(f,.x) with a,(f,x), i.e. ^^(a^f,*),^,*),...,^,*))^ for all (f,jr)G [^,oo)xR^. (2.4.8) Note that matrix y4(f,.x) satisfying Eq. (2.4.7) need not have the s a m e properties as those of solution A(f,.x) of Eq. (1.9.5) (see also (1.9.7) and the text above it). In particular, the latter matrix is generally nonsquare. In other words, solving (2.4.7) for y4(;,x) is a simpler task than solving (1.9.5) for A(f,.x). A s s h o w n below, matrix ^f(f,.x) serves solely to improve the computational efficiency. O n e can readily check with the help of (2.4.7) that the trace of matrix in Eq. (2.3.7) can be described with equation <^ ;-i

for all ^ = 1,2,...,^, (f,.x)€ [^,<*,)XR^.

(2.4.9)

75

M x M s /or JVb:se-/K<^Hceoi PAenomena m Expecfa?:oH

T h e terms s u m m e d over on the right-hand side of Eq. (2.4.9) are coupled with values of function g^ as follows

g^.(f,.x± ^a,(;,x))=g^(f,.x)± ^ [3^(f,*)/3*]a,(;,.x)

for all A:,/ = 1,2,...,J, (f,%)€ [;„,<*,)xR'',

(2.4.10)

w h e r e ^ is a parameter which m a y depend o n f, x, g^ a n d a,. Quantity Q/(^'^) depends on functions g%. a n d H^ a n d is independent of $. T h e last two terms on the right-hand side of (2.4.10) point out that parameter y determines h o w accurate the first three terms o n the right-hand side describe the left-hand side. S o ^ should be chosen to be sufficiently small. It follows from (2.4.9) that

= ? ^ [&('.*+ ^,(f.*)) * 2g,t(f,*) +g^(;,.x- $<:,(;,.*))], for all &,/ = l,2,..,
(2.4.11)

[,4(;,;c)]T[3ga(;,;r)/3;<: )T^=

2? g^(f,^ + ^^(^,^)) -g^(f,^ - ? a^(^,^)) forall % = 1,2,...,
(2.4.12)

a n d the error of every of these approximations is O ( y ^ ) . Relation (2.4.12) presents 6? systems of linear algebraic equations for ^ r o w vectors dg^f,.x)/3x,...,dg^(f,.x)/3x with the s a m e matrix [,4(f,x)]T. Clearly, these systems can be solved uniquely provided that the matrix is nonsingular. T o assure the latter property, w e a s s u m e that det[#(f,.x)]?i0, for all (f,x)e [f ,o°)XR^,

(2.4.13)

i.e. non-negative definite symmetric matrix 77(f,.x) is positive definite. T h e n Eq. (2.4.7) can always be solved for nonsingular matrix y4(f,x) at a n y fixed (f,x) e [f^, 00 )x R^. If this procedure is implemented b y m e a n s of the Choles-

76

Dt/^MSMn Processes

ki method (e.g., §4.6 of Churchhouse, 1981), then matrix y4(f,x) is lower triangular. In this case, solving systems (2.4.12) with upper triangular matrix [/l(;,x)]T for any %: = 1,2, .,<% is very simple. For improved efficiency, these systems can be solved in parallel. Note that, at every fixed (f,x)€ [f,'°°) XR'', the Choleski triangular factorization (e.g., (4.2.19), (4.2.20) of Churchhouse, 1981) should be used for Eq. (2.4.7) only once. T h e above considerations show that evaluation of matrix 3g(;,.x)/d.x and vector (2.3.6) in (2.3.10) requires to calculate only N - = (5(f^+ J)/2 nonlinear scalar functions of d scalar variables, i.e. N - - 5J2/2,

for high d (see (1.2.14)),

(2.4.14)

that agrees with strategy (1.11.2). N u m b e r N - in (2.4.14) is in <^/5 times less than in case of (2.4.6). This, due to (1.2.14), means that the proposed method is qualitatively more efficient and thereby opens a w a y to practical application of the second-order O D E (2.3.11). In so doing, entries (2.3.7) of vector (2.3.6) are determined with (2.4.9) and approximate relations (2.4.11) whereas rows 8g^(f, x)/3x of matrix dg(f,x)/d.x are uniquely evaluated from J systems (2.4.12) of linear algebraic equations under condition (2.4.13). Note that value (2.4.14) of number N - is only 2.5 times greater than that in case of the first-order O D E (A.1).

2.5

Ordinary Differential Equation S y s t e m for Variance

Sections 2.3 and 2.4 propose the general recipes to derive approximate descriptions for expectation e(f) in the form of O D E systems. However, similar recipes, being applied to expression (2.2.8) for c?V(?)A% do not enable one to obtain O D E descriptions for variance V(f). The reason is term .x-e(f) in (2.2.4). Thus, to treat the variance, one should choose another way. For example, one m a y try to express matrix (2.2.4) in (2.2.8) by means of the variance matrix. 2.5. i

D a m p i n g wmfrMc

This section considers the damping matrix corresponding to function g. L e m m a 2.1 Let the assumptions below be valid: (1) dg/dx6C°([;„,<x.) X R^), the integral in equality

Ordinary Dt#ere^^o^ Eq'uafMn /or Vctrtance

77

P(f,*) = f ^ - ^ W * - ' ( ^ J K , forall(^)e[^,o.)x^, (2.5.1) 0

3M

where M presents the second variable of function g (i.e. the values of g are supposed to be g(f,H)) converges uniformly with respect to (f,x)€[f ,oc) x R'', and hypothesis of Theorem 2.1 holds. (2) Hypothesis of L e m m a 1.1 (see Section 1.6) where / z [f,,°°), process % is specified with initial condition (1.4.24) (see also (1.4.23)) and function Z is described with expression Z(;,.x) = D(f,.x) holds. Then the following assertions are valid. (1) DeC°([^,oo)xR^). (2) Expectation D^(f)=E[D(f,x(^))]=/D(^Jc)p,(^^^)Jj<;,

for all f > ^ ,

(2.5.2)

exists and equality D^J=E[D(^0,x(',^0))] = limE[D(4x(^))] = f D ( ^ ^ ) p ^ ) ^ , (2.5.3) '"'

R'

holds. i*roo/ Assertion (1) follows from assumption (1) and the continuity of function e (see assertion of Theorem 2.1). Assertion (2) follows from assumption (2) and results (1.6.9), (1.6.10) of L e m m a 1.1 applied to function Z(;,x) = ,P(;,.x).

2.5.2

T%e MWCorre%ct%e<%-wMtfrt3;es approaiHMtftoM

To express matrix M(f) by means of variance y(f), w e introduce the uncorrelated-matrixes assumption with the help of Theorem 2.4 below. This theorem applies covariance 7?(f) of random matrixes [%(-,?)-g(f)] x[x('.?W(f)f and D(f,x(',?)). In view of (1.6.13) and (2.5.2), this covariance is written as ^)=E{{[x(-,f)-<^)][X(^)-<^]^-^)}[D(^x(V))-D^)f}. T h e o r e m 2.4 Let the assumptions below be valid. (1) Hypothesis of L e m m a 2.1 (see Section 2.5.1) is fulfilled.

(2.5.4)

78

D:^*MstoK Processes

(2) The integral in (2.5.2) converges uniformly with respect to f > ; . (3) Hypothesis of Theorem 2.2 (see Section 2.2) holds. (4) Hypothesis of L e m m a 1.1 (see Section 1.6) where 7 3 [f ,°o), process % is the same as in L e m m a 2.1 and function Z is described with expression Z(f,x)s[jc-g(;)][.i-e(f)]T'[D(f,.x)]T hoMs. (5) Inequality (2.4.13) holds. Then the following assertions are valid. (1) D,<EC°([;„,M)xR'). (2) Function 7? exists for all f > f and is presented as R(f)=Af(f) - ^ ) [D,(;)]T,

for all ; > ;„.

(2.5.5)

(3) Relation R(f)=0,

for all f > ^ ,

(2.5.6)

is equivalent to Af(f) = H^)[D,(f)]T,

for all ;> ;^,

(2.5.7)

and if (2.5.6) holds, then F(f) is the solution of linear O D E system ^y=D^)^)^(f)[D^)]^^(^),

for all f ^ ,

(2.5.8)

with initial condition (1.6.14), i.e.

F(;)^(;,;J^[<%;J]T+ J* c(;,;-K)N(;,;-K)[C(f,f-x)]T
for all f > ^ ,

(2.5.9)

where 1^, is described with (1.6.8) and (^(f,^) is the Cauchy matrix of linear O D E system

^=D,(;)M,

(2.5.10)

where w E R ^ , i.e.

^^=D^)^(r,rJ,

for all r ^ ^ ,

C(^,rJ=7. (2.5.11)

79

Ordty:ayy D:^ereH^otZ Eq'ua^ton /or Vartctnce

(4) Matrix V(f) and is positive definite for every f > f„. J?roo/

is presented in Appendix D.6.

H o w restrictive is the uncorrelated-matrixes ( U M ) condition (2.5.6)? If, for instance, every column of matrix [x( ^)"^(^)][X('^)"^(^)]^ and every column of matrix D(f,%( ,f)) are uncorrelated vectors for all f^f^,

(2.5.12)

then (2.5.6) is obviously valid. Properties (2.5.12) and (2.5.6) are equivalent at J = 1. However, if J > 1, then (2.5.12) implies Eq. (2.5.6) but does not generally result from (2.5.6). Thus, the U M condition (2.5.6) is weaker than "by-column" uncorrelatedness (2.5.12). Substituting (1.6.20) into (2.2.3) and (2.5.2), one obtains 7^(f)=77(f,e(f)), D,(f)=.D(f,e(;))=^(^)),

forallf>;„, (2.5.13) forallf>f„. (2.5.14)

This m e a n s that representation (2.5.8) corresponds to Point (2) in Section 8.2 and thereby to the purpose of the present work (see Section 1.13). T h e U M assumption (2.5.6) is the feature which enables one to obtain expression (2.5.7). If (2.5.6) does not hold, one m a y apply (2.5.7) only as some approximation. So (2.5.7) can be called the U M approximation for matrix Af(f). W e also emphasize the following fact. Corollary 2.1 If assumptions (1), (2), and (4) of Theorem 2.4 are replaced with the features that 3g/3.xeC°([; ,oo) x R'') and function g is linear in x, i.e.D(f,x) is independent of* (D(f,A:) = D(f,e(f))),

(2.5.15)

then all assertions of Theorem 2.4 and U M feature (2.5.6) are valid. Proof involves (2.2.4), (D.6.2), (2.5.15), (2.5.2), (2.5.14), (1.4.28) and is obvious. This corollary emphasizes that, in the linear-drift case (2.5.15), the U M

80

D:^*Hs:oK Processes

assumption (2.5.6) automatically holds.

If hypothesis of Theorem 2.4 and condition (2.5.6) hold, then variance H(;) of process x satisfies O D E (2.5.8) with initial condition (1.6.14) (see (1.6.8) for t^). T h e solution of this initial-value problem is given with (2.5.9). If hypothesis of Corollary 1 holds, then drift vector g(f,x) is linear in x and variance matrix F(f) is governed by the linear O D E system (2.5.8) regardless of the fact whether diffusion matrix #(f,x) (involved in (2.5.8) by m e a n s of (2.2.3)) is independent of x or depends on x. This is an unexpected and surprising fact! If, on the contrary, drift vector g(f,x) is nonlinear in x, then the exact O D E for the variance is generally not available. O n e has only the exact expression (2.2.8) for JV(f)/<^f and approximate description (2.5.8), (1.6.14) (or (2.5.9)). T h e above comparison of the linear and nonlinear cases demonstrates that the property of drift function g to be nonlinear qualitatively complicates analysis of the variance.

(2.5.16)

Moreover, this complication concerns not only the variance. For instance, the results of M a m o n t o v and Willander (1997e) show that nonlinearity of the drift function can be responsible for the long asymptotic correlation time of the stationary process. This correlation time is not predicted by the linear treatment and m a y lead to the so-called flicker effect (discussed, for example, in Section 2 of M a m o n t o v and Willander, 1997e). Further considerations on the stationary case in Section 4.10 stress the importance of the drift nonlinearity even more. They include the example where this nonlinearity causes the long, non-exponential asymptotic behavior of the covariance of the hard-sphere velocity in the hard-sphere fluid. Section 4.10.4 presents the corresponding procedure to approximately determine the fluid-particlevelocity covariance and spectral density in the general case. Theorem 2.4 proposes a reasonable w a y to overcome difficulty (2.5.16). This w a y is essentially based on the U M approximation (2.5.7) which provides O D E system (2.5.8) where matrix D^,(f) is given with equality (2.5.2) that, in case of (2.5.15), is reduced to (2.5.14).

OrdtfMMy Dt#erenttaJ Equation /or Vartanee

2.5.4

81

/?Mn<%aFMenf
System (2.5.8) under assumptions (2.5.13), (2.5.14) is sometimes applied to the case as functions D and 7/ do depend on x. To be specific, we point out paper Demir ef a?. (1996). It discusses the solution of ISODE (1.9.2) with initial probability density p under the conditions that p^ is Gaussian and its variance is of a specific form (see Section V.E and (61) of Demir e? a?., 1996). In so doing, expectation e (i.e. x^ in terms of Demir e? a?., 1996) is determined as the solution of initial-value problem (A.l), (A.6). A special care should be taken over the description of the variance matrix proposed by Demir e? ot^. (1996). These authors use J7(;,e(f)) instead of 7f(f,x) that leads to Eq. (2.5.13), linearize g(f,x) in (1.9.2) in a neighborhood of point x = e(f) (x=x^ therein) that leads to Eq. (2.5.14), and apply the well-known (e.g., (8.2.8) in Arnold, 1974) O D E for variance. It is similar (see also (1.9.5)) to O D E system (2.5.8) which, however, is derived in Theorem 2.4 for the general nonlinear case in the U M approximation without any assumptions on disabling the dependences of /f or D on the statespace vector x. As directly opposed to our alternative (2.2.3) and (2.5.2), Eqs. (2.5.13) and (2.5.14) fully ignore the influence of x-dependences of diffusion #(f,x) and damping D(f,x) upon variance F(;) thereby disregarding both warning (2.5.16) and the nonlinear nature of ISODE (1.9.2) per se. Nevertheless, this purely linear treatment which in essence contradicts the nonlinearities is proposed in Demir ef aJ. (1996) for nonlinear ISODE systems. Moreover, the same, heuristic-linearization technique is applied to self-oscillating nonlinear ISODE systems by Demir and Sangiovanni-Vincentelli (1996) (see (3)-(6) therein). Clearly, the approach of Demir e? aJ. (1996) and Demir and Sangiovanni-Vincentelli (1996) cannot be recommended for nonlinear ISODE systems. As for linear ISODE systems, the corresponding theory and methods have been developed many decades ago (e.g., Section 8.2 in Arnold, 1974). Demir ef a J. (1996) considers the numerical aspects concerning O D E (2.5.8) under conditions (2.5.13), (2.5.14). They treat system (2.5.8) as the linear O D E system for
82

D:/^Ms:on Processes

included in the corresponding future development (see Section VIII therein). In contrast to this, our expression (2.5.9) requires the amount of m e m o ry proportional to 6^ rather than %P and hence complies condition (1.11.2). In the general, nonlinear case, i.e. as function D or 7^ does depend on the state-space vector x, application of (2.5.8) under assumptions (2.5.13), (2.5.14) suggested in Demir ef a^. (1996) can obviously lead to inadequate results for y(f). However, in some exceptional cases w h e n accuracy of determination of H(f) is not very important, combination (2.5.8), (2.5.13), (2.5.14) can be used. O n e such example is presented in Section 3.6.1. O u r formulas (2.2.3) and (2.5.2) involves probability density p.. A s is discussed above, the simplest approximation (1.6.20) is definitely not the best one. Which approximation can be used instead?

2.6

T h e Steady-State A p p r o x i m a t i o n for the Probability Density

To obtain a reasonable approximation for probability density p^(f ,f,jc), one can involve the issues discussed in Section 1.12.2, especially in Point (1) therein and (1.12.8) as well as in the text between (1.12.10) and R e m a r k 1.17. In the deterministic case (see Appendix A), relation (1.12.8) m e a n s that O D E (A.l) is convergent, i.e. property (A.10) holds.

(2.6.1)

Duration ofthe settling density p,(f ,f,x) to invariant density p (f,.x) expressed with (1.12.8) is usually proportional to T,„, in inequality (1.11.3) where parameter T; characterizes time dependence of p.. Hence, by virtue of (1.11.3), one can apply the steady-state approximation p,(;^,;,.x) = p,(4x) for all % and all f^f<,. Since density p, is defined for all te R (see Definitin 1.11 in Section 1.7), Eqs. (2.2.3) and (2.5.2) can be generalized to the whole time axis. The corresponding approximations are

; % ; ) =/*H(;,.x)p,(f,.x)&t,

for all ; e R ,

(2.6.2)

forallfER.

(2.6.3)

R*

D,(;) = fD(;,x)p,(;,*)t?.x,

In so doing, the feature of matrix !^(f) mentioned in assertion (4) of Theo-

83

7%e <Sfeat%y-.Sfafe Appro^ima^toT: /or (/te ProoaM:'(y Denst^y

r e m 2.4 is provided by condition (cf., assumption (5) of Theorem 2.4) det[^(f,^)]#0,

forall(f,*)eR^.

(2.6.4)

W e in what follows use (2.6.2)-(2.6.4) instead of (2.2.3), (2.5.2), (2.4.13) respectively. The general w a y to determine invariant probability density p, is described in Point (2) in Section 1.12.3 and associated with (1.12.9) and (1.12.10). However, as is noted at the beginning of Section 1.13, this recipe is impractical in the present, high-dimensional case (1.2.14) accompanied by conditions (1.11.1), (1.11.2). This naturally poses the following question: can density p, or its suitable approximation be determined in any, say, practice-friendly way? The answer is positive. It is described in Section 3.5. O n e can construct such approximation for p. rather than for p . The constructed approximation is practically suitable in the high-dimensional case (1.2.14). That is w h y w e involve feature (1.12.8) and subsequent relations (2.6.2), (2.6.3). T h e above considerations are continued in Chapter 3.

Chapter 3

Invariant Diffusion Processes

3.1

Introduction

This chapter considers the case w h e n D S P % with drift and diffusion functions g, J? is invariant (see Definition 1.11 in Section 1.7). Its importance is discussed in Section 1.7. The corresponding invariant probability density p. is also used Section 2.6 as the steady-state approximation for density p,. In particular, p, is involved in Eqs. (2.6.2), (2.6.3) for matrixes # . (f), D^ (f) in initial-value problem (2.5.8), (1.6.14) for variance F(f) of the non-invariant D S P s with the s a m e drift and diffusion functions.

3.2

Preliminary R e m a r k s

T h e present chapter assumes that hypotheses of L e m m a t a 1.4 and 1.5 hold and subsequently applies Eqs. (1.7.7), (1.7.8) instead of Eqs. (1.6.11), (1.6.13). A n y relation employing time value f is considered to be valid for all f e R provided that the relation does not involve quantity p(f.,.x.,f,.x) for some ;.<; (see also (1.3.16)). For instance, Eq. (2.2.4) becomes Af(^)=j*[jc-e(f)][g(r^)-g(^e(f))]Tp.(^jc)^,

for all?.

(3.2.1)

Chapter 2 develops the efficient recipes to approximately determine expectation e(f) and variance V(f) of non-invariant D S P %. H o w can these recipes be modified if the process is invariant? Because of (1.12.8) and (2.6.1), 85

86

/HMtr:a;:f D:/^MSK)n Processes

one should consider O D E systems (2.3.1), (2.5.8) on the whole axis R under the uniform-boundedness conditions (1.7.9), (1.7.10) respectively rather than with initial conditions (1.6.12), (1.6.14) (see also R e m a r k 1.17 in Section 1.12.2 and the discussion below it). In so doing, expression (2.5.9) should be considered under condition (1.6.18) in the limit case as f ^ - o o . Section 3.3 is devoted to expectation e(f) of invariant D S P %- Its variance V(?) is considered in Section 3.4.

3.3

Expectation. T h e Finite-Equation M e t h o d

Existence and uniqueness of the solution of problem (2.3.1), (1.7.9) is a challenging topic for purely qualitative research. In this field, Theorem 2 of M a montov and Willander (1997c) for the case as function g is norm-coercive uniformly in f€ R (see Definition 1 in Mamontov and Willander, 1997c) can be helpful. The present section discusses the corresponding practical approach. It is assumed that O D E (2.3.1) is convergent (see (A. 10)) and hence problem (2.3.1), (1.7.9) has the unique solution. This solution (i.e. the steady-state solution of O D E system (2.3.1)) can efficiently be obtained by means of the finite-equation method (see Section 4 of M a m o n t o v and Willander, 1997c). This technique involves notion of uniformly stable matrix (see Definition 2 in Mamontov and Willander, 1997c) which is also formulated in Definition 3.1 below and used in Theorem 3.1 in Section 3.4. Definition 3.1 (Definition 2 in M a m o n t o v and Willander, 1997c). If dg/8xeC°(R*^), then matrix dg(f,.x)/3.x is called uniformly stable on R*^ if and only if there are symmetric nonsingular matrix A and scalar A > 0 which are independent of (f,x) and such that the m i n i m u m eigenvalue of symmetric matrix {/(f,.x) + [./(f,.x)]T}/2 is not less than A for every (f,.x) € R ^ where ./(;,*) =A[-3g(;,*)/d.x]A-i. The property of matrix 3g(f,x)/3.x to be uniformly stable on R*^ in particular assures (see assertion (1) of Theorem 3.1 below) that

det 3g(f,*) ^ 0,

R e m a r k 3.1

for all (;,*) E R ^ .

(3.3.1)

It follows from Definition 3.1 and Eqs. (2.5.1), (2.6.3),

Expecfa^on. 7%e Fwt^e-Eq'ua^toH Afe^Aod

87

(1.7.1), (1.7.2) that if matrix 3g(;,.i)/3.x is uniformly stable on R ^ at the matrix A a n d scalar X mentioned in the above definition, then both matrixes (2.5.1) and (2.6.3) (if they are continuous functions) are also uniformly stable on R^** at the s a m e A a n d A. T h e key advantage of the finite-equation method is that it enables one to determine solution e(f) of (2.3.1) with property (1.7.9) as a solution of the finite, i.e. non-differential, equation system which is constructed by m e a n s of the analytical procedure completely described with function g. For example, the simplest approximation according to the finite-equation approach is 3g(f,x) '3g(;,*) g(?,*) 3f

(3.3.2)

where the matrix is non-singular (see (3.3.1)). Comparison of (3.3.2) with (2.3.1) shows that the left-hand side of (3.3.2) presents a n approximation for
(3.3.3)

w h e r e the limit is uniform with respect to f € R a n d vector-functions e(°\e^),. ,e(*\ are described with the following iterative procedure 0=g(f,e(")(;)), 3g(f,g(*-j))

3x

for any fixed f E R, (3.3.4)

3g(f,e (*-*)) g(f,^) 3f

for all it = 1,2,3,... a n d any fixed f e R. (3.3.5) T h e well-known Newton-Raphson method can be applied to calculate e ^ (;) at each %: > 1 a n d fixed f. T h e numerical results obtained by the finite-equation technique can be adequate if condition (1.11.3) is fulfilled. This is easily illustrated with application of the technique to the particular case of (2.3.1) as matrix dg(f,x)/d.x is independent of (f,x). If m o r e complex O D E system (2.3.9) under condition (1.7.9) a n d the condition that function de/d; is uniformly bounded for all r e R

(3.3.6)

88

Znuartanf D:/%SM)H Processes

is analyzed instead of problem (2.3.1), (1.7.9) (see Section 2.3.2 for the details), then the finite-equation method can also be applied to problem (2.3.9), (1.7.9), (3.3.6). In so doing, the corresponding procedure similar to (3.3.4), (3.3.5) can readily be constructed. More generally, the finite-equation method can be used for the O D E systems of the higher orders discussed in Section 2.3.3. This method does not assume the solution to be periodic: the solution is considered as a time-dependent function of a general form. This is a valuable advantage for the following two reasons. First, the nonperiodic (e.g., quasi-periodic, almost-periodic) steady-state responses are in the focus of m a n y applied fields. Second, engineering techniques developed in specific fields [for example, the approaches discussed by Telichevesky ef aJ. (1996) in connection with high-frequency semiconductor circuits] have not overcome the above periodicity restriction yet. In so doing, the general, nonperiodic case is still considered as a topic for future work (see Section 4 in Telichevesky ef a?., 1996). T h e finite-equation method also provides a deeper insight in the singularly perturbed O D E s (Section 5 of M a m o n t o v and Willander, 1997c). It can be used to approximately construct stable integral (or invariant) manifolds of nonlinear O D E systems (p. 459 in M a m o n t o v and Willander, 1997c; M a montov and Willander, 1998b). In this field, it significantly differs from the techniques developed earlier such as the theory of Pliss (1966) or the center -manifold method (e.g., Carr, 1981). For example, the conditions enabling one to consider the center manifolds (e.g., Chapter 1 of Carr, 1981) have no relation to the conditions which underlie the finite-equation method (Sections 2-4 of M a m o n t o v and Willander, 1997c).

3.4

Explicit Expression for Variance

The theorem below gives the sufficient conditions which assure that property (1.7.10) holds for variance (2.5.9) as f^-*-oo. This theorem applies the property of matrix dg(f,x)/3x to be uniformly stable on R ^ (see Definition 3.1 and R e m a r k 3.1 in Section 3.3). T h e o r e m 3.1 Let the assumptions below be valid: (1) Hypothesis of Theorem 2.4 under conditions (2.6.2), (2.6.3) and property (2.5.6) hold. (2) Matrix 3g(f,.x)/d.x is uniformly stable on R**'*. W e denote m i n i m u m and

ExpZ:ct% Expresstort /or Var:aKce

89

m a x i m u m absolute values of eigenvalues of matrix A from Definition 3.1 with m and P respectively. (3) There exists scalar & > 0 which is independent of (f,x) and is such that H/7,(;)ll<3 for all f E R . Then the following assertions are valid. (1) For any (f,x) e R*", real part of every eigenvalue of matrix dg(f,.x)/d.x does not exceed some negative number, and inequality (3.3.1) holds. (2) At any f,€R, inequalities H^(f,fJH>(a/P)exp[-A.(;-;J] for all ff„ hold where scalar X is from Definition 3.1. (3) Quantity (l/2)w^A^M is the Lyapunov function for linear O D E system (2.5.10). This system is exponentially asymptotically stable in the large and also unstable to the left (i.e., as f-^-oo) in the large. (4) Problem (2.5.8), (1.7.10) has the unique solution which is Eq. (2.5.9) in the limit case as f.-* * °°, i.e.

t-^f) = f^(;,;-K)^(f,;-K)[^(;,;-K)fJK, for all ;e R. (3.4.1) 0

This solution satisfies inequality Ht^f)] < (p/m)^W(2A) and is positive definite for all re R. froof is based on Definition 3.1 and R e m a r k 3.1 and is simple. W e only mention a few aspects. Equality (3.4.1) follows from (2.5.9) in the limit case as f -> - oo because of the second of the estimations in assertion (2). Assumption (2) and assertions (l)-(3) are similar to the hypothesis and the assertions of L e m m a 1 in M a m o n t o v and Willander (1997c). The first of the estimations in assertion (2) assures that zero solution of exponentially asymptotically stable O D E (2.5.10) is its unique solution uniformly bounded on R. This provides the uniqueness of matrix (3.4.1) as the solution uniformly bounded on R. Theorem 3.1 results in explicit expression (3.4.1) for variance H(f) of the invariant D S P . This expression is used in what follows.

90 3.5

/nuartani Dt^MSMH Processes

T h e Simplified Detailed-Balance Approximation for Invariant Probability Density

Equations (2.6.2) and (2.6.3) involve invariant probability density p.. This section proposes the approximation for p. which can be implemented in high-dimensional case (1.2.14) for realistic computing expense. In so doing, hypothesis of Theorem 3.1 is assumed to be fulfilled. D o w n to at least (3.5.26), w e assume that 7 7 e C ^ ( R ^ ) .

/or %oF6trtfA?M o/f&e <%ens#y Property (1.7.1) can be assured with the following change of variables p,(f,x) = exp[f/,.(;,.x)],

for all (f,x)€ R ^ .

(3.5.1)

Density p,(;,.x) can have zero values due to value -oo of quantity t/.(;,;c). Representation (3.5.1) precludes some irrelevancies in numerical results, for example, in the data denoted with sign "n" in Fig. 3 of Langtangen (1991) (see also the corresponding issue on p. 44 therein). Relation (3.5.1) enables one to rewrite (1.12.9) as the equation for t/, = lnp,., namely, 8t/.(;,.t)

.-

-r

—^+[^tW.*)]^(^)+v,\-(;.*)=o, forall(f,.x)ER'",

(3.5.2)

where \7. is described with (1.10.1) and v(f,^) = F(;,A:)-(l/2)^^)^C/,.(r^), forall^eR^.

(3.5.3)

In so doing, the assumptions in R e m a r k 1.16 (see Section 1.12.1) where p is replaced with p,. are regarded to be valid. In particular, the Fichera drift vector F(;,;t) is determined with (1.12.7). B y virtue of (3.5.1), Eq. (1.7.2) and the convergence mentioned in condition (1.2.10) are transformed into fexp[t/,(;,x)]
for all ;e R,

(3.5.4)

TTte S:mj9Zt/ted Def<MM-Bc&tnce Appro^tnta^on

91

and convergence of the integral in (3.5.4) is uniform with respect t o f S R .

(3.5.5)

It follows from (1.12.6), (3.5.1) and (3.5.3) that G(f,.x)p,(f,x) = p,.(f,.x)v(f,.x). Since G(f,.x)p,(f,x) is the probability flux corresponding to density p, (see R e m a r k 1.16), vector (3.5.3) m a y be interpreted as the probability velocity. Transformation (3.5.1) reduces problem (1.12.9), (1.2.10) for p, to problem (3.5.2), (3.5.4), (3.5.5) for LA which is assumed to be bounded in a certain sense. However, in contrast to (1.12.9), P D E (3.5.2) for t7. is nonlinear. This is a penalty paid for the guarantee that (1.7.1) holds. T h e advantage concerning the non-negativity of p,. is not the only one associated with (3.5.1). The other one concerns inequality V^F(f, *) < 0,

for all (f, *) (E R ^ ,

(3.5.6)

which seems to be natural. Indeed, if J = l and the processes are described with I S O D E (1.9.2) where both functions g and A are time-independent and linear with respect to jc, inequality (3.5.6) is k n o w n (e.g., (8.4.3.b), (11.2.15b), (11.2.17) in Arnold, 1974) as necessary and sufficient condition for any of the I S O D E solutions to be stochastically asymptotically stable in the large. Property (3.5.6) likely holds for arbitrary cf in m a n y applied problems. If (3.5.6) is valid, then change of variable (3.5.1) grants one more advantage. T h e point is that one of the difficulties arising, for example, in n u m e rical treatment (e.g., p. 35 of Langtangen, 1991) is that K F E (1.12.9) has zero solution regardless of whether inequality (3.5.6) holds or does not hold. In contrast to this, inequality (3.5.6) does assure that P D E (3.5.2) which involves (3.5.3) has no constant, i.e. (f,x)-independent, solutions. This fact m a y be helpful in practice.

T h e time behavior of term 3C/,(f,.x)/d; in (3.5.2) is to a great extent determined by internal properties of the D S P s and hence can be characterized with time parameter T. . Unlike this, the time dependences of coefficients g(f,.x) and /7(f,.x) and hence (see (3.5.3)) of vector v are described with the

92

/HuarKmf Dt/^uston Processes

time scale determined with time parameter t . In view of (1.11.3), the lefthand side of (3.5.2) can approximately be replaced with zero. Then one gets [V^,(;,.x)fy(f,x) + ^(;,.x)=0,

forall^eR^,

(3.5.7)

instead of (3.5.2). W e shall call P D E (3.5.7) truncated equation for t/, = In p,. Truncated P D E (3.5.7) is obtained by means of omitting the time-derivative in (3.5.2) in view of the "fast" motions (1.11.3). W e note that similar equations are well known, for example, in approximate asymptotic methods. Khasminskii and Yin (1996a, 1996b) apply truncation in the above sense to the version of K F E (1.12.4) as the basis to analyze transition probability densities of singularly perturbed K F E s [see (1), (2) in Khasminskii and Yin, 1996a; (3), (4) in Khasminskii and Yin, 1996b]. They employ [(8) in Khasminskii and Yin (1996a); (9) in Khasminskii and Yin (1996b)] the scheme of the boundary-function method (Vasil'eva and Butuzov, 1985; O'Malley, 1991; Vasil'eva ef a?., 1995) to study one- and two-dimensional DSPs. Khasminskii and Yin (1996a, 1996b) reveal n e w capabilities of singular- perturbation theory. However, practical application of the asymptotic "small"-parameter techniques, in particular, the techniques for singularly perturbed equations is associated with the well-known problem. All these methods assume that the analyzed equation inculdes a "small" parameter. The point is that, in applied problems, it is not always easy to determine this parameter. A more detailed discussion on this topic can be found, for example, in Section 5 and Ref. 3 of Mamontov and Willander (1997c). Khasminskii and Yin (1996b) also suggest (see p. 1816 therein) to apply their approach to multidimensional DSPs. However, it essentially envolves solutions of the corresponding P D E s in 6?-dimensional space. Treatments of this kind are unsuitable in high-dimensional case (1.2.14) because of the reasons discussed in Section 8.2. O n e can readily note that (3.5.7) holds if v(;,x)=0 for all (r,x)6R^ or, due to (3.5.3) (cf., eq. (8) on p.191 of Soize, 1994) if F(;,jt)-(l/2)jy(;,*)V^,(f,x)=0,

forall(?,*)ER'".

(3.5.8)

In view of (1.12.5), (1.12.6) and (3.5.1), Eq. (3.5.8) is equivalent to the D B equation (1.12.13). Equation (3.5.8) includes values of the derivatives of t/, with respect to x but does not include values of function t/,. Hence any x -independent function of f is a solution of (3.5.8). This means that t/. can be presented in the following w a y

7%e S t m p ^ / M De^atJe^-Ba^aHce Appro3ctma^:on

93

{7,. (f, x) = t/(r, x) - W,. (^), for all (;, x) e R*^,

(3.5.9)

where: *

function C/ is any function satisfying version F(f,x)-(l/2)/7(;,*)V,t/(;,x)=0,

forall(f,x)eR^, (3.5.10)

of the D B equation (3.5.8) under condition (see (3.5.4)) fexp[t/(f,x)]J.x<<*', y

for a l l ^ e R ,

(3.5.11)

and the condition that (see (3.5.5)) convergence of the integral in (3.5.11) is uniform in f 6 R; *

(3.5.12)

function W , is defined with equality exp[W',(;)] = fexp[!7(;,.i)],ac, forallfSR.

(3.5.13)

The rest of Section 3.5 is devoted to problem (3.5.10)-(3.5.12).

3.5.3

Cose off&e dc^a^eJ 6a^ancc

To resolve the D B equation (3.5.10) for t/, w e involve property (2.6.4). It holds since assumption (5) of Theorem 2.4 is valid because of Theorem 3.1 (see the very beginning of Section 3.5). Then (3.5.10) is equivalent to (cf., eq. (11) on p. 191 of Soize, 1994) V,t/(f,.x) = c(f,x),

for all (?,x) e R*",

(3.5.14)

where c(f,x) =2[/7(f,.x)]*i.F(;,.x). B y virtue of properties 3g/8.x E C°(R"'*) in Definition 3.1 (see Section 3.3) involved by Theorem 3.1 and 7feC^(R"^) (see the very beginning of Section 3.5), one has y4 € C°(R*^) where matrix-function v4 is defined with expression ^(^)=^2fN(^)]^(^)}^ 3x In view of (3.5.15), Eq. (3.5.14) implies

R,raH(,,*)eR-M.

( 3 5 ^

94

/nuartanf Dt/^Ms:on Processes

^ ^ * ) =^(^^) +^.(f.^),

for all (;,*)e R^',

(3.5.16)

w h e r e ^(f,x) = {^(r,x) + [^(r,%)]T}/2 and y!„(;,*) = {,4(f,x)-[,4(;,;<:)]T}/2 are the symmetric and skew-symmetric components respectively of matrix y4(f,x).

Equation (3.5.14) is not always solvable for (/. It has solutions if and only if matrix y4(f,x) is symmetric or, that is the same, the skew-symmetric component is zero, i.e. y^(;,*)=0,

forall(;,*)eR^.

(3.5.17)

forall^eR^.

(3.5.18)

In this case, one has (see (3.5.16)) 3^(f.*)=^(;^),

Equality (3.5.17) is k n o w n as the condition of the detailed balance (e.g., p.191 pf Soize, 1994). This n a m e is due to the fact that (3.5.17) enables the D B equation (3.5.14) to become solvable for {7. The D B condition is fulfilled if, for instance, g is gradient mapping and #(f,.x) is scalar matrix for all (f,x) E R ^ [e.g., see (4.1.5), (4.1.6) in Ortega and Rheinboldt (1970) on notion of gradient mapping]. If (3.5.17) holds, then function C/ can readily be determined from (3.5.14). For instance, this can be done by means of the Taylor formula for function ?7(f,-) in a neighborhood of point x = e(f), namely (e.g., (10) on p. 71 of Ortega and Rheinboldt, 1970), 1

t/(;,x) =LT(;,e(;)) + [X -e(f)f f V^ t/{f,e(f) +n[.x -e(f)]}
(3.5.19)

where M presents the second variable of function {7 (i.e. the values of t/ are regarded as t/(f,M)). Without loss of generality (see (3.5.9) and (3.5.13)), one can apply t/(;,e(;)) = 0,

forallfSR.

Substitution of (3.5.20) and (3.5.14) into (3.5.19) leads to

(3.5.20)

7%e NtfHpZt/M De^at^ed-Ba/a?:ce Appro^:ma(;oy:

95

^(f,.x) = [jc-e(f)]Tfc{f,e(f)+K[.x-e(f)]}^K, 0

forall(f,;c)eR^.

(3.5.21)

The D B condition (3.5.17) holds in rather exceptional cases. Typically, in applied problems it is not even k n o w n if this condition is valid. So w e concentrate on the question: what can be done if (3.5.17) is not used? In so doing, we, however, involve the main idea of the detailed-balance treatment.

3.5.4

T%e <%ef(M%e(%-&a%ance approarwMzfMM

If (3.5.17) need not be valid, then matrix (3.5.16) is generally nonsymmetric and hence Eq. (3.5.14) need not have solutions. In this case, one m a y try to follow the simple recipe that skew-symmetric matrix y4 (f,.x) in (3.5.16) is neglected.

(3.5.22)

This leads to the approximate version of the D B expression (3.5.18), i.e.

3 ^ - * ) ^^(r,;c),

forall(f,jc)eR^.

(3.5.23)

W e therefore call (3.5.23) the D B approximation. Since Eq. (3.5.14) need not have solutions, neither function V {7 nor any of its value are known. Hence w e should rely on (3.5.23) as the equation for C/. The Taylor formula for function !7(f, ) in a neighborhood of point x = e(f) which involves its second-order .x -derivatives is written as (e.g., Eq. (10) on p. 80 of Ortega and Rheinboldt, 1970)

y(f,.x)=t/(;,e(;))+[.* -
forall(;,.x)ER'^.

(3.5.24)

Since function V^C/ remains unknown, the only thing which can be undertaken on the values of V^t7(f,e(f)) in (3.5.24) is to prescribe them in some

96

7?wartan^ D:/%s:o?t Processes

way, for instance, as follows V^(7(f,e(f)) = 0,

for all re R.

(3.5.25)

Substitution of (3.5.20), (3.5.25) and (3.5.23) into (3.5.24) leads to

^ , x ) = l[x-e(f)]^ 2 J * ( 1 - K ) ^ , ^ ) + K[x-^)]}^K [x-e(r)],

forall(r,x)eR^.

(3.5.26)

Assumption (3.5.25) m a y seem to be rather arbitrary. It in particular m e a n s that, for any f e R, point x = e(f) is sufficiently close to the point of a local extremum of the corresponding (see (3.5.9), (3.5.1)) probability density as a function of x. This, however, is really the case for s o m e c o m m o n multidimensional probability densities. So, one can expect that approximation (3.5.26) (which results from assumption (3.5.25)), being applied to (3.5.9), (3.5.1) will not cause catastrophic errors in evaluation of integrals (2.6.2) and (2.6.3). Formula (3.5.26) is discussed in Section 3.5.5.

3.5.5

TTte s%mpH/!e<% <%ef<M%et%-&
Since property (1.2.14) is the case, expression (3.5.26) is still impractical. T h e reason is that matrix ^4 (f,x), the symmetric component of matrix y4(f,x), necessitates to evaluate not only J(J+l)/2 entries of symmetric diffusion matrix LT(f,.x) but also (see (1.12.7) and (3.5.15)) all the first- and second-order jc -derivatives of these entries. In so doing, the total n u m b e r of the scalarfunctions to be determined is J(J+l)^(J+2)/4 (e.g., 18at J = 2 and 60 at <% = 3). In the present case (1.2.14), this n u m b e r is approximately equal to <%^/4 that contradicts requirement (1.11.2). T o remove this problem, w e apply the following operation: matrix 77(f,.x) in (3.5.15) is replaced with matrix #(f,e(f)). (3.5.27) Obviously, this simplification can only be adequate if dependence of matrix N(f,.x) on x at any fixed re R is not too strong. Condition (3.5.27) leads to (see also (1.12.7)) ,4(;,x)=2{//[f,e(;)]}*i8g(f,.x)/ax for all (;,*) and hence

forall(;,A:)eR^.

(3.5.28)

F r o m this, the assumption that # E C ^ ( R ^ ) (see the very beginning of Section 3.5) is no longer required. Substituting the obtained expression for y^(;,.%) into (3.5.26) and replacing sign " = " in the resulting equality with sign "=", one obtains ^ ( f - * ) = "(l/2)[*-e(0]T{[R(^e(r))] 'D,(y,^) +D^(f,^)[7/(f,e(f))] '} [^-e(f)], forall(4Jc)€]R^,

(3.5.29)

where

D,(;,x) = - 2 f ( l - K ) M M M ^ ^ K , forall(r,jc)eR^. (3.5.30) T h e subscript "sdb" (i.e., simplified detailed balance) emphasizes the approximate nature of (3.5.29) stemming not only from the D B approximation (3.5.23) but also from simplification (3.5.27). W e note that, under assumption (3.5.27), expression (3.5.3) for probability velocity v(f,x) and condition (3.5.6) are written as y(r^)=g(^A:)-(l/2)^,e(r))^(/^^), V^g(f, *) < 0,

for all ( f . ^ e R ^ , for all (f, *) (E R ^ ,

where the inequality holds because of assertion (1) of Theorem 3.1. W e also note that truncated P D E (3.5.2) has no (t, ^-independent solutions if operation (3.5.27) is involved. Since t ^ is an approximation for {7, the corresponding approximations W ^ and p ^ for W , and p; are determined according to (3.5.13) and (3.5.9), (3.5.1) respectively, i.e. e*P [H^<(')] =/exp [C/^(f,Ar)]^,

P , ^ ^ ) = P ^ ^ ) = ^p[^(4^)-^^(f)], provided that (see (3.5.11))

for all f E R,

for all (f,*)eR^,

(3.5.31)

(3.5.32)

98

/nuartaHf D:/%ston Processes

fexp[t/^(f,A;)]Jjc<<x.,

forallfER,

(3.5.33)

and (see (3.5.12)) convergence of the integral in (3.5.33) is uniform in f € R.

(3.5.34)

Are (3.5.33) and (3.5.34) valid and is function p ^ (see (3.5.32)) probability density? The conditions which give the positive answer are formulated in Theorem 3.2. T h e o r e m 3.2 Let the assumptions below hold. (1) Hypotheses of Theorem 3.1 is fulfilled. (2) There exist numbers 0 < y^ ^ y^ such that, for any (;,%) E R*^, every eigenvalue of symmetric matrix (3.5.28) is in interval [y^y^]Then the following assertions are valid. (1) y„„EC°(R'% (2) -(y^/2)[^-^(f)f^-a(r)]<^(^)<-(y^/2)^-e(r)]T^-^)] for all (y,jc)ER^.

(3) Inequality (3.5.33) holds and property (3.5.34) is the case. Moreover, one has (2Tc/y^)^<J^exp[C/^(f,^)]^< (2it/y^)^ for all f € R . (4) W ^ e C ( R ) a n d W ^ is such that (J/2)lm(2n/y^)<^^)<(J/2) x!n(2n/y,J for all f E R . (5) p,^ e C ° ( R ^ ) , inequalities [Y^/(2^)]^exp{-(y^/2)[^-e(f)]T^-e(r)]} < p ^ , ^ ) ^[Y^/(2n)]^exp{-(y^/2)[^-^)]T[^-g(f)]}, forall^eR*^,

(3.5.35)

hold and p^^ is probability density. Proof is simple and mainly of a technical nature. Property 8g/3x E C ° ( R ^ ) required (see (3.5.29), (3.5.30)) for assertion (1) is the case because of assumption (1) of this theorem and Theorem 3.1. It should be emphasized that Theorem 3.2 states nothing on differentiability of functions ^
Ana/yftcaZ-AfMfnertca^ Approach ^o D^MStoT: Processes

99

bounded in the sense of Gaussian estimations (3.5.35). Replacing p,. in (2.6.2) and (2.6.3) with p^,, one obtains /V,(r) = exp[-W^(r)] TV ^(r), D^(r) = exp[-^„(r)]D^(r),

for all re R, for all r e E ,

(3.5.36) (3.5.37)

where 7V^^(r) = f/V(;,*) exp[t^^(r, jc)] J^,

for all re R,

(3.5.38)

R^

D^^(f) = fD(f,x) exp[^7^^(r,jc)] J%,

for all re R.

(3.5.39)

R"

Thus, the above derivation of expression (3.5.32) and the related terms opens a w a y to apply O D E (2.5.8) under conditions (3.5.36), (3.5.37) for variance matrix V(?) of the invariant process even in the high-dimensional case (1.2.14).

3.6

Analytical-Numerical A p p r o a c h to Non-Invariant a n d Invariant Diffusion Processes

This section discusses h o w the above results can be used as an analytical basis of the analytical-numerical approach mentioned in Sections 1.11,1.13 and 8.1. It also considers some related questions. The main result of Section 3.5 is the derivation of the S D B approximation p ^ (see (3.5.32)) specified with (3.5.29M3.5.31) and Theorem 3.2 on the basic properties of function p ^ . In view of Section 2.6, this result meets the requirements of the purpose of the book mentioned in Section 1.13. The remarkable fact is that equality (3.5.29) involves matrix D,(r,Ar) (see (3.5.30)) which is similar to damping matrix D(r,.x) (see (2.5.1)) used in (2.6.3). So stressed application of the terms based on nonlinearity of drift function g can be regarded as a reply to challenge (2.5.16). It should also be emphasized that simplification (3.5.27) concerns only the terms involved in (3.5.15). It is not applied to (2.6.2). Thus, expression (2.6.2) does enable to take .x -dependence of diffusion function 7V into account. Relation (3.5.32) leads to approximate expressions (3.5.36) and (3.5.37) for matrixes 7V.(r) and D,(r) involved in O D E (2.5.8). In so doing, scalar W ^ ( r ) and matrixes 77^(r), D^(r) are calculated by m e a n s of integral

100

ZnuartatK^ D:/yus:o?: Processes

equalities (3.5.31), (3.5.38), (3.5.39). In computational practice, the integration over R^ is usually replaced with the corresponding integration over some bounded domain EcR*^ (see also the text around (1.2.4)). A s demonstrated by the numerical results on stationary K F E for stationary probability density (1.7.11) (e.g., p. 44 of Langtangen, 1991), this domain m a y be chosen in the form of a rectangular parallelepiped, i.e. 2 =[H^wJ x [MywJ x... x [M^,wJ

(3.6.1)

where H^<w^, %: = 1,2, ...,<%. Nonstationary D S P is usually analyzed on some bounded interval, say, [f„,f.] where f„
(3.6.2)

The corresponding choice of M^ and w^ in (3.6.1) is discussed in Section 3.6.1 below. For domain E , equalities (3.5.31), (3.5.38), (3.5.39) are rewritten as e x P ^ ^ C ) ] =/exp[t/,„,(;)]
for all ;<E R,

(3.6.3)

for all re R,

(3.6.4)

for all f(E R,

(3.6.5)

E

^ ( ? ) =/^(f.*) exp[!7^(f)M*, E

D^(f) = fD(f,x) exp[t/^(f)]6?x, E

where damping matrix D(f,x) is determined with integral representation (2.5.1).

3.6. J

CAotce o/fAe &OM/M%et% d o w M M w o/fAe wfegrafton

The estimations for H^ and w^ in (3.6.1) for the stationary one-dimensional process is proposed, for instance, on p. 44 of Langtangen (1991). This technique can be generalized for the nonstationary high-dimensional process on interval (3.6.2) in the following way. The well-known Bienayme—Chebyshev inequality (used, for instance, in Appendix B ) prompts one to choose M. and n^ in (3.6.1) according to relations MA = <^.-<;.0^

"A = e*.M^.°*.

& = l,2,...,
(3.6.6)

Ana%y%caJ-M47?ter:caJ Approach ^o D:^*Ms:o?t Processes

101

where $ > 0 is a number independent of f, e^=min{<^(f)}, <e[<.,(.]

^^=max{e^.(^)}, '

A; = l,2,...,^,

(3.6.7)

;€[;.,<.]

e. (f) is the /cth entry of expectation vector e(f), o^=max{Jt^)},

R = l,2,...,af,

(3.6.8)

K.(^) is the %th entry of the principal diagonal of variance matrix M(^). If e ^ = e^^, then the probability of, for example, 0.04 in the Bienayme-Chebyshev inequality corresponds to c =5 that agrees with the values approved by the finite-element treatment of stationary one-dimensional probability density (see p. 44 of Langtangen, 1991). In general, number Q should be at least of a few units. Even in the stationary case, Langtangen (1991) suggests to evaluate the expectations and variances by m e a n s of direct numerical simulation of the corresponding I S O D E (see Section 3.2 of Langtangen, 1991). In contrast to this, w e consider another approximate procedure which presents a purely deterministic and m u c h more simple alternative. Namely, vector e(f) involved in (3.6.7) can be determined as described in Sections 2.3 and 3.3, i.e. from (A.l), (A.6) or (2.3.11), (A.6), (2.3.12) if the process is not invariant and from (2.3.1), (1.7.9) or (2.3.9), (1.7.9), (3.3.6) if the process is invariant. To calculate o, in (3.6.8), it is sufficient to have only entries t^(f) of the principal diagonal matrix y(f) which is described with O D E (2.5.8). In so doing, w e consider only such equations in system (2.5.8) that corresponds to entries of the principal diagonal of F(f) and neglect all the terms on the right-hand sides of these equations which are associated with the off-diagonal entries. The resulting system has the following form

-^=2D^(;)^(;)+7^(;), for all A; = 1,2,..., J, ;e[;^,;.],

(3.6.9)

where ^,^(f) and D, ^(f) are the %th entries of the principal diagonals of matrixes 7f^(f) and D^,(f) respectively. The corresponding initial conditions follow from (1.6.14), i.e.

102

Znuartanf D:/fMs:on ProeeMes

^ ( 0 = ^.M-

forall* = l,2,...,J.

(3-6.10)

T h e accuracy of evaluation of !^(f) for the purpose of (3.6.8) is not a critical issue because of the above arbitrariness in choice of n u m b e r ^. This is one of the rare cases as the linearizing assumptions (2.5.13), (2.5.14) can be adequate. So w e apply expressions

for all ^ = 1,2,...,J, ;E[;„,;.].

(3.6.11)

Substitution of (3.6.11) into (3.6.9) gives

for all ^ = 1,2,...,J, ^S[^J.].

(3.6.12)

For the Arth equation in (3.6.12), the Cauchy matrix <^.(f,fj is scalar and is obviously written as Q(;,;J = exp{2j/[dg^(K,e(K))/3xj6?KJ, ^ = 1,2,...,^, for all ^^ ^. Thus, w e have d initial-value problems (3.6.12), (3.6.10) for J scalar functions K . . Each of these problems is similar to initial-value problem (2.5.8), (1.6.14) for matrix F and hence is solved by m e a n s of the formula similar to (2.5.9) for the non-invariant process or (3.4.1) for the invariant process. The correponding solutions K . can be determined numerically. T h e n they are used in (3.6.8).

3.6.2

EuttZMCtftom o/fAe w m M / o Z d tw^e^raZs. 7%e MtMtfe CarZo fecAfMqfMe

Formulas (3.6.3)-(3.6.5) present the d-fold integrals of the following scalar functions: the integrand in (3.6.3), <%(;%+1)/2 entries of symmetric matrix N(f,.x) in (3.6.4), and 6p entries of matrix dg(f,.x)/9.x in (2.5.1) involved in (3.6.5). T h e n u m b e r of these scalar functions is proportional to ^ for high J (see (1.2.14)). In this case, the only realistic technique to evaluate the above d-fold integrals is the Monte Carlo method proposed by Metropolis and U l a m (1949) [see also Rubinstein (1981) or pp. 95-99 of Sobol' (1994)]. The reason can briefly be explained in the following way.

Ana^yftca^-JVumertca/ Approach (o Dt^hston Processes

103

Let JV be the number of points in domain (3.6.1) involved in the calculation of a ^-fold integral. Each point is associated with evaluation of the above scalar functions. Hence the amount of the calculations for (3.6.3) -(3.6.5) at every fixed f is proportional to JV ^ . The error of a deterministic technique is proportional to NJ^ ^ where ^ is the order of the applied quadrature formula. Unlike this, the error of the Monte Carlo integration is, according to the well-known statistical estimation (e.g., pp. 16, 93 of Sobol', 1994), independent of d and proportional to JVp . (In the Monte Carlo case, JV is the n u m b e r of stochastically independent random points (i.e. trials) in rectangular parallelepiped (3.6.1) or, after the variables are properly changed, in the d-dimensional unit cube.) Since <; is usually independent of d and not too high, one typically has <;/^^l/2 (see also (1.2.14)). This m e a n s that, roughly speaking, at the same integration accuracy, the Monte Carlo approach requires radically less number JV of the points than the deterministic, quadrature treatment. In other words, it is computationally m u c h more advantageous and, if d is high enough (see (1.2.14)), the only reasonable choice. D u e to the above statistical estimation, number Af should not be too small. In fact, the greater JV, the less the error of the integration. So, the corresponding amount of calculations at fixed f which is proportional to JV <%2 (see above) will be large especially in the typical case as f is not a single time point but varies within interval (3.6.2) (at f„
p-(f) = J* d"(;,; - K) N,(;,; - x) [ c(;,; - n ) ] ^ ,

for all r e R,

(3.6.13)

o where number g is the f -independent number of at least a few units. T h e reason of large amount of calculations required to implement formulas (2.5.9) or (3.6.13) is that the integrals in both equalities m a k e it necessary to solve initial-value problem (2.5.11) for ^Xtf-matrix d%,f ) at

104

/?:uar:an^ Dt/^Ms:o?t Processes

fixed ; and m a n y different values of K. Indeed, initial m o m e n t ^ in (2.5.11) is expressed as f „ ^ " * where K increases from 0 to f-f, in case of (2.5.9) and from 0 to eT,„, in case of (3.6.13). Thus, evaluation of the integrals in (3.6.3)-(3.6.5) and (2.5.9) or (3.6.13) characterizes our approach as computationally intensive. O n the other hand, w e have imperative condition (1.11.1). Is it possible to resolve this problem? Appendix H points out a possible w a y to a positive answer.

3.6.3

S u m m a r y o/'fAe a p p r o v e A

This section shows h o w to combine the above analytical results in the algorithm to evaluate expectation e(f) and variance ^ ) of the nonstationary high-dimensional D S P on interval (3.6.2). (1) If the process is not invariant, then vector e in (A.6) and matrix P' in (2.5.9) are calculated with the help of (1.6.7) and (1.6.8) respectively. In so doing, properties (1.6.17) and (1.6.18) should hold. Initial probability density p„ (see (1.4.20), (1.4.21)) is assumed to be available. (2) O n interval (3.6.2), vector e(f) is determined from initial-value problems (A.l), (A.6) or (2.3.11), (A.6), (2.3.12) if the D S P is not invariant or from problems (2.3.1), (1.7.9) or (2.3.9), (1.7.9), (3.3.6) if the D S P is invariant. In the latter case, the finite-equation method (see Section 3.3) can be applied. T h e calculated results are stored since they are used at the steps below. (3) N u m b e r s M^oi?^, % = 1,2,...,J, are evaluated according to (3.6.6) -(3.6.8). In so doing, initial-value problems (3.6.12), (3.6.10) are solved. The integration domain E is formed by m e a n s of (3.6.1). (4) Time f discretely increases from ^ to f, (see (3.6.2)) and, for any fixed f, matrix F(f) is calculated according to (2.5.9) if the process is non-invariant or (3.6.13) if it is invariant. In so doing, matrixes 7^(f) in (2.5.9) or (3.6.13) and Z^(f) in (2.5.11), necessary to evaluate (7(f,^), are determined with (3.5.36) and (3.5.37) where the integrals in expressions (3.6.3)-(3.6.5) for W^^(^), ^
DtSCMSStOM

105

ant with (1.11.1) and (1.11.2). This procedure presents the general description of the analytical basis pointed out in connection with the purpose of the book in Section 1.13 (see also Section 8.1). T h e corresponding comparison with the treatment of other authors, for example, Demir e? a^. (1996), is s u m m e d up in Table 1 of Mamontov, Willander and Lewin (1999). W e also note the circumstance which can m a k e implementation of the proposed algorithm more flexible. This algorithm employs only the coefficients of the D S P s , namely: *

*

one <%-vectors g(f,x), the drift vector of the D S P s (see (1.8.2)); it is used in (A.l) (or (2.3.1)), (2.3.3), (2.3.10), (2.3.12), (3.3.2) (or (3.3.4), (3.3.5)) as well as to calculate matrix 3g(;,.x)/3.x (see (2.3.3), (2.3.10), (2.5.1), (3.3.2) (or (3.3.5)), (3.5.30)) and vector (2.3.6) by m e a n s of the F D techniques (see R e m a r k 2.2 in Section 2.3.1) such as the method proposed in Section 2.4.2; vector dg(f,x)/3f used only in (2.3.10) and (3.3.2) (or (3.3.5)) can be evaluated with the help of the F D procedures as well; one symmetric ^f X ^-matrix /7(f, x), the diffusion matrix of the D S P s (see (1.8.3)); it is assumed to be nonsingular (see (2.6.4)), applied to evaluate (2.3.7) as described in Section 2.4.2 and used in (3.5.29) and (3.6.4) involved in (3.5.36).

T h e above vector and matrix depend only on features of a specific family of diffusion processes, i.e. a specific applied problem. Hence their calculations can be provided by the dedicated, say, D I F C O D (Diffusion Coefficients and their Derivatives) subprogramme which is clearly independent of the computational part described with the above Steps (1)—(4). In so doing, different versions of D I F C O D can be developed, each version for a specific problem. This can be helpful if the above analytical-numerical approach is employed in various (and dissimilar) engineering fields. This stresses that the approach can equally be used in different applications.

3.7

Discussion

The main result of this chapter is the analytical-numerical method to determine expectation and variance of a nonstationary high-dimensional D S P . T h e key points on the w a y to this method are:

106

/fn?ar:an% D:/^Ms:on Processes

*

representations (2.2.2) (Theorem 2.1) and (2.2.4), (2.2.8) (Theorem 2.2) derived directly from basic diffusion equalities (1.8.2) and (1.8.3) within D S P theory (see also R e m a r k 2.1 in Section 2.1); * the U M approximation (2.5.7) for matrix (2.2.4) (Theorem 2.4); * derivation of the S D B approximation (3.5.32) (Section 3.5 and Theorem 3.2) for the invariant probability density. T h e results of the chapter correspond to the points mentioned in the purpose of the book in Section 1.13. The s u m m a r y on the comparison with the results by other authors is presented in Table 1 of Mamontov, Willander and Lewin (1999). The proposed approach is applied to covariance and spectral density of stationary D S P in Chapter 4. Various aspects of the present chapter can be the topics for future development. For instance, one of the next steps can be computer implementation of the proposed analytical-numerical method in a parallel m o d e (discussed in Appendix H ) for one of the engineering fields. This research would enable one to m a k e efficient analysis of high-dimensional D S P s available that m a y lead to results of a significant practical importance. They can be even more valuable since the literature still offers little information on stochastic simulation in parallel-workstation environments.

Chapter 4

Stationary Diffusion Processes

4.1

Introduction

This chapter considers the case w h e n D S P % with drift and diffusion functions g, H is stationary (see Definition 1.12 in Section 1.7). The key deterministic characteristics of % are expectation (1.7.17), variance (1.7.18), covariance (1.7.26), (1.7.20), (1.7.28) and spectral density (1.7.21) or (1.7.30), (1.7.31). They are the figures of merit listed in Points (1) and (2) in Section 1.11. They involve both stationary probability density p^ and homogeneous transition probability density p. of the process. The c o m m o n analytical recipes to determine them are described with Theorem 1.3 and Corollary 1.2 respectively in Section 1.12.3. Section 8.2, however, explains w h y these recipes are unsuitable for application under conditions (1.12.4), (1.11.1) and (1.11.2). A s for stationary density p^, it, being a particular case of invariant probability density p; (see (1.7.11)), can approximately be evaluated with the help of the corresponding version of the procedure summarized in Section 3.6. This version is presented in Section 4.11. Unfortunately, the w a y which would enable one to efficiently determine transition density p^ and, at the same time, be compatible with the above conditions is still unknown. The present chapter is focused on another solution of this problem. It develops the approximation for covariance (1.7.26) which does not involve density p^ and is intended to be used in (1.7.21) (or (1.7.30), (1.7.31)) instead of precise covariance C(A). Sections 4.3^1.7 and 4.9 are devoted to this topic. Sections 4.3-4.6 ana107

108

.SfafMnary Dt/%s:oM Processes

lyze various properties of matrix ^C(A)/^A . The asymptotic formula as Aj,0 is considered in Section 4.3. Section 4.4 discusses the flicker effect in connection with this formula. Section 4.5 is devoted to genera! description of 0. Section 4.6 formulates the uncorrelated-matrixes condition and derives the related results. The representations for spectral density under this condition are described in Section 4.7. Under less restrictive conditions, the spectral density can be evaluated on the basis of the deterministic-transition approximation developed in Section 4.9. Sections 4.8 and 4.10 consider the corresponding examples in fluid physics. The proposed approach is summarized and discussed in Sections 4.11 and 4.12 respectively. Section 4.2 below analyzes the results by other authors related to the topic of the present chapter.

4.2

Previous Results Related to Covariance a n d Spectral-Density Matrixes

There are a few papers devoted to covariances and spectral densities of stationary D S P s with nonlinear coefficients. This literature does not help too m u c h if (1.2.14) holds. For example, Zhang (1990) and Dimentberg e? aJ. (1995) deal only with two-dimensional case. Zhang (1994) generalizes the results of Zhang (1990) for multidimensional systems. However, both Zhang (1990) and Zhang (1994) consider only quite particular forms of the diffusion coefficients and the drift vector with "small" parameter. Besides, Zhang (1990) and Dimentberg ef a?. (1995) discuss spectral densities in the case as the diffusion matrixes are singular (in contrast to (2.4.13) or (2.6.4)). A s is well-known, this usually requires a special treatment (e.g., Chapters X - X V of Soize, 1994). Both Zhang (1990) and Zhang (1994) do not take into account this fact. The procedure proposed by Roy and Spanos (1993) m a y seem to be general. It is based on the representation of transition probability density p,,(x ,A,.xJ as the solution of problem (1.12.16), (1.5.7) in the form of formal power series with respect to A (see (5) in Roy and Spanos, 1993). In so doing, the rigorous mathematical justification is not attempted (see the text just below (5) in Roy and Spanos, 1993). O n e can note that the Roy-Spanos approach is also impractical in the present, high-dimensional case (1.2.14). The reason is that the coefficients of the series involve the corresponding

Der:ua^:ue o/'Couar:ance m ^Ae L:m:^ Case o/'Zero T:me Nepara^on

109

powers of operator [X(.x)]* so that the complexity of the resulting expressions becomes prohibitive even if only the first three terms of the series are applied (see (1.11.2)). The Pade approximation and continued fractions discussed on pp. 360-362 in Roy and Spanos (1993) (with no proofs either) do not contribute to overcoming the above difficulties. The formal-series technique is illustrated in Roy and Spanos (1993) with a few examples (see (35), (39), (43), (44) therein). Each of them is associated with a singular diffusion matrix like in Zhang (1990) and Dimentberg ef c^. (1995) (see above). Most of the examples includes the second and third powers of the solutions. A s is well-known (e.g., see R e m a r k 1.15 in Section 1.9), the so-called explosions of the solutions can be the case. However, Roy and Spanos (1993) do not discuss this problem. In other words, the sufficient conditions for existence of the global solutions (e.g., (6.3.4) in Arnold, 1974) are not assured. Thus, the problem pointed out in Section 4.1 were not considered in the literature before. The present chapter is intended to fill this gap. This chapter (similarly to Roy and Spanos (1993)) is completely within D S P treatm e n t (1.9.7). W e also stress that the results below are not limited to one or another specific application field. For example, they can equally be used in microelectronics or economics -or other D S P applications such as physics, medicine, sociology.

4.3

Time-Separation Derivative of Covariance in the Limit C a s e of Zero T i m e Separation

Theorem 4.1 below proposes the exact expression for the derivative of matrix (1.7.26) in the case as A^O. T h e o r e m 4.1 Let the following assumptions be fulfilled. (1) Hypotheses of L e m m a t a 1.6 and 1.7 (see Section 1.7) hold. (2) Function ((r_,^_,^,jc ) = jc_ j^ is regular with respect to stationary D S P X (see Definition 1.5 in Section 1.3 and (1.7.16)). (3) Limit relations (1.8.5), (1.8.6) hold uniformly with respect to x_e R'' (cf., assumption (1) of Theorem 1.3 in Section 1.12.3). (4) Function (x_*eJ[g(xJ-g(e^)]T regular with respect to process % (see Definition 1.5 in Section 1.3). (5) Matrix M (see (1.7.18)) is nonsingular. W e denote

110

Sfaftonary Dt/j^ston Processes

D^=AfJ^

(4.3.1)

where

M, = /(*-'J [g(*W(e,)fp, (*)
(4.3.2)

is the stationary version of (3.2.1). Then the following assertions are valid. (1) Expression (1.7.26) is presented as

C(A)=j* J*(*_-eJ(^-eyp^jp^,-,A,^)^ ^=/ /(*.- e j ( ^ - ^ f p,(^_) P^(^.,A, j;J^_ dx,, for all A > 0 . (4.3.3)

(2) Matrix (4.3.2) exists and equality

lim^(A)=M, A^O

(4.3.4)

^A

holds. (3) Relation

h4c(A)]<^pD.

(4.3.5)

Aj.0

is valid. J"roo/^ is similar to that of Theorem in Mamontov and Willander (1997e) and involves (1.7.26), Definition 1.5, (4.3.3), (1.5.4), (1.8.5), (1.8.6), (1.7.17), (1.7.18), (4.3.2), (1.7.27), and inequality Jl.x-eJp,(.r)
(4.3.6)

that stems from the existence of matrix (1.7.18) (e.g., see (11.3.3) in Arnold, 1974).

111

F/tcAer E/%cf

4.4

Flicker Effect

Theorem 4.1 generalizes the results related to J = 1 of Mamontov and Willander (1997e) for arbitrary
IfgeC^R^then g(*)-g(
for all*,

(4.4.1)

where

D(,) = f ^ * ( ^ ^ K ,

f^u^

(4.4.2)

is the time-independent version of damping matrix (2.5.1). Vector H in (4.4.2) stands for variable e^+n(.x- ej of function g. In this case, matrix Af (see (4.3.2)) in (4.3.4) becomes M,=j*(*-eJ(*-ey[D(x)fp^(;<.)^. y

(4.4.3)

If one assumes that (cf., (2.5.15)) drift function g is linear, i.e. damping function D is independent of* (D(x) = D(eJ),

(4.4.4)

then (4.4.3) is transformed (see (1.7.18)) into M , = !^[D(eJ]T. This leads (see (4.3.4), (1.7.27)) to equation Hm{[C(A)] ^C(A)/JA}^=D(eJ that, compared AJ.0

to (4.3.5), points out that D„=D(e,) if (4.4.4) holds. If (4.4.4) does not hold, then, generally speaking, D ^ D ( e J . In other words, nonlinearity of drift function g causes the difference between D^ and D(eJ. The well-known flicker effect (e.g., see the discussion on pp. 400-402 of Mamontov and Willander, 1997e) can in m a n y cases be associated (see Sections 2 and 3 of Mamontov and Willander, 1997e) with the property that

112

SfaftoHttry Dt/^MSto?: Processes

the real part of every eigenvalue of matrixes D^ and D(eJ is negative

(4.4.5)

and the fact that matrix D greatly differs from matrix D(e ) that, in the scalar case, is expressed as D(eJ/D^l.

(4.4.6)

R e m a r k 4.1 The fact that, in the scalar case, the flicker effect can be caused by the strongly different values of the damping was revealed in the pioneering analysis by M . Surdin (1939). Surdin's treatment is, however, approximate and rather schematic. In a more consistent reading, its results mean that the damping must be nonlinear (cf., (4.4.2)) with very different values for instance, shown in (4.4.6). A n example of the physical problem where (4.4.6) holds is described in Section 4.8. More details on the nonlinear approach to the flicker effect can be found in Section 2.1 of Mamontov and Willander (1997e). W e return to the flikcer effect and its nonlinearity-related origin in Section 4.12 in connection with the example in Section 4.10.

4.5

Time-Separation Derivative of Covariance in the General Case

This section develops the general expression for JC(A)A%A for all A > 0. W e first consider auxiliary L e m m a 4.1 and then present Theorem 4.2 on the formula for dC/dA. L e m m a 4.1

W e denote 9(A,xJ=J*(x_-eJp/.x_)p,,(x_,A,.xJdx_,

for all ^ e R ^ , A > 0 , 3(0,*J = lima(A,*J,

forall^SR^,

(4.5.1) (4.5.2)

where the limit is assumed to be uniform with respect to ^ . Let the assumptions below hold. (1) Hypothesis of Theorem 4.1 is fulfilled.

Der:ua?tue o/' Couartance :n ^Ae Genera/ Case

113

(2) Hypothesis of Theorem 1.3 (see Section 1.12.3) is fulfilled. Then the following assertions are valid. (1) Function 9 exists for all A > 0 , equality C(A)=fu(A,^)(^-eJ^,

forallA>0,

(4.5.3)

holds and u(0,*J = (;t—e,)p,(*J,

for all ^ .

(4.5.4)

(2) Function 3 is the solution of K F E (1.12.16) under initial condition (4.5.4), i.e.

^ ^ - ^ = [7sT(^)]^(A,^), 9A .Proo/

forall^, A > 0 .

(4.5.5)

is presented in Appendix E.l.

This lemma is used in the following theorem. Theorem 4.2 Let functions 7f and t)(A, -) be differentiable and the assumptions below be valid. (1) Equalities Km

{(^^-^J[C(^)^(A,^)]+(l/2)u,(A,^)77(^)eJ = 0,

<xj-*°°

forallA;,? = l,2,..., 0, x ^ and e^ ^ are the %th entries of vectors x^ and e^, &,(A,.xJ is the J th entry of vector &(A,xJ, e^_ is the M h unit vector, and G(x) is the time-independent version of differential operator G(f,.x)(see (1.12.6), (1.12.7)). (2) Hypothesis of L e m m a 4.1 is fulfilled. (3) Integral J^&(A,^J[g(jcJ]^jc^ converges uniformly with respect to

A>0. Then the following assertions are valid. (1) C(ECt([0,=.)). (2) Equality

^^Lt/(A), JA

for all A > 0 ,

(4.5.7)

114

<Sfaf:o?mry Dt/^ston Processes

holds where

^(A)=j*jj*(^-^)[g(^)-g(^)]^p,(^p,(^,A,^)^A^, R'' [ R"

J forallA>0,

(4.5.8)

and lim t/(A)=Af^. jProo/

(4.5.9)

is presented in Appendix E.2.

Theorem 4.2 provides general representation (4.5.7) for JC(A)/JA . It is valid not only in the limit case as AJ,0 but also for all A > 0 (see (4.5.7), (4.5.9), (4.3.4)). In principle, covariance matrix C(A) for all A > 0 can be calculated by means of (4.5.7), (4.5.8) and (1.7.27). However, as is noted in Section 4.1, this w a y is impractical in the present, high-dimensional case (1.2.14) since it involves density p.(x ,A,xJ determined as the solution of Eq. (1.12.16) in 6?-dimensional space (see Theorem 1.3 in Section 1.12.3). Is it possible to avoid explicit application of this solution? Sections 4.6 presents the positive answer resulting from expression (4.5.7).

4.6

Case of the Uncorrelated Matrixes

Let us consider covariance ^ of random matrixes [x(' ;f)"^ ][X('-^ A)-e ]T and D[x(',f+A)] associated with stationary D S P %. This covariance is written as ^(A)=E{{[x(^)-eJ[x(^)-^f-K}{D(x(',f+A))-E[D(x(,^A))]}^} (4.6.1) and is the stationary version of nonstationary covariance (2.5.4). It enables one to express t/(A) (see (4.5.8)) in (4.5.7) by means of C(A) and D,, = E[D(x(<,f))]=jD(*)p,(x)J*. R'

L e m m a 4.2 Let the assumptions below hold. (1) Hypothesis of Theorem 4.2 is valid.

(4.6.2)

115

Case o/ f/:e U^correZa^ed Ma^rt^ces

(2) If matrix D(x) introduced with (4.4.2) exists for all x, then function (x_-e )(j^-e )^[D(xJ]^ is regular with respect to process xThen the following assertions are valid. (1) Function D introduced with (4.4.2) exists for all x. (2) Matrix D introduced with (4.6.2) exists. (3) Function 7^exists for all A > 0 and is presented as R,(A) = JC(A)/
for all A > 0.

(4.6.3)

(4) Relation R,(A) = 0,

for all A > 0 , (4.6.4)

is equivalent to L(A) = -C(A)Dj for all A > 0 and if (4.6.4) holds, then JC(A)/JA = C(A)Dj, i*roo/

forallA>0.

(4.6.5)

is similar to that of Theorem 2.4 (see Appendix D.6).

Quantity R^(A) is important for the following two reasons. *

In many applied problems, relations limC(A) = 0,

(4.6.6)

lim^)=0

(4.6.7)

A-+M

JA

hold, so (see (4.6.3)) one obtains lim ^ (A) = 0. This means that matrixes [x(',f)-eJ[x(',f+A)-ejTandD[x(-,f+A)] are usually uncorrected for any f in the limit case as A -* oo. Relation (4.6.4) presents the stationary version of the U M condition (2.5.6). * Quantity 7^ (A) enables one to point out the family of D S P s with nonlinear coefficients which, however, admits the linear-coefficient treatment of the covariance. This topic is discussed in the theorem below. T h e o r e m 4.3 Let the assumption below hold. (1) Hypothesis o f L e m m a 4.2 is valid. (2) T h e U M condition (4.6.4) holds. (3) Inequality det [H(x)] ^ 0,

for all x E R*',

(4.6.8)

116

<S^a%onary Dt^Hston Processes

is valid and matrix "„=.f"MP,(*)^

(4-6.9)

R"

exists. (4) The condition that real part of every eigenvalue of matrix D

is negative

(4.6.10)

forallA>0,

(4.6.11)

is fulfilled. Then the following assertions are valid. (1) Matrix 7^ is positive definite and equalities C(-A) = exp(AD„)^„

C(A) = ^ e x p ( A D j ) , D,, = Z),, ^ ; , ^ K ^ +^

(4.6.12) = 0.

(4.6.13)

hold. (2) Relations (4.6.6) and (4.6.7) are valid. (3) Variance matrix P^ (see (1.7.18)) is the unique solution of Eq. (4.6.13), it is expressed by means of formula ^ = j*exp(KDJR,,exp(KDj)
(4.6.14)

and is positive definite. Proo/

is presented in Appendix E.3.

Note that matrixes (4.6.9) and (4.6.2) are the stationary versions of matrixes (2.6.2) and (2.6.3) respectively. H o w restrictive is uncorrelated-matrixes ( U M ) condition (4.6.4)? If, for instance, every column of matrix [/(-,f)-ej [%(''?+A)-ejT ^ ^ every column of matrix and D [%(',?+A)] are uncorrected for all f, A > 0 ,

(4.6.15)

then (4.6.4) is obviously valid. Properties (4.6.15) and (4.6.4) are equivalent at ;f = l. However, if J > 1 , then (4.6.15) does not generally follow from

Specfra/ Denst(y m (Ae C/^eorrg/at^-Ma^r^es Case

117

(4.6.4). Thus, U M condition (4.6.4) is weaker than "by-column" uncorrelatedness (4.6.15). W e also note that properties (4.6.4) and (4.6.15) are similar to properties (2.5.6) and (2.5.12) respectively. If hypothesis of Theorem 4.3 is fulfilled, then its assertions show that stationary process % has the same covariance and spectral density as the stationary Ornstein-Uhlenbeck process with drift vector D ^ x and diffusion matrix //,, [e.g., (8.2.12) and Section 8.3 of Arnold (1974); (4.4.54) and (4.4.58) in Gardiner (1994)]. This fact is quite unexpected and surprising! Indeed, it demonstrates that even in the nonlinear case and even if both diffusion N and damping D depend on x, the linear treatment (4.6.11) -(4.6.14) can be applied provided that the U M condition (4.6.4) holds. This feature greatly simplifies analysis of spectral density (1.7.21). The corresponding representations are summarized in Section 4.7. The U M condition (4.6.4) implies (see (4.6.12)) that matrixes D^ and D ^ are equal to each other. The above approach can also be applied even if they differ slightly, i.e. D , = D„.

(4.6.16)

However, in general, this relation need not hold. Comparison of (4.6.2) and (4.3.1) (see also (4.4.3)) points out that matrix D ^ generally differs from matrix D^.

4.7

Representations for Spectral Density in the Uncorrelated-Matrixes Case

If hypothesis of Theorem 4.3 (see Section 4.6) is valid, then (4.6.13) and (4.6.11) hold. In this case (e.g., see (4.4.6.e) in Gardiner, 1994) S(/)=2(D,+ tM/)-\f7,(Dj-tM/)"i,

for allf,

(4.7.1)

where t is, as in Section 1.7, the imaginary unit, / is the identity matrix, M is coupled with / by means of (1.7.22), and matrix D^+to/ is nonsingular for all / because of (4.6.10). Since matrix ^ is positive definite (see assertion (1) of Theorem 4.3), matrix ^(/) is positive definite for all /. It follows from (4.7.1) that

118

<S?a%onary D:/^Ms:on Processes

Re^(/) = 2 ( D ^ ^ 7 ) i ( D ^ ^ D j + ^ ^ ) [ ( D j ) ^ ^ 7 ] - i and Im^(/) = 2 M ( D ^ ^ / ) - ^ ( D „ ^ ^ D j ) [ ( D j ) ^ ^ 7 ] ^ for all f,

(4.7.2)

where matrix D ^ + to^/ is nonsingular for all /. Expressions (4.7.2) show that properties Im^(/) = 0,

for all/,

(4.7.3)

and matrix D^77^ is symmetric

(4.7.4)

are equivalent. If any of (4.7.3) and (4.7.4) holds, then ^/)=Re^(/) = 2 ^ [ ( D j ) ^ ^ 7 ] - ^ 2 ( D ^ M ^ ) - ^ ,

for all/.

The above formulas can help to evaluate the spectral-density matrix in practice. 4.8

Example: Comparison of the D a m p i n g s for a Particle N e a r the M i n i m u m of Its Potential Energy

This section presents the example of the applied problem where damping D ^ (see (4.6.2)) can significantly differ from the asymptotic damping D^ (see (4.3.1)). Let us consider a particle with mass /M > 0 and position z in R in the field of its potential energy w(z). The particle undergoes stochastic motion described with the following version ^z=^f, ?w T

(4.8.1)
(4.8.2)

of I S O D E system (1.9.3) where T > 0 is the relaxation time of m o m e n t u m w v of the particle, x is the velocity of the particle, and the last term on the right-hand side of (4.8.2) presents the stochastic driving force multiplied by <%;. Parameters w , T and & are independent of z, .x and f. It is assumed that function w is defined and sufficiently smooth on R.

Co?nj9ar:so?: o/' ^Ae D a m p m g s /or a Par^tc^e

119

In m a n y real-world problems, this function is also uniformly bounded on R and has at least one local minimum. Let point z = 0 be one of these minima and 6pw(0)/^z^>0. Thus, if z is close to 0, then M,(z)="l^ T 2

(4.8.3)

a>0.

(4.8.4)

where

In what follows, w e consider behavior of the particle only in a neighborhood of point z = 0. In the case as 5 = 0, i.e. as equation system (4.8.1), (4.8.2) is deterministic, one can show by means of applying (4.8.3) to (4.8.2) that the left part of (4.8.2) can be neglected if 2a-r
(4.8.5)

A s a result, the fast component is ignored so that the corresponding model describes slow component of the particle motion. W e assume that (4.8.5) holds and (heuristically) omit the left part of (4.8.2) also in the stochastic case, i.e. as & ;* 0. Then velocity x is eliminated from (4.8.2) with the help of (4.8.1) and system (4.8.1), (4.8.2) is rewritten as a single equation, namely,
^gg)

Jz

A similar model in fact derived by means of assumption (4.8.5) is applied, for instance, to study particles in colloidal suspensions (e.g., (8) in Lowen ef aZ., 1991) and some other particle systems (e.g., (10.6) and Section 10.2 of Klimontovich, 1994). It is also used by Malakhov (1995) to study the phase transitions of the first kind. Since function w is uniformly bounded on R, it inevitably deviates from parabolic law (4.8.3) if z is not too close to 0. To be specific, w e consider, for example, the following form of force -
(4g?)

120

^a^onaTy D^Msm?: Processes

in (not too small) neighborhood of point z=0. Equation (4.8.7) implies w(z) = w(0) + (w/T)(a/2)(c/&)^ln[l + (c/^z^]

(4.8.8)

and transforms (4.8.6) into ^z = — ^f+A^WS,!'). [l + (c/&)2z2]

(4.8.9)

Damping coefficient (4.4.2) becomes D(z) = . [l + (c/6)2z2]

(4.8.10)

If c =0, then energy (4.8.8) is parabolic, i.e. relation (4.8.3) holds, and (4.4.4) is valid. If c^O,

(4.8.11)

then energy (4.8.8) is nonparabolic and (4.4.4) is not valid. In other words, the nonparabolicity is equivalent to the property of damping (4.8.10) to be dependent of z. Beyond the above neighborhood, potential energy w has the z -dependence which differs from (4.8.8) and can in general be of a quite complex form. However, w e concentrate below on the effect of nonparabolicity of energy w due to parameter c. For this reason, we disregard actual behavior of w far from zero and assume that (4.8.8)-(4.8.10) hold for all z e R. Thus, the corresponding results can be applied to the DSPs granted with Eq. (4.8.9), that is a simplified model of the ISODE system (4.8.1), (4.8.2) describing the particle near point z=0 of the minimum of its potential energy. Some properties of the stationary solution of ISODE (4.8.9) were studied in Section 4 of Mamontov and Willander (1997e). The corresponding results include the following statements. *

Solutions of ISODE (4.8.9) are the DSPs with drift and diffusion coefficients described with equations g(z) = -az/[l+(c/6)2z2],

*

,r/(z) = &2.

(4.8.12)

If c^/(2a)
Comparison o/^Ae Dampmgs /or a Parf:cfe

p^(z) = p/0)[l + ( c / ^ z ^ ] ^ \

for all z.

121

(4.8.13)

T h e stationary D S P determined b y it h a s zero expectation, i.e. e, = 0,

(4.8.14)

D(e,) = -a.

(4.8.15)

and

*

Inequalities 3c^/(2a)
(4.8.16)

2a [l-3c2/(2a)]

(4.8.17)

then

^

and asymptotic zero-time-separation damping (4.3.1) (see also (4.3.5)) has the form D„=-a[l-3c2/(2a)].

(4.8.18)

Comparing (4.8.18) with (4.8.15), one can readily see that (4.4.6) holds if and only if any of inequalities l/[l-3c^/(2a)]^l,

t^2/(2a)

(4.8.19)

hold. Thus, if any of (4.8.19) is valid, one can expect (see R e m a r k 4.1 in Section 4.4) that spectral density of the above stationary D S P performs the flicker effect. Let us examine relation (4.6.16). To do that, w e calculate damping (4.6.2). In so doing, w e assume that the variance exists, i.e. inequality (4.8.16) is valid. It follows from (4.6.2), (4.8.10), (4.8.16) and (4.8.13) that

D „ = -ap,(0).f[l+(c/6)2z2]-(i^)Jz,

(4.8.20)

where n u m b e r p,(0) is determined from condition (1.7.13) for density (4.8.13), i.e.

122

.Sfa?:onary D:/%s:o?: Processes

P,(0)j*[l+(c/^z^] ^ ' j z = l .

(4.8.21)

The integrals in (4.8.21) and (4.8.20) can be expressed by means of the beta function B (Euler's integral of the first kind). Namely, in view of evenness of the integrands, the well-known formula 3.251.11 on p. 343 in Gradshteyn and Ryzhik (1994) and inequality (4.8.11), one obtains f[l + (c/6)2z2]-'/''jz=(&/c)B(l/2,a/c2-l/2),

f[l + (c/6)2z2]-(tW'')Jz=(A/c)B(l/2,a/c2 + l/2).

Since B(l/2,H/c2-l/2)=r(l/2)r(a/c2-l/2)/r(a/c2), B(l/2,a/c2 + l/2)=r(l/2)r(3/c2+l/2)/r(a/c2 + l) where T is the g a m m a function, equality (4.8.21) becomes p/0) = (c/6)r(a/c2)/[r(l/2)r(a/c2-l/2)].

(4.8.22)

Then (4.8.20) is transformed into D ^ = -a [r(a/c^)r(a/c^ + l/2)]/[r(a/c^+l) xT(a/c^ -1/2)] or.becauseofproperty r(y+ l) = yT(y) forall y > 0 (seealso (4.8.4), (4.8.11), (4.8.16)), into D „ = -a[l-c2/(2a)]<0.

(4.8.23)

Comparison of (4.8.18) and (4.8.23) leads to the issues below. Condition D = D holds if and only if c = 0, i.e. potential energy w (see (4.8.8)) is parabolic, function g in (4.8.12) is linear, and damping D (see (4.8.10)) is independent of z . Then Theorem 4.3 m a y be applied. If c ^ 0, then D ^ ^ D^. In this case, potential energy w (see (4.8.8)) is not parabolic, function g in (4.8.12) is nonlinear, damping D (see (4.8.10)) does depend on z. Hypothesis of Theorem 4.3 cannot hold since its assertion (4.6.12) is not valid. Inequality (4.8.16) prescribes the range for c^ in the following way: 0 < c ^ < 2 a / 3 . If c^ monotonously increases from 0 to 2a/3, then ratio D^/D^ monotonously increases from 1 to oo and variance (4.8.17) monotonously increases from &^/(2a) also to oo.

7%e DeterwMn:s(tc-7!rans:f:o?t Approxtma^ton

4.9

123

T h e Deterministic-Transition Approximation

The above example not only illustrates some terms used in Theorem 4.3 (see Section 4.6) but also shows the limits of applicability of the practical method suggested by this theorem and specified in Section 4.7. O n e of the key issues is that, if the nonlinearity of the drift function is sufficiently weak, then the method can be adequate. If, on the contrary, the nonlinearity is strong (matrix D^ greatly differs from matrix D ^ ) , then the method is not very useful. Other techniques should be applied instead. O n e of the them is developed in the present section. If hypothesis of Theorem 4.3 fails, one m a y try to apply such w a y to determine C(A) that does not involve the U M condition (4.6.4). Representation (4.3.3) is not suitable either because it is impractical (see Section 4.1). Hence some other, perhaps approximate expression for C(A) must be constructed. This approximation should agree with some of the basic properties of the covariance, namely, (1.7.27), (4.3.4), (4.3.5). Let us involve the deterministic-transition (DT) approximation (A.4) in (4.3.3). For stationary D S P x< this approximation has the form p^_,A,^) = 8 [ ^ ^ _ , A ) ] ,

foralljc,^eR^, A > 0 ,

(4.9.1)

forall^jrJeR^, A > 0 ,

(4.9.2)

where ^(x_,A) = ri(;,x_,;+A),

and .x^=i)y(.x ,A) is the unique solution of initial-value problem J.x -^=g(*J, aA

A>0,

*JA=o^-

(4.9.3) (4.9.4)

which is the stationary version of problem (A.l), (A.6). Substituting (4.9.1) into (4.3.3), one obtains C^(A)=j*(^-eJ[^_,A)-ejTp(^)^^

forallA>0.

(4.9.5)

Subscript "dt" in (4.9.5) and the formulas below point out that the corresponding relations are based on the D T condition (4.9.1). Function C . is analyzed in the following theorem.

124

.%a?:onary Dt/^us:on Processes

T h e o r e m 4.4 Let the assumptions below hold. (1) Solution \];(.x ,A) is defined for all A > 0 at every fixed x_E R*^. (2) geCi(R'). (3) Hypothesis of L e m m a 1.7 (see Section 1.7) holds (so vector e can be determined with (1.7.17)). (4) There exists number 0, > 0 such that HljF(*_,A)-eJ<e.ll.x_-eJ,

forall^_e^, A > 0 .

(4.9.6)

(5) Matrix P^ is nonsingular. (This is the same as assumption (4) of Theorem 4.1.) Then the following assertions are valid. (1) lj7eC^(]R''x[0,oo)). (2) Function C ^ introduced with (4.9.5) is defined and continuous for all A > 0 , i.e. Q,€C°([0,oo)), and relations IIC,,(A) II < O. tr!^,

for all A > 0,

C„(0)=!imC„(A) = ^,

(4.9.7) (4.9.8)

A^O

hold where the matrix norm is spectral. (3) If integral ^(A)=j*^_-eJ[d^,A)/3Afp^(^)^

(4.9.9)

converges uniformly in A > 0 , t h e n Q,eC\[0,oo)). If, besides, vector e^ (see assumption (3)) is such that g(e,)=0,

(4.9.10)

then ^^=^(A)^.-^){g[^,A)]-g(eJ}^p,(^.)^., R**

for all A > 0 ,

^Q,(A)

lim—^-L=M,, A^O

<^A

(4.9.11) (4.9.12)

7%e De^erHMfMs^c-Trans/fM)?: Appro^cma^toH

H m j [ Q , ( A ) ] ^ ^ ^ j =D..

froo/

125

(4.9.13)

is presented in Appendix E.4.

Theorem 4.4 is illustrated in Section 4.10 with the example from Appendix C. W h a t is the meaning of Theorem 4.4? Comparison of (1.7.27) and (4.9.8), (4.3.4) and (4.9.12), (4.3.5) and (4.9.13) points out the following. Matrix C. (A) introduced with (4.9.5) by means of the D T condition (4.9.1) can serve as a viable approximation for covariance matrix C(A) (see (4.3.3)). This result is used in the analytical-numerical approach described in Section 4.11 below. If J*llCa(A)ll
(4.9.14)

then the corresponding spectral density ^ ( / ) = 2fc,,(A)exp(-t2n/A)JA, for all/,

(4.9.15)

exists. W e hereafter call function C^, ^ and ^ the D T approximations for C, ^ and t/ respectively. Difference between ^(/) and <S^(/) generally depends on specific applied problem. Apparently, the most practically meaningful w a y to estimate this difference in various engineering fields is to undertake the following steps: * to select a few typical problems in a specific field for which the experimental data for spectral density 3(/) can be obtained; * to numerically determine the D T approximation (4.9.15) in the corresponding frequency ranges; * to compare the experimental and numerical results. This can be a topic for future research. Equation (4.9.5) is based on (4.3.3) but, in contrast to (4.3.3), has the advantages crucial in practice: it does not involve transition probability den-

126

-Sfaftonatry D:/?Mston Processes

sity p. and eliminates the integration over R'' with respect to x^ in (4.3.3). The latter reduces the 2^-fold integral in (4.3.3) to the ^-fold integral in (4.9.5). Both the advantages are especially important in the present, highdimensional case (1.2.14). This means that, under hypothesis of Theorem 4.4, representation (4.9.5) resolves the problem in the focus of the present chapter (see Section 4.1).

4.10

E x a m p l e : Non-exponential Covariance of Velocity of a Particle in a Fluid

To illustrate the results of the previous section, w e show what they can give to the example of the fluid-dynamics problem considered in Appendix C. It is even more important since Appendix C does not provide any technique to evaluate the covariance and the corresponding spectral density (see the last paragraph of Appendix C.4). 4.i0.i

C o u a W a n c e wt f/te generaZ case

In the case described in Appendix C, function i]j (see (4.9.2)-(4.9.4)) depends also on position z, i]f(x_,A) = ^(z,x_,A) = ^v(z,H,y?,A)yJ^, for every fixed z E Q ,

(4.10.1)

where v(z,M,y_,A) is the solution of scalar O D E =<M

, for every fixed z € Q ,

(4.10.2)

T(z,n,y)

with initial condition Y]A. 0 is the square root of y at x=.x_, i.e. y^^l/^=c^\_.

(4.10.4)

127

Afon-exponenfKt^ CoMaWatHce o/ tAe Par^c^e VeJoe:^y

For the reasons discussed in R e m a r k 4.2 (see Section 4.10.2 below), w e consider only the case as the above solution is defined for all A > 0. Then Q(z,n,A)=j*^v(z,M,y^A)y^Jt_^p^_)^_,

for every fixed z E Q and all A > 0, or, in view of (C.3.1), (4.10.4) and (C.3.7),

1

Q , (z,"-A) = ^. - t°^")] f Jv(z,w,y^,A)y^y'T (z,n,y') exp 3 ^ n Tg(z,n) { ^ for every fixed z E Q and all A > 0 .

^y.2 (4.10.5)

This is the D T approximation to determine covariance (C.3.7). If quantity (4.10.5) is of property (4.9.14) (see also (C.3.12))

f]Q,(z,H,A)ldA<°o,

for every fixed z e U ,

(4.10.6)

then the corresponding spectral-density function <$^, can be written analogously to (4.9.15) (see also (C.3.11)), i.e.

^(z,n,/) = ^(z,/:,-/) = 4J'C^(z,^A)cos(2Tt/A)JA, o for every fixed z E U and all / > 0 .

(4.10.7)

To our knowledge, Eqs. (4.10.5) and (4.10.7) are the first analytical (though approximate) formulas for covaraince and spectral-density matrixes of stochastic three-dimensional velocity of the fluid particle with nonlinear friction, i.e. with the momentum-relaxation time dependent on the particle kinetic energy. The formulas allow for time-independent position z of the particle and time-independent concentration n = n(z) of the particles in the fluid. The concentration can be subject to a boundary condition at the boundary 3Q of the bounded or unbounded domain Q occupied by the fluid. W e cannot point out the analogous result in statistical mechanics.

128

.Sfaftonary Dt/^Ms:on Processes

rc^a^c
for every fixed z e Q ,

(4.10.8)

so one readily obtains

v(z,yAA)={l + 2 e [ A / ^ ( z , n ) ] y ^ p ^ , for every fixed z S U and all A > 0.

(4.10.9)

In this case, solution (4.10.1) is written as

HF(z,jc_,A)={l + 2 e [ A / ^ ( z , H ) ] ^ p ^ _ , for all x_, every fixed z e U and all A > 0 .

(4.10.10)

Accounting (C.2.4), one can readily check that property (4.9.6) at 0.= 1 holds for (4.10.10). W e also stress that quantity (4.10.10), as a function of A , is defined for all A > 0. R e m a r k 4.2 Equation (4.10.8) at e = l/2 is similar to the Ridley O D E (see (4.140) in Ridley, 1988) in the particular case of low energies. The latter equation can be written in the form of (4.10.2) but is more general than (4.10.8). At high energies, it suggests such asymptotics for the relaxation time in (4.10.2) which can roughly be described still with Eq. (4.10.8), however, at another value of e, namely, e = -3/2 (cf., Eq. (C.4.1) in this work and Eq. (4.138) in Ridley, 1988). O n e can show that initial-value problem (4.10.8), (4.10.3) has solution v(z,n,yAA) defined for all A > 0 if and only if e > 0 (cf, (C.4.2)). At Ridley's value e = - 3/2 (and at any negative e), the solution which is the normalized kinetic energy of the fluid particle is determined for A only on some bounded time interval depending on fixed y_ (see (4.10.4)) in (4.10.3). Beyond this time interval, the energy merely does not exist! This imperfection comes from the qualitative inadequacy of Ridley's momentum-relaxation time for the electron-electron scattering (e.g., see (4.138)

129

M)H-exj90Henf:a^ CouarMnce o/' ^Ae Par^tcfe Ve/oc;fy

in Ridley, 1988) at high energies. Note that Ridley refrains at all from analysis of his above O D E , even in simple particular cases (see also the discussion in Appendix C.4). Curiously, in spite of this severe problem, the above Ridley treatment is sometimes referred with no criticism (e.g., see p. 79 in Lundstrom, 1992). The resulting issue is that the physically consistent models for the momentum-relaxation time in (4.10.2) (or (C.1.8)) should assure the property of solution v(z,n,y_, A) of initial-value problem (4.10.2), (4.10.3) to be defined for all A > 0 rather than on a bounded interval of values of A (on this topic, see also R e m a r k C.l in Appendix C.l). Substitution of (4.10.9) and (C.4.1) into (4.10.5) leads to

2''

C„(z,",A) =

3(l-2e)r[(l-2e)/2]

[o(z,H)]'

xj*{l + 2e[A/^(z,7!)]y^p^'^exp

<^-,

for every fixed z S Q and all A > 0.

(4.10.11)

This is the expression for covariance C.(z,n,.x) in quadratures. O n e can show that covariance (4.10.11) meets requirement (C.3.12) at C = C ^ if and only if

ee[0,l/2).

(4.10.12)

So, w e shall replace (C.4.2) with more restrictive condition (4.10.12). If (4.10.12) holds, then both standard deviation (C.4.6) and covariance (4.10.11) exist. The latter enables one to evaluate diffusion parameter (C.3.10) as (see also 191 and 860.17 in Dwight, 1961) r,,

, l-4e/3r[(3-4e)/21

Z)(Z,M)=

^

^

LI

2'

Z

,

,,/

\i2

i ^ (z,H)lo(Z,M)r,

T[(3-2e)/2] °*" ^ ^' ^ ' for every fixed z € Q ,

or, equivalently, in the form of (C.2.6) where standard deviation o(z,n) and effective relaxation time T(z,n) (see R e m a r k C.3 in Appendix C.3) are determined with (C.4.6) and

130

<S%a%o?mry Dt/^ston Processes

-,

,

-C(Z,M) =

^

^

2 ' ' (l-4e/3) r[(3-4e)/2] ^

^ —)A

/

,

,

J. -c^(Z,^)

l - 2 e ( l - 2 e / 3 ) T[(l-2e)/2] °^

^

for every fixed z s U .

(4.10.13)

T h e diffusion parameter also provides the value of spectral density (4.10.7) at zero frequency (cf., (3.13.3)). Equation (4.10.11) also points out (again by m e a n s of 860.17 in Dwight, 1961) the following asymptotic formula (see also (C.3.9) at C = C ^ )

for every fixed z e U and e € (0,1/2) in the limit case as A - ^ oo.

(4.10.14)

T h e next section considers a specific particular case of simple fluid.

4.10.3

Cnseo/f%te%mr<%-spAere/ZMM%

A particular case of a simple fluid is the hard-sphere fluid. S o m e of its parameters are discussed in Appendix C.l (see the text on (C.1.16)-(C.1.19)). T h e present section assumes that, for the hard-sphere fluid, Eq. (C.4.1) holds at e = l/3 that is allowed by condition (4.10.12). T h e n all the results of Section 4.10.2 are applied at this value of e. W e suggest the reader to implement the corresponding substitutions. This section points out only the t w o resulting expressions:

C ., (z,w, A) = — [o(z,n)l ^ ^ ^ T(l/6)^ ^ ^ /

?\

X / {1 + (2/3) [A/T.(z,n)] y ?" }*'" y _"^ exp y

2,

for every fixed z S Q and all A > 0,

<^y-,

(4.10.15)

JVbft-exponertf ta? Couartance o/° ^Ae Par^c^e Ve/oet(y

131

C^(z,n,A)^^n^ [ o ( ^ ) ] ^ - ^ - ^ , ^^ ^ 3 T(l/6) ^ ^ [3 ^(z,n)j for every fixed z E Q in the limit case as A-* oo.

(4.10.16)

For the hard-sphere fluid, the non-exponential (i.e. different from (C.2.8)) asymptotic formula similar to (4.10.16), namely, C(A)-A*^,

in the limit case as A^-oo, (4.10.17)

was discovered in the molecular-dynamics numerical simulation by Alder and Wainwright (1970) (see also the references therein). The corresponding experimental confirmations are discussed in Section 3.2 of Weitz e? 6^. (1993), the paper devoted to the diffusing-wave spectroscopy (DWS). The D W S technique is perhaps the most advanced one to measure the particlevelocity covariance. Statistical fluid mechanics undertook many attempts to analytically describe the Alder-Wainwright result (e.g., (XII. 151), (XII. 177), (XII. 194) in Resibois and De Leener, 1977; (8.7.25) in Hansen and McDonald, 1986). In spite of the fact that these attempts to a certain extent clarify the phenomenon, they cannot be considered as completely satisfactory from the modelling viewpoint for the following reasons. *

*

The derivations are based on the time- and space-related frequencydomain representations. The core tool is the Fourier transform. As it is well-known (e.g., pp. 267-268 of Landau and Lifshitz, 1987); Sections 7.3.7 and 7.3.8 of Vol. II of Sedov, 1971-1972), the space-related-frequency formalisms, for example, the wave-vector one, are meaningful only if the fluid is arbitrarily assumed to occupy the whole three-dimensional space, i.e. the universe, or, in terms of domain Q (see Appendix C), Q = R^. This automatically ignores the boundary condition for concentration w (see (C.1.29)) in (4.10.16). O n the contrary, they are naturally accounted in the present approach (see the text between (C.1.7) and (C.1.8)).

N o fully time-domain results are available (probably due to the lack of the corresponding techniques in statistical fluid mechanics). * The effects of nonlinearities are not considered. More specifically, if linear Markov models are not sufficient, the statistical fluid mecha-

132

Sfaftonary Dt/^Ms:on Processes

*

nics jumps over a vast field of nonlinear Markov descriptions directly to non-Markov models which, in so doing, are tied down to the linear-only ones with the help of the memory-function formalism. This scheme is prescribed, for instance, on pp. 324-325 of Resibois and D e Leener (1977) (see also R e m a r k 1.6 in Section 1.3) and employed by m a n y authors, sometimes in a fairly complex form (e.g., see (4.2) and Section 4 of Bocquet, 1998) with no discussion on the corresponding practical methods. N o sharp connections to mathematical theory of stochastic processes, random fields and S D E s are pointed out (that m a y seem to be a kind of tradition in statistical mechanics).

The treatment in Appendix C and the present section attempts to fill the above (fairly huge) gap. The corresponding analytical version of the Alder-Wainwright asymptotics (4.10.17) is Eq. (4.10.16) obtained by means of assertions (1) and (2) of Theorem 4.4 (see Section 4.9). This theorem also provides full description (4.10.15) of the covariance, i.e. at any A > 0 . W e stress that both functions o and tr in (4.10.15) and (4.10.16) depends on concentration (C.1.29). The first of these functions is described in Appendix C.l. The second one can in principle be derived by means of theory of particle scattering in fluids. The A -dependence in Eq. (4.10.15) for the covariance for all A > 0 points out that it is able to describe the numerical results presented with the triangles in Figure 2 of Alder and Wainwright (1970) (or in Figure XII.3 of Resibois and D e Leener, 1977). Comparison of Eq. (4.10.16) with the analogous analytical results of the statistical fluid mechanics cited above involves the physics-related details which are beyond the present book. This comparison can be the topic of a separate research. The Alder-Wainwright non-exponential asymptotics (4.10.17) gave birth to a great enthusiasm in fluid theory. M u c h efforts have been spent to reveal, experimentally, numerically or theoretically, similar asymptotics in m a n y fluids. Not every of these attempts turned out to be successful. O n e of the reasons is that a lot of fluids are dispersed in other media and hence are not simple, i.e. they have m u c h more complex scattering pictures. More specifically, the total momentum-relaxation time (C.l.8) (cf, (C.4.1)) in dispersed fluid is generally contributed not only by the particle-particle scattering (as is the case in simple fluids) but also with other (and usually very different) scattering mechanisms. The mechanisms other than the particle-

Nw-exponenftaZ CouarMtnce o/' ^Ae Pctr^tc^e Ve/oe:^y

133

particle one can significantly reduce its relative contribution. For instance, Ferry's (1980) intention to reveal behavior (4.10.17) in the electron fluid in crystalline silicon pays no attention to that this fluid is dispersed rather than simple. A s a result, the A ^-dependence w a s found only within a bounded interval of values of A (see the figures in Ferry, 1980). This fact obviously has no relation to the limit case as A-* o° and, thus, to asymptotic formula (4.10.17). Before trying to discover non-exponential asymptotics for the particle-velocity covariance in a fluid, one m a y w a n t to thoroughly determine which specific scattering mechanisms in the fluid form the total relaxation time (C.1.8) and w h a t are the specific expressions for the individual m o m e n t u m relaxation times T^ involved in it. T o evaluate the covariance and the corresponding spectral density in the general case, one can apply the results of Appendix C.3 and Section 4.10.1.

4.i0.4

S M M M M a r y o/fAe procecfMrc tn f/te Fettered case

T h e results of Appendix C.3 and Section 4.10.1 present the analytical basis to approximately analyze covariance C^(z,w,A) and spectral density ^^(z,n,/) of any entry of the stationary D S P with drift vector (C.1.26), diffusion matrix (C.1.28) and properties (C.1.8)-(C.1.10), (C.1.29). T h e basic points of the corresponding procedure are summarized below. T h e input data are: arbitrary point z (see (C.1.2)) in Q , m a s s (C.1.3), absolute temperature (C.1.4), concentration function ?! (see (C.1.29)) (which need not be homogeneous in z), the velocity-of-sound function o(z,n) (see (C.1.10)) and the particle-momentum-relaxation time function Tr(z,n,y^) (see (C.1.6), (C.1.8), (C.1.9)). Quantity o(z,n) is determined with Eq. (C.3.5) which presumes that condition (C.3.6) holds. Function v(z,H,y_,A) of A is determined as described in Section 4.10.1, i.e. as the solution of initial-value problem (4.10.2), (4.10.3) defined for all A > 0. If it is defined only on a bounded interval of non-negative values of A , then the y^-dependence of function T is physically inconsistent. In this case, the model for T should be revised and the calculations should be repeated. Covariance (C.3.7) is approximated by C^(z,w,A) that, as a function of A > 0, is evaluated by m e a n s of v(z,H,y_^,A) according to (4.10.5). Note that expression (4.10.5) takes into account the nonlinearities caused by the y^-

134

Sfaftonary D:/fMs:on Processes

dependence of -r in (4.10.2) and (4.10.5). Diffusion parameter D(z,n) is determined with Eq. (C.3.10) at C s C .. It also provides the zero-frequency value (C.3.13) at ^ = ^% of the spectral density. If necessary, effective momentum-relaxation time i;(z,M) is calculated with (C.2.6). This parameter can be used as discussed in R e m a r k C.3 (see Section C.3). Spectral density ,S(z,H,/) is determined with Eq. (4.10.7) which presumes that condition (4.10.6) holds. Note that, since the covariance in (4.10.7) is nonlinearities-aware (see above), the spectral density can also allow for the nonlinear effects, for instance, the flicker noise (see R e m a r k 4.1 in Section 4.4). The I S O D E system (C.l.l) underlying the above treatment and Appendix C is rather simple model for the particle velocity x. The more sophisticated stochastic models based on ISDEs can be constructed in connection with the wave-diffusion treatment discussed in Chapter 6.

4.11

Analytical-Numerical A p p r o a c h to Stationary Process

The above considerations present the results which can be combined in the analytical-numerical approach to analysis of the high-dimensional stationary DSPs. This approach is described in Section 4.11.2. Section 4.11.1 discusses some related aspects.

4.ii.i

J^rae^tcct^ MSMes

M a n y of the integrals mentioned in the previous sections apply stationary probability density p^ and deals with integration over R^. In some particular cases, for instance, w h e n specific forms of functions g and /f allow exact analytical expression for p^ or ^ in (1.2.14) is not too high (say, between 10 and 15), the special techniques can be applied (e.g., Parts II and III of Soize, 1988). However, they are not suitable for higher J. Other methods have to be used instead. O n e of them which is practically suitable even under condition (1.2.14) is proposed in Section 3.5.5 in the form of the S D B approximation. In the present case as the invariant process is stationary, this approximation is described as (see (3.5.32), (3.5.31), (3.5.29), (3.5.30))

135

A/ta^y^tca/Numertca^ Approach (o S^a^tonary Process

P,(*) = P.^M = e x p [ ^ * M - H ^ J - ^

aH ^ e R^, (4.11.1)

where exp(^J=/exp[C/^)]^,

(4.11.2)

< ^ M = -(i/2) (^ - ^ ) 1 [^,)r'D. (x) +D. M [7/(^)]i}(^- ej, forallxeR^, (4.11.3) where matrix 7f(eJ is assumed to be nonsingular and ^ 3c[e +K(x-e )1 D,(x) = -2f(l-K) ^ _ ^L^K, o dM

. for all ^ER'', (4.11.4)

where H represents variable e + K (x - e ) of function g. It should be emphasized that the S D B approximation p ^ for p^ obviously coincides with p^ if diffusion function 77 is independent of x and g is gradient mapping (e.g., see (4.1.5), (4.1.6) in Ortega and Rheinboldt (1970) on notion of gradient mapping). As for the integration over R*', it is in practice usually (see Section 3.5.6) replaced by the integration over rectangular parallelepiped (3.6.1). Subsequently, expressions (4.11.2), (1.7.18), (4.3.2), (4.6.2), (4.6.9), (4.9.5) are rewritten as (see (4.11.1) for p^ and (4.4.2) for D(.x)) e*p(t^)={exp [(/,,(*)] <^,

(4.11.5)

E

M = exp(-^Jj*(x-ej(x-^f exp[C/^)]^,

(4.11.6)

E

Af - exp(-^Jj*(x-eJ[g(x)-g(eJ]Texp[C/^M]^,

(4.11.7)

E

D ^ = exp(-^,)j*D(x)exp[^^)]^,

(4.11.8)

E

^ , - exp(-^)j*^)exp[C/^(x)]^,

(4.11.9)

E

C„(A)-exp(-^J^-^)[^,A)-ejTexp[t/^M]^. (4.11.10)

136

<Sfaf:onary D:^us:o/t Processes

The choice of numbers M. and w, in (3.6.1) is presented in Section 3.6.1. In the present, stationary case, they are evaluated according to equalities (see (3.6.6)) fora!lA; = l,2, ,
(4.11.11)

where number Q is of a few units, e^ ^ is the ^th entry of vector e^ and numbers o ^ are such that (see (3.6.12)) 2[3^(eJ/arJo^+^(e,)=0,

forallR = l,2,...,,f,

(4.11.12)

scalars #t%(e ) and 3g%(e )/3x% are the Arth entries of the principal diagonals of matrixes J7(eJ and 3g(
4. i J. 2

SMFMMMtry o / f&e a p p r o a c h

The analytical-numerical method includes the following basic steps. (1)

Expectation vector e is determined as the unique solution of equation (4.9.10). This vector is stored since it is used at the steps be-

low. (2) (3)

Domain E is formed according to (3.6.1), (4.11.11), (4.11.12). Matrix D(eJ, number W ^ and matrixes t^, Af^, D^, and D ^ are calculated by means of (see (4.4.2)) D(eJ = 3g(e,)/ax,

(4) (5)

(6)

(4.11.13)

(4.11.6), (4.11.7), (4.3.1), (4.11.8) respectively. If (4.6.16) holds, then go to Step 5. Otherwise go to Step 6. The treatment for spectral density ^(/) from Section 4.7 is applied. In so doing, matrixes .H^ and C(A) are evaluated with (4.11.9) and (4.6.11). O n e should note that, since variance matrix !^ w a s determined at Step 3, it is unnecessarily to re-evaluate it as solution (4.6.14) of matrix equation (4.6.13). This case is apparently the most typical in m a n y applied problems characterized by nonlinear drift function g. Theorem 4.4 provides the corresponding representations. Namely, the D T approximations C^(A) and ^ ( / ) are used instead of covariance matrix C(A)

D:scMss;on

137

and spectral-density matrix 5(/). They are calculated according to (4.11.10) and (4.9.15) where i)f(.x,A) is determined as the solution of initial-value problem (4.9.3), (4.9.4). In so doing, formulas (1.7.30) and (1.7.31) can also be applied after replacing C(A) and ^(/) therein with Q,(A) and -$%(/) respectively. T h e <%-fold integrals in (4.11.5)-(4.11.10) are evaluated with the Monte Carlo method. This can efficiently be done in the parallel environment discussed in Appendix H . The parallelization-efficiency analysis is presented in M a m o n t o v and Willander (1998a). Clearly, the proposed algorithm is suitable if (1.2.14) holds and can be carried out in the amount of computer m e m o r y proportional to 6p that agrees with conditions (1.11.1), (1.11.2). Thus, the above method meets the requirement formulated in connection with the purpose of the book in Section 1.13. T h e s a m e as noted in Section 3.6.3 on independence of specific applications and on the D I F C O D subprogram in the nonstationary case is also valid in the present, stationary case. The subprogram should evaluate only one ^-vector, g(.x) (used at Step 1 to determine ijy(.x,A), and in (4.11.7)), and two 6?x ^-matrixes, #(x) (used in (4.11.3) and (4.11.9)) and dg(.x)/d.x (used in (4.11.4) involved by (4.11.3) and (4.4.2) involved by (4.11.8)).

4.12

Discussion

T h e main results of the present chapter are: Theorem 4.4 on the D T approximations (4.9.5) and (4.9.15) for the covariance and spectral density, the analytical basis of the analytical-numerical approach (Section 4.11.2) and the procedure for the fluid-particle velocity (Section 4.10.4). The D T approximation (4.9.5) is m u c h simpler than c o m m o n expression (1.7.26) (or (4.3.3)) for the covariance and meets requirements (1.2.14), (1.11.1), (1.11.2). The above theorem and analytical-numerical approach stem from the analytical results developed in Sections 4.3^.7 and 4.9. A m o n g these results, asymptotics (4.3.5), representation (4.5.7), and Theorem 4.4 should be emphasized. T h e theorem is examplified with the specific problem in particle physics (Sections 4.8) and such field as the stochastic velocity of the fluid particle (Section 4.10). Section 4.6 also shows that in the particular, U M case, the well-known linear treatment of the covariance and spectral density can be applied to stationary D S P even with nonlinear coefficients. In so doing, the linear-analysis terms are determined with these coef-

138

<Sfa(tonary Dt/^Mston Processes

ficients in a certain way. W e also note the circumstance which is to a considerable extent clarified owing to the D T approximation. The D T covariance (4.10.5) takes into account the nonlinearity of the particle friction, i.e. the dependence of t on the normalized kinetic energy (4.10.4) of the particle. In the particular case of a simple fluid (see (C.4.1), (C.4.2)), result (4.10.5) is specified to (4.10.11) that thereby corresponds to the particle-particle scattering. T h e energy dependence (C.4.1) prescribes both high and low values of function T. Hence, according to R e m a r k 4.1 (see Section 4.4), one can expect that spectral density (4.10.7) applied to covariance (4.10.11) performs the flicker effect. This m e a n s that the flicker noise in the particle systems can be explained with the particle-particle scattering (or, perhaps, other scattering mechanisms which lead to dependence (C.4.1) with property (C.4.2)). This issue well agree with the experimental results by other authors. For instance, the flicker noise in very different semiconductor devices is in m a n y cases associated with the scattering of electrons or holes (e.g., Min, 1981; Mihaila, 1986; Pawlikiewicz ef a?.; 1988; Berntgen e? a?., 1999). T h e high levels of this noise explained by m e a n s of the particle-particle scattering (e.g., Min, 1981) or the particle-phonon scattering (e.g., Mihaila, 1986) are confirmed by the measurement data in Figures 6 and 7 in Martin e? otf., (1997). These data were obtained for the short-channel metal-oxidesemiconductor field-effect transistors ( M O S F E T s ) and point out that the shorter the channel is, the higher the noise is. This problem is addressed to by Burghartz ef a^., (2000) (see Figure 8 therein) w h o proposed a possible w a y to reduce its severity. However, to what extent this suggestion is capable to suppress the undesirable features of the scattering near the channel-oxide interface and thereby to prevent prohibitive levels of the flicker noise in the future very-short-channel (or weakly-mesoscale; see Section 6.3.3) M O S F E T s remains to be seen. W e also note that the particle-phonon scattering has a special influence upon physical phenomena in advanced, mesoscale and microscale semiconductor devices (e.g., see Section V of Iafrate and Stroscio, 1996). T h e stochastic fluid modelling for these devices is discussed in Chapter 6. T h e present chapter opens the w a y to approximate analysis of high-dimensional stationary D S P s in dissimilar applications at realistic computing expenses. Various aspects of this analysis can be the topics for future development. O n e of them is computer implementation of the proposed analytical-numerical method. This direction concerns the corresponding parallel

DtscMss:on

139

numerical algorithms (e.g., see Mamontov and Willander, 1998a) and comparison of the simulation results with the corresponding experimental data. It will also enable one to obtain not only the results of a significant practical importance for engineering but also the information which will be highly valuable to quantitatively estimate accuracy of the method and to improve its overall performance.

Chapter 5

Ito's Stochastic Partial Differential Equations as Non-Markov Models Leading to High-Dimensional Diffusion Processes

5.1

Introduction

Chapters 1^4 deal with DSPs. They can in particular be described by means of ISODEs (see Section 1.9). However, notion of ISDE can be extended to more general families of stochastic equations important in engineering and the applied sciences. For instance, Dean (1996) and Garcia-Ojalvo and Sancho (1999) describe various applications of stochastic partial differential equations to physics and chemistry. Qualitative theory of Ito's stochastic partial differential equations (ISPDEs) is well developed (e.g., Bellomo, Brzezniak and De Socio, 1992; Belopol'skaya and Daleckii, 1980, 1990; Gliklikh, 1997; Grecksch and Tudor, 1995; Krylov, 1996; Krylov and Rozovskii, 1979; Rozovskii, 1990; Sobczyk, 1991). The corresponding short introductions and reviews can be found in Albeverio (1996), Pardoux (1993), Streit (1996). Theory of ISPDEs includes the treatments of ISPDEs as ISODEs in appropriate function Hilbert spaces, or more general, Banach spaces. The theorems on existence and uniqueness of the solutions of ISODEs in Banach spaces (which need not be the Hilbert ones) can be found, for instance, in Belopol'skaya and

141

142

7M's Pctrtta/ D:^eren^:ctZ Eoua^to^s as JVon-AfarAou Mo^e/s

Daleckii (1980, 1990), Krylov and Rozovskii (1979). The present chapter deals with the topic closely related to practical implementation of ISPDEs. It considers approximate reduction of these equations to high-dimensional I S O D E s in Euclidean spaces. The latter ones can be analyzed with the help of the methods developed in Sections 2^1. Solutions of I S P D E s regarded as I S O D E s in Banach spaces are usually Markov processes in Banach spaces (e.g., see the above references). H o w ever, in view of R e m a r k 1.6 (see Section 1.3) and the text below it, the above reduction shows that solutions of I S P D E s in Euclidean spaces are, at every fixed space point, non-Markov stochastic processes. The origin of the non-Markov behavior is that the values of a solution of an I S P D E at different time points are in general stochastically dependent because of the spacial coupling inherent in P D E s . This dependence is the case regardless of a specific number of the time points. A scalar I S O D E is inherently unable to describe the non-Markov behavior whereas a scalar I S P D E with solutions regarded in a Euclidean space can do that. In engineering and the applied sciences, the latter issue is important in constructing such nonMarkov models which are supplied with not only consistent theory but also efficient practical procedures. Section 5.2 briefly describes the main differences between various types of ISDEs including ISODEs, I S P D E s and Ito's stochastic partial integro-differential equations (ISPIDEs). Section 5.3 focuses on reduction of an I S P D E to a high-dimensional system of I S O D E s by means of the stochastic-adaptive-interpolation method in conjunction with countable bases in function Banach spaces. The corresponding practical issues are discussed in Section 5.4. Section 5.5 concludes this chapter.

5.2

Various T y p e s of Ito's Stochastic Differential Equations

Ito's stochastic differential equations are considered in Section 1.9 (e.g., see (1.9.1) or (1.9.2)). They can generally be described in the following way. The left-hand side of I S D E presents differential of its solution. The right-hand side linearly depends on differential of the Wiener stochastic process (see Section 1.5), and this process is the only term in the equation which explicitly depends on elementary event ^ e E . The initial condition for solutions (1.2.12) of I S D E at the initial time point f„ is (1.4.25). Solutions of I S D E are determined for all f > ^ in some interval 7 s ^ (used in the preceding

VartoMS Types o/Vfo's S^oc^asi:c Dt^ergH^a/ Egua^ioHS

143

chapters) which m a y be bounded (like (3.6.2)) or semi-bounded (like [^,°°)) or m a y coincide with the whole axis R (like in the case as the solution is invariant D S P ; see Section 1.7). If the right-hand side of I S D E also includes functions explicitly dependent only on time or the solutions, then the I S D E is I S O D E . This equation is written in the form of (1.9.1) or (1.9.2). If the right-hand side of an I S D E includes functions explicitly dependent not only on time or the solutions but also on the space derivatives of the solutions and perhaps on the space-coordinate vector z e Q (introduced in Section 1.2; see the text around (1.2.13)), then the I S D E is I S P D E . Solutions of I S P D E are random fields (see (1.2.13)). In m a n y cases, domain Q does not explicitly depend on ^ but m a y explicitly depend on f, i.e. Q = Q(f). W e assume that Q(f) is defined for all ;e R. Thus, solutions of I S P D E are analyzed on set

z={(;,z)eR^:;e/,zeQ(F)}.

(5.2.1)

The property that the conditions for solutions at boundary 3Q(f) do not explicitly depend on elementary event $

(5.2.2)

is valid in m a n y applied problems and is assumed in what follows. If the right-hand side of an I S D E includes not only the terms usual in I S P D E s (see above) but also functions explicitly dependent on the integrals of solutions or their space derivatives over space domain Q(f), then the I S D E is called Ito's stochastic partial integro-differential equation, briefly, ISPIDE. For example, let us consider the following equation system 3A^= J^df,

(5.2.3)

3^ 3*2 = H,

- 3; +

"

-

^3z

6?Z

32*. ^3;+ 3 3 ^ ( 6 , ; ) ,

(5.2.4)

3z

where ;f=l, Q(f) = (0,l), f > 0 , scalar real quantities a^Hya^ are independent of (^,f,z), «^>0, #2>0, the boundary condition is limx^limx. = 0 for all ? > 0 and the initial conditions consists of equalities .xJ,-, = x i(''Z), *2l,=o = X..2(''Z) forallzS(0,l) where Xo.iC'Z), X„.2(''Z) are random vari-

144

7^o's Pctr?:a/ D:^*ere7:^ta/ EoMa^ons as TVon-MarAou Models

ables. Clearly, I S D E system (5.2.3), (5.2.4) represents a nonlinear ISPIDE system. It m a y be regarded as a stochastic generalization of the deterministic (#3=0) partial integro-differential equation studied by Fitzgibbon and Parrot (1997). Ito's stochastic partial integro-differential equations enable one to take into account nonlocal phenomena in a more flexible w a y than that can be achieved by m e a n s of ISPDEs. Ito's stochastic partial integro-differential equations can be examplified with Ito's versions of the linearized stochastic Boltzmann equation discovered by Kadomtsev (1957) (see also Sections 22 and 23 of Klimontovich, 1982) and the nonlinear stochastic Boltzmann and Boltzmann-Enskog equations derived by U e y a m a (1980, 1981). Stochastic partial integro-differential equations which are in a certain respect more general than the above ISPIDEs are discussed in Section 6.3 of Bellomo, Brzezniak and D e Socio (1992).

5.3

M e t h o d to R e d u c e I S P I D E to System of I S O D E s

Ito's stochastic partial integro-differential equations (see (5.2.3), (5.2.4) for an example) can in m a n y cases be considered as I S O D E s in appropriate Banach (i.e., complete normed linear) spaces. This means that every solution of the equation, as a function of z, belongs, say, almost surely to a Banach space which w e denote ^ . For an ISPIDE of the type described in Section 5.2, space ^ depends on both domain Q(f) and time f, i.e. ^=^[n(^),f],

forallfER.

(5.3.1)

In so doing, the explicit dependence of ^ on f is due to the boundary condition which generally depends on ;. However, J?* does not explicitly depend on elementary event ^ because of (5.2.2). Solution jc of the ISPIDE under consideration can be expressed with Eq. (1.2.13) and relation X(i;,f,')€<^[Q(;),f] holds almost surely. To discuss the method in the title of the present section, w e need notion of countable (i.e., either finite or infinitely countable) basis of Banach space. Stochastic processes in Banach spaces with these bases are the topic usually related to qualitative analysis (e.g., Spruill, 1977; Gihman, 1980; Ahmed, 1992; Pugachev, 1995) or briefly mentioned in connection with quantitative techniques (e.g., Grecksch and Kloeden, 1996; K w a k and Son, 1991). W e consider this topic more focusing it on the issues which can help

Method to Reduce /SPZDE to System ofJ-SODEs

145

to better direct the corresponding practical strategies. Let us assume that Banach space (5.3.1) has a uniform in f E R countable (or Schauder) basis (Schauder, 1927a, 1927b) (see also Section 3.6 of Lusternik and Sobolev, 1974; pp. 93-94 in Dunford and Schwartz, 1988; Sections 6.8.2 and 6.8.3 of Edwards, 1995) {^(t,)EF"[Q(f),f]:^ = l,2,3,..},

forallfER.

(5.3.2)

This m e a n s that, for every function (p(;,-)E.^[Q(;),f] defined for all ;e6((t)) s R , there exist the unique function sequence c^f),c^(^), , c^.(f),... such that

$(f,z)=2^(;)6,,(;,z).

(5.3.3)

A: = l

In so doing, convergence of the series is understood in the sense of relation (e.g., p. 48 in Schauder, 1927a; p. 93 in Dunford and Schwartz, 1988)

lim

E ',(W,-)

= 0,

uniformly in ;E8(
where norm HH^. is the norm of space (5.3.1). Note that, if one or another property holds uniformly in f E R, it holds for all f, negative or positive, with sufficiently large ]f]. Thus, the uniformity in f (see (5.3.3)) of basis (5.3.2) allows to apply the basis also in the limit cases as ;-* -oo and ;-* oo if the corresponding limit analysis is meaningful and necessary. W e also stress the following obvious fact: both space ^"[H(f),f] and any of its basis (5.3.2) are independent of the equations considered in the space and the initial conditions for these equations.

(5.3.5)

However, definition of the space m a y include a family of the boundary conditions for the equations. If it does, then every function &%(?,') of its basis (5.3.2) has to satisfy this family. Such dependences on the boundary conditions are a must, for instance, in case of the Bubnov-Galerkin (BG) method. (The s u m m a r y on the classification of the Galerkin-type techniques can be found, for example, on pp. 163-164 of volume 4 of EncycJopedta o/*Afa^e7naf:cs, Kluwer, Dordrecht, 1989). It follows from (5.3.2) and (5.3.3) that

146

7fo's Par?:
all functions &%(?,') are linearly independent uniformly in (r,z)e$.

(5.3.6)

Quantities c^.(f) in (5.3.3) are called the coordinates of function (t)(r,) in basis (5.3.2). There is the linear one-to-one correspondence between coordinate set c^;), c^(f), ..., c^.(f), and function (])(;,'). The former ones depend on the latter one continuously (e.g., pp. 113-115 in Lusternik and Sobolev, 1974). It is well-known that any Banach space with a countable Schauder basis is separable. However, not every separable Bahach space has such basis (Enflo, 1973). If a Banach space with scalar product (,) is Hilbert space, i.e. it is complete in norm H' H=y(',-), then the space is separable if and only if it has a countable basis. If space (5.3.1) is Hilbert one, then basis (5.3.2) is called orthonormal if and only if !!&J=-/(^A) = 1 R"" all ^ = 1,2,3,... and (^,&,)=0 for all A:,/ = 1,2,3, such that A:^/. In so doing, series (5.3.3) and coefficients c^(f) are called the Fourier series and the Fourier coefficients of function ())(?,-) in basis (5.3.2). There is no general recipe to construct bases in Banach spaces. H o w ever, the following facts are well-known and can be used in practice. * *

At ;?=1 function set {z*:%: = 0,l,2, } is a basis of space C°([-l, 1]). At ^=1 functions y% + l/2 P^(z), A:=0,1,2,..., where P^ are the Legendre polynomials, i.e.

P.(z) = — ! — ^ * f ( ^ ^ ) l * , ' ^ Ar!2* ^z'

*

are an orthonormal basis of L^((-1,1)). At;?=l functions exp(-z/2)L^(z),A; = 0,l,2,...,where.L^ aretheChebyshev—Laguerre polynomials, i.e. Lrz) = ^ ^ ^ ' e x p ( - z ) l , '^ ^! Jz'

*

forallzfER,

forallzER,

are an orthonormal basis of space L^((0,oo)). At^f=l functions [/c! 2 ^ e x p ( z ^ ) ] ^7/^(z), A;=0,l,2,..., w h e r e ^ are the Chebyshev-Hermite polynomials, i.e.

147

Met/tod fo Reduce 7SP7DE (o Sysfem of/SODEs

^(z) = ( - l ) ^ e x p ( z ' ) ^ ^ ^ ^ R , dz*

forallzSR,

are an orthonormal basis of space L ^((-00,00)). If^f=l,then function set {l,cosz,sinz, ,cos^z,sinA:z, } is an orthonormal basis of L ^((- n, ?t)) for any real / e (1,00). O n the contrary (e.g., see p. 453 in Edwards, 1995), the above function set is not a basis of spaces L ^ ((- n, n)), L " ((- n, n)) or the space of 2 n -periodic continuous functions. If ;?=1, then functions S^.(y,z)/yy where y > 0 is arbitrary fixed number, S,,(y,z) = s i n c ^ M ,

for all integer /c and all z <

and _

sine C! =

smna

,

are a basis of a certain Banach space (Theorem 2.8 of McNamee, Stenger, and Whitney, 1971). They are associated with Whittaker's cardinal functions. * At ^f = 3 the set of the spherical functions (e.g., (26.28) in Vladimirov, 1984) divided by their norms (e.g., (26.30) in Vladimirov, 1984) is an orthonormal basis of L^({jrSR^: Hjcll = l}) where the vector norm is Euclidean. * At
148

7M's ParfMtJ Dt/yeren^ta/ EqMa(:oKs as Mw-AfarAou Afoc^e/s

can be generalized for the two- or three-dimensional cases. Various Banach spaces of continuously differentiable functions of m a n y real variables, for instance, the spaces listed in L e m m a 12 by Mityagin (1970) have the bases. This important result w a s discovered by Mityagin (see Proposition 3 in Mityagin, 1970). The spaces also include C*((0,1)'), %= 0,1,2, ., ? = 1,2,3,..., for which Ciesielski and Domsta (1972) constructed even orthonormal bases. However, they have not found a wide-spread use in applied problems yet. More details on the bases in Banach spaces can be found in Singer (1970, 1981). In the case of stochastic equation, expression (5.3.3) is generalized correspondingly. For solution x(^.')^-^[ &(?),?] °f the ISPIDE mentioned above, expression (5.3.3) becomes

X(6,f,z)^(i;,;)6,(;,z),

(5.3.7)

*=i

where Z is described with (5.2.1). In so doing, convergence of the series is understood in the sense of relation (cf., (5.3.4))

lim

E<^,W,-)

= 0,

uniformly in fEV,

(5.3.8)

which holds almost surely. The key feature of representation (5.3.7) is that basis functions &. are independent of ^ whereas coordinate functions c^ are independent of z. Owing to this, coordinates c^,f) are stochastic processes, i.e. the quantities which m a y be described as solutions of I S O D E systems.

&3.i

i*rq/ec%MM a p p r o a c h

Availability of countable basis (5.3.2) and the corresponding relation (5.3.7) are precisely the features constituting the theoretical ground of the method mentioned in the title of Section 5.3. This method: (1)

applies a finite number of terms of the series in (5.3.7) and thereby seeks solution x(i[,;,z) in the form of approximation

X ( M = x,(^.z)=E';.*(^)^). *=i

foraH(f,z)eZ;

(5.3.9)

AfefM ?o Reduce M P 7 D E ^o -System o/*/^ODEs

(2)

149

(note that approximate coordinates c, ^.(^,^) are generally differs from coordinates c^.($,f) in (5.3.7), (5.3.8)); assumes that functions ^ in (5.3.2) are sufficiently smooth in ; and z, more specifically, they have continuous partial derivative with respect to f and all those partial derivatives with respect to entries of vector z which are included in the I S P I D E under consideration.

Since Y M c^(i;,f)&^(f,.x) is the projection of solution (5.3.7) into such subspace of space .F*in(f),f] which corresponds to basis functions &](?,'), ,&,(?,-), quantity (5.3.9) can be regarded as the approximate /-term projection of solution %(?[,f,z). R e m a r k 5.1 only if

Clearly, approximation (5.3.9) converges to (5.3.7) if and

limll^(^,r,-)Hy=0,

uniformlyinfe/,

(5.3.10)

where e,(i;,f,z)=x(i;,f,z)-x,(i;,f,z). T h e convergence m e a n s that the error of the approximation can be reduced to an arbitrary small value by m e a n s of choosing sufficiently large / . W e note that the error e,(?[,f,z) of approximation (5.3.9) for (5.3.7) can be expressed as the s u m e,($,f,z) = e,^,;,z) + e,,(5,f,z),

(5.3.11)

of the projection error e, (^,f,z),

% ( ^ - z ) = E M$,f)-c,.^,f)]6,(;,z),

(5.3.12)

and the truncation error e, (^,t,z),

2,.,(M=Ec^-;)\(;-*).

(5.3.13)

R e m a r k 5.2 B y virtue of R e m a r k 5.1 and (5.3.11), approximation (5.3.9) is convergent if and only if both the limit relations

150

7?d's ParftaZ D:^ereK^:'a^ Eouct^toKs as ATbn-AfarAou Models

]imlje,(^^,)ll^ = 0,

uniformly in r e / ,

(5.3.14)

liml{e;,(^,^)ll^-=0,

uniformly in ^ E 7 ,

(5.3.15)

hold. Since, however, set (5.3.2) is a basis of space .y7il(f),f], equality (5.3.15) is automatically valid because of (5.3.13) and limit relation (5.3.8) in the definition of the basis. Thus, to rely on approximation (5.3.9) if &. are the basis functions, it is sufficient to assure that (5.3.14) holds. R e m a r k 5.3 If set (5.3.2) of linearly independent (or even orthonormal) functions is not a basis of the space (cf., set {l,cosz,sinz, ..,cos^z,sinA;z,...} is not a basis of space L i((-n,n)) as noted above), then one should verify not only (5.3.14) but also (5.3.15) for every specific solution x (see (5.3.7)) under consideration. The latter circumstance substantially complicates proper application of (5.3.9). Indeed, multiple analysis (e.g., due to variations of the equation parameters or the initial data) c o m m o n in applied sciences and engineering leads to a large number of the above specific solutions. Moreover, if set (5.3.2) is not the basis, then its advantage pointed out in (5.3.5) is no longer the case. This means that set (5.3.2) can depend on the equations or the corresponding initial conditions. If it does, analysis of (5.3.15) turns out to be even more difficult. A s a result, an efficient practical computing based on (5.3.9) becomes quite problematic if (5.3.2) is not the basis. R e m a r k 5.2 demonstrates that, if set (5.3.2) is a basis of space (5.3.1), then one should not worry about validity of feature (5.3.15) of truncation error (5.3.13). R e m a r k 5.3 points out a pronounced practical importance of the answers to the questions below. * * *

Which specific Banach spaces can the ISPIDE in hand be considered in? Which countable bases of the suitable spaces are at one's disposal? H o w to construct a basis for the space with no available bases?

These questions do not always get proper attention. For instance, works by Axelrad (1990) and Rahmat, White and Antoniadis (1996) devoted to solutions of some semiconductor problems by means of the expansions in the trigonometric functions and the spherical harmonics respectively. In so doing, they discuss neither in which Banach spaces the employed functions form the bases nor in which Banach spaces the semiconductor equations

Met/tod fo Reduce 7SPZDE ^o Sysfem of/SODEs

151

are considered. In other words, Axelrad (1990) and Rahmat, White and Antoniadis (1996) unintentiy lead to the problems described in R e m a r k 5.3 (see Section 5.3.1). A n example of the countable-basis construction can be found in Appendix G.

5.3.2

Sfoe/Mtsfte coMocafton wte^Aod

T h e collocation method for deterministic problems w a s proposed by L. V. Kantorovich (1934) (see also §3 in Chapter IV of Kantorovich and Krylov, 1958). T h e present section straightforwardly generalizes it for ISPIDEs. Since the basis is smooth (see Point (2) in Section 5.3.1), it follows from (5.3.9) that

forall(;,z)EZ,

(5.3.16)

for all a = l,...,;?, (;,z)€Z,

(5.3.17)

for all K,p = l,..,^, (f,z)SZ.

(5.3.18)

Partial derivatives of x? with respect to z of the higher orders are expressed analogously. T h e idea of the collocation method is to determine / approximate coordinates c,^(i;,f), A:=l, ,/, from / equations for / values of approximation (5.3.9) at J points in set Q(;). According to this, one chooses J > 3 generally time-dependent discretization points as follows: z^€C°(7),

for all ee =1,2,.,/,

(5.3.19)

z^(<)eQ(;),

forallae=l,2,..,<^, l<6?
(5.3.20)

z^(-)E3H(;),

f o r a l l a e ^ + 1,,/, l<;/
(5.3.21)

152

Zfo's Parf:'aZ Dt^eren^ta/ E^rua^toKS as JVoM-AfarAou Afode^s

where interval / is the s a m e as in Section 5.2. T h e space discretization (5.3.19)—(5.3.21) dependent on time can be necessary if the domain does depend on time. Applying (5.3.9), (5.3.16)-(5.3.18) (and the formulas for the higher-order derivatives if the latter are included in the I S P I D E ) to every of the points (5.3.20), (5.3.21), one obtains equations

for all ae = l,...,?, (f,z)EZ, (5.3.22) 36,(f,z)

3x,(6,;,^(;)) = E 3c„(6,;)6,(;,zjf)) + E ',.*(^)

3f, 3f

for all se=l, ,J, (f,Jc)SZ, (5.3.23)

3X,(^^)) 3^a

' ^ „ . 36*M)) *-i

3z^

for all se=l,...,/, K=l,...,^, (f,z)EZ, (5.3.24)

3^<,3^p

*=i ^

'

3j;^d^p

for all $=1,..J, <x,P = l,..,
(2) (3)

replaces x(S.^(f)), 3x(^.z.(;)), 8x(^,^(f))/3z^, 3 ' x ( ^ ^ ^ ) ) /8z„3zp with X,(6.;-^)), 3x,(ii,f,z^)), 3x,(6,f,z^(;))/3zK, 3'x, ($,f,z (f))/3z 8zg respectively (this rule is also used in case of the higher-order z -derivatives); applies (5.3.22)-(5.3.25) (and analogous expressions for the highorder z -derivatives if they are involved) to the latter quantities; replaces x(^.f.z). 3x(^f,z)/3z„, d ^ x ^ . ^ ^ S z p ^ n X?(^.z), 3X;(^,4z)/dz^, 32x,(^'2)/3z„3zp respectively ifthe I S P I D E is not I S P D E , i.e. includes the former quantities in the integrands of the

153

Method to Reduce LSP7DE to System of/SODEs

(4)

integrals with respect to z over domain Q(f); applies (5.3.9), (5.3.17), (5.3.18) to quantities x,(^.z), 3X;(^.z) /dz„, 3^X/(^^.^)/3z^&jj respectively.

T h e equation system obtained in this w a y is a stochastic O D E system with respect to approximate coordinates c^(^,f), &: =1, ,/. However, this system turns out to be unresolved for differentials 3c^(^,f), A:=1,...,L Indeed, the J equations which correspond to cf points (5.3.20) include the /><^ differentials whereas the /-<^ equations which correspond to /-cf points (5.3.21) does not include the differentials. This feature noticeably complicates practical reduction of the obtained system to Ito's form (see (5.3.37), (5.3.38) below). T o remove these difficulties, one can apply the method discussed in the next section.

5.3.3

<Sfoc/Mtsftc-6M%apftpe-tnfefpoZafMM wtefAocf

T h e stochastic-adaptive-interpolation (SAI) method w a s proposed in Chapter V I of Bellomo and Riganti (1987) (see also Bellomo, D e Socio and M o naco, 1988; Bellomo and Flandoli, 1989; Bellomo, Brzezniak and D e Socio, 1992; Preziosi, Teppati and Bellomo, 1992; Bellomo, 1997). T h e S A I method is a stochastic generalization of the differential quadrature ( D Q ) technique (Bellman, Kashef and Casti, 1972) (see also Section XII of Bellman and Adomian, 1985). T h e S A I method can be regarded as the reformulation of the stochastic collocation method (Section 5.3.2) in terms of vector X,($-f) = [X;(^^^(r)),x,(e^^)),...,X;(^4^(f))]\ for all f€ 7.

(5.3.26)

Let us consider this topic. In view of (5.3.26), expressions (5.3.22) are equivalent to equality

X,(i;,;) = M,(;)

where

for all f € / ,

(5.3.27)

154

/M's Par?:aZ Dt^eren^:a^ Eq'uattons as JVon-AfatrAou Moc!eZs

^^(4z/r)) ^ ( ^ ( f ) )

^(f,z^(^)))

M,(f) =

for all f E / .

^i(^^/M) ^2^-^;M)

(5.3.28)

^/(^^M)

W e need the definition below. Definition 5.1 W e call space discretization (5.3.19)-(5.3.21) regular (with respect to basis (5.3.2)) if a n d only if there exists f -independent n u m ber f)o > 0 such that ] det [M,(f)] [ ^ il„ for all f E 7 w h e r e matrix Af,(f) is described with (5.3.28). W e a s s u m e that discretization (5.3.19)-(5.3.21) is regular with respect to basis (5.3.2). In view of Definition 5.1, equality (5.3.27) c a n be rewritten as

f',i($,;)l C,.2^.?)

[M,(;)]^x,(^,f),

forallfEJ.

(5.3.29)

c„(6,f) T h e n the collocation procedure (l)-(4) in Section 5.3.2 is modified in the following w a y . O n e : (1) does not apply the right-hand sides of (5.3.22), (5.3.23) since these representations are not needed; (2) substitutes (5.3.29) into (5.3.24), (5.3.25), (5.3.9), (5.3.17), (5.3.18); this in particular leads to

/dx,(^(;)) 3z_ = {^(f)[M,(;)]-'}x,($,f),

3x,(^;-z,(;)) 3z..

for all K = 1, ...,
(5.3.30)

155

Mefhod ;o Reduce 7SPZDE ^o Sysfem of/SODEs

3'x,(^f-^M)l 3z^3zp

3'x,(^.;,(;))

for all a,P = l, ,^f, ; e / ,

(5.3.31)

3z^3zp where p^i(^z^(f))

3&,(^z^(f))

3z.

3z.

d^i(^z^))

36,(f,z,(f))

3z.

3z^

R„(f)

for all a = !,,
3^,(;,zi(;))

3z^3zp

3z^3zp

3'^(^z;(r))

3^;(^z,M)

3z^3Zjj

dz^3zp

(5.3.32)

^.pM ^

for all <x,p = l, ,
(5.3.33)

(3) determines entries of vectors x€ER^ and x € : R ^ as follows ^(^-f) = X ; ( ^ ^ ( f ) ) ,

forallse=l,2,...,J,

(5.3.34)

^-^(^^) = X ; ( ^ ^ ^ ) ) -

forallse^+1,...,/,

(5.3.35)

forallae=l,2,..,J,

(5.3.36)

so, in particular, J^($,f) = ^ X , ( ^ K ( f ) ) .

and applies (5.3.34)-(5.3.36) to the system resulting from step (2); the model obtained in this w a y is presented as I S O D E system Jx =^(f,.x, x ) & +^(f,%, Jc)^W^(^,^),

(5.3.37)

156

ZM's Par?:aZ Dt/yereHtta/ FoHa(:o?n as JVon-AfarAou Afode^s

E(^,A:,^)=0,

(5.3.38)

w h e r e vector x is in m a n y cases uniquely determined from equation (5.3.38) as a function of (f,x), i.e. * = e(f,.x);

(5.3.39)

this feature allows one to interpret system (5.3.37), (5.3.38) as I S O D E system (1.9.2) w h e r e g(;„x)=,#(f,*,e(;,.x)),

A(;,*)=^(;,.x,e(f,x)).

(5.3.40)

S y s t e m (1.9.2) (accompanying b y (5.3.39), (5.3.40)) is resolved for differentials (5.3.36). T h e resolved form is granted b y the fact that (5.3.23) is not involved. S y s t e m (1.9.2) can be solved b y m e a n s of the special techniques developed in Chapters 2-4 for high-dimensional D S P s . R e m a r k 5.4 The above passing from an ISPIDE to I S O D E system (1.9.2), (5.3.39) presents the SAI method. The key operations of this method in conjunction with basis (5.3.2) are as follows. System (1.9.2) obtained as the result of the above steps (l)-(3) is solved for vector (5.3.34). Then vector (5.3.35) is evaluated according to (5.3.39). Entries of vectors (5.3.34) and (5.3.35) form vector (5.3.26). It by means of (5.3.29) provides approximate co-ordinates c, ^ , f ) in (5.3.9) (and the related equations like (5.3.17) or (5.3.18)). Then one can calculate the basisbased approximation (5.3.9) for (5.3.7) at arbitrary z E Q(f). A s is noted in Section 1.9, entries of vector (5.3.26) which stem from the I S O D E system can usually be regarded as entries of a D S P . In view of Rem a r k 1.6 (see Section 1.3) and the text below it, the mentioned approximation (5.3.9) at any fixed z E Q(f) presents a non-Markov stochastic process. The time-depth (or the so-called m e m o r y ) of the non-Markov effect modelled in this w a y is proportional to number / of the terms in (5.3.9). W e also note that equations like I S P D E s or ISPIDEs as non-Markov m o dels still did not become wide-spread in some treatments in engineering and applied research. The latter treatments confine themselves to the statistical-mechanics approach (possibly under the influence of those scientific schools in theoretical physics which are unwilling to invest efforts in application of stochastic science). In fact, statistical mechanics presents an attempt to describe random phenomena solely in deterministic terms such as collision integrals, probability densities, solutions of c o m m o n kinetic equati-

MetAod to Reduce ASP/DE to System o/'Z&ODEs

157

ons and the related ones. Notion of stochastic process is not even mentioned in the standard texts on statistical mechanics, for instance, Balescu (1975) or Resibois and D e Leener (1977). In spite of a certain usefulness of the above theories, they are not comprehensive. Theoretical physics is more than statistical mechanics (that is explained, for instance, on pp. 1-2 in Honerkamp, 1998). However, the restrictive mathematical limits of statistical mechanics can be overcome. O n e of the corresponding examples is the interesting approach of Honerkamp (1998) which reformulates statistical physics in terms of stochastic processes and random fields. Compared to the original SAI method, the n e w point developed in this chapter is that (5.3.26), (5.3.28), (5.3.30)-(5.3.33) express respective quantities in a m u c h more general form, namely, in terms of the time-dependent discretization (5.3.19)-(5.3.21) and basis (5.3.2) of time-dependent Banach space (5.3.1). The original version applies the Karhunen-Loeve expansion (e.g., (1.6) and (6.6) in Bellomo and Riganti, 1987) instead of Eq. (5.3.7) (or (5.3.9)). However, the Karhunen—Loeve expansion is less general (cf, pp. 413-416 of Papoulis, 1991) and, in its nature, less fundamental than expansion (5.3.7) based on Schauder basis (5.3.2) of Banach space (5.3.1). All the well-known presentations of the D Q and SAI methods (e.g., Bellman, Kashef and Casti (1972); Bert and Malik, 1996; Bellomo and Flandoli, 1989) mention neither the above basis nor any connection to it, calling functions &. the test (or trial) functions. This lack does not enable one to reveal the theoretical meaning and practical capabilities of these methods to a full extent. R e m a r k 5.5 Expressions (5.3.34), (5.3.26), (5.3.30), (5.3.31) form the D Q s underlying the SAI method. The entries of the matrixes in the figure parentheses in (5.3.30) and (5.3.31) are the coefficients of the D Q s . In the D Q practice, there are some heuristic "rules" which m a y seem to be based on formal derivations. O n e such "rule", the multiplication one, is described, for instance, in Section 6 of Bellman, Kashef and Casti (1972) (see (3) on p. 48 therein). According to it, the D Q for the second-derivative vector on the left-hand side of (5.3.31) are obtained by means of successive application of formula (5.3.30), i.e.

158

Ro's Par^taZ Dt^erenttaZ Eoua^ons as Non-MatrAou Moc!eZs

/^<2

3'X;(^-^iM) 3z„3zp

for ail <x,P = l,...,
(5.3.41)

^

However, comparison of (5.3.41) with (5.3.31) demonstrates that (5.3.41) is valid if and only if

3„.p(;)=^)[M,(f)r^K(;). for all a,p = l,...,
(5.3.42)

Condition (5.3.42) is obviously very particular (see (5.3.28), (5.3.32), (5.3.33)). It can hold only for quite special bases (5.3.2) and discretizations (5.3.20), (5.3.21). This means that the above multiplication "rule" is not general. Generally speaking, it is not true. This example shows that the countable-basis approach can help not only to discover a real meaning of some computational recipes but also to reveal the conditions w h e n these recipes are correct. W e also note that entries of vector x in (1.9.2) present the stochastic generalization of the state variables. In other words, I S O D E system (1.9.2), (5.3.39) (obtained from (5.3.37), (5.3.38) by m e a n s of (5.3.40)) corresponds to the state-space approach (see also Section 4.3 of M a m o n t o v and Willander, 1997b). Notably, this fact agrees with the recent research interest (e.g., K a n g and Lacy, 1992; Kang, 1992; Shi and Jastrzebski, 1996) in the advanced state-space formulations for semiconductor circuits. A m o n g the currently available commercial products, one can point out the P H O E N I X electronic circuit simulator by SimExel Corporation (Laguna Beach, C A , U S A ; see <St?M.Exe/. <St7MMJa?:oM etna* .BeAautora/ Afoc^e/t^g wt^A fAe .ExfeMoM <Sfa?eSpace Approach at U R L : http://www.simexel.com/) based on the treatment by Schwarz (1989). A noticeable practical advantage of the SAI method is the following fact. It m a k e s use of the feature that definition of Banach space (5.3.1) need not include the boundary condition thereby enabling one to strongly simplify determination of basis (5.3.2). Indeed, the basis functions need not satisfy the boundary condition where boundary 3Q(f) m a y be of an extremely com-

Reefed ContpHfa^toKaZ Tssues

159

plex shape. To choose basis (5.3.2), one can consider some other domain in R**, say, P(f) B 0(f) with boundary 9P(f) of a m u c h more simple shape than that of 8Q(f) and such that one can easily construct a countable basis of the Banach space corresponding to P(f). Then this basis can be used as basis (5.3.2). If at least the following three conditions hold: * * *

the boundary condition is linear; space (5.3.1) is a Hilbert one; basis (5.3.2) is orthonormal and every function &^. in (5.3.2) satisfies the boundary conition for all f,

then one can also apply the B G method (e.g., §2 in Chapter IV of Kantorovich and Krylov, 1958). This method has found wide-spread use in deterministic problems after the studies of Galerkin (1915); it was formerly used by I. G. Bubnov in solving specific problems in elasticity theory.

5.4

Related Computational Issues

This section discusses some computational issues related to the outcomes of Section 5.3. Stochastic model (1.9.2), (5.3.39) resulting from application of the SAI method to ISPIDE (see (5.2.3), (5.2.4) for an ISPIDE example) is typically high-dimensional in the sense of (1.2.14) (see also (5.3.34)). It can be treated by means of the techniques developed in Chapters 2-4 for high-dimensional I S O D E systems. Thus, thanks to the SAI procedure, I S O D E system (1.9.2) presents a unified description closely associated with ISPIDEs that enables one to practically analyze both ISODEs, I D P D E s , ISPIDEs within the same computational environment. This is the /trsf issue. .Second, the DQ/SAI expressions (see Remark 5.4) involve such /x/matrixes (e.g., (5.3.32) or (5.3.33)) which are dense. Subsequently, the m e mory advantages associated with sparse matrixes typical, for example, in the F D approach cannot be utilized. O n the other hand, the accuracy of the basis-based representation (5.3.9) is determined with the rate of the convergence in (5.3.10) and, thus, with number / of the terms in (5.3.9), i.e. the number of the discretization points (see (5.3.19)-(5.3.21)). A large amount of the DQ/SAI-simulation results [e.g., see Olaofe and Mason (1988), Bert and Malik (1996), Bellomo (1997), as well as the references therein] demon-

160

7M's Parf:a^ Dt^ereH^a/ EgMa^:ons as Nbn-AfarAou Afoc^e/s

strates that the same numerical accuracy as in the F D case can be achieved at noticeably lower values of / , for instance, 9 instead of 480 (see the left column on p. 18 of Bert and Malik, 1996). The point is that the accuracy of the F D schemes is determined mainly by the space-mesh characteristic interval that leads to high values of /. Unlike this, the accuracy of the D Q /SAI procedures is also contributed by their unique feature, namely, the interpolating capabilities of the basis functions not available in the F D techniques. A s a result, the DQ/SAI-discretization system is typically m u c h smaller, i.e. the values of ^ in (1.2.14) are m u c h less, than the FD-discretization one. Thus, the total amount of the m e m o r y required for the matrixes in the DQ/SAI method is usually not very large. This issue is confirmed for the D Q method (the deterministic version of the SAI method) by the results of the comprehensive comparison (e.g., the right column on p. 21 of Bert and Malik, 1996). It shows that the D Q approach stands out in numerical accuracy and computational effciency over both the F D and finite-element approaches. W e also note that, since one need not apply the sparse-matrix techniques, the computational environment becomes more simple, compact, and easily maintanable. TTMrd, performance of the SAI method can significantly be improved if one applies the basis functions as functions &^(f, -) (see Remarks 5.2 and 5.3 in Section 5.3.1). This emphasizes importance of the abilities to efficiently construct countable bases for Banach spaces. .FoMrf A, the nonlinearities-aware analytical-numerical treatment of the unified model (1.9.2), (5.3.39) includes the nonstationary, steady-state, and stationary cases (e.g., see the summaries in Sections 3.6.3 and 4.11.2). These analyses apply multifold integrals. The most efficient technique for this task is the Monte Carlo method. To improve its efficiency, one can use parallel computing. The specific parallel strategies m a y involve the sharedm e m o r y or message-passing approaches and generally depend on specific applications, available computer systems, and m a n y other factors. M a m o n tov and Willander (1998a) analyzes a parallelization efficiency of the implementation of the results of Chapter 5 by means of Parallel Virtual Machine ( P V M ) (Geist ef a?., 1994; see also Appendix H ) .

5.5

Discussion

The main outcome of the present chapter is the procedure to reduce a nonli-

DtSCMSSMft

161

near ISPDE or ISPIDE to a system of nonlinear ISODEs. This procedure is based on the innovative combination of the SAI method and countable basis of an appropriate function Banach space (see Remark 5.4 in Section 5.3.3). The importance of the basis for the SAI methods is stressed in Remarks 5.2 and 5.3 in Section 5.3.1. The resulting ISODE system is typically high-dimensional and its solutions are usually DSPs. This system can be analyzed by means of the special techniques developed in Chapters 2-4 for high-dimensional DSPs. Thus, the analytical-numerical approach of Chapters 2^1 is equally suitable for both ISODEs and ISPDEs/ISPIDEs. It can form the basis of the corresponding unified computational environment. Importance of a consistent and efficient reduction of an ISPDE or ISPIDE to a high-dimensional system of ISODEs (e.g., see Eqs. (1.113) a (1.115) in Garcia-Ojalvo and Sancho, 1999) is stressed by the following fact. Such reductions usually underlie the presently available ways to understand and simulate fairly complex noise-induced phenomena (e.g., stochastic resonance) in ISPDEs (e.g., Section 1.5.2 of Garcia-Ojalvo and Sancho, 1999; see also Lindner e? a/., 1995). Application of the Banach-space basis (rather than the F D schemes like in the mentioned works) can supply the corresponding analysis with a solid mathematical ground and make the analysis less arbitrary. The above reduction procedure straightforwardly points out that ISPDEs or ISPIDEs can serve in various applied problems as such models of non-Markov phenomena which are meaningful and based on the consistent mathematical theories. This can extend a set of the options available to engineers and applied researchers in modelling non-Markov effects. A few issues related to the corresponding computational aspects are discussed in Section 5.4. It in particular emphasizes the advantages of the SAI approach compared to the finite-difference and finite-element treatments. Importance of application of parallel computing to improve the overall efficiency of the corresponding computational techniques is also pointed out. Application of the fluid-dynamics ISPDE system to modelling noise in semiconductor systems is considered in Chapter 6. This is one of the many examples of engineering problems which can benefit from the above results concerning practical treatment of ISPDEs or ISPIDEs.

Chapter 6

Ito's Stochastic Partial Differential Equations for Electron Fluids in Semiconductors

6.1

Introduction

Nonrelativistic quantum mechanics erected around the Schrddinger equation presents the theoretical basis of modern physics and, according to Laughlin and Pines (2000), is the theory of everything. More specifically (p. 28 of Laughlin and Pines, 2000), it describes the everyday world of human beings—air, water, rocks, fire, people, and so forth; it may easily include light, possibly gravity but, regrettably, cannot include nuclear interactions and perhaps some other things; however, the missing parts are irrelevant to human-scale phenomena, so nonrelativistic quantum mechanics is suitable for all practical purposes. There is a great variety of the ways connecting D S P theory to physics. One of them, and perhaps the most important from the physics viewpoint, is the inherent coupling of the Schrddinger equation with DSPs. This fact is revealed and discussed in detail by Nagasawa (1993) on the basis of the rigorous mathematical analysis. The above coupling is so deep (e.g., Section 4.2 of Nagasawa, 1993) that can be regarded as the equivalence (in a certain sense) of the two treatments. One can note (Section 4.4 of Nagasawa, 1993) that D S P theory provides a better understanding of quantum mechanics; moreover, its nonrelativistic part is a D S P theory containing the

163

Schrodinger equation intrinsically. Because of the above equivalence and the issues in the previous paragraph, one should believe that D S P theory is merely another version of "the theory of everything". The present chapter deals with some applications of D S P modelling to semiconductor electronics, the science based on nonrelativistic quantum m e chanics and the Schrodinger equation. This equation is used for various purposes, from basic semiconductor research to engineering software for design of quantum devices and the corresponding electronic circuits. In principle, the Schrodinger equation can also be applied to multiparticle systems. However, the required computational power rapidly grows with the number of the described particles. In fact, if this number exceeds about 10, then the corresponding computing expenses become prohibitive. More precisely (p. 28 of Laughlin and Pines, 2000), no computer existing, or that will ever exist, can break this barrier because it is a catastrophe of dimension [see Laughlin and Pines (2000) for the details]. In other words, the Schrodinger -equation approach is unsuitable to analyze multiparticle systems in practice. T h e related difficulties also include the fact that the Schrodinger-equation-based simulation usually deals with m a n y details which are not always (if at all) needed in engineering. Other microscopic, i.e. by-particle, descriptions, for example, the molecular-dynamics simulation (e.g., Hansen and McDonald, 1986) or the statistical-mechanics formalism (e.g., Resibois and D e Leener, 1977; Klimontovich, 1982) do not resolve the above problems either because of still high complexity of the applied models. A possible solution can be I S P D E s (or ISPIDEs). This is pointed out, for example, by mesoscopic modelling proposed by Fraaije ef af. (1997) and based on it the C^'MesoDyn software, a commercial package developed by Molecular Simulations, Inc. (San Diego, C A , U S A ; http://www.msi.com /materials/cerius2/MesoDyn.html). This package (available only for the S G I and I B M workstations running in a single or multiprocessor mode) includes solving a pair of stochastic P D E s for particle concentrations (e.g., (21)-(24) in Fraaije ef a/., 1997). Application of I S P D E s to semiconductor problems is presented, for instance, by M a m o n t o v and Willander (1995b, 1997d), Willander and M a m o n t o v (1999). S o m e of the application-related aspects of the latter approach are discussed in the present chapter. Section 6.2 discusses capabilities of deterministic macroscopic models to allow for microscopic effects. This section derives the conditions for the domain occupied by a fluid to be regarded within macroscopic modelling as macroscale, mesoscale or microscale. Stochastic generalization of the above

Aftcroscopcc Phenomena tH Afaeroseop:c J^ode/s

165

deterministic models provides the n e w options as discussed in Section 6.3 in connection with M -type semiconductors. In so doing, the focus is on the derivation (not formal incorporation) and the physical meaning of the stochastic source in the particle-concentration equation. Section 6.4 surveys the approach of Section 6.3 in connection with various types of noise in semiconductor systems and notes some issues for the future development of this approach.

6.2

Microscopic P h e n o m e n a in Macroscopic M o d e l s of Multiparticle Systems

Macroscopic, the continuum-mechanics-based models of multiparticle systems are more compact and transparent, better balanced in accuracy and complexity, better suited for computational implementation, easier in optimization than their microscopic counterparts [e.g., see the concise s u m m a r y on pp. 11-12 of Volume I of Sedov (1971-1972) and the detailed criticism in Section 1 of Ancona (1995)]. The disadvantage of the macroscopic models is that they are not intended to describe the underlying microscopic phenomena. They can be taken into account (e.g., see Appendix F). However, this is usually possible only to a limited, in m a n y cases insufficient extent. Therefore, in applied fields, it is highly desirable to endow the macroscopic descriptions with better capabilities in modelling microscopic phenomena. There are various ways to do that. Sections 6.2.1-6.2.3 discuss two of them. Section 6.2.1 considers some microscopic-effect-related features of deterministic macroscopic models. The corresponding classification of the scales of domains occupied by fluids is introduced in Section 6.2.2. Section 6.2.3 points out the option which is beyond deterministic formulations. It is stochastic generalization of macroscopic fluid models to ISPDEs.

6.2. i

Aftcrosco%MC /Yt7M%owt wctZAs w tZeferwtMMsfte

Haifa century ago Goldstein (1951) reported mathematical analysis of the asymmetric random-walk ( A R W ) model for a particle moving stepwise with the same velocity v in the one-dimensional particle-position space as follows. At every time step, the particle moves in the same direction as at the previous time step with probability 7^ or, alternatively, in the direction

166

7^6's Par^ta^ D:^eren^ta/ EgMa^tons /or E/ec^rons m -Semteon^ue^ors

opposite to that at the previous time step with probability 7? , so 7*„+R„=l.

(6.2.1)

Probabilities 7" and 7? are usually termed transmission and reflection ones respectively. The direction alteration and probabilities 7^ and 7?^ are due to individual scattering acts, i.e. scattering of the particle with another particle or, if the fluid is dispersed, with an element of the dispersion mediu m (e.g., potential barrier). The above random walk is also called persistent (e.g., Section 3e of Weiss, 1994). In the particular symmetric case as 7^=7? =1/2, the above model coincides with the standard, symmetric random walk (e.g., pp. 345-346 of Papoulis, 1991) which, in the well-known limit case, becomes Brownian motion (see R e m a r k 1.13 in Section 1.9). However, asymmetricity 7^# 7^ generally better allows for microscopic features of the scattering. Goldstein (1951) showed that the A R W model in a certain limit case leads to P D E

d'p.(r,z)

i 8p.(;,z) +

a;'

2 3'p.(,,z) =y

T„

df

(6.2.2)

a;'

where z E Q =R and p. are the same as in Section 1.2 and Appendix F, i.e. the particle position in domain Q and the probability density of this position, and T > 0 is the parameter of the model. More details on the A R W description and other models of random walks can be found, for example, in Section 3e of Chapter 2 and Section 4d of Chapter 3 of Weiss (1994). R e m a r k 6.1 R a n d o m walks are usually very schematic descriptions of the real phenomena. Only a limited number of them are accounted in the above A R W . For instance, it does not include: *

the C R / G R phenomena (discussed in Apppendix F ) because of (6.2.1); * boundary condition for the walking particle; * Eq. (6.2.2) is derived as the limit case of A R W only under rather idealizing assumptions.

The first feature points out that parameter T can generally be determined with the effects which need not be accounted in A R W . The first and second features m e a n that, if A R W is applied to a fluid, i.e. a multiparticle

Aftcroscoptc Phenomena m Afacroscop:'c Models

167

system, then the number of particles JV(;) (e.g., see Appendix F) is independent of time, 7V(f) sJV. The third feature can be regarded as a direct consequence of the limitations associated with the first and second features. Note that Eq. (6.2.2) is macroscopic. This is stressed by the fact that, in view of R e m a r k 6.1 and (F.9), it can be rewritten as P D E 3^(f,z)^ 1 dn(f,z)_^2d^!(f,z) 3;2

T„

3f

' "

(623)

g;2

for concentration w. According to Remark 6.1, this equation is valid on the whole z-axis, i.e. as Q =R. However, Eq. (6.2.3) can also be considered on bounded domain Q or unbounded domain Q such that Q # R (e.g., see Section 2d of Chapter 5 of Weiss, 1994). Since Eq. (6.2.3) is macroscopic, there are, of course, recipes which are not associated with A R W and, nevertheless, enable one to derive the equations analogous to (6.2.3). S o m e of them are well-known, for instance, in continuum fluid mechanics. O n e such example is considered in Section 6.3.2. R e m a r k 6.2 The key advantage of the A R W model is that it aggregates the details of the discrete stochastic microscopic description in the compact and sharp form of the deterministic macroscopic Eq. (6.2.2) (or (6.2.3)). This grants microscopic-modelling capabilities to macroscopic techniques, the mathematical apparatus commonly used in engineering applications, and thereby significantly increases the range of applicability of the macroscopic approaches. Equation (6.2.2) (or (6.2.3)) is of a combined nature: it describes the macroscopic quantity by means of the ARW-related parameters p and T . For this reason, it is usually regarded as mesoscopic, i.e. intermediate compared to macroscopic and microscopic models. Equation (6.2.2) (or (6.2.3)) is often called the telegraph one. However, another term, the wave-diffusion equation, is more relevant for it because of the following reasons. If the first term on the left-hand side of P D E (6.2.2) is neglected, then this equation becomes the standard diffusion one, i.e. K F E for Brownian motion (see the second equation in (1.10.7)) 3p.(;,z)_^d2p.(;,z) (6.2.4) 3;

*

dz'

where quantity D =^

(6.2.5)

is the particle-position diffusion parameter (C.2.6). If, alternatively, the second term on the left-hand side of P D E (6.2.2) is neglected, then (6.2.2) becomes the standard wave equation

It in particular points out that y can be regarded as the velocity of the waves of the fluid with concentration n determined from (6.2.3). For a wide family of fluids, this velocity is velocity of sound (C.1.12) or, more generally, its nonlinear-friction-aware version (C.3.5), i.e. f„ = ^,

(6.2.7)

D = T;„^.

(6.2.8)

so (see (6.2.5))

Subsequently, one obtains (see (C.2.6)) T =T where, as before, T is the effective momentum-relaxation time (see R e m a r k C.3 in Appendix C.3 for the details). Equations (6.2.4) and (6.2.6) show that the nature of Eq. (6.2.2) is inherently associated with both diffusion and waves. Therefore, a more sharp term for (6.2.2) is the wave-diffusion equation ( W D E ) . This is also emphasized by the fundamental solution of the Cauchy problem for (6.2.2) (e.g., p. 695 of Courant and Hilbert, 1962) since its shape describes diffusion of the wave. The latter is in a general agreement with the well-known particle-wave duality (e.g., see Sections 1.2 and 1.3 of Nagasawa, 1993), the important principle of quantum mechanics. T h e A R W model became very fruitful in physics. It w a s applied to some problems in fluid dynamics such as the fluid flows through porous media (Scheidegger, 1958). In so doing, it w a s noted (p. 654 of Scheidegger, 1958) that the generalization of the one-dimensional A R W and W D E for three dimensions is obvious. For instance, if the fluid is isotropic, then the particle can be considered as undergoing the above A R W of the s a m e parameters along each of the three coordinate axis independently. So, the corresponding W D E can be written for each of the coordinates.

Microscopic P/tenofHena in Macroscopic Afooieis

169

Another field of physics which makes use of the A R W model is quantum mechanics. The discrete, ARW-based approximation of quantum mechanics introduced not long ago naturally leads to W D E (6.2.2) (e.g., (6.5) in Godoy and Garcia-Colin, 1996; (6.3) Godoy and Garcia-Colin, 1998; (37) in Ord, 1996). The mentioned approximation has many advantageous features. One of them is that it significantly clarifies and noticeably extends the wellknown quantum-mechanics-related theory by Landauer (1957, 1970) developed irrespectively of random walks. Interestingly, some of Landauer's results regarded before as purely quantum are rederived by means of classical fluid-transport methods (e.g., Godoy and Garcia-Colin, 1999). W e note that the above discretization of quantum mechanics applies the A R W model where velocity v is the Fermi velocity which, in terms of particle concentration n available from (6.2.3), is expressed with (C.1.24). This agrees with (6.2.7) since velocity o therein is determined by generalization (C.3.5) of Eq. (C.1.12) that includes both (C.1.24) and (C.1.15) as the limit cases. There are some mesoscopic treatments which applies transmission and reflection probabilities 7^ and 7?^ but do not involve the above A R W model and the resulting W D E (6.2.2). Among them, one can note both purely qualitative theories (e.g., Datta's approach, 1995) and the considerations inspired by such theories and which focuses on engineering applications (e.g., Lundstrom, 1997, 1998). Since the latter works cannot make use of the advantages pointed out in Remark 6.2, they have to keep complexity of the employed formalism at an engineering-relevant level by means of other ways. One of them is oversimplification. For instance, Lundstrom outlines the model for M O S F E T s which includes precisely two individual scattering acts (e.g., see p. 361 and Fig. 1 in Lundstrom, 1997). This model assumes that the fluid transport inside both the source and channel of the device are without scattering. In so doing, no condition assuring the transport to be of this kind (e.g., similar to inequality (6.2.19) below; see also Remark 6.6 in Section 6.2.2) is verified. Subsequently, the consistency and the limits of applicability of Lundstrom's model still need a detailed discussion. A n example of a much more general and therefore less restrictive scattering scheme which can be used for any conducting region including sources or channels of M O S F E T s can be found, for instance, in Figure 1 of Gupta (1994). R e m a r k 6.3

It should be stressed that relation T^= T (see the text below

170

TM's ParfMtZ Dt^ereH^a^ Eq'ua^oKS /or EJecfrons m 5em:coK<^MC^ors

(6.2.8)) is valid only under the limitations inherent in A R W . S o m e of them are listed in R e m a r k 6.1. Subsequently, the above relation need not hold in more general cases w h e n more phenomena than those described at the ARW-levelaretakenintoaccount. Note that, if relation T =T does not hold, expression (6.2.8) need not hold either. Quantity T^ depends on the level of detail of the model in a specific problem and can be available, for example, as the coefficient of the specific W D E included in such model. There are interesting attempts to m a k e A R W more adequate, both physically and mathematically (e.g., Balakrishnan and Chaturvedi, 1988). Still more research on such modifications is needed. Luckily, there are other treatments which allow for more features and where W D E s like (6.2.3) are common. In connection with this, w e note the following four issues on continuum fluid mechanics and electrodynamics equations (e.g., Sedov, 1971-1972; Jackson, 1975): *

*

The continuum-based, macroscopic theories provide such derivations of the W D E s analogous to Eq. (6.2.3) (see also (6.2.7), (6.2.8)) which are independent of the A R W model. From this viewpoint, contribution of A R W is that it sharpens the microscopic-modelling capabilities inherent in the W D E s . The continuum-based W D E s are valid for different physical quantities (particle concentration, velocity, pressure, vector potential of electromagnetic field, absolute temperature, etc.) and model a lot of fluid transport properties. These W D E s are, in contrast to (6.2.3), generally nonlinear. Even their linearized versions are in m a n y cases m u c h more complex than (6.2.3). The homogeneous parts of these linear equations can usually be written in the form of P D E 3 M

1 3M +

3; ^

^ = -c?M,

^ ,-. zEQ,

/c n n\ (6.2.9)

-r„ 3;

where M is the modelled physical quantity, O is a linear differential expression which, together with the boundary condition for M at boundary 3Q of domain Q , describes the operator in an appropriate function Banach space. * Sometimes W D E s for the same physical quantity but pertained to different materials correspond to different limit cases of (6.2.2). For example, if M is vector-potential of electromagnetic field, then the

Microscopic Phenomena in Macroscopic M o c M s

171

corresponding homogeneous W D E for the field in semiconductor is written as (6.2.9) (e.g., see (A.7) in Willander and Mamontov, 1999) and hence corresponds to (6.2.2). Unlike this, the versions for the field in ideal conductor and ideal insulator (e.g., see (A.5) and (A.6) in Willander and Mamontov, 1999) are

— = -T()M,

zeO,

(6.2.10)

—

zEO,

(6.2.11)

= -OM,

a;' *

and correspond to limit cases (6.2.4) and (6.2.6) respectively. In the macroscopic modelling built on top of the equations similar to W D E (6.2.9), it is important to interpret and analyze these equations from the viewpoint of the interrelation of the underlying macroscopic and microscopic phenomena. This information can help to better justify the choice of one or another model depending on a specific problem and the specific purposes of applied research or engineering design.

R e m a r k 6.4 The importance of the ARW-based W D E (6.2.2) or more general W D E (6.2.9) in connection with the above issues can be formulated as follows. In spite of the fact that Eq. (6.2.2) stems from fairly schematic discrete microscopic description, it is the core model which must be included in a macroscopic treatment in the form of Eq. (6.2.9) (or similar to it) if the treatment has to take into account the microscopic effects in quantity M properly. Comparison of (6.2.2) and (6.24) or (6.2.9) and (6.2.10) points out the second-order-time-derivative term in (6.2.9) is the inevitable component of microscopic-effect-aware modelling that can facilitate construction of more adequate descriptions. Section 6.2.2 presents more details concerning the topic of Remark 6.4.

6.2.2

AfaeroseaZe, wtesosca^e ct7M% MMcroseaZe tfowMM/ts

This section discusses the conditions in terms of o and T^ enabling one to reduce W D E (6.2.9) to diffusion equation (6.2.10) or wave equation (6.2.11).

172

7K5's ParftaZ Dt^ere^tta^ EgMa^:'ons /or ^ec^roms w <Sem:condMc(ors

These conditions also apply the effective angular frequency of quantity M, notion introduced in the definition below. Definition 6.1

W e assume that:

*

Linear operator O in vector Eq. (6.2.9) is homogeneous and independent of (;,z), and the set of its eigenvalues has no finite limit points. * Every eigenvalue of operator O is positive. * Quantity M„ denotes max{M„^...,M„^} where M„^, A: = 1, ..,<%, is the m i n i m u m of the eigenvalues corresponding to the eigenfunctionvectors with the Arth entries which do not identically equal to zero. * Every eigenfunction of O corresponding to eigenvalue M„ is linearly independent of any other eigenfunction of O . Then the above positive scalar o is called effective angular frequency of quantity H described with W D E (6.2.9). B y virtue of Definition 6.1, the component of the relaxation of quantity M related to the right-hand side of Eq. (6.2.9) and hence to domain Q is associated with M^. This quantity depends on the shape and size of the domain and the boundary conditions which determine operator O . For example, if 0 = (P^+;p/

(6.2.12)

where <2 is a scalar, 7 is the identity operator and operator ^ is described in Appendix G and associated with domain Q in the form of a rectangular parallelepiped (G.3), then effective frequency M is determined as 2

^

(6.2.13)

^mm^.^JJ that follows from Definition 6.1 and Eqs. (6.2.12) and (G.16). Note that, in case of (6.2.13), M^ > a ^ where the equality holds if and only if Q = R^. Equation (6.2.13) examplifies the following remarkable fact. R e m a r k 6.5 Effective angular frequency M„ introduced above for operator O associated with domain Q is, as a rule, inversely proportional to the characteristic linear size /7, of the domain, i.e. its linear size which is determined by the operator itself (in case of (6.2.13), ^ ^ m i n i p ^ y ^ } ) .

M:croscop:c PAenomenct M Mctcroscoptc ModeZs

173

The approximations (6.2.10) and (6.2.11) for W D E (6.2.9) can be obtained by means of analysis of the corresponding time scales. In so doing, one can apply the principal-eigenmode treatment, i.e. to consider such coordinate y of solution M of W D E (6.2.9) which corresponds to any eigenfunction for eigenvalue M„. Obviously, quantity y is described with the following O D E

A ± ^ L ^ y = 0.

(6.2.14)

The characteristic equation of Eq. (6.2.14) is A^ + irj'A + M ^ O .

(6.2.15)

Analysis of its roots leads to the following results. If 4T„
(6.2.16)

where ^=(V^)*',

(6.2.17)

then both roots of Eq. (6.2.15) are real and general solution of O D E (6.2.14) presents a linear combination of the decaying exponential functions of f with the characteristic times approximately equal to T and Tp. In view of (6.2.16), the former, fast decaying function can be neglected. Then the solution is described with the latter, slow decaying function like in case of the first-order O D E ^ + ^ = 0. 6?f

(6.2.18)

Tp

B y virtue of (6.2.17), (6.2.14) and Definition 6.1, Eq. (6.2.18) corresponds to (6.2.10). This shows that parameter (6.2.17) can be regarded as the characteristic time of the diffusion of solution H of (6.2.9). Thus, if domain Q is sufficiently small and hence (see Remark 6.5) effective angular frequency M^ in (6.2.14) is sufficiently large to provide (6.2.16) by means of (6.2.17), then W D E (6.2.9) can approximately be reduced to diffusion equation (6.2.10). The latter is c o m m o n in macroscopic modelling. Inequality (6.2.16) is the condition in terms of solution M of (6.2.9)

174

/fo's Patrftm/ Dt^eren^aJ Eq'Ma^ions /or E/ee^rons tn SemteoK^HC^ors

for the fluid transport to be diffusion. In case of (6.2.16), domain Q is called macroscale with respect to M. The feature of the transport to be diffusion, or of the domain to be macroscale, with respect to some quantity does not imply that the same property holds with respect to other quantities. The relation opposite to (6.2.16) is i:^4i;„.

(6.2.19)

The fluid transport is regarded as wave-diffusion with respect to solution M of (6.2.9) if and only if neither 4t^
(6.2.20)

Under condition (6.2.20), domain Q is called mesoscale with respect to M. In this case, W D E (6.2.9) cannot be approximated with diffusion equation (6.2.10), should generally be considered as it is and is sometimes called m e soscopic. The feature of the transport to be wave-diffusion or of the domain to be mesoscale with respect to some quantity does not imply that the same property holds with respect to other quantities. In other words, the domain m a y be macroscale and mesoscale with respect to different variables. If 4^<^,

(6.2.21)

then both roots of the characteristic equation (6.2.15) are real. In case of (6.2.21), the fluid transport with respect to solution M of (6.2.9) is called incoherent. In particular, the transport in a macroscale domain is always incoherent (cf., (6.2.16), (6.2.21)). If the inequality contrary to (6.2.21) holds, i.e. Tp<4T„,

(6.2.22)

then both roots of the characteristic equations (6.2.15) are complex and constitute a complex conjugate pair. Then the general solution of O D E (6.2.14) is the product of the decaying exponential function with the characteristic time 2 T and the harmonic with a certain angular frequency M . The harmonic points out a coherence, the feature opposite to the exponential dissipation. Therefore, in case of (6.2.22), the fluid transport with respect to solution M of (6.2.9) is called coherent. It follows from (6.2.22) and (6.2.20) that the coherent transport can be the case in mesoscopic domains. The inhomogeneous generalization of W D E (6.2.9) with the driving harmonic sig-

Microscopic Phenomena in Macroscopic Models

175

nal of the angular frequency equal (or close) to M naturally leads to the resonance in the amplitude of M. Note that evolution (6.2.16), (6.2.20), (6.2.21), (6.2.22), (6.2.19) is, by virtue of R e m a r k 6.5, developed in the course of decrease in the characteristic linear size of domain Q . If the domain is very small or, equivalently, the coherence is sufficiently strong, i.e. (6.2.19) holds, then the above frequency M is approximately equal to effective angular frequency M^ of operator O . This not only stresses the meaning of the latter parameter but also points out the following important issue. In view of (6.2.17), inequality (6.2.19) is equivalent to inequality (2T„/o*')2>l.

(6.2.23)

Definition 6.1 points out that quantity o\ has the meaning of the characteristic time of the particle travelling through domain Q . In these terms, condition (6.2.23) merely means that the domain is so small and hence the travelling time is so short compared to the above characteristic time 2 T^ that the exponential relaxation has almost no influence upon variable M. Thus, the T -related term in Eq. (6.2.14) can be neglected that results in

^ + M^y = 0. J;2

(6.2.24)

Obviously, the version of W D E (6.2.9) corresponding to Eq. (6.2.24) is w a v e equation (6.2.11). The wave character of a multiparticle system can in particular be a manifestation of its quantum-mechanical nature. The above discussion shows that (6.2.19) is the condition for the fluid transport to be the wave one with respect to solution M of (6.2.9) and for domain Q to be regarded as microscale with respect to M. In view of R e m a r k 6.3 (see Section 6.2.1) and the physical meaning of -c, the wave transport is not associated with the particle scattering. The coherent behavior is the property which was usually not focused on in continuum-mechanics-based modelling. A growing interest to mesoscopic analysis attracts more attention to this behavior. More details on the physical picture of the coherence can be found, for instance, in Godoy and Garcia -Colin (1996, 1998). The approach in the previous and present sections enables one to resolve the difficulties of understanding the transistion between incoherent and coherent regimes in a more simple and practically tractable w a y than

176

7M's ParftaJ Dt^ereH^:aZ Eq'Ma^oKs /or E^ec^rons m -SemteoHductors

can be done by means of the introduction of a non-Hermitian component into the Hamiltonian in the Schrbdinger equation (Ferry and Barker, 1999). The above transition is in fact determined by the comparison of quantities T^ and 4 T„ as pointed out with (6.2.21) and (6.2.22). R e m a r k 6.6 The above analysis presents the classification of the fluidtransport types and the domain scales (with respect to variable w). Its key feature is that it is based on Definition 6.1 which applies some elements of the eigenstructure of a linear operator in a function Banach space. The latter circumstance brings the present approach beyond everyday usage of m a thematics in theoretical physics. The point is that this science is not always well-prepared to treat fluids in bounded domains (or unbounded domains rather than the whole physical space) because of too m u c h emphasis on the wave-vector formalism and the parameters inherently related to it such as the de Broglie wavelength (focused, for instance, by Yu, Dutton and Kiehl, 1998). Strictly speaking, this formalism is applicable only to the domains coinciding with the whole physical space (e.g., the Universe if the space is three-dimensional). The notion of wave vector is associated with travelling (or progressive) waves in the whole space [e.g., see pp. 267-268 of Landau and Lifshitz (1987) or Sections 7.3.7 and 7.3.8 of Vol. 2 of Sedov (1971 -1972) for the details]. The advantage of the above classification based on linear operators in function Banach spaces is that it allows the domain to be bounded or unbounded, coinciding or not coinciding with the whole physical space depending on the boundary conditions. This enables one to distinguish the fluidtransport types and the domain scales in terms of the quantities standard in the most c o m m o n , macroscopic theories including such parameters as the shape and size of the domain. The next section introduces the stochastic generalization of the deterministic model considered above.

6.2.3

<S%oc/msftc gemeraHzafton o/'fAe (ZeferwMwisftc macroscopic models o^FMM%tparfte%e sys^eFMs

Sections 6.2.1 and 6.2.2 discuss some microscopic phenomena in deterministic macroscopic models of multiparticle systems. These considerations lead to the mesoscopic descriptions. However, the deterministic representations allow for mesoscopic effects only partially. Indeed, Nagasawa (1993)

7%e 7 & P D E System /or F/ectroKS tn K-!Type Semiconductor

177

stresses (see p. 219 therein) that the system mesoscopic model, i.e. intermediate compared to macroscopic and microscopic models of the system, should be stochastic. This idea leads to one more option for microscopic effects in macroscopic descriptions. This field is contributed by many authors, for instance, Bellomo, Brzezniak and De Socio (1992), Bellomo and Preziosi (1995), Ermak and M c C a m m o n (1978), Espaiiol (1995), Espanol and Warren (1995), Fraaije e? a?. (1997), Groot and Warren (1997), Lindenberg and Seshadri (1981). Chapter 5 of the present book also follows the above direction. It emphasizes that stochasticity incorporated into PDEs, in particular, in the way of ISPDEs, enables one to describe stochastic effects in both time and space. The next section presents the corresponding example concerning fluctuations in semiconductors.

6.3

T h e I S P D E System for Electron Fluid in M-Type Semiconductor

This section presents the stochastic generalization of the deterministic P D E system for electron fluid in H -type semiconductor based on continuum fluid mechanics. The generalization is obtained in the form of ISPDE system. Section 6.3.1 describes the deterministic model. The W D E s inherent in it are examplified with the W D E for electron concentration derived in Section 6.3.2. This allows one to specify parameters t^, M^ and the classification of the fluid transports and the domain scale in Section 6.2.2 for the case of the electron fluid. Section 6.3.3 presents two versions of the ISPDE system which correspond to two cases in the above classification.

6.3. i

DeferwMwtsftc wtodeZ /or eZecfrom /!MM% w sewMco?M%Mcfor

Electron (or hole) fluids in semiconductor are usually assumed to be of the kind considered in Appendix C.l. Hence, the basis of the fluid model in the electron case is the following continuum-fluid-mechanics P D E system (e.g., Landau and Lifshitz, 1987; Anile and Pennisi, 1992; Ancona, 1995) considered at every point z of domain Q:

178

7M's Patrtta/ D:^ergH^ta^ JE^Ma^ons /or E/ec^rons M NemtcoKduc^ors

trajectory equation dz ^ *

(6.3.1)

mass-volume-density conservation equation ^M

^

*

^,

momentum-volume-density ^

(V^)n = -r,

(6.3.2)

conservation equation

+ ( ^ ) ^ = ?"^4<

V,n-^--w^,

(6.3.3)

where

df

df

^

vector v = v(^,z) is the electron-fluid velocity, V is the Hamilton operator (cf., (1.10.1)) with respect to entries of vector z, scalar r = r(f,z,n) is the rate of change in concentration M due to the recombination and generation of electrons, quantity /7 = fMMV

(6.3.5)

is the volume density of the fluid m o m e n t u m ,
(6.3.6)

where T(z,n) is determined with (C.2.6). T h e energy-volume-density-conservation equation is not involved in the above model because of (C.1.4). T h e first term on the right-hand side of (6.3.3) is the volume-density of the Lorentz force acting on an electron with charge -
?%e 7 S P D E System /br E/ee^rons tn n-Type Sem:conc!Hctor

179

donor atoms (e.g., phosphorus, arsenic or antimony), is described with the Coulomb-law P D E e„E^V,()) = ,7(M-JV;)

(6.3.7)

where e is the permittivity of vacuum, e is the relative permittivity of the semiconductor and JVj is the concentration of the charged donor atoms. The latter quantity generally depends not only on z but also on other variables in a fairly complex w a y (e.g., Mamontov and Willander, 1994, 1995a). For the sake of simplicity, w e assume that it depends only on z , i.e.

^ J = ArJ(z).

(6.3.8)

T h e conduction-current surface density j a n d total-current surface density ; for the electron fluid are determined as follows j' = -<7Hv,

3Vd) ;'=j'-e e-^-L ° 3;

(6.3.9)

(6.3.10)

where nv is the flow vector for the electrons and the second term on the right-hand side of (6.3.10) is the so-called displacement-current surface density. In view of (6.3.1), (6.3.4)-(6.3.6), (C.l.ll), generalization (C.3.5) of (C.1.12) and (C.1.13), P D E s (6.3.2) and (6.3.3) are equivalent to — + V ^ w ) = -;-, 3f

(6.3.11)

^+3,,= ^ o ^ - = ^ - . ^ g ^ 3? 3* "! M Tr(z,n) Trajectory equation (6.3.1) is not employed below. The left-hand side of (6.3.12) is the total acceleration of the fluid: the first and second terms are instantaneous and convective accelerations respectively. The latter one is nonlinear in v. W e in what follows neglect this term for simplicity. Then Eq. (6.3.12) becomes ^ J L v A - a ^ - ^ - . 3f "i " T(z,n)

(6.3.13)

Thus, the basic equation system consists of P D E s (6.3.7), (6.3.11) and

180

Ac's Far^a^ D:/yeren(:'aZ Equa^toyts /or Electrons m Re??n'coy:dMc^ors

(6.3.13) for potential $ , concentration n and velocity-vector v respectively. Note that Eq. (6.3.12) can be rewritten by m e a n s of velocity (e.g., (16) in M a m o n t o v and Willander, 1995b) Vn v = nV(()-D^

(6.3.14)

in a m o r e compact w a y (see also (C.2.6), (C.2.7)) 3v

v-v

3^

T(Z,M)

(6.3.15)

Vector (6.3.14) can be regarded as the generalization of the Darcy velocity (Darcy, 1856). T h e latter notion is well-known, for instance, in modelling flows of fluids through porous media (e.g., see Fowler, 1998). In the filed of semiconductor devices, the above model can be applied, for instance, to n -type channels of junction field-effect transistors or the socalled series resistances of M-type regions of p-n-junction diodes. T h e m o del is readily generalized for hole fluids in the p-type semiconductors and, moreover, for non-uniform, electron-hole fluids in semiconductors (e.g., M a montov and Willander, 1995b, 1997d; Willander and M a m o n t o v , 1999). In the latter case, the treatment involves s o m e results of the well-known theory (e.g., C h a p m a n and Cowling, 1990). T h e next section considers an example of W D E inherent in the above deterministic model. 6.3.2

Afesoscoptc wape-tH/jfMstoM eqMaftons

Applying operation 3/3? to Eq. (6.3.11) and accounting (6.3.13), one obtains HP

3;2

' ^ 3;

T

= 0, (6.3.16) T 3;

3f

w h e r e parameter ir;' = -^3/!

(6.3.17)

is usually called the electron lifetime. T h e subsequent transformations focuses on simplification of the left-hand side of (6.3.16). N a m e l y , omitting

77:e /&PDE System /or E^ec^royts tn K-T]ype Se;HteornfMctor

181

the second term a n d assuming that the z -dependence of n, o ^ a n d t in the third, fourth a n d fifth terms are negligible, w e get (see also (6.3.17)) 3^7!

1 ^T ^(-M^)+

3r'

T 3^

-O V^VH+3—V,V(j)+ =U. W 3f

This equation is rewritten as 3^ 3f'

fl 1 )3n o ^ ^ M + -^L-^(^-7Vj) — + —

=0

we^e

because of (6.3.11) and (6.3.7) or, after linearizing it in n under the assumptions that the semiconductor is quasi-neutral (i.e. n=JVj) and 3r/3f is approximately independent of n, as 3^ 3^

— + —

— -o \7Vn + M H + 3f ^ ^ -

3f

0.

(6.3.18)

Quantity q*^/! M_

^ in (6.3.18) is k n o w n as the (angular) p l a s m a frequency of the fluid (cf., (2.96) in Ridley, 1988). T h e h o m o g e n e o u s version of Eq. (6.3.18) is 3^n

1\ f1 113w n=0 — + -or2y-,Tr V'V+(o"+

(6.3.19)

3; 2 that can be represented as Eq. (6.2.9) where M = n, -l

—-i

-l

(6.3.20)

a n d (see (6.2.12)) -^%

"2

2

,-

^-i

(6.3.21)

A p p e n d i x G discusses the specific version of ^ in connection (see (G.6)) with the Laplace operator g? described with (G.4), (G.5). This in particular implies that E q . (6.2.13) is valid under condition (6.3.21). R e m a r k 6.7 T h e result of the above derivation can be formulated in the following w a y . T h e deterministic macroscopic continuum-fluid-mechanics-

182

7to's Par^a/ Dt^ere^^oZ Eq'Ma^toKS /or JS/ec^ro^s :n 5e??t:eondMC^ors

based model considered in Section 6.3.1 includes in particular mesoscopic W D E (6.2.9) (in the form of (6.3.19)) which stems from the A R W discretization of quantum-mechanics discussed in Section 6.2.1. This presents another example (the first one is described in Appendix F) of the hidden randomness in the standard, nonrandom fluid equations. T h e full, not simplified, mesoscopic W D E corresponding to homogeneous linear W D E (6.3.19) is Eq. (6.3.16). It does not include any n e w information in addition to that presented with the basic Eqs. (6.3.7), (6.3.11), (6.3.13) (or (6.3.15)) since it is combined from them. Inequality T^-r,.

(6.3.22)

usually holds for semiconductors, except perhaps some special cases which are beyond the present book. Relations (6.3.20) and (6.3.22) imply T„ = T

(6.3.23)

(cf., R e m a r k 6.3 in Section 6.2.1). Applying (6.3.23) and evaluating effective angular frequency M^ (see Definition 6.1 in Section 6.2.2), one can employ the analysis presented in Section 6.2.2. In particular, Eq. (6.2.17) and relations (6.2.16), (6.2.20)-(6.2.22), (6.2.19) become ^ = (^f)-i,

(6.3.24)

the diffusion-transport/macroscale-domain case:4T^Tp,

(6.3.25)

the wave-diffusion-transport/mesoscale-domain case: neither 4 f^Tp nor Tp<4T,

(6.3.26)

the incoherent-transport case: 4 T < Tg,

(6.3.27)

the coherent-transport case: T^< 4 T,

(6.3.28)

the wave-transport/microscale-domain case: Tp-^4T.

(6.3.29)

This classification is with respect to concentration n since the above relations are associated with W D E s (6.3.19) and (6.3.16) for H. In so doing, the meaning of characteristic time Tp of the diffusion associated with n is the relaxation time of the fluid-particle position. Note that condition (6.3.29) is

7%e 7 S P D E Nys^em /or E/ec^rons tn K-Type 5em;condMc^or

183

also the condition for the transport to be almost without scattering. To better interpret the classification in terms of theoretical physics, one can replace the terms "diffusion" and "wave" in (6.3.25), (6.3.26) and (6.3.29) with "dissipative" and "quantum-like" respectively. W e use the term "quantumlike" rather than "quantum" for the reason explained in the remark below. R e m a r k 6.8 In case (6.3.29) the fluid transport is of wave type. Let us examine w h e n the eigenstructure of the corresponding wave equation reproduces the eigenstructure of the quantum-mechanical version of the homogeneous linear Schrbdinger equation of classical mechanics, namely (e.g., (9.9.5) in Reif, 1985), i ^ t <3;

=-_^_gt/,

forallzgQ,

(6.3.30)

47t7M

where scalars w , A are the same as those in Appendix C.l, i is the imaginary unit and 9? is the Laplace differential expression (see (G.4)). The core quantum-mechanical feature is that parameter A (Planck's constant) in (6.3.30) is nonzero. Equations (6.2.11), (6.2.12) under conditions (C.2.5) and <3 = 0 (i.e. w h e n the charged nature of the fluid particles and their scattering is unimportant) and the first equality in (6.3.21) and relation (G.4) point out that the generic wave P D E for fluid is —

= -o2gM,

forallzgQ,

(6.3.31)

where the H-dependence of o is described with (C.1.12), (C.1.23), (C.1.20) (see also (C.1.15) and (C.1.24) for the particular limit cases). W e assume that the fluid concentration in bounded domain Q does not deviate too m u c h from its average, time-independent value n, so the n -dependence of o is its dependence on H, i.e. o = o(n). Domain Q can also depend on n, i.e. Q = Q(n). Variable M in (6.3.31) can in particular be n-M. Let both (6.3.30) and (6.3.31) be equipped with the same boundary condition (if necessary for that, w e disregard a possible difference in the physical meanings of t/ and M). Then operator g in both cases (6.3.30) and (6.3.31) has the same eigenvalues and eigenfunctions. W e also assume that the operator O corresponding to (6.3.31) (i.e. O = o^ g ) admits (see Definition 6.1 in Section 6.2.2) effective angular frequency M . It can be expressed (cf., Section 27.2(a) of Vladimirov, 1984) as M

184

7Ki's Pctr^:aZ Dt^ere^^a/ Equa^tons /or EJec^rons w Sem:'cono!Mc^ors

= o(n)(n/^J where;?, is thecharacteristiclinearsize of domain Q (cf.,Rem a r k 6.5 in Section 6.2.2). Frequency M^ determines the time development of M in (6.3.31). The time development of C/ in (6.3.30) is determined (e.g., (9.9.6) in Reif, 1985) with angular frequency M ^ = [/t/(4itw)](7t/j9j^. Thus, the eigenstructures (the sets of eigenfrequencies and eigenfunctions) corresponding to (6.3.30) and (6.3.31) are identical if and only if M ^ = M^ or, equivalent^ (see the above expressions for M^, and M ), if and only if ^,= A/[4wo(M)].

(6.3.32)

This value enables one to reproduce the core space-time characteristics of the Schrodinger P D E (6.3.30) with the wave P D E for fluid. O n e can show by means of the mentioned description of o that, for the electron and hole fluids in silicon in the wide ranges of the charge-carrier concentration M and at the absolute temperature T above the open-space value 3 ^ K, characteristic linear size (6.3.32) does not exceed 1—2 n m (nanometers). This is the case when, first, the fluid transport performs pronounced quantum features that is confirmed by the molecular-dynamics-simulation results (e.g., Yano, Ferry and Seki, 1992) and, second, the quant u m description of the transport can be adequate. If, on the contrary, the m i n i m u m linear size of the domain is noticeably greater than 1-2 n m (the least size achievable by the modern semiconductor technologies is about 18 nanometers), then the quantum-only modelling is not sufficient. The dissipative, i.e. diffusion, transport component has to be taken into account properly. The latter is, however, not a very easy task if one confines oneself to the frames of quantum mechanics. Note that, in the particular case (C.1.24), quantity (6.3.32) is expressed as /?,= ( a, /n) ^ and, importantly, can be obtained in the semiclassical limit, i.e. as A^O . This fact leads to the following issue. The wave behavior of the fluid transport caused by the small-size effect is not inherently associated with the core quantum-mechanical feature /: # 0. Subsequently, it can be described with the models of classical mechanics modified specifically for this purpose and, thus, considered as quantum-like. A n example of the latter, quantum-like approach is the W D E treatment in the present chapter. In contrast to quantum mechanics, it provides the deep and flexible coupling of the wave (quantum-like) and diffusion (dissipative) descriptions. In case (6.3.25), the particle m o m e m t u m is settled for small fraction

7%e Z S P D E System /or EZec^rons :n K-T^pe Sem:conc!MC^or

185

(proportional to T/Tp) of the particle-position relaxation time Tp. So, the evolution of the m o m e n t u m roughly does not affect the course of diffusion of the particle (i.e. the settling of its position) as it should be in the diffusion phenomenon. This corresponds to neglecting the acceleration in velocity equation (6.3.15) that simplifies it to v = ^.

(6.3.33)

T h e resulting deterministic model (6.3.7), (6.3.11), (6.3.33) is well-known in semiconductor theory (e.g., Shockley, 1949; V a n Roosbroeck, 1950; Section 1.7 of Sze, 1981) and is often called drift-diffusion (DD) due to the first and second terms on the right-hand side of Eq. (6.3.14) for the Darcy velocity. It is commonly believed that the range of applicability of the D D description is rather limited. For instance, to model strongly nonequilibrium phenom e n a , one m a y want to apply the initial system (6.3.7), (6.3.11), (6.3.15) (or even more complex one which also includes the equation for D . However, the sharp theoretical discussion and careful numerical analysis reported by Ancona (1995) provides a deeper insight. It demonstrates that capabilities of the D D model are m u c h wider than previously thought. This is a helpful issue for choice of a proper practical model. In case (6.3.26), the above diffusion picture is not valid since the particle m o m e n t u m has no time to be settled in the course of the particle-position relaxation which is n o w comparable with the m o m e n t u m relaxation. So, the usual, diffusion interpretation cannot be applied. Subsequently, there is no ground to neglect the acceleration in Eq. (6.3.15). The basic equations remain to be the same as before: (6.3.7), (6.3.11), (6.3.15). This model can be used in both cases (6.3.26) and (6.3.29). S o m e specific examples of macroscopic descriptions related to the microscale-domain case (6.3.29) can be found, for instance, in Likharev (1986). S u m m i n g up this section and the previous one, w e stress the following issue. R e m a r k 6.9 The equations similar to W D E s (6.3.16) and (6.3.19) for concentration n can also be obtained for other physical variables like velocity v or pressure II. The classification analogous to (6.3.25)-(6.3.29) with respect to M can be constructed for these variables as well. The W D E s for various quantities offer m a n y n e w options. For instance, the wave-transport version of the velocity W D E m a y be used as a consistent mathematical model for spiralling and quantization of the particle trajecto-

186

Ac's Par?:a? D:/?Bre;tf:a/ EgMa^:'ons /or E^ec^roKS tn Semteonduc^ors

ries and thereby contribute to the investigation on the vortexes in quantum transport (e.g., Barker, 2000). In so doing, the discreteness can result from the discrete eigenstructure of linear differential operator (6.2.12) in the above velocity equation which corresponds to the wave P D E (6.2.11). The key point is that the macroscopic-fluid-dynamics P D E system (6.3.7), (6.3.11), (6.3.13) (or (6.3.15)) describes all the cases (6.3.25)-(6.3.29). This feature of the macroscopic-fluid-dynamics model for semiconductors w a s not pointed out before. It opens the w a y to model mesoscopic and microscopic effects in multiparticle fluids in macroscopic terms. This enables one to use the advantages of the efficient practical methods well-developed for the macroscopic models. T h e classification of the fluid-transport types and the domain scales (with respect to M ) for semiconductors presented in this section stems from the classification developed in Section 6.2.2 and discussed in R e m a r k 6.6 therein. It helps to better specify the limitations and the development directions of the stochastic generalization discussed in the next section. This classification is more detailed and mathematically sharper than the transport classification proposed by Grinberg and Luryi (1992b). These authors, in particular, do not apply the effective angular frequency (see Definition 6.1 in Section 6.2.2) associated with linear differential operator O . Instead, they involve the so-called scattering m e a n free path, a semi-qualitative parameter which depends on neither the size and shape of the domain where the fluid transport is considered nor the conditions at the domain boundary. O u r classification can equally be applied to various semiconductor systems including the short-channel M O S F E T s mentioned in Section 6.2.1 (see the text above R e m a r k 6.3), bipolar transistors (BTs) with thin bases [e.g., Grinberg and Luryi (1992a, 1993,1994), Vaidyanathan and Pulfrey (1997)] or the corresponding multitransistor circuits. W e , however, note that the classification is not comprehensive in the sense that it is developed for linear representations like W D E s (6.2.9), (6.2.14), (6.3.19). Analysis of m u c h more complex, nonlinear equations like (6.3.16) m a y enrich it with n e w features. There are other ways to study microscopic phenomena with the help of the fluid-dynamics models. O n e of them is quantum hydrodynamics ( Q H D ) by Barker and Ferry (1998) (see also Barker, 1998). The present approach, however, differs from it in some respects. T h e most pronounced difference is that our modelling stems from the

7%e Z S P D E System /or Electrons tn K-Type Semtcon^Mctor

187

general, wave-diffusion picture of the fluid-particle transport originated from the A R W discretization and, thanks to this, inherently includes the options related to both the diffusion and wave cases. In so doing, the macroscale, mesoscale and microscale domains as well as the incoherent and coherent transports are naturally described in the form of the sharp formulations. Such a flexibility m a y enable one to analyze various transport phenom e n a in systems of the domains of strongly different sizes from the unified viewpoint and within the same modelling environment. This option can become crucial for efficient engineering simulation in m a n y applied fields like the advanced telecommunication circuits described by Freeman ef a?. (1999) or Meyerson (2000). Unlike this, Q H D stems from quantum-mechanical treatments and thereby is inherently associated only with the wave transport. The dissipative (or diffusion) effects are incorporated into Q H D in one or another w a y from beyond the limits of its quantum-mechanical origin. O n e of the related m o difications is discussed by Ferry and Barker (1999). The main focus of Q H D is on the quantum, wave transport. The wave-diffusion and diffusion phenom e n a and hence the mesoscale and macroscale domains are not easily described within this treatment. This is examplified by another modification (Barker and Watling, 2000) presenting the formalism which, from the modelling viewpoint, is less consistent and more arbitrary than the W D E approach. A regrettable feature of Q H D is that the specific shape and size of the space domain as well as the corresponding boundary conditions are not involved as parts of the model. The sharp mathematical criteria (similar, for example, to (6.3.25)-(6.3.29)) are not available either. Regarding the advantages of the Q H D vision, one can note that it attracts attention to some topics which m a y improve adequacy of the fluid-dynamics interpretations of the wave transport (e.g., Section 8 of Barker and Ferry, 1998; Barker, 2000).

6.3.3

Sfoc/tasfte geweraHza^ton of fAe <%eferwMWM?tc /%Mt
Overwhelming majority of semiconductor systems in modern engineering corresponds to diffusion transport (6.3.25) or to the case w h e n the inequality in (6.3.25) is not very pronounced. The latter feature can be formulated more precisely as follows:

188

Ro's ParfiaZ Dt^ere^^ta/ EgMa^:o?n /or E/ec^rons m S'emMonJuc^ors

the quasi-diffusion-transport/weakly-mesoscale-domain case: quantity T^/(4i;) is not less than a few units.

(6.3.34)

T h e present section focuses o n the stochastic generalization adequate for (6.3.34). This generalization includes the four steps below. (1) Following the treatment in Appendix C, w e extend Eq. (6.3.3) with dependence of T, on velocity-vector x of purely random motion of electron (see (C.1.5)-(C.1.9)). So, w e replace Eq. (6.3.6) with -r, = T(z,M,y^) where y^ is the normalized kinetic energy of the fluid particle (see (C.1.6)) where velocity o of sound is according to (C.1.12), (C.1.23), (C.1.20). Subsequently, Eq. (6.3.15) becomes ^ 3?

^ . -r(z,n,y2)

(6.3.35)

In so doing, w e assume that the Darcy velocity (6.3.14) remains unchanged. Notion of the purely-random-motion velocity m e a n s that velocity v is stochastic and vector x is determined from equality v = E(v)+*

(6.3.36)

where E(v) is the expectation of f, so E(*)E=0.

(6.3.37)

(2) W e replace E(y) in (6.3.36) with velocity v (see (6.3.14)) for the sake of simplicity a n d efficiency of the corresponding practical methods. T h e point is that the strongly nonlinear nature of the model in Section 6.3.1 m a k e s evaluation of E(v) very c u m b e r s o m e a n d time consuming. A s a result of the above modification, E q . (6.3.36) becomes (see (37) in M a m o n t o v a n d Willander, 1995b) v = iJ+x

(6.3.38)

that is in fact a change of variables representing passing from velocity v to velocity *. Substitution of (6.3.38) into (6.3.9), (6.3.11) a n d (6.3.35) leads to j = -
— + ^ ( ^ ) + (Vn)^+/:^=-r,

(6.3.39)

(6.3.40)

TVte Z-SPDE Nys^em /or E/ec(ro?ts m H-Tlype Semtcondue^or

^

=-

^

- ^

189

(6.3.41)

where (6.3.39) is a particluar case of Eq. (39) in M a m o n t o v and Willander (1995b). (3) A s is assumed above, both velocities v and x are stochastic. If vector x is stochastic, then, by virtue of (6.3.38), vector v is also stochastic. T o describe x in (6.3.41) as a stochastic quantity, w e generalize deterministic equation (6.3.41) to the following one

3x = -

df - 3v -

2

-o(z,M)JH^,;),

T(z,w,y2)

-c(z,H,y2)

for all z E U ,

(6.3.42)

which can be regarded as the extension of (C. 1.1) with term 3v. (4) T o m a k e feature (6.3.37) possible at least in the case as (;,z) -dependent concentration n is independent of chance ^ , w e , following, for instance, (8.2.6.a) in Arnold (1974), have to omit 3v in (6.3.42). This results in

3x = -

-

df-

T(z,M,y^)

^

0(z,H)JW(^,f),

\) T(z,H,y2)

for all z e H .

(6.3.43)

For the sake of simplicity, w e neglect the last term on the left-hand side of (6.3.40), i.e. — + V ^ M ) + (VH)Tjr = -7-. 3f

(6.3.44)

Thus, the resulting semiconductor-fluid I S P D E (SF-ISPDE) system includes the following equations: (6.3.7), (6.3.43), (6.3.44) and (6.3.38) where the Darcy velocity y is determined with (6.3.14). Regarding this system, w e note the following six aspects. F:rsf, vector x in Eq. (6.3.44) described with Eq. (6.3.43) presents the stochastic noise source which, as is pointed out by Eq. (6.3.38), is associated with the velocity fluctuations. Its mathematical properties and physical meaning are discussed in Appendix C in detail, examplified in Section 4.10

190

7^6's Parf:a^ D:^ergK^:a/ Equa^oKS /or E/ec^roHS tK SemtcoHduc^ors

in connection with the nonlinear friction and equipped with the practice-oriented procedure in Section 4.10.4. This approach to noise sources in nonlinear semiconductor equations is qualitatively different from more c o m m o n theories (e.g., V a n Vliet, 1971a, 1971b, 1994) which confine themselves to purely formal incorporation of the sources. In so doing, the suggested sources represent white noises in both time and space that automatically leads to stochastic nonlinear P D E s in the sense of the generalized random fields. In spite of that, no meaningful mathematical reading is attempted. Moreover, the well-known difficulties of theoretical and practical treatment of the nonlinear S D E s in the generalized sense (e.g., Section 3 of Streit, 1996) are merely ignored. N o problems of this kind exist in the above SF-ISPDE system. In particular, it does not apply any generalized stochastic processes. SeeoMd, Eq. (6.3.43), at T independent of y^, was first applied to semiconductor problems by Mamontov and Willander (1995b) (see (24), (28), (29) therein). The similar description w a s introduced in fluid theory by Pope (1983) as Eqs. (9)-(ll) therein [see also Pope (1985, 1994) for details and discussion in connection with Lagrangian P D F method]. Pope's model is also used for dispersed fluids by Pozorski and Minier (1999) (see Section rV therein). However, both the Pope and Pozorski-Minier models do not point out the Darcy velocity (6.3.14), important quantity in m a n y applied problems, as the component of velocity y (see (6.3.38), (6.3.44); cf, (16) in M a montov and Willander, 1995b). They do not allow for the nonlinear friction (presented in (6.3.43) with the y^-dependence of the momentum-relaxation time T) either. Unlike this, the corresponding practice-oriented procedure in Section 4.10.4 opens a w a y to consistent analysis of nonlinear noise in the fluid-particle velocity. TA:rd, since Eq. (6.3.36) w a s replaced with Eq. (6.3.38) at the above step (2), velocity component x is, strictly speaking, no longer the velocity of the fluid-particle purely random motion. However, this component can still be regarded as a good approximation for the mentioned velocity. The key feature of % is pointed out at step (4), namely, Eq. (6.3.37) holds at least in the case as (;,z)-dependent concentration n is deterministic (i.e. independent of chance ^ ). Note that the above SF-ISPDE system shows that the Darcy velocity v is generally stochastic. It can be deterministic, for instance, in the case of the deterministic concentration. FoMrf A, as follows from R e m a r k C.3 in Appendix C.3 and Section 4.10.4, the typical y^-independent value of momentum-relaxation time T(z,w,y^)

77:e 7 R P D E System /or E^ec^ro^s :n K-yype 5em:coy:dMc^or

191

in (6.3.43) is effective momentum-relaxation time T(z,n). The corresponding value of o(z,n) is, due to Appendix C.3, quantity o(z,n) (see (C.3.5)). If, for simplicity, one replaces i(z,n,y^) and o(z,n) in (6.3.43) with i(z,n) and o(z,w) respectively, then (6.3.43) is reduced (see also (C.2.6)) to T(z,M)djc=-jcd^ + y D ( ^ ) J ^ , r ) ,

forallzeQ. (6.3.45)

In the macroscale-domain case (6.3.25), the relaxation of x is m u c h faster than the relaxation of n. Hence one can apply such values of x to Eq. (6.3.44) which are obtained by means of neglecting the left-hand side of (6.3.45), i.e. passing to the white-noise-like, Brownian-motion description for x. A s a result, one obtains equation dH+\^(vH)3f+(^H)TyD(z,H)^W^,;) = -r3;

(6.3.46)

instead of (6.3.44). The details on rigorous qualitative theory of the whitenoise limit in linear stochastic P D E s can be found, for instance, in Bouc and Pardoux (1984) (see also Bouc and Pardoux, 1981). Thus, if (6.3.25) holds, then the SF-ISPDE system (6.3.7), (6.3.43), (6.3.44), (6.3.38) is reduced to its m u c h simpler, macroscale-domain version (6.3.7), (6.3.43), (6.3.46), (6.3.38). If condition (6.3.25) is not valid, then system (6.3.7), (6.3.43), (6.3.44), (6.3.38)isadequatenot at any value of quantity 4ir/ip. Indeed, the analysis of the roots of the characteristic equation (6.2.15) shows that this system can be an adequate description only if the weakly-mesoscale-domain condition (6.3.34) holds. The reason of this limitation is the two simplifications involved at the above step (4). Subsequently, the SF-ISPDE system (6.3.7), (6.3.43), (6.3.44), (6.3.38) can be regarded only as the weakly-mesoscaledomain model. Ft#A, the SF-ISPDE system (6.3.7), (6.3.43), (6.3.44), (6.3.38) has a crucial advantage compared to the Boltzmann-equation-based modelling of electron (or hole) fluids in semiconductors. Indeed, Eq. (6.3.44) models concentration H as stochastic quantity because of stochastic-noise-source term x governed by Eq. (6.3.43). In contrast to this, the c o m m o n , deterministic Boltzmann (or Boltzmann-Enskog) equation is inherently unable to describe the concentration as stochastic quantity: the concentration stemming from this equation is always nonrandom (e.g., Bellomo, Palczewski and Toscani, 1988). To m a k e it random, one should turn to the stochastic Boltzm a n n or Boltzmann-Enskog equations (e.g., U e y a m a , 1980, 1981) thereby

192

/M's Parfta/ D:^eren^:aJ Eq'Hat^oKS /or E/ec^roKs m 5em;condHc^ors

leaving the scope of the traditional statistical mechanics. However, the latter models are m u c h more complex than the above SF-ISPDE system. StxfA, both systems (6.3.7), (6.3.43), (6.3.46), (6.3.38) and (6.3.7), (6.3.43), (6.3.44), (6.3.38) can be treated in practice by means of the method considered in Chapter 5 that transforms these SF-ISPDE systems to the corresponding high-dimensional I S O D E systems. Its solutions, the high-dimensional D S P s can be analyzed with the help of the approach described in Chapters 2-4. There are rigorous qualitative treatments of the fluid-velocities in various fluid-dynamics stochastic P D E systems. O n e of the first work on this topic is Bensoussan and T e m a m (1973). It enables one to allow for the fluidviscosity-related terms but considers only the linear systems. The issues on potential practical implementations of the models developed in the qualitative theories are usually not discussed at all in these theories either. The next section focuses on various issues related to the SF-ISPDE system (6.3.7), (6.3.43), (6.3.44), (6.3.38).

6.4

Semiconductor Noises a n d the S F - I S P D E System: Discussion a n d Future D e v e l o p m e n t

Section 6.4.1 considers various semiconductor noises and the role of the SFI S P D E system from Section 6.3.3 as a n e w option to better understand them and to more efficiently treat the corresponding problems in semiconductor engineering. Section 6.4.2 discusses some directions for future development of the SF-ISPDE system.

6.4.J

TTte S F - V S P D E sys^gFM tw coyt/tcc^oK

Thermal noise w a s discovered experimentally by Johnson (1927, 1928) two years after the flicker (or 1//) noise w a s discovered, also by Johnson (1925). The term "thermal" is due to the fact (see (1) in Johnson, 1928) that variance P'; of electric current 7 flowing through a homogeneous two-terminal resistor is proportional to the resistor absolute temperature, i.e.

v?-r.

(6.4.1)

SeHHCondMc^or JVo:ses and ^Ae RF-7SPDE Sysfew:

193

Relation (6.4.1) can also be obtained from the particular version of velocityfluctuation-based Eq. (6.3.39). To do that, one assumes that both concentration H and Darcy^ velocity y are deterministic and, besides, involves standard deviation (C.3.5) of * and expression (C.1.15) for parameter o in (C.3.5). This means that the SF-ISPDE system in Section 6.3.3 describes thermal noise. The latter fully agrees with the well-known fact that the physical origin of thermal noise are the velocity fluctuations. Note, however, that these fluctuations do not always result in thermal noise as pointed out by the general expression (C.1.12), (C.1.23) for o in (C.3.5) and the temperature-independent asymptotic formula (C.1.24). More details on this topic can be found in Appendix B of Willander and M a m o n t o v (1999). Johnson's experimental result (6.4.1) w a s first theoretically explained by Nyquist (1928) with the help of thermodynamics. A kinetic derivation of the well-known Nyquist formula can be carried out by means of the SFI S P D E system that is shown in Section V of M a m o n t o v and Willander (1995b). It is usually believed that there are m a n y types of noise in semiconductor systems, for instance, thermal, shot, generation-recombination, flicker, etc. (e.g., Horowitz and Hill, 1997; Fish, 1993). This classification results from different manifestations of electronic noise. However, it does not stem from the differences in the physical phenomena underlying this noise. To the best of our knowledge, it has not been proved yet that the above different types of noise are caused by different physical effects. The only feature which is certain is that thermal noise is originated from the particle-velocity fluctuations (as is mentioned above). This fact is well-known and important in statistical physics (e.g., Resibois and D e Leener, 1977). Regarding the other types of noise, the corresponding physical pictures are not very clear. A n example is shot noise. The first model for it w a s suggested by Schottky (1918). It explains shot noise by random emission of electrons through a potential barrier. In spite of the fact that it is not proved (up to now, by the way) that the above emission is the only origin of shot noise, the possible alternatives were not considered for a long time. Only twenty years later, pioneering work by Rack (1938) showed that the Schottky model can be rederived and interpreted as a result of the velocity fluctuations. Gupta (1982) reported the thermodynamic derivation of shot noise in nonlinear two-terminal resitor on the basis of the velocity fluctuations, in other words, as a particular manifestation of thermal noise. The corresponding kinetic derivation w a s developed in Secti-

194

Z^o's ParftoJ D:^ere^^a/ E^Ma^ons /or EZee^rons m Semtconduc^ors

on V of M a m o n t o v and Willander (1995b) with the help of the above SFI S P D E system. Interestingly, semiconductor-engineering community (e.g., Sarpeshkar, Delbruck and Mead, 1993) has also recognized that thermal and shot noises have the same physical origin. The flicker noise is even more complex to be interpreted in terms of specific physical phenomena (e.g., see the discussion and references in M a m o n tov and Willander, 1997e). The diversity of the approaches to this noise seems to be unlimited. The discussions sometimes reach "the infrared and ultraviolet catastrophes" predicted by Mandelbrot (e.g., see Section 2.2 of M a m o n t o v and Willander, 1997e) and the emitted photons "lost to the universe" (e.g., p. 11 of V a n Vliet, 1991). The authors of the present book prefer the viewpoint that the flicker noise is due to the nonlinearities of stochastic systems (see Section 2.1 of Mamontov and Willander, 1997e). This idea w a s first proposed and discussed by Surdin (1939) w h o schematically explained the flicker noise in terms of the nonlinear friction (cf., the dependence in Eq. (6.3.43) on the normalized energy (C.1.6)), i.e. as a feature of the particle-velocity fluctuations in nonlinear systems. The corresponding practice-oriented procedure presented in Section 4.10.4 can serve as the efficient basis for quantitative analysis of this noise. Generation-recombination noise in electronic systems is not very distinct notion. Indeed, c o m m o n texts in statistical mechanics (e.g., Balescu, 1975; Resibois and D e Leener, 1977) usually include neither physical pictures nor mathematical treatment for this noise. This fact can by no means be disregarded. Another point to be stressed is that the works concentrated on generation-recombination noise usually skip the analysis of the standard deterministic models for the fluid-particle position (e.g., similar to that presented in Appendix F) whereas this analysis is logically the first, simplest and inevitable step in any corresponding research. It turns out that the velocity fluctuations, the basis of thermal noise, can also be the physical origin of shot noise and the flicker noise. Moreover, more special types of noises in semiconductor systems, for instance, the socalled microplasma one can be explained with these fluctuations and in terms of the above SF-ISPDE model (e.g., Section 3 of Mamontov and Willander, 1995c). The SF-ISPDE approach enables one to analyze noises in more complex systems. For example, Section 2 of Mamontov and Willander (1995c) develops the stochastic noise model for B T s in the form of I S O D E system intended for semiconductor circuit simulation. The next step is the interpreta-

SenMcondMcfor No:ses an<^ ^Ae SF-7SPDE .Sysfem

195

tion of noise in a single B T and multidevice circuits as multidimensional D S P carried out by Mamontov and Willander (1997b). This treatment also includes derivation (not formal incorporation) of the stochastic noise sources in the corresponding I S O D E systems and serves as the analytical basis for the Q u A n T (Quick Analyzer for Transistors) software (Appendix 2 of M a montov and Willander, 1997b). The software m a d e it possible to develop the practical engineering recipes to reduce the high-frequency values of the spectral densities for terminal currents of B T s (Section 3.2 of Willander e? a?., 1996; M a m o n t o v and Willander, 1997a). A compact presentation of the above results, i.e. a brief outline of Chapter 2 of this book and a comparison with the approach by Demir, Liu and San-giovanni-Vincentelli (1996), is reported in M a m o n t o v and Willander (1997d). A s is noted in this paper, a part of the practical recipe has already been confirmed experimentally by other authors. The n e w point of the recipe related to improvement in the corresponding S N R (see Appendix B ) is developed in Appendix B of Willander and M a montov (1999). This work also generalizes the D D version of the SF-ISPDE system for the non-quasi-electrostatic electromagnetic field. In particular, the influence of the field upon the covariance and spectral density of velocity component % is expressed analytically with the help of the equations which are similar to stochastic description (6.3.43).

6.4.2

S o m e tftree^oTts /br /Mfure tfeueZopmenf

A s is stressed in R e m a r k 6.9 (see Section 6.3.2), deterministic P D E system (6.3.7), (6.3.11), (6.3.13) (or (6.3.15)) is able to model all the cases (6.3.25) -(6.3.29). However, it is traditionally used in semiconductor applied research and engineering only in cases (6.3.25) and in the "weakened" version (6.3.34) of case (6.3.26). So, one of the directions for future development is to apply the above system to cases (6.3.26) (without limitation (6.3.34)), (6.3.28) and (6.3.29), i.e. to semiconductor fluids in mesoscale and microscale domains allowing for coherent transport. Manifestations of the microscopic effects in semiconductor devices are diverse and depend on the device structure, the operating conditions and other factors. This is demonstrated, for instance, by the molecular-dynamics simulation (e.g., Yano, Ferry and Seki, 1992), the experimental results (e.g., Matsuoka ef a?., 1994) and the semiconductor numerical studies (e.g., Naveh and Likharev, 1999). The latter paper is based on quantum-mechanical

196

/to's Par%:a/ D:^ereK^:a/ Eguo^tons /or EZec^ro^s m SemtcoTtduc^ors

modelling whereas the first one shows that the approaches totally different from quantum mechanics can successfully be used as well. In the light of these works, the following question is naturally posed: w h y does one need to apply the wave-diffusion and wave transport capabilities of semiconductor P D E system (6.3.7), (6.3.11), (6.3.13) (or (6.3.15))? T h e point is that the practical semiconductor engineering systems, single devices and muitidevice circuits, have m u c h more complex structures than those which are in the focus of the basic semiconductor research. The practical systems usually consist of different materials configured as the space domains of rather intricate shapes and very different sizes subject to external excitations, for instance, driving electric signals, temperature, irradiation and others. At the same time, research and engineering applications are usually concentrated on typically a few key parameters of the system so m a n y details provided by quantum-mechanical treatment remains beyond the interest. The discussion in Section 6.1 sharpens these issues. More aspects of the practical advantages of macroscopic modelling of the meso- and microscopic phenomena are reported by Y u , Dutton and Kiehl (1998). O n e of the possible solutions of the above problem can be application of the mesoscale- and microscale-domain capabilities inherent in semiconductor P D E system (6.3.7), (6.3.11), (6.3.13) (or (6.3.15)) not only to c o m m o n cases (6.3.25) or (6.3.34) but also to cases (6.3.26)-(6.3.29). This is a vast field where the future results will give more issues on h o w to improve and modify the approach. This is a n e w direction. In particular, it is not pointed out by Y u , Dutton and Kiehl (1998). The main advantages of the present approach is its flexibility in accounting various physical phenomena (including those prescribed by specific engineering applications), practical efficiency for multiparticle systems and the fact that it goes back to A R W , the discretization of quantum mechanics (see Section 6.2.1). Another direction for future development is the stochastic extension of the previous one. T h e SF-ISPDE system in Section 6.3.3 is suitable for the weakly-mesoscale-domain case (6.3.34). The D D version of the system is suitable only for the macroscale-domain case (6.3.25). So, the problem is to generalize the SF-ISPDE system for all other cases, i.e. (6.3.26)-(6.3.29). This development can siginificantly extend capabilities and adequacy of the macroscopic modelling in semiconductor research and engineering. To what extent the corresponding results will be quantitatively complete and which modifications will have to be done to m a k e the approach mature are the questions to be answered in future research.

Chapter 7

Distinguishing Features of Engineering Applications

This chapter discusses the features mentioned in its title. They include issues (1.2.14), (1.11.1)-(1.11.3). Each of them is considered in a dedicated section below. These features do not characterize all the details of engineering applications. They merely present the conditions which should necessarily be taken into account by developers of the corresponding practical techniques, both analytical and numerical.

7.1

High-Dimensional Diffusion Processes

Engineering deals with design of products. Most of them is complex, multicomponent systems. They are characterized with a big number of variables. So, the corresponding models are formulated in terms of multidimensional mathematical objects. For multidimensional stochastic processes (see Section 1.2), this means that its dimension ^ is high or, more specifically, condition (1.2.14) holds. For instance, number J can be proportional to the number of the particles in a multiparticle ensemble or to the number of the p-M junctions in semiconductor circuits. Another example is boundary-value problems for I S P D E (or ISPIDE) systems. These models are considered in Chapters 5 and 6. After a proper discretization of the involved space-derivatives (for example, by means of the SAI method discussed in Chapter 5), the ISPIDE system is presented with I S O D E system (1.9.2). Then cf is equal

197

198

D:sf:ngM:s/Etng FeafMres o/'E^gmeermg App/:cat:oKS

to the number of the discretization points in the interior of the domain associated with the boundary conditions and is typically high (see (1.2.14)).

7.2

Efficient Multiple Analysis

Design of the products is usually developed as a sequence of design iterations. Each of them is aimed at evaluation of a specific version of the product or the product model. If the current version is satisfactory, the design is completed. Otherwise, a new version is suggested and analyzed at the next iteration. In so doing, application of the mathematical model at least to some of the design iterations is highly desirable. The reason is that the model enables one to replace the prototype-product manufacturing which is usually expensive with analysis of the model of the current version of the product that is usually much less expensive. This makes it possible to reduce design cost and duration and, as a result, to offer the product to market in time and at a lower price. Mathematical models are usually implemented in simulators, the computer software simulating the models on a computers. It follows from the above picture that, as a rule, the design is associated with multiple analysis of the model and hence requires multiple running of the corresponding simulator. Within the design cycle (which must be as short as possible), it may be necessary to simulate the model a lot of times. However, not all engineering institutions have the budgets sufficiently large to purchase and maintain (or rent) computer systems of exceptional computing power. Most of engineers and scientists relies and will rely on reasonably-priced computers of standard workstation capabilities. This leads to the issue formulated in (1.11.1).

7.3

Reasonable A m o u n t of the M a i n Computer M e m o r y

Hardware capabilities of the computer systems in (1.11.1) are typically medium. There are various ways to increase their computational speed. One of them is discussed in Appendix H. However, one cannot do too much with the corresponding memory requirements. In principle, if the logical address space of a simulation software exceeds the actual space of the computer main memory, the so-called virtual memory has to be applied.

Rea/-TY?ne StgMa/ TratKs/brma(:oy: &y Dt^hston Process

199

This technique provides storing temporarily unused sections of the main m e m o r y on the hard drive in a swap space dedicated to the virtual m e m o r y system. However, moving data to and from the hard drive is very slow compared to the access time for the main memory. Too m u c h disk swapping can dramatically slow the simulation. Engineering computing is usually based on the iterative algorithms. Multiple application of the virtual m e m o r y (e.g., at each iteration) m a y significantly decrease the speed. That is w h y one should rely mainly on the space of the main memory. A typical space of the workstation main m e m o r y is not very big, say, 64 megabytes or so. Hence, taking this and (1.2.14) into account, w e formulate one more distinguishing feature of engineering applications as condition (1.11.2).

7.4

Real-Time Signal Transformation b y Diffusion Process

Features (1.2.14), (1.11.1) and (1.11.2) certainly complicate the analysis. However, there is an issue which can help to simplify it. Indeed, in m a n y engineering systems, time dependences can be separated (rather approximately) into two groups. Each group has its o w n time parameter characterizing duration of the corresponding time changes. The first group is determined by external driving excitations and described with parameter tr^., > 0. For instance, it m a y be the characteristic time of the external signals in drift vector g(f,x) and diffusion matrix #(f,x) in (1.8.2) and (1.8.3). The second group is determined by internal time-dependent phenomena and described with parameter T,„, > 0. For example, in the model based on linear differential equation, it m a y be the relaxation time of the corresponding fundamental solution. Parameter T. is usually m u c h less than T since only in this case the internal behavior of the system can adequately follow time changes of the external control variables. If one considers the time scale of the external signals as that of the surrounding-related, real time scale, one can say that the system operating in the above w a y provides a real-time signal processing (or transformation). Thus, the next distinguishing feature of engineering problems can be formulated as semiqualitative inequality (1.11.3). It can help to overcome the difficulties caused by (1.2.14), (1.11.1) and (1.11.2).

Chapter 8

Analytical-Numerical Approach to Engineering Problems and C o m m o n Analytical Techniques

8.1

Analytical-Numerical A p p r o a c h to Engineering P r o b l e m s

The analytical-numerical approach to engineering problems presents replacement of a part of a purely numerical treatment with appropriate analytical results. The purpose is to solve some part of the computational task analytically thereby eliminating their numerical, time-consuming solving. In so doing, numerical procedures are normally applied only to the part which cannot be solved analytically. The fractions of the analytical and numerical solvings within an entire algorithm m a y be very different depending on the specific problem, the applied theory, the level of theoretical and computational background of the algorithm developers and a lot of other factors. Generally, the less complex the numerical parts is, the more efficient practical computing is provided. The analytical-numerical approach have m a n y benefits compared to the purely numerical one. W e notice only the following four of them. * *

It can significantly speed up the computer simulation. It naturally offers a wider set of modelling techniques and thereby provides more freedom and flexibility in construction of simulation

201

202

Ana^y^ca^-NHmertca^ yecAytiq'Hes /or EKgtneermg ProMems

*

*

algorithms. The analytical representations that it includes can also help to better understand numerical results. This provides deeper insight into behavior of the basic characteristics of the modelled system and enables one to achieve a greater success in the subsequent engineering analysis and design. The analytical-numerical treatment stimulates not only theory of numerical methods but also the basic theory and mathematical modelling of the underlying phenomena thereby contributing to, say, harmonization of theory and practice. In so doing, the theory is thought to be developed vertically, i.e. for the different levels of description d o w n to the analytical results suitable for efficient practical implementation. The numerical procedures should, in their turn, allow for the basic qualitative features of the simulated analytical expressions.

The obvious disadvantage of the analytical-numerical approach is that it is usually based on approximations. Understandably, analytical expressions for parameters of complex systems can be derived and applied only if simplifications (sometimes noticeable) are involved. There is, however, the circumstance which significantly reduces severity of this problem. The point is that the engineering models (drift and diffusion functions g, TV in the D S P case) correspond to the modelled phenomena with an error typically of 1 0 % . Such a high percentage of the model error is c o m m o n in various applications, for example, semiconductor bipolar transistors (p. 55 in Flower, 1990). Therefore, it is in general fruitless to pursue the accuracy of, say, 0.1-1% by m e a n s of highly time-consuming purely numerical m e thods. For a huge class of the real-world systems, the above advantages prevail over the limited increase in the simulation error. In other words, the analytical-numerical approach presents a compromise between the moderate accuracy of the involved analytical approximations and the noticeably reduced complexity and computing expenses of the applied numerical algorithms. In some engineering problems, such treatment m a y be the only reasonable option, even in spite of the fact that its overall accuracy is worse than that of the numerical-only alternative. Natural candidates to form the analytical basis of the above analytical-numerical techniques are the c o m m o n analytical results of D S P theory listed in Section 1.12 and summarized at the end of Section 1.12.3.

Ltm:faf:ons o/ CoyKmon Tec/tHtgues m ;Ae NigA-D:mens:'oKa^ Case

8.2

203

Severe Practical Limitations of C o m m o n Analytical Techniques in the High-Dimensional Case. Possible Alternatives

The common analytical recipes formulated in Points (l)-(4) in Section 1.12.3 can serve as the analytical basis of the approach discussed in Section 8.1 only if dimension J is at most of a few units. They, however, turn out practically inapplicable if conditions (1.2.14), (1.11.1), (1.11.2) are the case. The reason is explained below. The common analytical results are based on the corresponding KFEs. The latter, under rather mild assumptions (see Remark 1.16 in Section 1.12.1), present PDEs in Euclidean space which is high-dimensional because of (1.2.14). Multidimensional PDEs are usually solved numerically. The method commonly used for this purpose is the F D technique. This technique for PDEs was discovered by Fubini (1903) and is widely applied since long ago. The F D method was applied to various PDEs, for instance, by Dean, Glowinski and Li (1988). They found that the advanced numerical solution of the particular version (see (5.4) and Section 5.4.5 therein) of nonstationary K F E (1.12.3) in
204

AnaZy%caJ-AfH;ner:ca^ Tec/tMMyues /or E?tg:KeerMg P r o M e m s

perhaps rather rough approximations for them. The corresponding error inevitably affects the figures of merit in Points (1) and (2) in Section 1.11. (For example, application of the simplest approximation (1.6.20) for p, to (1.6.7) and (1.6.8) gives identity in the first case and zero in the second one, i.e. the useless results.) (2) It seems to be reasonable to consider additional options to develop the analytical basis for high-dimensional DSPs. O n e such option can be based on the ideas from those parts of D S P theory where the analytical results, say, practice-friendly are common. For instance, theory of linear I S O D E systems (e.g., Section 8 of Arnold, 1974; (4.4.71) in Gardiner, 1994) describes the expectations, variances, covariances of D S P s by means of O D E systems (also linear). Generalizing this idea, one can attempt to derive O D E systems for the D S P figures of merit even if the forms of the x-dependences of the drift and diffusion functions are nonlinear. The above systems are deterministic like the above P D E s but, at the same dimension of the state space, incomparably m u c h more tractable than P D E s . Sometimes, the ODE-system idea in Point (2) is understood too literally. For example, Demir, Liu and Sangiovanni-Vincentelli (1996) and Demir and Sangiovanni-Vincentelli (1996) use I S O D E system (1.9.2) to simulate noise in semiconductor circuits. In so doing, they linearize the system in a neighborhood of the expectation vector and then apply the well-known results of the narrow-sense-linear-ISODE theory (Section 8.2 of Arnold, 1974) to the linearized system. The numerical results for the latter system are then declared as the results for the initial, nonlinear system. A n obvious and severe drawback of this treatment is that there is generally no basis to justify the above declaration and to apply the above linearization to nonlinear I S O D E systems. The ODE-system idea in the above Point (2) does not prescribe any linearization of drift and diffusion functions g, // in (1.8.2), (1.8.3). O n the contrary, it presumes adequate allowing for their nonlinearities. Points (1) and (2) to a considerable extent determine the purpose of the present book formulated in Section 1.13.

Appendix A

Example of Markov Processes: Solutions of the Cauchy Problems for Ordinary Differential Equation System

This appendix examplifies Markov process / in Section 1.4 with the solution of O D E system

^=g(;,*),

f2f„,

(A.1)

with initial condition (1.4.25) where g:R"^-*R", g,3g/ax€C°(R"), *, presents values (see (1.4.22)) of initial random variable x^ with initial probability density p^. To be considered as Markov process %, the solution of the above Cauchy problem (A.l), (1.4.25) has to be completely described with joint probability density (1.4.35). Initial density p^ is available. Transition probability density p is constructed below. B y virtue of the well-known theorem (e.g., Pontryagin, 1962), initialvalue problem (A.l), (1.4.25) at every fixed (f ,.x ) € R"^ has the unique solution (defined on some time interval which includes point ^ as the left boundary point). W e denote this solution with T)(f^,^,f), i.e. *=il(f..*o.?). HmTt(fo'*o^W.-

4'. 205

^'..

(A.2) (A.3)

206

Example o/'AfarAou Process

Eqaution (A.2) can be regarded as the one which determines values (1.2.12) of stochastic process x depending on values (1.4.22) of initial random variable Xo. This can be expressed by m e a n s of the following transition probability density P('o.*o''-*) = 8[*-il(;,.*o-;)]'

f^f..

(A.4)

T h e n Eq. (A.3) shows that property (1.4.34) holds for density (A.4). Since the right-hand side of (A.4) presents values of delta-function 8 , w e shall associate this density with the deterministic transition (from vector x to vector .x). Thus, the solution of O D E (A.l) with initial condition (1.4.25), as Markov process x, is described with joint density (1.4.35) where the transition density is determined by m e a n s of (A.4). In the present, deterministic-transition case (A.4), randomness of process x is solely due to the random nature of initial value A^ in (1.4.25) associated with initial random variable x or initial probability density p . T h e corresponding subsequent probability density (1.4.31) is

p,(^^^)=j*p^)8^-f,(^,^^)]^,

;>;„.

(A.5)

If initial random variable x„ is in fact not random, i.e. its values j^ are reduced to a single, deterministic vector e , then Eq. (1.4.37) holds for initial density p (see R e m a r k 1.8 in Section 1.4). In this case, initial condition (1.4.25) is replaced with

and equation p,(;.,f,*) = 8[*-Ti(;^,f)],

f>^,

(A.7)

follows from Eqs. (A.5), (1.4.37) and the property that (see Section 1.2) the convolution of any generalized function (in particular, delta-function 8 ) with delta-function 8 is equal to the function itself. Equation (A.7) is the simplest, deterministic expression for probability density p,(f^,f,.x). If one denote e(;) = il(;„,e„,;),

f^.,

(A.8)

then (A.7) is written as Eq. (1.6.20). T h e latter equation is the simplest ex-

207

Examp/e of MarAou Process

pression for density p,. Let us assume n o w that unique solution T](^,e^,f) of problem (A.l), (A.6) at any fixed (f„,ejE R"^ is defined for all f > ^ .

(A.9)

Then the above considerations are valid for all f^ f„ - They do not apply function f) at f < L. However, this function is sometimes involved not only for all ; > L but also for all K f g , i.e. for every f€ R. The point is that, in m a n y applied problems, O D E system (A.l) is convergent in the sense of §16 in Chapter IV of Demidovic (1967) (see also Section 1 in Chapter 7 of Pliss (1966) for the particular periodic case). This convergence by definition m e a n s that feature (A.9) holds, there exists the unique solution Tt.(f) of system (A.l) such that it is uniformly bounded on R, time-independent if g is time-independent and is asymptotically stable in the large.

(A. 10)

Property (A. 10) points out that any solution of (A.l) tends, or converges, to solution Ti. as ;-* oo. In m a n y , sufficiently regular cases, it can be described by m e a n s of equation !],(;)= limr](;„,e„,f),

for all ^ E R ,

(A.ll)

where the limit is independent of e^ and is uniform in an appropriate sense. In terms of quantities (A. 7) and p,. (*) = 8 [* - Ti, (;)],

for all ; E R,

(A. 12)

Eq. (A.ll) can be expressed as (1.12.8) in Section 1.12. Solution ri; is sometimes called steady-state. In m a n y engineering systems, the steady-state solutions are very important. T h e stochastic generalization of the steady-state solution leads to the stochastic process defined for all ;e R. If this process is a Markov one with transition probability density p(f_, - ,;^, -) defined for all f_,^E R under condition (1.3.2), it is necessarily invariant (see also the text below (1.7.10)).

208

E x a w ^ e o/AfarAou Process

T h e importance of the steady-state deterministic responses to a great extent determines the importance of invariant Markov stochastic processes.

Appendix B

Signal-to-Noise Ratio

Signals in engineering systems (e.g., currents and voltages in semiconductor circuits) are usually described by means of entries of one-dimensional or multidimensional stochastic processes. S o m e of the entries (for instance, the input or output signals) are especially important. They should be carefully analyzed and highly predictable, ideally, deterministic. Therefore, design of the system for low noise has the purpose to first of all m a k e the important signals as m u c h deterministic as possible. H o w to quantitatively measure the extent of randomness of the signal? Which practical figure of merit can be applied? The answers to these questions can be obtained with the help of the Bienayme-Chebyshev inequality (e.g., (15.7.2) in Cramer, 1946; p. 10 of Arnold, 1974), one of the fundamental results of probability theory. This inequality was discovered independently by I. Bienayme in 1853 and P. L. Chebyshev in 1866. Let X.(^) be the stochastic (generally, neither Markov nor even twopoint) signal under consideration. Let random variable X.('^) have finite expectation e„(f) and variance ^,(f). The Bienayme-Chebyshev inequality quantitatively describes the extent of randomness of the signal in the following way. It gives the upper bound on the probability of deviation of the signal from its expectation in terms of its standard deviation Jl^(f). For signal X.(^'f)' this inequality is written as P({6EE:]x,(^)-e.(;)I^S^^})^l/4',

for all;,

(B.l)

where P is the probability of random event (cf., (1.2.4)) and <;> 0 is a (i;,f)independent number. Estimation (B.l) provides the precise and best pos209

210

S:gnaZ-(o-.2Vo:se Eafto

sible bound for arbitrary signal X*(^) of the above properties. Importantly, the signal need not be Markov, stationary or invariant (see Section 1.7 on Markov processes of these types). In engineering, inequality (B.l) is usually rewritten in terms of the relative deviation ][x,($,f)-e,(f)]/e,(f)j (rather than the absolute one ] X,(^) *e,(f) I) provided that e,(f) ^ 0. In this case, one has F ( { ^ e E : ] [ x ^ J ) ^)]/.,(f)l>5[/F^)/^)]})^l/<', for all?: e,(f)#0, or, which is the same,

F({^eS:][x^,r)-^)]/^(f)l>y}) 0 is a ($,^-independent number. Quantity [e,(f)]^/t^(f) in (B.2) is closely associated with what is called signal-to-noise ratio. Namely, parameter (e.g., the left column on p. 434 of Horowitz and Hill, 1997; Section 11.3.3 of Benson, 1988) SNR(;) = 101g{K(f)]2/F,(;)} dB, forallf:e,(f)^0, (B.3) (where "dB" stands for decibel) is called the signal-to-noise ratio (SNR) of stochastic signal x, at time f. If the signal at time f is deterministic, i.e. !^(f)=0, then S N R (;) = <=.. B y virtue of (B.3), the Bienayme-Chebyshev inequality (B.2) is transformed into P({$e E : ] [x,($,;) -e,(f)]/e,(f) ]>y}) < l O - s t W / y ^ , forallf:e,(;)#0.

(B.4)

Thus, S N R (B.3) determines the upper bound of the probability. For instance, if SNR(f) =30 dB, then, in view of (B.4), the probability that signal x, at time f deviates from its expectation e,(f) more than for 10 % (y=0.1) does not exceed 10*^°/0.1^ =0.1. In other words, relative deviation ][X,(',f)-e,(f)]/e,(f)l is within 10 % with probability at least 0.9. T o continue with specific examples, w e also mention that S N R s of the output signals of c o m m o n consumer audio systems are typically about 45-60 dB.

<StgHa/-fo-M):se Eat:o

211

For the high-fidelity (so called Hi-Fi) equipment, the S N R s m a y reach 90 d B or greater. Clearly, the higher the S N R is, the less random the signal is and the closer it is to its deterministic component (expectation). If the S N R is low, then it can be difficult to detect its deterministic part in the actual, noisy signal. The detectability bound is usually determined (e.g., p. 370 of Larson, 1996) as 10 dB. Thus, a "good" signal has S N R not less than 10 dB. The above discussion demonstrates that S N R of a signal shows h o w meaningful, both practically and theoretically, the signal is.

(B.5)

Thus, achieving high values of S N R s is the task of paramount importance in design for low noise. In engineering, other parameters, not very well suited to be the noiserelated figures of merit of noisy systems, are sometimes propagandized as these figures. To mention a few, w e point out the so-called noise factor, noise figure, noise temperature and equivalent input noise described, for example, by Horowitz and Hill (1997) or Fish (1993). The corresponding notions, in contrast to the S N R notion, proceed from in fact arbitrary assumptions on fairly complex and generally nonlinear stochastic systems (e.g., electronic circuits). A s a result, they are formulated in the heuristics-inspired terms rather than in a proper connection with rigorous probability theory. The lack of such connection manifests itself in particular with the application rules disclosing the arbitrariness of the underlying assumptions (e.g., pp. 434-435 of Horowitz and Hill, 1997). Even the advocates of the above "figures of merit" point out that such figures can easily become "very deceptive" (e.g., p. 435 of Horowitz and Hill, 1997). The inability of this approach to apply the key quantities related to S N R , i.e. e„(f) and y,(;) in (B.3), forces to employ other parameters, not so general and sharp as the mentioned ones. Clearly, issue (B.5) is in so doing not involved. The traditional w a y to improve S N R is to reduce the variance. It is prompted by (B.3). Another, m u c h less c o m m o n option which also agrees with (B.3) is discussed in Section 2.4 in connection with stochastic resonance. This phenomenon is the increase in S N R because of the increase in the expectation in some range of not very low values of the variance.

Appendix C

Example of Application of Corollary 1.2: Nonlinear Friction and Unbounded Stationary Probability Density of the Particle Velocity in Uniform Fluid

This appendix discusses application of Corollary 1.2 and R e m a r k 1.18 (see Section 1.12.3) to an example in stochastic fluid dynamics. The example deals with stochastic velocity of particle in fluid and is described in Appendix C.l. Appendixes C.2 and C.3 focuses on the particular case of the linear particle friction and the general case of the nonlinear particle friction respectively. The nonlinear friction in a simple fluid is analyzed in Appendix C.4.

C.l

Description of the M o d e l

Let us consider an isothermal inviscid isotropic uniform fluid located in domain Q with piecewise smooth boundary 3Q introduced in Section 1.2 (see the text above (1.2.13)). A s in Section 1.2, point in ^-dimensional Euclidean space is denoted with z. The example below deals with the case as ^f=3. The uniformity of the fluid means that the fluid consists of identical particles. Let vector x € R^ at 6? = 3 be the purely random velocity of a particle at time-space point (f,z) and be described with equation (e.g., M a m o n t o v and 213

214

A^oH/:'near Frt'e^ton a n d C/KAoMKded DenstYy o/'^Ae Par^te/e Ve/oc^y

Willander, 1995b; Mamontov and Willander, 1997d; Willander and M a m o n tov, 1999) 8* = --rijtd;+/2l^o;?W($,f), for every z e Q ,

(C.l.l)

where the Wiener-process vector is three-dimensional and the partial differentials point out that vector z is independent of f ,

(C.1.2)

and, thus, z is merely a parameter in Eq. (C.l.l). (Purely random velocity % described by Eq. (C.l.l) is in statistical mechanics usually called the velocity of chaotic motion of the fluid particle.) Since z denotes the particle position, assumption (C.1.2) also means that trajectories of the particles are not modelled. The fluid viscosity is not taken into account in (C.l.l) either. The property of the fluid to be isotropic is presented with the facts that, first, real quantities T and o in Eq. (C.l.l) are scalars rather than matrixes and, second, the matrix of masses of the fluid particle is scalar matrix ?n/ where identity matrix 7 is a 3x3-matrix and w > 0 is c o m m o n , scalar mass of the particle. In what follows, w e assume that mass w is independent of (i[,f,z,.x),

(C.1.3)

absolute temperature T is independent of ($,f,z,x).

(C.1.4)

The latter feature means that the fluid is isothermal. Feature (C.1.3) enables one to represent Eq. (C.l.l) in the following equivalent form J(wjc)= tr ^(w^)^+y2r*^wo JW($,r), i.e. as the particlemomentum-conservation (or the second Newton's law) equation. Its physical meaning can be described by the second paragraph of R e m a r k 1.13 (see Section 1.9) with replacement of .x with a particle m o m e n t u m m x . More specifically, scalar t is the relaxation time of particle m o m e n t u m due to scattering of the particles, T = T(z,M,y2)>0,

(C.1.5)

where dimensionless variable y > 0 is the square root of the normalized kinetic energy

Descrtpfton o/' (Ae Mode^

y^ =

/

215

= o jt'x

(C.1.6)

of the purely random motion of a particle, quantity n is concentration (or n u m b e r volume density) of the fluid particles which is generally a random Held, i.e. (ci., (1.2.13)) n=M(^,;,z)>0.

(C.1.7)

Application of concentration w means that every point z e Q is associated with the ensemble of the particles each of which occupies (on the average) volume n *^. This volume containing one particle can be regarded as w h a t is sometimes called dot. (This notion can be used, for example, in connection with the eigenstructure analyzed in Appendix G under the condition that ^.=^=pg=n *^\) T h e randomness in the ensemble manifests itself in various ways, in particular, with the above dependences on elementary event ^ . The concentration is not modelled with Eq. (C.l.l) and can be determined as the solution of the boundary-value problem for the corresponding P D E with the boundary conditions at 3Q . This option is discussed in Chapter 6. W e also note that if the particle scattering is contributed by a few, say, JV > 1 mechanisms, described with respective relaxation times T., A: = 1, . ,N , then relaxation time T is evaluated according to the well-known formula (e.g., Eq. (6f.l) in Seeger, 1973) T-' = E ^ a=i

(C.1.8)

where ^ = ^(z,n,y2)>0,

R = 1,...,N.

(C.1.9)

T h e above mechanisms always include at least one type of scattering, the particle-particle scattering. It is present in any fluid. ( H o w dominating it is compared to the other mechanisms depends on a specific fluid.) W e denote the corresponding relaxation time with t^. If no other types of scattering are present, i.e. JV^ = 1, the fluid is called simple. More details on the scattering can be found, for instance, in Chapters 3 and 4 of Ridley (1988) or Chapters 6 and 7 of Seeger (1973).

216

AfonZtnear Frtc^tort aytd t/KhouKded De^st^y o/'(Ae Par^teZe VeZoct^y

R e m a r k C.l Dependences (C.1.9) on the normalized energy y^ are usually derived [e.g., see Seeger (1973) or Ridley (1988)] by means of the corresponding techniques of theoretical physics of fluids of charged or uncharged particles. Thus, the physical relevance, qualitative adequacy and quantitative accuracy of the resulting representations depend mainly (if not solely) on the available achievements of theoretical physics. The potential lack or imperfections of the expressions for one or another scattering mechanisms can be due to still unsolved problems in development of this science. Scalar o in Eqs. (C.l.l) and (C.1.6) is the velocity of sound waves propagating in the fluid, o = o(z,n)>0.

(C.1.10)

Allowing for features (C.1.5), (C.1.7) and (C.1.10), w e see that (1.2.13) holds, i.e. velocity X is a random field. The sound-wave velocity o can be described, for instance, with the wellk n o w n expression (e.g., (48) and Appendix A of Espanol, 1995)

^ ^] 8(/nn)

(C.l.ll)

where Tun is the mass volume density of the fluid, II is the pressure in the fluid, and the derivative is evaluated at constant temperature (C.1.4) in accordance with the discussion on p. 1739 of Espanol (1995). In what follows, w e assume that II = II(M). Then Eq. (C.l.ll) is written as

- ^ - T(n)

(C.1.12)

"\ where Ag is the Boltzmann constant and

J-311^.

(c.i.13)

Note that limit relation limT(7!) = l n-*0

usually holds and hence (see (C.1.12))

(C.1.14)

Desertion of ^e Mode^

limo = ^ .

217

(C.1.15)

To be specific, w e point out a few examples of expression (C.1.13). For the fluid constituted with hard spheres of radius g, dependence II(n) has the following form (see (10) in Carnahan and Starling, 1969)

n(w) = ^ r n t + c + a'-si'

(C.1.16)

(1-3)3 where t3 = u n is the volume fraction of the hard spheres and v = (4/3) np^ is the volume of a sphere. A surprising accuracy of formula (C.1.16) is noted, for instance, by Balescu (1975) (see (8.6.3) and pp. 282-283 therein) and Hansen and McDonald (1986) (see (4.6.14) and pp. 95-96 therein). Because of purely geometrical reasonings, parameter o can never be greater than its value 63^ in the densiest package, i.e. a < 6!^. Indeed, in the densiest package of the spheres of radius p, each sphere touches exactly twelve neighboring spheres (e.g., see p. 218 and Fig. 1 in Albeverio et al., 1989) and each of the twelve touch points corresponds to a face of the regular dodecahedron circumscribed about the sphere. This means that C3^ = u /u. where v . is the volume of the dodecahedron. The corresponding analytical calculations lead to the following results: t3^ = (^/6)6,

6=

5 4 5 6 + 2 4 4 0 ^ UKQ2, ^ 5250+2350/5 s ^ = 0.7547.

(C.1.17)

(C.1.18)

(C.1.19)

Curiously, these equations seem to be unknown before. For instance, Reichl (1980) suggests (see p. 381 therein) value c3^=(y2n/6), i.e. Eq. (C.1.17) at 8 = 1 , with no discussion. Balescu (1975) points out (see p. 283 therein) another value, namely, t3^=0.742 not explaining it either. Unlike this, the above derivation and Eqs. (C.1.17)-(C.1.19) provide the exact description of O.,. In the hard-sphere-fluid case (C.1.16), Eq. (C.1.13) becomes

218

ATonHnear Frtc^on anc! E/n&OMnde<^ Denst^y o/'^Ae Par^tc/e VeJoc^y

a,/ x \Zl+4a+4a^-4a^+a^ T(^) = -!

. (1-1)2 This quantity obviously has property (C.1.14). Another example is the electron (or hole) fluid in semiconductor. In this case, dependence II (n) is described parametrically with the following two equations (e.g., (2), (3), (9), (10) in Mamontov and Willander, 1995b) M=JV(r)$^(Z),

(C.1.20)

n=X;,7W(r)$^(Z)

(C.1.21)

/V(r)=2A'(2nw^r)^,

(C.1.22)

where Z E R ,

$ ^ and
(C.1.23)

3-i/2(2) where *P^<2 is the Fermi-Dirac function of index -1/2 and quantity Y has property (C.1.14). T h e third example of the influence of dependence (C.1.13) upon velocity (C.1.12) follows from the previous one. In the limit case as Z-*oc or, that is the same (see (C.1.20)), as H/JV(r)-3-oo, Eqs. (C.1.20), (C.1.22), (C.1.23) and the properties of the Fermi-Dirac functions transform (C.1.12) into asymptotic representation \l/3

lim o =- ^

[-

(C.1.24)

4m ^ "(7*)

where K,= / 3 n / 8 = 0 . 6 8 . Note that property (C.1.15) cannot be valid for quantity (C.1.24) since the latter corresponds to the directly opposite case, i.e. the one as n/JV(r) -> oo. T h e limit value (C.1.24) of velocity (C.1.12) is called the Fermi velocity and applied to the electron (or hole) fluids in the

Description o/ t^e ModeZ

219

so-called degenerate semiconductors or to the electron fluids in metals. More details on representation (C.1.20), (C.1.23) and (C.1.24) can be found in Willander and M a m o n t o v (1999). Returning to Eq. (C.1.1), w e note that this equation is generally not I S O D E since parameters (C.1.5) and (C.1.10) applies random concentration (C.1.7). Equation (C.1.1) can, however, be one of the equations in an I S O D E system. This is the case discussed in Chapter 6. If the concentration H is nonrandom, i.e. (C.1.7) becomes H=n(f,z),

(C.1.25)

then, for every fixed z e Q , Eq. (C.1.1) is a particular version of I S O D E system (1.9.2) with functions g and A determined as follows g(z,H,.y) = -[T(z,n,y2)]-ix,

(C.1.26)

A(z,H,x) = \/2[T(z,?!,y2)]^o(z,,!)Z,

(C.1.27)

where, as above, scalars y and ir(z,/!,y^) are determined by means of (C.1.6) and (C.1.8), (C.1.9) respectively. The corresponding velocity-diffusion matrix is (see (1.9.5)) N(z,M,A:)=2[T;(z,H,y2)]*i[a(z,n)]2/.

(C.1.28)

T e r m (C.1.26) in I S O D E (C.1.1) is associated with the particle friction. If T(z,n,y^) is independent of y and hence (see (C.1.6)) of x, then (C.1.26) is linear in x. This corresponds to the linear friction of the fluid particles. If, on the contrary, -c (z,n,y ^) does depend on y and hence on x, then one deals with the nonlinear friction. For the purpose of the whole Appendix C devoted to the stationary modes, w e assume that concentration (C.1.7) is not only nonrandom (like in (C.1.25)) but also time-independent, i.e. M=n(z).

(C.1.29)

Then Eq. (C.1.1) for every fixed z € Q becomes a particular version of autonomous I S O D E system (1.9.3) (see also (1.9.6)) where functions g and A as well as 7i are determined as before, i.e. with (C.1.26)-(C.1.28). W e shall assume that every solution of I S O D E system (C.1.1) under conditions (C.1.8)-(C.1.10), (C.1.29), (C.1.26), (C.1.27) is a homogeneous D S P with drift and diffusion functions g, 77 described with (C.1.26), (C.1.28). To

220

M w ^ n e a r FrtcftoH OTtd C/nAoun^ed Denstfy o/ tAe Par(:c/e Ve/oc:fy

study the stationary solutions of the above system, one can apply Corollary 1.2 and Remark 1.18 (see Section 1.12.3). They include the results on the corresponding stationary probability density p^. W e also assume that assumption (1) of Corollary 1.2 holds and concentrate on its assertion (2) and assumption (2). Appendixes C.3 and C.4 below analyze density p in connection with dependence of T on the normalized particle kinetic energy y^ (see (C.1.6)). The case as T is independent of this quantity is well-known. It is briefly summarized in Appendix C.2. To assure validity of assertion (2) and assumption (2) of Corollary 1.2, w e shall, according to Remark 1.18, solve the D B equation (1.12.18) for p under conditions (C.1.26), (C.1.28), (C.1.8)-(C.1.10), (C.1.29), (1.7.12)1 (1.7.13) and then verify if this density meets the requirements formulated in assumption (2) of the corollary. This approach is used in Appendixes C.2 -C.4 below.

C.2

Energy-Indpendent M o m e n t u m - R e l a x a t i o n T i m e . Equilibrium Probability Density

If all T^ in (C.1.9) are independent of quantity (C.1.6) (i.e., of the particle kinetic energy), then T (see (C.1.8)) is also independent of velocity x, i.e. T(z,H,y2)=T(2,n),

for every fixed z € Q .

(C.2.1)

In this case, as it is well-known (e.g., p. 126 of Gardiner, 1994), the D B equation (1.12.18) for p, under conditions (C.1.26), (C.1.28), (1.7.12), (1.7.13) has the unique solution p^(z,n,x) = pg(z,n,.t), for any fixed z € U and all x € R^,

(C.2.2)

where

/ \= p^(z,H,jc)

1 [^0(Z,H)]3

f x^x )h exp^! 2[c(z,H)fJ

for any fixed z e U and all x e R^, i.e. density p^ is Gaussian with expectation vector (see (1.7.17))

(C.2.3)

Energy-7H&pendenf Re/axa?toH TSme. E^M^:&rtun Denst^y

e,= 0

221

(C.2.4)

and variance matrix (see (1.7.18)) T^ = ^ (z,n) which coincides with its diagonal [o(z,n)]^7 where o(z,w) = o(z,w),

for every fixed z e Q .

(C.2.5)

Probability density (C.2.3) is of a sharp physical meaning. Indeed, in statistical mechanics or kinetic theory of low-concentration fluids, density (C.2.3) with value (C.1.15) of parameter o(z,n) (cf., (1.28)inBellomo, Palczewski and Toscani, 1988) is called the equilibrium probability density since, being substituted into the so-called collision integral, m a k e s it equal to zero. F r o m this viewpoint, Gaussian density (C.2.3) with value (C.1.12) (rather than (C.1.15)) of parameter o(z,n) can be thought as the corresponding generalization for the fluid of arbitrary (rather than low) concentration. T h e advantage of this option is that it combines simplicity of Gaussain density with the ability to allow for some high-concentration effects. Note that probability density (C.2.3) with the limit value (C.1.15) is also called Maxwellian (e.g., p. 12 of Bellomo, Palczewski and Toscani, 1988). T h e corresponding (particle-position) diffusion parameter D(z,n) is described as (e.g., (3.21) in Willander and Mamontov, 1999) D(z,n) = T;(z,w)[o(z,n)]^, for every fixed z E Q \

(C.2.6)

This equation under conditions (C.2.5) and at the limit value (C.1.15) is k n o w n as the Einstein relation (e.g., (3.8) on p. 219 of K a m p e n , 1981). If the particles in the fluid are charged, then Tr(z,n) also provides the so-called drift mobility. For instance, if the particles are electrons or holes, then the particle mobility n(z,w) (e.g., (2.29) in Willander and Mamontov, 1999) H(z,H) =
for every fixed z e U ,

(C .2.7)

where % is elementary charge. Density p,(z,n,<) (see (C.2.2)) under conditions (C.1.26), (C.1.28), (C.1.8) -(C.1.10), (C.1.29) obviously satisfies assumption (2) of Corollary 1.2. Then, by virtue of assertion (2) of this corollary, the corresponding D S P specified

222

Nb?tH?:ear .Frtc^to?: a?:d t/H&ou?:de^ Dens:?y o/ ^Ae Parf te/e Ve/oe:^y

w i t h density p (z,M,-) as the initial o n e is stationary a n d its stationary probability density is p^(z,/!,-). T h e stationary D S P is the stationary Ornstein-Uhlenbeck process (e.g., Section 8.3 of Arnold, 1974). Its covariance matrix C(A)=C(z,n,A) (see (1.7.26), (1.7.20), (1.7.28)) a n d spectral-density matrix ^ ( / ) = ^(z,n,/) (see (1.7.25) or (1.7.30), (1.7.31)) coincide with respective diagonal c o m p o n e n t s C(z,n, A ) / a n d ^(z,/?,/)7. T h e diagonal scalars are w e l l - k n o w n a n d expressed as follows

C (z,n, A ) = [o(z,n)] ^exp

for all A S R ,

(C.2.8)

for all/eR,

(C.2.9)

*r(z,7!)

^(z,n,f) =

i^(^) , {l+[^(z,n)M]'}

where M is described with (1.7.22). In contrast to the present appendix, the next one does not assume the y-independence in (C.2.1).

C.3

Energy-Dependent Momentum-Relaxation Time: General Case

Let us now consider the general case, i.e. as t^. in (C.1.9) depend on quantity (C.1.6) (i.e., the normalized particle kinetic energy), so they and hence T (see (C.1.8)) also depend on velocity .x. In this case, the D B equation (1.12.18) under conditions (C.1.26), (C.1.28) is written as 3 3*,.

P,(Z,M,X)

P,(Z,H,.X) -r(z,M,y3)

[°(z.")f

T(z,n,y2)

for all %: = 1, ...,<^ a n d every fixed z G Q , w h e r e y d e p e n d s o n j; (see (C.1.6)). T h e general solution of .this equation u n d e r probability-density conditions (1.7.12), (1.7.13) is

223

Energy-Dependent Re/ajca^on T*:me.' Genera/ Case

T^(Z,/!)

for every fixed z € Q ,

(C.3.1)

where function p - is described with (C.2.3) and quantity Tg(z,n) is determined from property (1.7.13) applied to (C.3.1), i.e. with equation Tg(z,M)= f-c(z,n,y^) Pg(z,M,.x)J.x,

for every fixed z E Q ,

or, in view of (C.2.3) and (C.1.6),

To(2'") =

fy^T(z,n,y^)exp

\

ZJ

Jy,

for every fixed z E Q .

(C.3.2)

W e shall assume that the integral in (C.3.2) exists, i.e.

fy2-r(z,H,y2)exp

zi

Jy < oo,

for every fixed z E Q . R e m a r k C.2

(C.3.3)

It follows from (C.1.26), (C.1.28) and (C.3.1) that Pg(z,n,j;)

g(z,n,.x)p^(z,H,.x) = Tg(z,w)

p^(z,/!,^) j;, /f(z,/!,jc)p (z,H,x)=2[o(z,M)]2^C 7, T^(Z,/!)

for every fixed z E Q . These quantities, as functions of x E R^, are infinitely smooth. Subsequently, assumption (2) of Corollary 1.2 is valid regardless of if momentum-relaxation time t (z,n,y ^ ) , as a function of x E R^ (see (C.1.6)), is smooth, continuous or bounded. Because of (C.3.1), stationary density p need not have any of these properties either. T h e only required property of T(z,n,y^) is that it m u s t be integrable in .x over R^ and the corresponding integrals exist, i.e. relation (C.3.3) (as well as (C.3.6) below) m u s t hold. T h e above remarkable feature provides a considerable freedom in constructing the momentum-relaxation-time functions in various applied pro-

224

JVoM/tnear Fr:c(:on aTtd UhAoKndetf Denst(y o/* (Ae Potr^:c^e VeJoet^y

blems. Appendix C.4 considers one of the corresponding examples where T(z,n,y^) is unbounded in y and hence in .x. In view of R e m a r k C.2, density p^(z,M,-) (see (C.3.1)) under conditions (C.1.26), (C.1.28), (C.1.8)-(C.1.10), (C.1.29) satisfies assumption (2) of Corollary 1.2. Then, by virtue of assertion (2) of this corollary, the corresponding D S P specified with density p (z,n,) as the initial one is stationary and its stationary probability density is p^(z,n,-). Unlike equilibrium density (C.2.2), stationary density (C.3.1), by m e a n s of the y^-dependence of the momentum-relaxation time in (C.3.1) and (C.3.2), does take into account redistribution of the velocity in the course of the particle scattering. Analytical Eq. (C.3.1) can be even more helpful since modern statistical mechanics does not include too m a n y analogous results (i.e. on the stationary but generally non-equilibrium probability densities). Density (C.3.1) (see also (C.2.3), (C.1.6)) is an even function of every entry of vector *. T h e n it follows from (1.7.17) that (C.2.4) holds. T h e corresponding variance matrix l^(z,M) is determined according to (1.7.18). Note that, in case of (C.3.1), its diagonal entries are equal to each other. So, returning to diagonal [o(z,n)]^7 of t^.(z,n) (see the text between (C.2.4) and (C.2.5)), one has [o(z,n)]2=(l/3)tr[^(;,,,)], for every fixed z e Q .

(C.3.4)

This m e a n s that [o(z,M)]^=(l/3)f j;^p^(z,^)^jc for every f i x e d z S Q or,in view of (C.3.1), (C.2.3) and (C.1.6),

o(z,/:) =

1 jy"T(z,H,y2)exp 3\ 7t Tc(z,H)


o (z,n),

for every fixed z e Q .

(C.3.5)

In applied problems, the variance is normally finite. So w e shall assume that the integral in (C.3.5) exists, i.e.

Energy-Dependent TMaxatton Ttme.' Genera/ Case

225

jy-r(z,n,y2)exp -3L Jy < oo,

for every fixed z E Q .

(C.3.6)

For the stationary D S P under consideration, the diagonal entries of covariance matrix C(z,M,A) (see (1.7.26), (1.7.20), (1.7.28)) are equal to each other and hence present scalar C(z,M,A) in diagonal component C(z,n,A)7 of C(z,w,A). Thus, scalar C(z,n,A) is the covariance of any entry of the stationary-DSP vector and can be determined as follows C(z,H,A) = (l/3)tr[C(z,H,A)], for every fixed z E U and all A E R,

(C.3.7)

that is similar to Eq. (C.3.4). Equation C(z,n,0)=[o(z,n)]2,

for every fixed z E Q \

(C.3.8)

stems from (1.7.28), (C.3.4) and (C.3.7). Note that C(z,H,-A) = C(z,M,A), for every fixed z S U and all A E R,

(C.3.9)

because of (1.7.20) and (C.3.7). It should be emphasized that notion of the particle-position diffusion parameter D (see (C.2.6)) is inherently associated with Brownian motion (see R e m a r k 1.13 in Section 1.9). This, in terms of entries of the stationary particle-velocity vector, means that the velocity is approximated with a scaled white noise. More specifically, velocity covariance C(z,n, A) is approximated with function 2D(z,n)8(A) for all A E R . In so doing, the diffusion parameter is determined from the condition that the integrals of the approximating and approximated functions with respect to A over R are equal to each other, i.e. (see (C.3.9))

D(z,n)=fC(z,n,A)JA, 0

for every fixed z e U .

(C.3.10)

This equation stemming from the above simple mathematical issues is regarded in statistical mechanics as one of its basic results and is k n o w n in

226

iVor^tnear frtc^ton and t/H6oH^ded Denstfy o/' tAe Par^tcZe VeZoctty

it as the Green-Kubo formula (e.g., (XI.5) in Resibois and D e Leener, 1977). T h e spectral-density ^(z,w,/) corresponding to scalar covariance (C.3.7) is evaluated as follows

^(z,n,/) = !S(z,H,-/) = 4fC(z,M,A)cos(2n/A) 0 ,

(C.3.11)

and represents any of the diagonal entries of spectral-density matrix (1.7.21) or of real part (1.7.30) of this matrix. According to (1.7.24), quantity (C.3.11) exists only if (see (1.7.24))

f I C(z,w,A) ] JA < oo,

for every fixed z e U .

(C.3.12)

It follows from (C.3.11) at /= 0 and (C.3.10) that 5(z,n,0) = 4D(z,H),

for every fixed z e U ,

(C.3.13)

i.e. diffusion parameter (C.3.10) provides the spectral-density value at zero frequency. R e m a r k C.3 Diffusion parameter (C.3.10) enables one to involve simplifications w h e n desirable. For instance, they can be associated with quantity T(z,w) determined from Eq. (C.2.6) where [o(z,w)]^ and D(z,n) are calculated with (C.3.5) and (C.3.10). Parameter i;(z,M) introduced in this w a y has the meaning of a momentum-relaxation time and can be interpreted as the characteristic time of the particle-velocity autocorrelation function [o(z,n)]*2C(z,H,A) (e.g., see (3.1), (3.4) and (3.8) in Scheidegger, 1958). For brevity, w e call momentum-relaxation time i(z,M) effective. Effective momentum-relaxation time T(z,n) can be regarded as the averaged in the above sense value of the y -dependent momentum-relaxation time t(z,H,y ^) and, thus, can serve as a reasonable approximation for it. In particular, effective time T(z,n) m a y be used in drift vector (C.1.26) and diffusion matrix (C.1.28) instead of -r(z,w,y^) to simplify the model or in Eqs. (C.2.7) and (C.2.8) to provide rather rough but simple estimations for covariance (C.3.7) and spectral density (C.3.11). More generally, effective m o mentum-relaxation time i:(z,H) enables one, if necessary, to reduce the present, general case where the momentum-relaxation time depends on the

Energy-DepeHdenf Re^axa^o?: T:me; Case of Stmp^e FYuM

227

normalized energyy to the particular one discussed in Appendix C.2 where the above time is independent of the energy. The next appendix shows h o w the above results can be applied to a simple fluid.

C.4

Energy-Dependent Momentum-Relaxation Time: C a s e of Simple Fluid

This appendix considers an example illustrating the treatment in Appendix C.3. Let the fluid be simple, i.e. A^=l, and momentum-relaxation time Ti(z,M,y^) in (C.1.8), i.e. (see the text below (C.1.9)) the particle-particlescattering parameter, is of the following specific form T(z,H,y2) = T,(z,H,y2) = ^(z,n)(y2)-'

(C.4.1)

which can be regarded as the generalization of Eq. (C.2.1) to nonzero values of n u m b e r e. W e assume that e e [0,3/2).

(C.4.2)

The particle-particle scattering is pronounced not in any fluid. For example, it is negligible in the electron (or hole) fluid in the so-called nondegenerate semiconductor (e.g., p. 175 of Ridley, 1988). This fact should be remembered w h e n one tries to apply the simple-fluid theories to the fluids with the scattering pictures strongly different from that in simple fluid. The expressions for the energy-dependent momentum-relaxation times due to the particle-particle scattering which are m u c h more complex than (C.4.1) are also available. S o m e of them like the Ridley formula (see (4.138) in Ridley, 1988) for the electron-electron interaction are well-known in semiconductor theory. Equations of this kind are, as a rule, based on sophisticated derivations which, however, fail to avoid strong physical simplifications, so the resulting approximation error is usually noticeable (e.g., p. 176 of Ridley, 1988). It turns out that one of the main difficulties is to construct an adequate (even qualitatively, not to mention quantitatively) description at high values of the particle kinetic energy including the limit case as the energy tends to infinity. This in particular forces one to replace the energy variable with a fixed energy value (see p. 176 of Ridley, 1988) that drastic-

228

JVon^near Frt'c(:o?: and t/^Aounded Densely o/' (/te Par(K;Ze VeZoct^y

ally reduces potential advantages of the above sophisticated expressions (cf., R e m a r k C.l in Appendix C.l). S o m e details on the asymptotic behavior at the high energy are discussed in R e m a r k 4.2 in Section 4.10.2. T h e present appendix applies Eq. (C.4.1) since it is sufficiently simple for analytical calculations and includes the typical, power dependence on the normalized energy (C.1.6). Quantities T^(z,n) and e in (C.4.1) are determined by the specific-fluid nature. For instance, Eq. (C.4.1) at e = 1/2 is the asymptotic representation of the above Ridley formula as the energy tends to zero. However, in what follows w e do not specify the nature of the fluid in connection with (C.4.1). Note that if e > 0 then stationary probability density (C.3.1) becomes unbounded at x = 0, more specifically, it tends to infinity as I .x ] -* 0. This is the case allowed by R e m a r k C.2 (see Appendix C.3). Let us examine which specific versions of (C.3.2), (C.3.5) and (C.3.1) correspond to the nonlinear-friction-related dependence (C.4.1). Substitution of (C.4.1) into the integrals in (C.3.3), (C.3.6) and allowing for the well-known formula 860.17 in Dwight (1961) result in the following equations



j.

Jy

r!^k(2-")'

2; for every fixed z € Q ,

jy"-r(z,H,y2)exp - ^

^

=^^^4-26^^

Jy = 2^

(C.4.3)

5-2e*-

for every fixed z e Q ,

T.(2'").

(C.4.4)

where T is the gamma-function. Integrals (C.4.3) and (C.4.4) exist under condition (C.4.2). Applying (C.4.3) and (C.4.5) to (C.3.2) and (C.3.5), one obtains

;(z.") = — r

^-[^(z,/!),

for every fixed z e Q ,

(C.4.5)

Energy-Depende/:^ Re/axa(ton Ttme; Case ofStmp/e F ^ M

229

o(z,M) = 1- — a(z,M),

\

3 for every fixed z e Q .

(C.4.6)

Equations (C.4.5), (C.3.1) and (C.1.6) lead to /it ! o^ p^(z,H,^) = Po^-"-*)' 2 ^ ^r[(3-2e)/2] T for every fixed z e Q .

(C.4.7)

Comparison of (C.4.7) and (C.4.6) with (C.2.2) and (C.2.5) respectively explicitly shows the influence of nonzero values of parameter (C.4.2) in Eq. (C.4.1). They not only m a k e density p^ unbounded as mentioned above. Increase in them also reduce standard deviation (C.4.6). More specifically, in the limit case as e f 3/2, density (C.4.7) tends to the delta-function 8(x) and standard deviation (C.4.6) tends to zero. This m e a n s that the corresponding stationary D S P in the above limit case becomes deterministic and its values are reduced to zero (cf., R e m a r k 1.1 in Section 1.2). These interesting facts is a manifestation of the nonhnearity in x of the friction-related term (C.1.26) under condition (C.4.1). Equations (C.3.10) (see also (C.2.6), (C.3.13)) and (C.3.11) apply covariance (C.3.7). T h e analytical approach developed in Appendix C.3 and illustrated in the present appendix will become complete only if the corresponding expression for the covariance on the right-hand side of Eq. (C.3.7) is available. This d e m a n d is m e t in Section 4.10.1 which derives the expression on the basis of the deterministic-transition approximation (see Section 4.9). T h e result is applied to simple fluids and the hard-sphere fluid in Sections 4.10.2 and 4.10.3 respectively. Section 4.10.4 summarizes the procedure to evaluate all the related quantities.

Appendix D

Proofs of the Theorems in Chapter 2 and Other Details

D. 1

Proof of T h e o r e m 2.1

T h e proof considers the Markov stochastic process in L e m m a 1.2 (see assumption (1)) where f , ^ are arbitrary such that f^>f_>^ and ^ is according to assumption (1). In so doing, Eq. (1.8.2) in assumption (2) is applied to this process. Multiplying (1.8.2) by p,(^,f_,^_) where f >f_>f^, taking integrals on both sides of the resulting equality with respect to x over R^, and accounting the uniformity in assumption (2), one gets lim

f f(x—.x_)p(;_,^,^,j;J^.x^ p,(^,f_,x_)Jx_=fg(f_,j;_)p,(^,f_,x_)6f.x_,

forallf_>^.

(D.l.l)

The double integral on the left-hand side can be presented as follows < <(.x+-x_)p(f_,.x_,^,.xJJ.Y^ P.(fo,f-,*.)6fX-

< fj^p(f,x_,f^,jtJ
fp(f,x,^,j)Qf_>^.

231

(D.1.2)

232

Proo/s o/^ (Ae Theorems tn CAap^er 2

T h e second integral on the right-hand side of (D.1.2) is equal to e(f ) because of Eq. (1.4.10) and L e m m a 1.2 in assumption (1). Expectation e(f ) exists due to assumption (3). The first integral on the right-hand side of (D.1.2) is equal to g(fj because of Eq. (1.4.28) at (;,*) replaced with (f_,.x_) and L e m m a 1.2 in assumption (1). Expectation e(f J exists due to assumption (3). Thus, Eq. (D.1.2) is equivalent to (

<(jt^-x_)p(;,jf_,^,.xj;f;^ P.(fo,f-,*-)
for all ; > ; > ; .

(D.1.3)

It follows from (D.l.l), (D.1.3) that

lim

J*g(f_,.x_)p,(^,f_,A:_)6f.x_,

for all ;_5=f„,

;,-?. or, which is the same,

^ ^ ^

=/g(f,*)p,(f.,f,A:)JA:,

for all r > ^

This quantity exists because of assumption (4). In view of assumption (5), right derivative ^e(f+0)/<%f is continuous on [fg,oo). This and (1.4.30) imply (2.2.1). (Note that assumption (5) is due to the well-known conditions (e.g., Section 1.5 of Vladimirov, 1984) sufficient for a parameter-dependent integral to be continuous function of the parameter.) Since the right derivative is continuous, the left derivative also exists on (fo,°°) and both derivatives are equal, i.e. ;fe(f-0)/Jf =f„. T h e n one has Je(f)/Jf=^e(f+0)/6?; for all f > ^ so (2.2.2) holds and eGC\[^,oo)). This completes the proof of the assertion of the lemma.

D.2

Proof of T h e o r e m 2.2

T h e proof considers the Markov stochastic process in Theorem 2.1 and L e m m a 1.3 (see assumption (1)) where f , ^ are arbitrary such that f^> f_> f and f is according to assumption (1). In so doing, Eq. (1.8.3) in as-

233

Proo/* of Theorem 2.2

sumption (2) is applied to this process. Multiplying (1.8.3) by p,(f^,;_,*_) where ^ > f _ > ^ , taking integrals of both sides of the resulting equality with respect to X over R'', and accounting notation (2.2.3) as well as the uniformity in assumption (2), one gets lim

f )(x^X_)(.X-X_Vp(f_,.x_f

r)6?r p,(^^_,j;_)^.=77^),

for all ;_>;_.

(D.2.1)

In view of identity x^-x_=[j^-e(fj] + [e(fj-e(f_)]-[x_-e(; )], the integral in (D.2.1) can be presented as follows

j^ J\*,-*-)(*,-*-f p(f.,*-,f,,*J<*x, P.(fo^-^-)^=^+^-^3+^+^-^-^-^7+^'

forall^>^>^,

(D.2.2)

where -^i^f f[^t"^(^)][^*"^(^)]^P(^'^-<^<^)^t P.(^'^-'-^-)^ ' R" R"

-^2=/ /K*^(^)]P(^-^-.f,.^)^ P.(^f--^)^--[e(^)-e(^_)]^,

^3=/ /[^-^(^)]K ^ ( ^ ) f p ( ^ - ^ - . ^ . ^ ) ^ P,(^.^.-^)^.,

A^^J-^JJMQ-^Jf^j'p^-^-.^-^)^ P,(^,^,^_)^_, /,= [e(^)-e(f_)]j'[j'^_-e(r_)fp(f^_,^,^)^ p^(f^,^_,jc )^jc

^ = / /K-e(^-)]^--e(f-)fp(^-^-.f^^)^ P.(f.-f..^)^.. R^ R'

234

Proo/s of t/:e TAeorems tn CAatp^er 2

W e also note that identity Jt^- e(^) = (x^-.x_) + [%_- e(f_)] - [e(;J - e(^ )] enables ons to rewrite the expression for 7- as (D.2.3)

^=^7^6^5

where

7? = f f(%^-Jc_)p(^_,Jr_,^,^J^Jc+ [^_- e(f_)f p,(^,f_, %_)^x_.

(D.2.4)

It follows from (1.4.10) and (1.4.28) that /,= [e(;j-2(f_)][6(fj-2(;_)f.

(D.2.5)

Equalities 72=0,

7,=0

(D.2.6)

hold because of L e m m a 1.2 (see assumption (1)) and the fact that function e exists (see Theorem 2.1 in assumption (1)). In view of L e m m a 1.3 (see assumption (1)), one has (D.2.7) Combining (D.2.2), (D.2.3) and (D.2.5MD.2.7), w e get f

f(.X^-.X_)(.X,r.X_)Tp(f_,.x_,^,.xj6?.x^ P,(fc.;_..l-)^.X-

forall^>;_>^,

(D.2.8)

where variances H(^) and ^ _ ) exist because of assumption (3). Passing in (D.2.8) to the limit as ^^_ and taking into account (D.2.1) and differentiability of e on [^,°°) (see assumption (1) and Theorem 2.1), one obtains

Jf

where (see (D.2.4))

8

8

A/

(D.2.9)

Green's FormH/a /or ^Ae Operator o/'Ko^mogorou's E^Ma^OK

235

^7

7. = lim

lim

f f(^^^.)p(^^-.^"^J^* [jc_- e(^_)]^p,(^,f_,^_)^_.

In view of (1.8.3) in assumptions (2) and (4), this equality is rewritten as /g =Jyg(f-,*J[*--e(f-)fp.(fo,'-,*J^- for all f_>f„ or 7g = [M(;_)]T,

forallf_>f„,

(D.2.10)

because of (1.6.11) and (2.2.4), so matrix (D.2.10) exists for all ;>;„. Assumptions (1) and (4) of the present theorem and assumption (5) of Theorem 2.1 show that function M determined with (2.2.4) is continuous, i.e.MeC([^,M)). Matrix 77^,(f) also exist for f > ^ and is continuous,i.e. 77 EC°([f ,oo))in view of (2.2.3) and assumption (5). (Note that assumptions (4) and (5) are due to the well-known conditions (e.g., Section 1.5 of Vladimirov, 1984) sufficient for a parameter-dependent integral to be continuous function of the parameter.) Equations (2.2.5) and (2.2.6) stem from property (1.4.30) applied to (2.2.3) and (2.2.4) respectively. Thus, assertion (1) is proven. Let us prove assertion (2). Substitution of (D.2.10) into (D.2.9) leads to ^

^

= M(;) + [Af(;)f+/^(f),

for all f ^ .

(D.2.11)

The continuity of/V,, Af and expression (D.2.11) point out that right derivative ;fV(;+0)/^f is continuous on [fo,°°). This implies (2.2.7). Since the right derivative of )^ is continuous, the corresponding left derivative also exists on (^,°o) and both derivatives are equal, i.e. JM(f-0)/Jf = JH[f+0)/^ for all ;>f^. Hence, one has
D.3

Green's F o r m u l a for the Differential Operator of Kolmogorov's B a c k w a r d Equation

The well-known Green's formula (e.g., pp. 235-236 in Courant and Hilbert,

236

Proo/s o/^ ^Ae TVteorems :n CAap^er 2

1962) can be used for operator X(f,jt) (see (1.10.2)). In so doing, the formula can be represented with the help of terms (1.12.6), (1.12.7) associated with operator ]X(f,.x)]* (see (1.12.1), (1.12.5)). W e give this representation below. Let the following assumptions hold. * *

*

Arbitrary scalar real function (p of (f,x)e7xR^ is such that
Then the well-known identity (e.g., p. 236 in Courant and Hilbert, 1962; (4.21) in Chapter 6 of Friedman, 1975) is written as ijr(f,;r){jqf,*)tp(;,*)} - (p(;,jr){[X(f,*)]'iK;,*)} =\fN(;,x), for all (f, x) e /x R'',

(D.3.1)

where N(r,jc) = (p(r,jc)[S(f,^)^(4^)]+(l/2)^(^x)^,A:)^(p(f^), for all (f,x) e /x R^,

(D.3.2)

and S(f,.x) acts on ijt(f,x) as described with (1.12.2). Integrating both sides of (D.3.1) with respect to .x over Q , one gets

n

n for all fSJ.

(D.3.3)

T h e right-hand side of (D.3.3) can be transformed into the integral over boundary 3 Q . Namely, by virtue of Ostrogradski's formula for a domain in ^-dimensional space (Ostrogradski, 1838) where ^ need not be equal to, or less than, 3 like in the well-known Gauss formula, w e have fvJ'N(f,jc)^= f [v(;c)f N(f,x) J&)(x), n an for all ; e / ,

(D.3.4)

where M(x) is an element of 8 0 and v(x) is the unit outward vector nor-

237

Proof o/ Theorem 2.3

m a l to 3 Q at point j;edQ. T h e n (D.3.3) is equivalent to j*{i],(;,.x) {^,jc)(p(^^)} - (p(f,jc) {[^(f,^)]*i);(f^)}}Jjr

= f[vMfN(;,jc)^M(^), an for all r e J.

(D.3.5)

Equality (D.3.5) is Green's formula for operator X(f,jt) on domain Q . It implies f^(t,^){^(f,^)(p(^,A:)}^ = f(p(f,^){[^,A:)]*i)y(f,^)}^, n n for all r e / ,

(D.3.6)

provided that [v(*)fN(;,*)],^ = 0.

(D.3.7)

O n e can also note that operator [X(f,A;)]* which is by definition (1.12.1) form a l adjoint of operator X(f,x) is adjoint of ^(f,.y) if and only if (D.3.6) holds. If H =R^, then the corresponding version of the above statement on the connection between (D.3.6) and (D.3.7) can be formulated as follows. If lim N(;,.x) = 0, uniformly with respect to f e / ,

(D.3.8)

then f^(r,A:){^,^)(p(f,^)}^ = f(p(;,^){[^,^)]*i)y(r,A;)}Jjc, R**

R-*

for all re/.

(D.3.9)

This is the fact which is applied in Theorem 2.3 (see Section 2.3.2).

D.4

Proof of Theorem 2.3

The theorem is based on Theorem 2.1 which can be applied due to assumption (1).

238

Proo/s o/ ^Ae TVteorems m Chapter 2

Assumptions (3) and (2) show that functions g^(f,') and p,(f ,r, ) have the s a m e smoothness as functions tp and i]j in Appendix D.3. So the corresponding results can be applied if tp(;,x)=g.(f,.x) and i];(;,x) = p^(f ,^,jc) for every A; = 1,2,...,^. In this case, notation (D.3.2) and assumption (4) demonstrate that (D.3.8) holds uniformly in ; > ; and hence (D.3.9) is also valid for all f^f , i.e. j*g(r,^){[^,jc)]*p,(^,f,^)}Jj; =j*p,(^,^,j;){^(r,^)g(^%)}^, R**

for all f>f„,

R^

or, in view of assumption (2) of this theorem and assertion (2) of Theorem 1.2,

Jg(f,x)

^

L j ^ ^ p,(;„,;,x){X(f,.x)g(f,*)};F*,

forallf>f.

(D.4.1)

O n e has

J

-

^ =J ^

^ P*(^o^-^) ^ + J g(f.^)

-

3^

for all ; > ;

^. (D.4.2)

because of assumptions (1) and (5) of the present theorem. B y virtue of assumption (6), the integral on the left-hand side of (D.4.2) converges uniformly in f>f . This in view of the well-known fact (e.g., Section 1.5 of Vladimirov, 1984) also used in Appendixes D.l and D.2 m e a n s that

Jg(f,*)p,(;„,;,F)jJ = / ^ ^ P X ' o - ' . * ) ^ + Jg(;,*) ^*('°'''*) <^, 3; for all f>f or, on the strength of (2.2.2) (see assumption (1)) and (D.4.1), J ^

= f[^^1

+ x(,,*)g(;,J p,(f.,;,*) ^.

(D.4.3)

Since the integrand in (D.4.3) is continuous function of f and the integral in (D.4.3) converges uniformly with respect to f^f„, the left-hand side of

Qucts:-.ZVeMfraZ Eqw^brtHfn P o m ^ o/ ^Ae Second-Orcfer Rys^em

239

(D.4.3) is also continuous with respect to ;> ^. Hence assertion (1) is valid. In view of (1.10.2) and assumption (3), w e have

^(;.F)&(;.F) = ^ ^(r,^) - ^ — - + ^ E E ^ . ( ^ ) ^ — ^ - for all ^ = 1,2,...,^ and (;,x)E/x]^. Then, owing to notations (2.3.6) and (2.3.7), expression (D.4.3) is equivalent to (2.3.8). Thus, assertion (2) holds.

D.5

Quasi-Neutral Equilibrium Point

This appendix deals with autonomous versions of O D E systems (A.l) and (2.3.11) independent of ;, i.e. with systems

§ = g(*).

(D.5.1)

^=g(*),

(D.5.2)

where (see (2.3.10))

gM = ^ ^ g M + ;,(*)

(DS3)

dx and A(x) is the f -independent version of vector (2.3.7), i.e. A(*) = (A^),A,M, ,/;^))T, A ^ ) = (l/2)tr[77(jc)d^(^)/^2^

^ = 1,2,...,^.

A n y solution of equation g(x) = 0 is called equilibrium point of O D E (D.5.1). A n y solution of equation g(*) = 0,

(D.5.4)

is called equilibrium point of O D E (D.5.2). Equilibrium point, say, ^ of (D.5.2) is called regular if and only if

240

Proo/s o/ ^ e TAeore^s :n CAap^er 2

det

Mi) ^0.

(D.5.5)

3* W e call a regular equilibrium point % of (D.5.2) neutral (with respect to function A) if and only if A(x) = 0.

(D.5.6)

If .x is a neutral equilibrium point of O D E (D.5.2), then it follows from (D.5.3)-(D.5.6) that x is not affected by the second term on the right-hand side of (D.5.3) and is also an equilibrium point of O D E (D.5.1). In m a n y applied problems, equality (D.5.6) need not hold precisely. So w e consider its weaker version. W e call a regular equilibrium point x of O D E (D.5.2) quasi-neutral (with respect to function A ) if and only if

number

Mi)

*(*) is m u c h less than

8x the typical values of H g(x) H for all % close to x.

(D.5.7)

If x is a quasi-neutral equilibrium point of O D E (D.5.2), then it follows from (D.5.3)-(D.5.5) and (D.5.7) that x is close to an equilibrium point of O D E (D.5.1) in the sense of relation g(x) = 0 that, in view of identity

Jx

77

= 0,

(D.5.8)

implies ^ x / ^ = g(x). For all x close to x, the latter relation leads to 6?X

J? =

3M,

for all x — x.

(D.5.9)

B y virtue of Eq. (D.5.3), one also obtains

^)=^W

Mi) 3x

A(x)

3A(x) (x-x), 3x for all x close to x,

or, due to (D.5.7) and (D.5.9),

241

Proof of T/teorem 2.4

3%

Jf

3x for all x close to x.

(D.5.10)

Combining (D.5.2), (D.5.8) and (D.5.10) and replacing the signs " = " with sign " = ", one readily obtains <^(*-.*) J^

3g(x)
-,

for all x close to x.

(D.5.11)

W e call quasi-neutral equilibrium points of O D E (D.5.2) asymptotically stable quasi-neutral one if and only if

det

A2j_3g(jr)^_3/:(.t)

3x

0

implies

R e 1 < 0.

3x

Obviously, linear O D E (D.5.11) with constant coefficients is asymptotically stable in the large if and only if quasi-neutral equilibrium point jt of O D E (D.5.2) is asymptotically stable. The above considerations m e a n that, in a sufficiently small neighborhood of asymptotically stable quasi-neutral equilibrium point x of nonlinear O D E system (D.5.2), this system can approximately be replaced with asymptotically stable linear O D E system (D.5.11).

D.6

Proof of T h e o r e m 2.4

Assertion (1) stems from assumptions (1) and (2) of this theorem and assertion (2) of L e m m a 2.1. It follows from (2.5.4), assumption (1), Eq. (2.5.2), and assumptions (3) and (4) that

forallf>;„, and function R exists for all f>f . Since

(D.6.1)

242

Proo/s o/ ^Ae Theorems tn CAap^er 2

g(^,^)-g(f,e(r))=D(^A:)[%-e(f)], forall(^,Jc)e[^,oo)xR^,

(D.6.2)

because of assumption (1) and (2.5.1), expression (2.2.4) is rewritten as Af(^) = f[^-e(r)][A;-e^)]^[D(4A:)]^p,(^^,Jc)^,

for all ?>;,,.

(D.6.3)

Substituting (D.6.3) into (D.6.1), one obtains (2.5.5) that proves assertion (2). Equivalence of (2.5.6) and (2.5.7) stems from assertion (2). If (2.5.6) holds, then (2.5.7) also holds for f > ; and hence expression (2.2.8) is transformed into O D E system (2.5.8). In view of continuity of D^ (see assertion (1)) and continuity of N , (see assertion (1) of Theorem 2.2), initial-value problem (2.5.8), (1.6.14) (see (1.6.8) for K J has the unique solution defined for all f>f . O n e can readily check that this solution is given with Eq. (2.5.9). This completes the proof of assertion (3). Assertion (4) follows from equality (2.5.9), the property of matrix P^ to be non-negative definite (see (1.6.8)), nonsingularity of the Cauchy matrix C'(f.f^), and assumption (5).

Appendix E

Proofs of the Theorems in Chapter 4

E. 1

Proof of L e m m a 4.1

In view of assumption (1) of this lemma, assumption (2) and assertion (1) of Theorem 4.1 and Eq. (4.5.1), equality (4.5.3) is valid and function (4.5.1) exists for all A > 0 and all ^ . It also exists at A = 0. Indeed, applying (1.5.7) to (4.5.1) and accounting (4.5.4), one obtains assertion (1). B y virtue of assumption (2) and equalities (4.5.1), (4.5.2), assertion (2) follows from the well-known fact (e.g., p. 44 of Arnold, 1974; Proposition 5 on pp. 117-118 of Soize, 1994).

E.2

Proof of T h e o r e m 4.2

Let us consider integral jg^u(A,^J[g(jcJ]^Jjc^ for all A > 0 . O n e obviously has 7sT(^ )(^-e)=g(j; ) because of (1.10.5). Hence the integral is written as

^(A,^)[g(^)]^=j-e(A,^)[^(^j(^-^)]T^^, R'

R**

for all A > 0 .

(E.2.1)

To m a k e the next step, w e need to involve assumption (1). Relations (4.5.6), (1.12.15) and the time-independent version of Eq. (D.3.2) at x=x^ and
243

244

Proo/s o/ ^Ae Theorems :n C^ap^er 4

be applied to the present case. A s a result, one obtains

for all A > 0 .

(E.2.2)

Relation (E.2.2), assumption (2) of this theorem, and assertion (2) of L e m m a 4.1 show that Eq. (E.2.1) can be rewritten as j*^(A,^)[g(^)f^ = j* ^ * ^ R**

(*,-
for all A > 0 .

R-'

O n the other hand, assumption (3) and (4.5.4) point out that f3(A,*J(^-eyjA:J = f — L z i i A ^ - g j T ^ ,

^

^

^ n.

3A Then by virtue of assumption (3), Eq. (4.5.2) and existence of matrix (4.3.2) (see assertion (2) of Theorem 4.1), w e have ^ ^

=j*9(A,xJ[g(j<:J]T^,

for all A > 0,

(E.2.3)

R"

where the right-hand side is continuous in A > 0 . Hence assertion (1) is valid. Expression (E.2.3) can be rewritten as (4.5.7), (4.5.8) because of (4.5.1), (4.5.2), and L e m m a 1.6 involved by assumption (2) and assumption (1) of L e m m a 4.1. Equality (4.5.9) follows from Eqs. (4.5.7) and (4.3.4). Hence assertion (2) is valid.

E.3

Proof of T h e o r e m 4.3

B y virtue of assumption (1) of this theorem and assertions (3) and (4) of L e m m a 4.2, property (4.6.5) is valid. This and (1.7.20), (1.7.27) imply (4.6.11). The second of equalities (4.6.11) and limit relation (4.3.5) entail (4.6.12). Equations (4.6.9) and (4.6.2) are the time-independent versions of (2.6.2) and (2.6.3) respectively. Besides, condition (4.6.4) prescribed by assumption

245

Proo/'o/TAeorent 4.4

(2) is similar to condition (2.5.6) of Theorem 2.4 leading to matrix O D E (2.5.8) for the variance. The time-independent version of this O D E is (4.6.13). Relations (4.6.8) and (4.6.9) in assumption (3) as well as (1.7.12), (1.7.13) point out that non-negative definite matrix J7^ is positive definite. This completes the proof of assertion (1). Assertion (2) directly stems from (4.6.11) and assumption (4). Assertion (3) follows from the above property of N,, assumption (4), Eq. (4.6.13) and presents the stationary version of expression (2.5.9) in Theorem 2.4 (see Section 2.5.2).

E.4

Proof of T h e o r e m 4.4

O n the strength of assumptions (1) and (2), solution ijy(.x_,A) of initial-value problem (4.9.3), (4.9.4) has the property pointed out in assertion (1). Let us prove assertion (2). It follows from (1.7.12) that

llj'(^-eJ[^_,A)-ey^^)^ Lj-ii^gjiii^^^gjip^^)^ . (E.4.1) IIR^ It R** In view of assumption (4), one has ll^-6jlll^.,A)-eJlp^(^_)<e.ll^-eJpp^(^_), forallA>0,*_.

(E.4.2)

Expression (1.7.18) prescribed by assumption (3) points out that Jll*_-e,ll2p,(*_)^_=tr^.

(E.4.3)

Equations (E.4.3), (E.4.2) and (4.9.6) means that the integrand in (4.9.5) is majorized (see also (E.4.1)) with the A-independent quantity of a finite integral over R^ (see (E.4.3)). The integrand in (4.9.5) is continuous in A > 0 due to assertion (1). Hence, function C ^ has property C^SC°([0,oo)). Equality

R**

R**

246

Proo/s o/ ^Ae r/:eorg?HS :M CAap^er 4

holds since solution ij;(.x ,A) satisfies initial condition (4.9.4) (see assumption (1)). In view of this and (1.7.18), Eq. (4.9.8) is valid. Estimation (4.9.7) follows from (4.9.5) and (E.4.1)-(E.4.3). This proves assertion (2). Property C^eC^([0,oo)) pointed out in assertion (3) follows from the uniform convergence and the continuity in A > 0 of the integrand in (4.9.9) and hence from the facts that <^(A) tT^(A),

for all A > 0 ,

(E.4.4)

JA

and !7a<=C°([0,°°)). If condition (4.9.10) holds, then (E.4.4) can obviously be rewritten as (4.9.11). Equation (4.9.12) stems from the continuity of &^, (see above), (4.9.11) at A = 0 and (4.3.2). Equation (4.9.13) follows from (4.9.8), (4.9.12), assumption (5), and (4.3.1). This completes the proof of assertion (3).

Appendix F

Hidden Randomness in Nonrandom Equation for the Particle Concentration of Uniform Fluid and Chemical-Reaction /Generation-Recombination Noise

The most popular model for concentration H of the particles of uniform fluid which is located in some time-independent domain, say, Q (described in the text above (1.2.13)) is deterministic equation

—=jr,

forallfe[;,°°),zeQ, (F.l)

with initial condition "L=,= ".(2).

^ r all z e Q ,

(F.2)

where n = n(f,z). Note that domain Q m a y coincide with space R** or be bounded, or unbounded but not coinciding with R**. Quantity ,Y in (F.l) usually depends on time f, point z, concentration n and its various z -derivatives. O n e of the phenomena often accounted in equation (F.l) is diffusion of the particle positions in domain Q . So, in m a n y cases, this equation is briefly called diffusion. T e r m ,y generally includes the rate of the concentration change due to the chemical reactions (similar to term r in Eqs. (6.3.2) and (6.3.3)) in the fluid. In the electron-hole fluid in semiconductors, this rate is associated with recombination and generation of the particles. T e r m Af can, depending 247

248

HYtMen .Randomness m ^Ae Co?:cenfra%:oM Eq'Ma^ion and Nbtse

on a specific problem, be of even more complicated form. For example, it can include integrals of (linear or nonlinear) functions of f, z, w and the z -derivatives of n over domain Q . However, as a rule, quantity A" does not include any time-derivatives of w. This feature is assumed to be valid in the present appendix. In so doing, the specific forms of dependences of ^f on its variables is unimportant. The appendix is focused on derivative dn/df. Concentration n is determined from (F.l), (F.2) under the additional conditions which are inherent in notion of concentration, namely, n(f,z)>0,

fora!!fe[;<,,oo),z€Q,

0 < 7V(f) < oo,

for all fS [;„,oo),

(F.3) (F.4)

where JV(f)=fw(;,z)Jz, n

for all re[^,oo).

(F.5)

Because of definition of concentration, quantity (F.5) (cf., (IV.3) in Resibois and D e Leener, 1977) is the number of particles in domain Q . If necessary, the corresponding conditions for w at boundary 3Q of Q are also applied. The above model does not include any random terms. Subsequently, it m a y seem that there is no randomness in this model. The latter often serves as the motivation to add one or another stochastic process to the righthand side of Eq. (F.l) as a noise source to endow (F.l) with randomness (e.g., (4.2) in V a n Vhet, 1971b; (96) in V a n Vliet, 1994). In m a n y cases, this is done in a heuristic way, i.e. with neither probabilistic analysis of Eq. (F.l) nor connection of the noise source to the classical statistical-mechanics results. This noticeably hampers a proper justification and adequate interpretation of the above heuristic incorporation. The present appendix discusses hidden randomness of nonrandom model (F.1)-(F.5) and thereby stresses the features which are not very widely employed in its applications. Let us introduce function p, with equality p,(;,z)=M^l, 7v(f)

forallfe[?,,o.),zEQ.

(F.6)

forall;e[f„,oo),zeO,

(F.7)

It follows from (F.3MF.6) that p.(;,z)>0,

N:dden Randomness tn (Ae Concentra^on Eoua(to?t and Notse

249

fp.(r,z)Jz=l, forallfe[f„,°o). (F.8) n Equations (F.7) and (F.8) m e a n that function p. is probability density of position z of a particle in domain Q . Thus, deterministic Eq. (F.l) does take into account the stochastic nature of the particle trajectories. This fact points out the probabilistic meaning of (F.l). Density p. enables one to carry out probabilistic analysis of the trajectories. Note that the particle-position probability density is a fairly deep stochastic characteristics of the fluid. Concentration n(f,z) described with the above deterministic model delivers both number of particles (F.5) and probability density (F.6). These equations show that the corresponding quantities are naturally affected by every phenomenon presented on the righthand side .Y of (F.l). In particular, if the chemical-reaction/recombinationgeneration ( C R / G R ) term is included, both number JV of the particles and the particle-position probability density p. are determined by this term and depend on it. Thus, Eq. (F.l) naturally allows for the noise due to chemical reaction in the fluid or recombination and generation of its particles. H o w complete and comprehensive is such accounting? To get the answer, one needs first to formulate which specific fluctuation phenomena are beyond the above model and why. To sharpen the above probabilistic picture, w e transform deterministic Eq. (F.l) to its equivalent form which explicitly describes JV and p.. Equation (F.6) can be rewritten as

M(;,z)=AT(;)p.(;,z), forall;e[f„,oo),zeQ. (F.9) This in particular points out that the time-evolution of concentration n m o delled by (F.l) is in fact a compound phenomenon which includes evolution of both TV and p.. W e assume function JV (see (F.5)) to be differentiable. Then, applying operation 3/df to both sides of Eq. (F.9), one gets

fora!lfE[^,°o),z(EQ. To evaluate derivative ^7V(r)/^r, w e assume that property

(F.10)

250

H:cMen Randomness m ine Concenfrafton Eoua^ton and JVotse

—

fn(f,z)Jz = f - ^ ^ J z ,

for all ^€[r ,°o),

(F.ll)

holds. (Note that Eq. (F.ll) takes into account the fact that domain Q is time-independent.) Equations (F.l), (F.5) and (F.9) transform (F.ll) into

forallfe[^,oo).

(F.12)

Substitution of (F.l), (F.6) and (F.12) into (F.10) leads to dp,(f,z)

i

P.(f,z) ^.

for all ;e[;„,<*,), z e O .

(F.13)

Thus, under conditions (F.ll), evolution equation (F.l) is equivalent to evolution equation system (F.12), (F.13). The corresponding initial conditions for JV(f) and p.(f,z) are obtained with the help of (F.2), (F.5) and (F.6) in the following form

(F.14)

JV(f.) = Af„ p.M=p...(z),

for all z e Q ,

(F.15)

where

(F.16) a "n(z)

for a l l z e Q .

(F.17)

In so doing, concentration w is determined on the basis of solution JV(f), p.(f,z) of (F.12), (F.13) according to (F.9). If the boundary conditions for concentration w are applied to Eq. (F.l), they after the substitution of (F.9) should also be applied in connection with the equation system (F.12), (F.13). System (F.12), (F.13) which is equivalent to Eq. (F.l) explicitly demonstrate the stochastic meaning of the latter by means of the particle-position

HtcMen Randowtftess tn ^Ae Co^cen^ra(:on Ei?Maf:on and JVo:se

251

probability density p.. Equation (F.l) m a y be more convenient in practice. However, system (F.12), (F.13) is more preferable qualitatively. Its important advantage is that it describes the interplay of probabilistic quantity p.(;,z) and physical quantity JV(f), and inherently takes into account all the phenomena represented on the right-hand side X of Eq. (F.l). The coupling of p.(?'2) related to a single particle and number JV(f) of particles results from the multiparticle nature of the fluid. The level of, say, sophistication of the stochastic modelling provided by system (F.12), (F.13) is sharpened by the following issues. *

This system is generally nonlinear. It is also integro-differential even if term X includes no integrals. T h e integral in (F.12) emphasizes non-locality of evolution of p.. * It is unclear which particle-position S D E underlies system (F.12), (F.13), i.e. has the solution with probability density p. described by this system. A certain family of partial integro-differential equations for probability densities of the S D E solutions is well-known in S D E theory (e.g., Corollary 2 on p. 299 in G i h m a n and Skorohod, 1972). To what extent they are related to system (F.12), (F.13) is not discussed in the present book since the above solutions are not D S P s . The corresponding research, if started, should pay attention to several features not accounted in the c o m m o n theory. To mention a few, w e point out the nonlinearity of A^ in n, coupling of JV and p. and the fact that domain Q m a y coincide with space R^ or be bounded, or unbounded but not coinciding with R^. *

If A" is linear and homogeneous in H, then Eq. (F. 13) is homogeneous in p. and independent of TV, Eq. (F.12) is linear and homogeneous in TV and, under initial condition (F.14), has the unique solution

^ ) =^expjj-[^]^^^]^L

for all r e [ ^ ) .

If, in addition, term ,Y includes the quantities related to the particleposition diffusion, the mentioned independence of TV is perhaps the most important difference between the linear and nonlinear diffusions of the fluid particles. The integral in (F.12) or (F.13) is zero if and only if the n u m b e r of the particles is time-independent, i.e. J7V(f)/J;=0 for all fE[^,oo).

252

H M d e n Randomness tn ^ne Concenfra?M)n E^MatMn and JVotse

*

If Jif is linear and homogeneous in M and the integral in (F.12) or (F.13) is zero, then Eq. (F.13) becomes linear and homogeneous in p. (see also (F.14)), 3p.(?,z)

for all f€[;„,<*,), z € G ,

(F.18)

and thereby is simplified to such level that makes its comparison with the c o m m o n probability-density equations reasonable. Indeed, not too complex versions of linear equation (F.18) present the equations which can be compared to K F E (1.12.4). The above features of probabilistic form (F.12), (F.13) of nonrandom equation (F.l) point out that this equation describes very deep and complex stochastic characteristics of uniform fluid. It allows for the stochastic phen o m e n a included in its right-hand side .Y by means of representation (F.9). O n e of them is the C R / G R noise. In spite of that, m a n y research works dealing with the C R / G R fluctuations are focused on incorporation of one or another stochastic noise source for concentration M directly into Eq. (F.l), with no regard of what stochastic phenomena are already hidden in it. In other words, the hidden randomness revealed by means of (F.12), (F.13) is merely ignored. Motivation, justification and meaning of this approach remains the topic for future discussion. Equation (F.l) can be derived in various and very different ways, for instance, by means of the continuum-fluid-mechanics techniques (e.g., Sedov, 1971-1972) or statistical-mechanics formalism (e.g., Resibois and D e Leener, 1977; Klimontovich, 1982). Generally, these derivations could help to specify and sharpen relation of the equation stochastic properties to the application-specific interpretations. However, to our knowledge, there is still no any universal, rigorous and closed derivation. A n y available one is not of this kind. Indeed, the continuum-mechanics derivations are usually not closed: they do not determine some parameters of Eq. (F.l). The statisticalmechanics derivations are not universal either. They are based on the inflexible formalism of limited analytical capabilities which, apart from this, is associated with still unovercome difficulties with high-density fluids and multiparticle phenomena [e.g., see Bellomo, Palczewski and Toscani (1988)

HtcMen Rctftdomness :n ^Ae Concen^ra^toK Equafton a^c! 7Vb:se

253

and Bellomo, Lachowicz, Palczewski and Toscani (1991) for details]. The hst of the problems can be continued. To be specific, w e mention the following example. A s follows from the well-known relations (e.g., (IV.2), (IV.3) in Resibois and D e Leener, 1977), any solution of the Boltzmann kinetic equation describe nothing but: * *

nonrandom number (F.5) of the fluid particles and the probability density of the particle position-velocity stochastic process (similar to the density in (1.4.27), (1.4.28) if vector .x includes both the position and velocity vectors); in so doing, the position probability density is precisely the same as (F.6).

O n the other hand, a rigorous and comprehensive stochastic reading of this equation has not been discussed in the literature yet. (The corresponding schematic outlines are, however, available (e.g., Section 8.5 of Gardiner, 1994).) T h e lack of the above reading leaves a lot of questions on the Boltzm a n n equation unclarified. In spite of this, some applied fields (for instance, the semiconductor modelling by Lundstrom, 1992), consider the equation as the universal description and strongly advocate it for practical use with no stochastic criticism. This inevitably limits proper understanding of its features important in the real-world problems. The previous paragraph points out one more drawback of the statisticalmechanics treatment: the lack of sharp considerations on rigorous connection to mathematical theory of stochastic processes, random fields and S D E s . A n interesting attempt to resolve the latter problem is Lagrangian probability-density-function method by Pope (1985) (see also Pope, 1994). It is the SDE-related alternative to a part of statistical fluid mechanics. The above issues show that, in the application-specific stochastic reading of equation (F.l), there is not too m u c h hope to strongly rely on its derivations. S o m e prompts can, however, be found on this way. For example, according to the statistical-mechanics treatment, the particle-position probability density p. is evaluated as the marginal density resulting from the averaging of a more general (dependent not only on the time-position vector) probability density with respect to the particle velocity (see (rV.2) in Resibois and D e Leener (1977) or (7.1) in Klimontovich (1982)) or the particle m o m e n t u m and perhaps other variables (e.g., similar to (3.80) in Pope (1985)). This m e a n s that the hidden randomness in Eq. (F.13) (or (F.l)) is affected by the zero-expectation component of the veloci-

254

# M M e n Ranc^OTHHess tn ^ e Conce^^ra^to?: Eq'Ma^toH anc^ JVotse

ty only indirectly and to a very limited extent. Thus, if a stochastic noise source is incorporated into equation (F.l), this is to be done to endow it with such stochastic phenomena which are not well presented in it. Hence, there is no ground to deduce the above source from the particle-position fluctuations or inherently associated with them fluctuations of n or N . Instead, a reasonable candidate for the noise source is a stochastic process or random field based on the zero-expectation, purely random component of the fluctuating velocity. Its application can, so to say, return Eq. (F.l) the stochastic features removed from it by the above averaging in the velocity. The velocity-based interpretation of the source underlies Chapter 6.

Appendix G

Example: Eigenvalues and Eigenfunctions of the Linear Differential Operator Associated with a Bounded Domain in Three-Dimensional Space

This appendix considers the following example of linear differential operator ^4 in (6.2.12): ,4Z = - v ^ Z ,

(G.l)

[v(z)fZ]^=0

(G.2)

where z, Z6E R \ ^^) is the same as in (D.3.4), i.e. the unit outward vector normal to boundary 3Q of domain Q at point z € 3 Q and domain Q is a rectangular parallelepiped, Q = (0^Jx(0,^)x(0,^),

^>0,

A-1,2,3.

(G.3)

To study operator ^4, w e consider the Laplace differential operator g? acting on real scalar function V of z as follows ^Y—^V,

(G.4)

[ v ( z ) f V , y ] ^ = 0.

(G.5)

Properties of operator S! are well-known (e.g., Section 22 of Vladimirov, 1984). W e in particular note that it is a non-negative definite self-adjoint 255

256

E:gB;:ua/Hes and E:gen/Mnc?:ons o/' fAe Lmear Operator

operator in Hilbert space L^(fl). It follows from (G.l), (G.2), (G.4), (G.5) that ^a?y=^4\y.

(G.6)

This points out the following features: *

Eigenfunctions of operator ^4 are gradients of such eigenfunctions of operator g which have gradients nonzero in norm H - L 2,^, of L ^(H). * Eigenvalues of operator y4 are such eigenvalues of operator g which correspond to its eigenfunctions specified above. * The other main properties of operator ^4 are the same as the main properties of operator §E. * Operator ^4 is an operator in Hilbert space L ^((1).

Eigenvalues and orthogonal eigenfunctions of operator g! are wellknown (e.g., Problem 1 on p. 268 of Landau and Lifshitz, 1987). The eigenvalues are numbers

\*^= (" W'+(n W ' ^ W ' ^0, for all integer Ar^A^A^O.

(G.7)

The orthonormal eigenfunctions are listed below: (G.8)

^ooo(^) -^/(Fl^2^3) ' ^oo(z) ="\/2/(^1^2^3) c o s ( n ^ z ^ ) ,

^>0,

(G.9)

^ o ( z ) = -\/2/(^^2^3) C0S(X^Z2^2) .

%2>0,

(G.10)

^ooj^h "^2/(^^^) c o s ( ^ ^ ^ ) ,

^>0,

(G.ll)

5^0(2)= -\/4/(^^^)cOs(7t^Z^JcOs(TI^Z^2),

%i,%2>°. (G.12)

^ot/^)" -^/4/(p^^3)cos(7t^z^Jcos(7i:^^),

^ , ^ > 0 , (G.13)

^ O R ^ ^ ) "= - ^ 4 / ( ^ 2 ^ ) COS (^ ^^2^2) COS (71 ^ Z ^ ) , /Cy^>0, (G.14) ^A^^(^) = -^8/(^)^2^3) C0S(7T^Z^J C O S ( X ^ 2 ^ 2 ) COs(TtA:gZg^) ,

A ^ A ^ O . (G.15) The physical meaning of eigenvalues and eigenfunctions of operator g! is described, for example, in Section 22.5 of Vladimirov (1984). Let us consider orthonormal gradients e^.^. of eigenfunctions E^ ^ ( z ) ,

E:geMua/Mes and .Eigen/HncfioMS o/' ^/te LtHear Operator

257

i-^- % ^ ^ ^ ) = [34^^)^zJ/!!3^^(-)/dzJI^Z(^, *-1,2,3. If ^. = ^ = ^ = 0, then the corresponding eigenfunction is (G.8) and the corresponding eigenvalue is (see (G.7)) A ^ ^ O . Gradient of jE,^ is zero vector and hence cannot be normalized. Thus, the normalized gradient does not exist if A. = ^ = ^ = 0. If ^,%yA; > 0 and A^+A^+A^X), then the corresponding eigenfunctions are (G.9)-(G.15) and the corresponding eigenvalues are (see (G.7))

for all integer ^ , % y ^ 5 0: A^+A^ A^> 0.

(G. 16)

These numbers are eigenvalues of operator ^4 (see (G.l), (G.2)). Its orthonormal eigenfunctions are the orthonormal gradients of eigenfunctions (G.9)-(G.15), i.e. %oo.i(2) = 72/(^2^3) sim(Tt^z^J,

^>0,

(G.171

%00.2(2) = 0.

A^>0,

(G.17")

%00.3^)^,

^>0,

(G.17n

^o.i(^) = 0.

(G.181

^0.2^) = \/2/(p^2^) sim ( m ^ ^ ) -

(G.181

^ . 3 ( 2 ) = ".

^>0,

(G.18"')

^.i(^) = 0,

^>0,

(G.191

^ . 2 ^ ) = 0.

^>0,

(G.19") (G.19"1

^ . 3 ^ ) = -/2/(p^2^) sim ( n ^ z ^ J, %^0.l(^) = ^ / ( P i ^ p A ^ X x ^ i ) Sim (71/C^ -1^1)^cos(n^z^),

^,^>0,

(G.201

%^0.2^) = ^/(p^^3\^o)(^^2^2) COS(x^Z^J sin(Tt^Z^). ^,^>0, %4o.3(^) = 0.

^,^>0.

(G.20") (G.20"')

258

Etgenuc^Mes and EtgeH/Hncttons o/ f/te Lt^ear Operator

%o^.i(^) = ^ 4 / ( ^ 2 ^ A ^ ) ( n ^ / p i ) sin(7t^z^ J cos (Ti^z^/p^),

#,^(z) = 0,

^,^>0,

(G.210

^,^>0,

(G.21")

%o*,.3(z) = ^4/(^^^3\o^)("^/P3) cos(7t^z^^) sin(^^z^J,

#^^(z) = 0,

^,^>0.

(G.21"l

^,^>0,

(G.221

^0^4.2^) = / 4 / ( ^ y ^ A ^ ) ( 7 t ^ ^ J sin(n^z^2) cos(n^z^^), ^-^>0,

(G.22")

^ . 3 ^ ) = ^/(F^2^0^)("^3^3)cOs(^^Z2/p2)S!n(x^Z^g),

^,^>0,

(G.22"l

%^^.l^) = ^/(p^2^3A^^)(7t^^Jsi]l(Tt^Z^Jc<)s(TI^Z^JcOs(^^Z^/pJ,

^,^,^>0,

(G.231

^ . 2 ( 2 ) = ^ / ( p ^ ^ F g A ^ ^ ) (n^/^J cos(^t/c^z^/p J s i n ^ ^ ) cos(it ^ z ^ J, ^,^,^>0,

(G.23")

% ^ . 3 ^ ) = ^8/(p^P3\^)("^3^3)cOS(^^^j/Pi)cOS(lt^2^2)s!m(^^Z3/pJ ,

^,^,^>0.

(G.23"l

W e also note some auxiliary formulas which can be helpful in practice. Vectors (1,0,0)\ (0,1,0)^, (0,0,1)^ can be presented by means of eigenfunctions ^. ^. of operator yl (see (G.I.), (G.2)) in the following way (e.g., see (22.20) in Vladimirov, 1984)

(l,0,0f=

^

^,^,^

(z),

(G.241

(0,l,0f= E

^.,^^A,4.^)-

^'^^

(0,0,lf= E

<^,*,*&*,^)-

^'^^

E:ge?wa^Hes a n d Etgen/Mnciions of ^Ae L m e a r Operator

259

where ^.^^^^.^.i^)^-

^ y ^ O .

^+^+^>0,

(G.25')

C2.^.^=j*%,^.2^)^' n

^ ' ^ 0 . ^+^+^>0,

(G.25")

^.^.^=f%^.3^^'

^ . ^ 0 , ^+^+^>o.

(G.25n

It follows from (G.3) and (G.17MG.23) that: *

the integral in (G.25') is non-zero (see those of (G.17)-(G.23) which are marked with " ' ") if and only if 4^ is odd that is A^=2% +1, A:> 0, and ^=A;-=0, * the integral in (G.25") is non-zero (see those of (G.17)-(G.23) which are marked with""") if and only if A^ is odd that is ^=2^+1, /r>0, and Ar^A^O, * the integral in (G.25"') is non-zero (see those of (G.17MG.23) which are marked with " "' ") if and only if ^ is odd that is ^ = 2^ +1, ^> 0, and ^=^2=0. Thus, Eqs. (G.24) are reduced to (1<0,U)

(G.26')

2 ^ ^l.(2Rtl)00^(2* + l)Oo(^)' ^>0

(U,1,U) =2^^2.0(2Atl)0^0(2Atl)o(^)' :.0(2Atl)0^0(2Atl)o(&>0

(G.26")

(U,U,1) =2^^3.oo(2A:tl)^00(2A:4l)(^)'

(G.26'")

where the involved eigenfunctions are ^.i)oo^) = {V27(f^Jsin[(2/c + l)7Tz^J,0,0}'*', R > 0 ,

(G.27')

A;>0,

(G.27")

^oo(n.i)^) = {0-0.^^^^2^)sm[(2/c + l)7tz^J^, ^ > 0 ,

(G.27"')

and Eqs. (G.25) are reduced to ^i.(H,i)oo =/^.i)oo.i(^)^ = ^i^2^/2{4/[(2^ + l)n]}, A:>0,

(G.28')

260

EtgeHua^Mes and E:genfMr:c?tOHS o/' ^ e Ltnear Operator

^.o(2.,i)o =^0(2^,1)0.2^)^ = V^^2{4/[(2^+l)7t]},

^>0,

^3.oo(2.*i) = ^ , i ) . 3 ( ^ ) ^ = ^ i ^ ^ / 2 { 4 / [ ( 2 ^ + l)7:]},

(G.28")

^ ^ 0 . (G.28"1

T h e corresponding eigenvalues (see (G.16)) are written as A(2*,i)oo = [(2^1)n/Pif.

A o ^ „ = [(2^1)n/p2]2, A ^ ^ ^ [ ( 2 ^ + l)it^f, ^>0.

(G.29)

The above results can be used in both theoretical studies and practical simulation algorithms.

Appendix H

Resources for Engineering Parallel Computing under Windows 95

The term "parallel computing" means implementation of an algorithm simultaneously, or in parallel, on a computer system with more than one processor. This, of course, assumes that the algorithm consists of the parts which can efficiently be parallelized. The number of the utilized processors usually varies from a few units up to a few thousands. Since a multiprocessor system accumulates the computational power of the involved processors, the term "multiprocessor computer" is often regarded as a kind of synonym of what is called supercomputer. As a rule, supercomputers are highly complex hardware systems which are built by means of the technologies significantly more sophisticated and expensive than those common in the world of ordinary computers. This appendix is devoted to another family of supercomputers, the family of, say, hidden supercomputers, i.e. those which have their major components and can be built at almost every business or university office. The underlying idea is to integrate a few single-processor personal computers (PCs) based on Windows 95, the most common of the Windows operating systems (OSs), into a multiprocessor computing system by means of the standard computer network. One week consists of 168 hours. One working week is typically 40 hours. Allowing for the fact that people at an office not only use the PCs but also have other activities, the pure computer time per person within a working week can be assumed of, say, 25 hours (5 hours each of 5 working days).

261

262

Resources /or ParaHe^ Compu^tng Mno!er W:Ho!oH;s 95

This means that the office PCs are used for only about 15 percent of the time. Slightly less optimistic and possibly more accurate estimation is suggested by the Los Alamos National Laboratory in USA: the computers are in use less than 10 percent of the time. The parallel computing enables one to employ the potential power of the strongly underutilized office PCs. This idea is not new. A certain progress has been achieved to implement it. The distinguishing feature of this approach is that it is focused on a fairly affordable parallel-computing environments. Indeed, the office PCs are usually connected to a corporate or department network (that is usually arranged according to the common Ethernet specification prescribing the 10 megabytes-per-second speed). The point is to supply this hardware with a software system which enables the networked PCs to operate as a multiprocessor computer. In so doing, the individual computers, or the nodes of the network, communicate with each other by sending and receiving messages (i.e., sets of typed data) with the help of the network. This explains why the above approach is also called message-passing parallel computing. Note that the resulting multiprocessor machine may be regarded even as a supercomputer if the number and computational capabilities of the nodes as well as the network speed are sufficiently high and the messaging time is not very much deteriorated by the ordinary corporate/department network traffic. As an example of the message-passing-based multiprocessing software, one can point out the Parallel Virtual Machine (PVM) system (Geist e? aJ., 1994; Dongarra and Fisher, 1997). One of the most recent versions of P V M is version 3.4 (Dongarra e? at?., 1997). The self-extracting executable file (of about 6 megabytes) for P V M 3.4.1 (i.e., Version 3.4 Release 1) can be downloaded as a .ztp file from Scott (1999). This version under Windows 95 supports the following compilers: * * *

Microsoft's Visual C++ 5.0 and 6.0; Compaq's Visual F O R T R A N 5.0 and 6.0; W A T C O M C/C++ and F O R T R A N (with the help of the included definition files); * and, hopefully, other compilers which follow the common standards for the C, C++ and F O R T R A N programming languages.

The corresponding messaging speed between the nodes is about 5x10^ bytes per second (see Fig. 6, "TCP Over Ethernet" in Dongarra and Fisher, 1997). The P V M environment is very popular in research and educational

Resources /or ParctHeJ C o m p u ^ n ^ MHo!er Wt?%%otus 9 5

263

community, is free and comes with a free (and efficient) technical support. The systems which to some (however, rather limited) extent resembles P V M is Paradise (httpy/www.sca.com/PAR_overview.html), commercial software by Scientific Computing Associates (New Haven, CT, USA; http://www .sca.com). The main difference is that this software unites the networked PCs based on Windows 95 by means of the virtual-shared-memory (VSM) environment. In so doing, the message-passing is used to implement the shared-memory model in distributed computing, so the resulting multiprocessor computer can be viewed as a shared-memory supercomputer. The above product includes proprietary compilers for the C, C++ and F O R T R A N programming languages. It is successfully used in a number of applications. For instance, it is employed in financial simulation (http://www.sca.com /simulation.html) and in the electronics industry to speed-up design of the so-called printed circuit boards (http://www.sca.com/pnc.html). Among the free software tools targeted at a fairly broad range of specific engineering and scientific problems, we note Scilab//. The Scilab// project (httpy/www.ens-lyon.fr/-desprez/FILES/RESEARCH/SOFT/SCILAB/) is intended to bring parallel computing to Scilab (Gomez, 1999; see also "Scilab H o m e Page" at http://www-rocq. inria.fr/scilab/), a scientific software environment for numerical computations. Scilab resembles Matlab/Simulink and MatrixX/SystemBuild family of products. It is an Open Source Software and is free for academic and industrial use, without any restrictions. It can be included in a commercial package (provided that proper copyright notice is also included). The Scilab// version for Windows 95 is available at the above Scilab// site. This version will include P V M 3.4 (e.g., see http: //www .ens-lyon.fr/-desprez/FILES/RESEARCH/SOFT/SCILAB /more.html). The Windows 2000 OS, a successor of Windows 95 (as well as Windows 98 and Windows N T 4), provides more options and flexibility in organizing parallel computing than Windows 95. This is achieved mainly by means of the symmetric multiprocessing (SMP) support like in Windows N T (e.g., P h a m and Garg, 1996). However, the advantage comes for substantially increased prices for both the O S and the corresponding minimum hardware configuration. A n administration and maintenance of Windows 2000 is not simpler and easier either. This should be accounted by the users who are not specialists in operating systems (OSs) and computer hardware. Anyway, P V M 3.4.1 for Windows N T can currently be downloaded (Scott, 1999) and will hopefully be available for Windows 2000 as well. In connection with engineering parallel computing under Windows 95,

264

Resources /or ParaMeJ Compu^Mg under Wmdows 95

there is a group of issues on the Monte Carlo method. The analytical-numerical approach to high-dimensional DSPs summarized in Sections 3.6.3 and 4.11.2 suggests to evaluate the applied multifold integrals (like (3.6.3) -(3.6.5) or (4.11.5)-(4.11.10)) by means of the Monte Carlo method in parallel. This involves implementation of random-number generators on multiprocessor systems. The corresponding results and related problems were surveyed, for instance, by Deng, Chan and Yuan (1994) and Luscher (1994). In so doing, it was noted that the main problems are: *

to provide highly random sequence of the generated pseudorandom numbers; * to assure that the pseudorandom-number sequence generated by one processor is stochastically independent of the pseudorandom-number sequence generated by any other processor.

A n interesting solution of both the problems was developed by Luscher (1994) and implemented in F O R T R A N by James (1994) in the R A N L U X program. The acceleration of R A N L U X proposed by Hamilton and James (1997) made it possible to develop the F O R T R A N version for the real- and protected-mode D O S O S and the Windows 95 O S (the R A N L U X I program). A n even more efficient version (the R A N L U X A S M program) written in the Intel i386/387 assembly language and callable from the Lahey, Microsoft and W A T C O M F O R T R A N compilers was developed by Hamilton (1997). This software can run under D O S , Windows 95 and Windows N T OSs and is available through the Computer Physics Communications Program Library under the catalogue identifier A D F O at http://cps.cs.qub.ac.uk/cpc/ cgi-bin/list_summary.pl/?CatNumber=ADFO . Another aspect affecting the overall performance of the parallel software for computational science and engineering is a choice of the programming language. Interesting and helpful issues on this topic can be found, for instance, in Kale (1998). In many cases, the choice is in favor of the F O R T R A N language. The F O R T R A N programming for the Windows OSs is described by Ribar, 1993. The details on the modern F O R T R A N compilers are available at Polyhedron Software, Ltd., http:/^www.polyhedron .co.uk/.

Bibliography

Ahmed, N. U. (1992) "An existence theorem for stochastic nonlinear evolution equations on Banach spaces", <S(ocnasftc Af:a^. App/. 10, 379. Albeverio, S. (1996) "Stochastic partial differential equations, infinite dimensional stochastic processes and random fields: A short introduction", in Proc. o/ f Ae Euroco^/Brewce on ATbn/tnear JHe:n-Gordon otftc? Scnrddinger Systems; TAeory atnc! App/tcaftons, eds. L. Vazquez, L. Streit and V. M . Perez-Garcia (World Scientific, Singapore), pp. 68-86. Albeverio, S., H0egh-Krohn, R., Holden, H., Kolsrud, T. and Mebkhout, M . (1989) "A remark on the formation of crystals at zero temperature", in S(ocAasf :c Afe^oc!s tn Afa^Aenta^tcs aK(f Pnystcs, eds. R Gielerak and W Karwowski (World Scientific, Teaneck, NJ, U S A ) , pp. 211-220. Alder, B. J. and Wainwright, T. E. (1970) "Decay of the velocity autocorrelation function", PAys. Reu. A 1, 18. Ancona, M . (1995) "Hydrodynamic models of semiconductor electron transport at high fields", VLS/Destgn 3, 101. Anile, A. M . and Pennisi, S. (1992) "Thermodynamic derivation of the hydrodynamical model for charge transport in semiconductors", PAys. Reu. B 46, 13186. Arnold, L. (1974) <SfocAasf:c D:^eren^ta^ EgMaftons; TAeory one! AppZtca^ons, John Wiley & Sons, N e w York. Arnold, L. (1998) R a n d o m Dynanttca/ Systems, Springer-Verlag, Berlin. Arnold L. and Lefever, R. eds. (1981) S^ocAastte Now/mear Sys^enn tn PAys:cs, CAemts^ry, and P:oJogy, Springer-Verlag, Berlin. Axelrad, V. (1990) "Fourier method modeling of semiconductor devices", Z E E E Trans. C A D 9, 1225. Balakrishnan, V. and Chaturvedi, S. (1988) "Persistent diffusion on a line", PAystca A 148, 581. Balescu, R. (1975) EqKtH6r:K?n and NonegMtH&rtHfn Sfaftsftca? MecAantcs, John Wiley & Sons, N e w York.

265

266

Bt6^M?gr<%)Ay

Barker, J. R. (1998) "Trajectory-based representations of quantum transport theory and their connection with semi-classical physics", in 7WCE-6. 7?xtended Aostracts o/J99S StxtA 7ntemat:ona/ Wbr^s/top on CofnpMfat:ona/ E/ectronMs, 19-21 October 1998, Osaka, Japan (IEEE). Barker, J. R. (2000) "A simple model for the quantum hydrodynamic simulation of electron transport in quantum confined structures in the presence of vortices", in 7fA 7nternaftonaf Wor%sAop on CofnpMfaftonaZ 7?fecfron:cs. Boo& ofAbstracts, 2225 M a y 2000, Glasgow, Scotland (University of Glasgow, Glasgow, Scotland ), pp. 83-84. Barker, J. R. and Ferry, D. K. (1998) "On the validity of quantum hydrodynamics for describing antidot array devices", Semtcond. Sc:. TecAno/. 13, A135. Barker, J. R. and Watling, J. R. (2000) "Three-dimensional modelling of dissipative quantum transport in quantum dots and atomistic scale devices using non-Hermitian generalized potentials", -SMper/aMtces and MtcrostrKctMres 27, 347. Barucci, E. and Landi, L. (1996) "Computational methods in finance: Option pricing", ZEE.E ContpM^aftonctJ Science and Engineering 3 (Spring), 66. Bellman, R. and Adomian, G. (1985) Parfta/ Dt^erenttaJ EoKaftons. ZVeHj AfetAods /or TAetr 7/reatntent and So^Mtton, D. Reidel, Dordrecht. Bellman, R., Kashef, B. G. and Casti, J. (1972) "Differential quadrature: A technique for the rapid solution of nonlinear partial differential equations", J. CompKf. PAys. 10, 40. Bellomo, N. ed. (1995) LecfMre /Votes on tAe MatAentctttcaf 7/Aeory of tAe Bo^tzntann EgMatton, World Scientific, Singapore. Bellomo, N. (1997) "Nonlinear models and problems in applied sciences from differential quadrature to generalized collocation method", MatA. Co/npMf. Modeling 26, 13. Bellomo, N., Brzezniak, Z. and D e Socio, L. M . (1992) TVon/cnear S(ocAas^:c Tf^o/utton Problems tn AppMed Sciences, Kluwer, Dordrecht. Bellomo, N., D e Socio, L. M . and Monaco, R. (1988) "Random heat equation: Solutions by the stochastic adaptive interpolation method", ContpMters AfatA. AppJic. 16, 759. Bellomo, N. and Flandoli, F. (1989) "Stochastic partial differential equations in contin u u m physics—On the foundations of the stochastic interpolation method for Ito's type equations", AfatA. Cofnput. <Stn::dat:on 31, 3. Bellomo, N., Lachowicz, M., Palczewski, A. and Toscani G. (1991) AfatAemattca/ Toptcs tn TVon^near 7
BtM/ograpny

267

^Hne^te Theory, World Scientific, Singapore. Bellomo, N. and Preziosi, L. (1995) Mode/Hng Ma^ema^tcct/ Afe^oc^s and Scten?t/:c CompHiot^to^s, C R C Press, Boca Raton. Bellomo, N. and Riganti, R. (1987) Mw/tnear SfocAasftc Sysfents /n PAystes an^ Mecnan/cs, World Scientific, Singapore. Belopol'skaya, Ja. I. and Daleckil, Ju. L. (1980) "Diffusion processes in smooth Banach spaces and manifolds. I.", Trans. Moscow. Mot?/:. Soc, Issue 1, pp. 113-150. Belopolskaya, Ya. I. and Dalecky, Yu. L. (1990) <S/oc/msf:c EgMafZons awe! Dt/j'erenftaZ Geowe^ry, PQuwer, Dordrecht. Benson, K.B. (1988) Audto Fngtneertng #and&ooA, McGraw-Hill, New York. Bensoussan, A. and Temam, R. (1973) "Equations stochastiques du type Navier-Stokes", J. FMMcttona? Ana/. 13, 195. Berntgen, J., Heime, K., Daumann, W., Auer, U., Tegude, F.-J. and Matulionis, A. (1999) "The 1//noise of InP based 2 D E G devices and its dependence on mobility", 7EEE Trans. E/ec?ron Detnces 46, 194. Bert, C. W . and Malik, M. (1996) "Differential quadrature method in computational mechanics: A review", App/. Mecn. Reu. 49, 1. Bocquet, L. (1998) "From a stochastic to a microscopic approach to Brownian motion", Acfa Pnystca Po/on/ca B 29, 1551. Bonzani, I. (1993) "The time evolution of random fields in stochastic continuum mechanics", Ma?n. Compuf. Mode//:ng 17, 37. Bouc, R. and Pardoux, E. (1981) "Moments of semilinear random evolutions", <S7AM J. App/. Ma?n. 41, 370. Bouc, R. and Pardoux, E. (1984) "Asymptotic analysis of PDEs with white-band noise disturbances, and expansion of the moments", Sfocn. Ana/. App/. 2, 369. Boulter, B. (1995) "Performance evaluation of H P F for scientific computing", in #tgnPer/brmance CornpM^tng and /Veftuor&tng, eds. B. Hertzberger and G. Serazzi (Springer-Verlag, Berlin), pp. 652-657. Brown, R. (1828) "A brief account of microscopical observations made in the months of June, July, and August, 1827 on the particles contained in the pollen of plants, and on the general existence of active molecules in organic and inorganic bodies", Edtn&Mrgn JVeu; P M . J. 5, 358. Bulsara, A., Hanggi, P., Marchesoni, F., Moss, F. and Shlesinger, M. ed. (1993) Proceedings of the N A T O Advanced Research Workshop: Stochastic Resonance in Physics and Biology, J. Sfaftsf. Pnys. 70, nos. 1/2. Burghartz, J. N., Hargrove, M., Webster, C. S., Groves, R. A., Keene, M., Jenkins, K. A., Logan, R. and Nowak, E. (2000) "RF potential of a 0.18-um C M O S logic device technology", 7EEE Trans. E/ec?ron Deu/ces 47, 864. Carnahan, N. F. and Starling, K. E. (1969) "Equation of state for non-attracting rigid spheres", J. Cnent. Pnys. 51, 635. Carr, J. (1981) App/:cafions o/Cenfre Mant/bZd Tneory, Springer-Verlag, New York. Casinovi, G. and Ho, R. J. (1996) "Computer generation of colored noise for time-domain analysis of integrated circuits", Ana/og Zn?egra?ed Ctrcutts and <S:gna/ Processing 11, 205.

268

Bt&Ztograp/ty

Chapman, S. and Cowling, T. G. (1990) 7%e AfafAefnaftcct^ TTteory o/'TVon-T/nt/br/n Gases, Cambridge Univ. Press, Cambridge. Chow, P.-L. and Khasminskii, R. Z. (1997) "Stationary solutions of nonlinear stochastic evolution equations", Stocn. Ana/. App/. 15, 671. Churchhouse, R. F. ed. (1981) 77ana'6oo& o/'App/tca6/eMat/tema^:cs, Vo/. 777; Numerical Afe^Aods, John Wiley & Sons, Chichester (England). Ciesielski, Z. and Domsta, J. (1972) "Construction of an orthonormal basis in C"(/^) and W,"(/'')", Studta Afatn. 41, 211. Courant, R. and Hilbert, D. (1962) Me^Aotfs o/'Afa^eyna^:ca/ PAystcs. Partta/ Di^Brentta/ Eauattons, Vo/. 3, Interscience, N e w York. Cramer, H. (1946) MatnemattcaZ Mecca's o/^.Sfa?tsf:cs, Princeton Univ. Press, Princeton. Darcy, H. P. G. (1856) Les Pbntames PuM:aues de Za VtHe de D:gon, Vector-Dalmont, Paris. Datta, S. (1995) 72/ectrontc Traytspor^ tn Afesoscoptc Systems, Cambridge Univ. Press, Cambridge. Dean, D. S. (1996) "Langevin equation for the density of a system of interacting Langevin processes", J. Pnys. A; Maf/t. Gen. 29, L613. Dean, E., Glowinski, R. and Li, C. H. (1988) "Numerical solution of parabolic problems in high dimensions", in Trans. 5fA Arwy Con/erence on Appftecf MatAeynattcs and Compattng, (ARO 7!ep. 85-07^ (U.S. Army Res. Office, Research Triangle Park, USA), pp. 207-285. Demidovic, B. P. (1967) Lectures on tAe Afatnemaftco^ Tneory o/Staot/tty, Nauka, Moscow (in Russian; Matn. Reu. 37 (1969) #1716). Demir, A., Liu, E. W . Y. and Sangiovanni-Vincentelli, A. (1996) "Time-domain nonMonte Carlo noise simulation for nonlinear dynamic circuits with arbitrary excitations", 7EEE Trans. C A D 7CAS 15, 493. Demir A. and Sangiovanni-Vincentelli, A. (1996) "Simulation and modelling of phase noise in openloop oscillators", in Proc. o/ tne 772E7? 7996 Custom 7ntegratea! CtrCMtts Con/erence (IEEE, N e w York), pp. 453-456. Deng, L.-Y., Chan, K. H. and Yuan, Y. (1994) "Random number generators for multiprocessor systems", 7nt. J. Mocfe/^ng ana* Stntt^atton 14, 185. Dimentberg, M., Hou, Z. and Noori, M. (1995) "Spectral density of nonlinear single -degree-of-freedom system's response to a white-noise random excitation: A unique case of an exact solution", 7nt. J. JVon-L:near Mecnan:cs 30, 673.

Dongarra, J. J., Fagg, G. E., Fisher, M., Geist, G. A., Kohl, J. A., Manchek, R. J., Mucci P., Papadopoulos, P., Scott, S. L. and Sunderam, V. (1997) "PVM Version 3.4; ParaMe/ Vtrtua/ Mdcntne System", Univ. of Tennessee, Knoxville, USA; Oak Ridge National Lab., Oak Ridge, USA; Emory Univ., Atlanta, USA, URL: http://www .netlib.org/pvm3/win32/ Dongarra, J. and Fisher, M. (1997) "Another architecture: P V M on windows 95/NT", in Proc. Caster Co?npMt:ng Con/erence, CCC'97 (Emory Univ., Atlanta, FL, USA). Dunford N. and Schwartz, J. T. (1988) Linear operators. Part 7; Genera/ TTteory, John Wiley & Sons, N e w York.

B:6t:ograpny

269

Dwight, H. B. (1961) T*aoJes o/7a^''"^ and Otner Afatneynattca/ Data, The Macmillan Company, New York. Dynkin, E. B. (1965) AfarAou Processes, Vots. 7-2, Academic Press/Springer-Verlag, N e w York/Berlin. Edwards, R. E. (1995) Functtona/ Ana/ysts; TAeory ana* AppZtcaftofts, Dover Publications, N e w York. Enflo, P. (1973) "A counterexample to the approximation problem in Banach spaces," Acta Maf/:. 130, 309. Ermak, D. L. and McCammon, J. A. (1978) "Brownian dynamics with hydrodynamic interactions", J. CAem. Pnys. 69, 1352. Espanol, P. (1995) "Hydrodynamics from dissipative particle dynamics", Pnys. Eeu. E 52, 1734. Espanol, P. and Warren, P. (1995) "Statistical mechanics of dissipative particle dynamics", EMropAys. LeM. 30, 191. Feller, W . (1954) "Diffusion process in one dimension", Trans. Anter. Mat^. Soc. 77, 1. Feller, W . (1968) A n /nfroa'Mcf:on to Pro6a6:Ztty Theory awe^ Zts App^tcattons, Vol. 1, John Wiley & Sons, N e w York. Feller, W . (1971) A n 7Mfroa*MctMM to Pro&aMtty 7%eory ana* 7ts AppZ:cat:ons, Vol. 2, John Wiley & Sons, N e w York. Ferry, D. K. (1980) "Long-time tail of the autocorrelation function for electron drift in high electric fields in silicon", PAys. Reu. Lett. 45, 758. Ferry, D. K. and Barker, J. R. (1999) "Issues in general quantum transport with complex potentials", App?. Pnys. Lett. 74, 582. Fish, P. J. (1993) EZectrontc Notse ana* Low JVotse Destgn, The Macmillan Press, Basingstoke. Fitzgibbon, W . E. and Parrot, M. E. (1997) "Convergence of singular perturbations of strongly damped nonlinear wave equations", Non?:near Analysts, 7%eory, Met/tea's and App/tcattons 28, 165. Flower, G. (1990) "An introduction to the limitations of the SPICE bipolar transistor model", in Proc. A F recAno?ogy Expo 90, 27-29 March 1990, Anaheim, CA, U S A (Cardiff Publishing, Englewood, CO, USA), pp. 47-72. Fokker, A. D. (1914) "Die mittlere Energie rotierender electrischer Dipole im Strahlungsfeld", Ann. Pnys. (Letpztg) 43, no. 5, 810. Fowler, A. C. (1998) Matnemattca/ AMe^s tn tAe AppJtea* Sctences, Cambridge Univ. Press, Cambridge. Fraaije, J. G. E. M., van Vlimmeren, B. A. C , Maurits, N. M., Postma, M., Evers, O. A., Hoffman, C , Altevogt, P. and Goldbeck-Wood, G. (1997) "The dynamic meanfield density functional method and its application to the mesoscopic dynamics of quenched block copolymer melts", J. Cnew. Pnys. 106, 4260. Freeman, G., Ahlgren, D., Greenberg, D. R., Groves, R., Huang, F., Hugo, G., Jagannathan, B., Jeng, S. J., Johnson, J., Schonenberg, K., Stein, K., Volant, R. and Subbana, S. (1999) "A 0.18um 90 GHz/^SiGe H B T BiCMOS, ASIC-compatible, copper interconnect technology for RF and microwave applications", in 7999 interna-

270

Biotio^rapny

f:o?m? E/ectro?t Debtee Meetwg. Tecnnicai Dtgest, 5-8 December 1999 (IEEE, N e w York), pp. 569-572. Freidlin, M . I. (1968) "On the factorization of non-negative definite matrices", TAeory Pro6a6. App/. 13, 354. Friedman, A. (1975) StocAasftcDt/j%renfta?Eq'MattonsandAppZtcafMns, VbJ. i, Academic Press, N e w York. Friedman, A. (1976) StocAasf:cDt/%renf:a?EoMa(:oMsa?:6?Applications, VoJ. 2, Academic Press, N e w York. Fubini, G. (1907) "Sugli integrali multipli", Rend. Ltncet, s. J" 16, 608-614. Fubini, G. (1903) "Di un metodo per l'integrazione e lo studio delle equazioni a derivate parziali", Rewdtconti de^ Circoio Afatematico & Palermo 17, 222-235. Galerkin, B. G. (1915) "On electrical circuits for the approximate solution of the Laplace equation", Vestnt& 7nz/teMerou i reA/m:Aou 19, 897 (in Russian). Gammaitoni, L., Hanggi, P., Jung, P. and Marchesoni, F. (1998) "Stochastic resonance", Reu. AfoderTt PAys. 70, 223. Garcia-Ojalvo, J. and Sancho, J. (1999) Notse :n SpatiaMy ExfencM Systems, Springer-Verlag, N e w York. Gard, T. (1988) introduction to Stocnastic Di^erentiai Equations, Marcel Dekker, N e w York. Gardiner, C. W . (1994) HandoooA o/Stocnastic Afetnods /or Pnysics, Cnemistry and tAe Afaturai Sciences, Springer-Verlag, N e w York. Geist, A., Beguelin, A., Dongarra, J., Jiang, W., Manchek, R. and Sunderam, V. (1994) P V M ; Parai/e/ Virtuai AfacAine. A t/sers' GMt^e and Tutortai /or Networked Parage/ Computing, M I T Press, Boston, U S A . See also "PVM: Parallel Virtual M a chine", U R L : http://www.epm.ornl.gov/pvm/ Gel'fand, I. M., Graev, M . I. and Vilenkin, N. Ya. (1966) Generated Functions, Voi. 5; /ntegrot/ Geometry and Representation TAeory, Academic Press, N e w York. Gel'fand, I. M . and Shilov, G. E. (1964) Generalized Functions, Voi. i; Properties and Operations, Academic Press, N e w York. Gel'fand, I. M . and Shilov, G. E. (1968) Generalized Functions, Voi. 2; Spaces o/Fundamental and Generalized Functions, Academic Press, N e w York. Gel'fand, 1. M . and Shilov, G. E. (1967) Generalized Functions, Vb?.3; 77:eory o/Di//erentiai Equations, Academic Press, N e w York. Gel'fand, 1. M . and Vilenkin, N. Ya. (1964) Generalized Functions, Voi. 4; Applications o/Harmonic Analysis, Academic Press, N e w York. Ghizzetti, A. (1981) "Aspects of the work of Guido Fubini in the field of mathematical analysis", in PuoM. Zstit. Mat. AppZ. Fac. /ngr. t/niu. Stud. R o m a Quaderno, No. iS, pp. 9-20 (in Italian). Gihman, 1. I. and Skorohod, A. V. (1972) Stochastic Di//erentiaZ Equations, SpringerVerlag, N e w York. Gihman, I. I. (1980) "A boundary value problem for a stochastic parabolic equation", Ukrainian AfatA. c/., 31, 383. Gikhman, 1.1, and Skorokhod, A. V. (1969) /ntroduction to tAe TAeory o/Random Processes, W . B. Saunders, Philadelphia.

.B:M:ograpAy

271

Gliklikh, Yu. (1997) GZo&a^Ana^ystsmAfa^Aematica^PAystcs.- GeowetrtcaytdS^ocAas^c Afe^Aods, Springer-Verlag, N e w York. Gnedenko, B. V. (1982) TAe 7/Aeory o/*ProAo:6d^y, Mir, Moscow. Godoy, S. and Garcia-Colin, L. S. (1996) "From the quantum random walk to classical mesoscopic diffusion in crystalline solids", PAys. R e n E 53, 5779. Godoy, S. and Garcia-Colin, L. S. (1998) "Mesoscopic diffusion as a non-Markov process", PAysMxz A 258, 414. Godoy, S. and Garcia-Colin, L. S. (1999) "Compatibility of Landauer diffusion coefficient with classical transport theory", PAys:cot A 268, 65. Goldstein, S. (1951) "On diffusion by discontinuous movements, and on the telegraph equation", Quart J. AfecA. AppZ. Ma(A. 4, 129. Gomez, C. (1999) ErtgMeertng a^d ScteMft/tc ContpKfmg MJtf A Se:/a6, Birkhauser, Boston. Gradshteyn I. S. and Ryzhik, 1. M . (1994) TctMe o/Znfegrc^s, ,Ser:es, and Produces, Academic Press, Boston. Grasman, J. and V a n Herwaarden, O. A. (1999) Asy?npfo?:c Afe^Aods /or ^Ae i^oA^erP/ctncA EquattoTt and f Ae Ex:( Pro6/em m Apj9?:caf:ons, Springer-Verlag, Berlin. Grecksch, W . and Kloeden, P. E. (1996) "Time-discretized Galerkin approximations of parabolic stochastic PDEs", BMH. AKsfra/. MafA. -Soc. 54, 79. Grecksch, W . and Tudor, C. (1995) <S(ocAasf:c Euo/M?:on Egua^ows; A 7f:/Aer^ Space ApproacA, Akademie Verlag, Berlin. Griewank, A. (2000) Eua^Mafmg DeWua^tues.' Prwctp/es awd TecAntaues o/A^or^Ayntc Dt^ereK^:o(toyt, 81AM, Philadelphia. Griewank, A. and Corliss, G. F. ed. (1991) AMfo?na(:c D:/?erenf:afMn.' 7/Aeory, 7mp/ewteK^a^ton, OTtd App/tcafton, SIAM, Philadelphia. Grinberg, A. A. and Luryi, S. (1992a) "Diffusion in short base", <So/td-S(a(e E^ec^ro^. 35, 1299. Grinberg, A. A. and Luryi, S. (1992b) "Ballistic versus diffusive base transport in the high-frequency characteristics of bipolar transistors", App/. PAys. Leff. 60, 2770. Grinberg, A. A. and Luryi, S. (1993) "Coherent transistor", 7 E E E TYaHS. E/ecfro?: Debtees 40, 1512. Grinberg, A. A. and Luryi, S. (1994) "Article comprising a 'ballistic' heterojunction bipolar transistor", t/n:(ed Sfafes Pa^en( JVo. 5,304,816. Groot, R. D. and Warren, P. B. (1997) "Dissipative particle dynamics: Bridging the gap between atomistic and mesoscopic simulation", J. CAewt. PAys. 107, 4423. Gupta, M . S. (1982) "Thermal noise in nonlinear resistive devices and its circuit representation", Proc. Z E E E 70, 788. Gupta, M . S. (1994) "Conductance fluctuations in mesoscopic conductors at low temperatures", Z E E E 7Y<ms. Electron. Deu. 41, 2093. Haken, H. (1975) "Cooperative phenomena in systems far from thermal equilibrium and in nonphysical systems", Reu. AfoderTt PAys. 47, 67. Haken, H. (1977) -Synergeftcs—An/n^rodMcfton. NoMegMt^t&rtMfnPAase Transports artd .Se^OrgaTMzafton :H PAysrcs, CAerrttsfry and Bto/ogy, Springer-Verlag, Berlin. Hale, J. K. (1980) Ordinary Dt/?erenfta/ EgKafMMs, Robert E. Krieger, N e w York.

272

BtMtograpAy

Hall, G. and Watt, J. M. (1976) Modern JVuntertca/ AfetAods /or Ordinary Dt^ere7t(ta/ Equations, Clarendon Press, Oxford. Hamilton, K. G. (1997) "Assembler R A N L U X for PCs", Computer Pnysics Communications 101, 249. Hamilton, K. G. and James, F. (1997) "Acceleration of RANLUX", Computer Pnysics ComwtM?Mcaf:ons 101, 241. Hansen, J. P. and McDonald, 1. R. (1986) TTteory o/*5tmp^e LtqMtoJs, Academic Press, London. Has'minskii, R. Z. (1980) S(oc7:<Mf:c Sta5tZi
B:MtograpAy

273

/'U%SS. JVbMueMe 5erte II (Avril-Juin), 532. Kantorovich, L. V. and Krylov, V. 1. (1958) Approxtmotfe Afe^Aods o/H^Aer Analysts, P. Noordhodd, Groningen. Khasminskii, R. Z. and G. Yin, G. (1996a) "Asymptotic series for singularly perturbed Kolmogorov-Fokker-Planck equations", SZAAf J. AjapA Afaf A. 56, 1766. Khasminskii, R. Z. and G. Yin, G. (1996b) "On transition densities of singularly perturbed diffusions with fast and slow components", SZAAf J. AppA Afa?A. 56,1794. Klimontovich, Yu. L. (1982) JHfte(:c TAeory o/'Nontdea/ Gases and AToyMdee? Plasmas, Pergamon Press, Oxford. Klimontovich, Yu. L. (1994) "Nonlinear Brownian motion", PAys:cs—E7spe%At 37, 737. Kloeden, P. E. and Platen, E. (1995) AfKHtertca? So%H(ton o/'S^ocAasttc Dt^ereni:a/ EaMaftons, Springer-Verlag, Berlin. Kolmogorov, A. (1931) "Uber die analytischen Methoden in der Wahrscheinlichkeitsrechnung", AfdtA. AnwaJen 104, 415. Kolmogorov, A. N. (1956) PoKna'aftons o^^Ae TAeory o/'Pro6ot6t/:ty, Chelsea Publishing, N e w York. Kovalenko, I. N., Kuznetsov, N. Yu. and Shurenkov, V. M . (1996) AfocMs o/'Rotwdont Processes. Hdnd&ooA /br A/a^Aemotftctaws ctytd Fytgmeers, C R C Press, Boca Raton. Krylov, N. V. (1980) Confro%Jed D:/?MSton Processes, Springer-Verlag, N e w York. Krylov, N. V. (1995) 7nfroa'Mc?toM fo ?Ae T*Aeory o/D:#Ks:ow Processes, A M S , Providence (RI, U S A ) . Krylov, N. V. (1996) "On L -theory of stochastic partial differential equations in the whole space", SZAAf J. Afa?A. AnaA 27, 313. Krylov, N. V. and Rozovskii, B. L. (1979) "Ito equations in Banach spaces and strongly parabolic stochastic partial differential equations", <Sowef AfafA. DoA%. 20,1267. Kwak, D. Y. and Son, S. (1991) "Prime sample scheme for almost sure convergence of a Galerkin approximation", Computers AfatA. App/tc. 22, 45. Landau, L. D. and Lifshitz, E. M . (1987) F^Htd AfecAa?Mcs, Pergamon Press, Oxford. Landauer, R. (1957) "Conductance as a consequence of incident flux", /AfB J. Res. DeueZop. 1, 223. Landauer, R. (1970) "Electrical resistance of disordered one-dimensional lattice", PAt?os. Afog. 21, 863. Langevin, P. (1908) "Sur la theorie du mouvement Brownien," C. R. Acotd. Set. Parts 146, 530. Langtangen, H. P. (1991) "A general numerical solution method for Fokker-Planck equations with applications to structural reliability", Pro6a6tZ:sf:c Fngrg. AfecA. 6,33. Larson, L. E. ed. (1996) R F awd Mtcrowaue CtrcMt Destgw /or WtreJess Communtco;^tons, Artech House, Boston. Laughlin, R. B. and Pines, D. (2000) "The theory of everything", P N A S 97, 28. Laughlin, R. B., Pines, D., Schmalian, J., Stojkovic, B. and Wolynes, P. (2000) "The middle way", PJVAS 97, 32. Lifshitz, E. M . and Pitaevskii, L. P. (1993) PAystca/ iOfteftcs, Pergamon Press, Oxford. Likharev, K. L. (1986) Dy?tot?Htcs o^ JosepAson Junc(to?ts and CtrcMtfs, Gordon and

274

BtMtograpAy

Breach, N e w York. Lin, H. and Yim, S. C. S. (1996) "Analysis of a nonlinear system exhibiting chaotic, noisy chaotic, and random chaotic behaviors", J. App/. AfecA. 63, 509. Lindenberg, K. and Seshadri, V. (1981) "Fluctuation-dissipation relations for systems with internal multiplicative noise", in A7P Con/erence ProceecHngs, no. &9 (American Institute of Physics, N e w York), pp. 333-337. Lindner, J. F., Meadows, B. K., Ditto, W . L., Inchiose, M. E. and Bulsara, A. R. (1995) "Array enhanced stochastic resonance and spatiotemporal synchronization", P/tys. Reu. LeM. 75, 3. Luchinsky, D. G., McClintock, P. V. E. and Dykman, M. I. (1998) "Analogue studies of nonlinear systems", Rep. Prog. PAys. 61, 889. Lundstrom, M. (1992) FMna'arnenta/s o/'Carrter Transport, Addison-Wesley, Reading (MA, USA). Lundstrom, M. (1997) "Elementary scattering theory of the Si MOSFET", /REE Electron Deu:ce LeM. 18, 361. Lundstrom, M. (1998) "The silicon M O S F E T from a transmission viewpoint", -Super^atf:ces ana* MtcrostrHctMres 23, 771. Lusternik, L. A. and Sobolev, V. I. (1974) Elements o/'Euncttonaf Analysts, Hindustan Publishing, Dehli. Luzin, N. (1972) LeconssMrZesEnsemMesAHa?yttgKeset^eHrsApp/:cat:oMs(Reimpression de l'edition de 1930), Chelsea Publishing, N e w York. Lowen, H., Hansen, J.-P. and Roux, J.-N. (1991) "Brownian dynamics and kinetic glass transition in colloidal suspensions", PAys. Reu. A 44, 1169. Ltischer, M. (1994) "A portable high-quality random number generator for lattice field theory simulations", Computer PAysics Co?n;?tMn:cat:ons 79, 100. Malakhov, A. N. (1995) "The effect of fluctuations on the kinetics of phase transistions of the first kind in physical systems", in Proc. o/'tAe Z3 Jntemattona/ Con/erence "No:se :n PAys:cct/ Systems and Z//'7<7MctMaf:ons" (World Scientific, Singapore), pp. 153-156. Mamontov, E. V. (1989) "An equilibrium solution of a system of nonautonomous linear partial differential equations", in TAeorettca/ and AfKmertcct^ StuaVes o/\BoMndary Va^Me ProoJems, ed. Yu. A. Klokov (Latv. Gos. Univ., Riga, Latvia), pp. 121-124 (Mathematical Reviews # 90j:00013; Current Mathematical Publications # 1 044 578). Mamontov, Y. V. and Willander, M. (1994) "Bandgap narrowing and incomplete ionization calculations for the temperature range from 40 K up to 400 K", ZE/CE Trans. E/ectrontcs E77-C, 287. Mamontov, Y. V. and Willander, M. (1995a) "Simulation of bandgap narrowing and incomplete ionization in strained Sii_,Ge, alloys on <001> Si substrate (for temperatures from 40K up to 400K)", So/td-Sfate E/ectrontcs 38, 599. Mamontov, Y. V. and Willander, M. (1995b) "Accounting thermal noise in mathematical models of quasi-homogeneous regions in silicon devices", ZEEZ? Trans. C A D /CAS 14, 815. Mamontov, Y. V. and Willander, M. (1995c) "Thermal noise in silicon bipolar transis-

BtM:ograpAy

275

tors and circuits for low-current operation—Part I: Compact device model", 7E7CE rrct^s. EZecfrontcs E78-C, 1761. Mamontov, Y. V. and Willander, M . (1997a) "On reduction of high-frequency noise in low-current silicon BTs", PAys. Scrtpfot T69, 218. Mamontov, Y. V. and Willander, M . (1997b) "Model for thermal noise in semiconductor bipolar transistors at low-current operation as multi-dimensional diffusion stochastic process", JE/CE Trans.EZecfron:cs E80-C, 1025. Mamontov, Y. V. and Willander, M . (1997c) "Asymptotic method of finite equation for bounded solutions of nonlinear smooth ODEs", Afa?A. JqpoTttca 46, 451. Mamontov, Y. V. and Willander, M . (1997d) "Application of ordinary/partial Ito's stochastic differential equations to modelling noise in semiconductor devices and circuits", in Proc. o/\Pro.R.f<SC '97, Z E E E Bene%Mx CtrctMfs and <Sysfe?ns CAapfer, 8fA awnua/ WorAs/top on CtrcKtts, Systems aytd Stgna^ Processtng, 27-28 November 1997, Eindhoven-Mierlo, The Netherlands ( S T W Technology Foundation, Utrecht, The Netherlands), pp. 401-409. Mamontov, Y. V. and Willander, M . (1997e) "Long asymptotic correlation time for nonlinear autonomous Ito's stochastic differential equation", JVo^Jwear Dy^oty?Mcs 12, 399. Mamontov, Y. V. and Willander, M . (1998a) "Algorithm to evaluate spectral densities of high-dimensional stationary diffusion stochastic processes with nonlinear coefficients: The general scheme and issues on implementation with P V M " , in App/:ed Parage/ ContpMftng. Large ScaJe -Sctenf t/:c and /Mdus^rtmZ ProMems, Lee?. Nofes tn Co^pu?. Set., VbJ. i54i, eds. B. Kagstrom e? a^. (Springer-Verlag, Berlin), pp. 315-321. Mamontov, Y. V. and Willander, M . (1998b) "Estimations of the stable periodic solution of high-dimensional nonlinear autonomous first-order O D E system by means of the finite-equation method", in Aos?rac?s o/'ECM7'98, iOfn Con/! o/*fAe Europecm CoHSorf:H?n /or Mct^emn^cs tn /ndus^ry, June 22-27, 1998, Gothenburg, Sweden. The P D F file at U R L : http://www.math.chalmers.se/Conf/ECMI98/standard /Abstracts/Telecommunications/mamontov.pdf Mamontov, Y. V. and Willander, M . (1999) "Modelling of high-dimensional diffusion stochastic process with nonlinear coefficients for engineering applications—Part II: Approximations for covariance and spectral density of stationary process", MafA. Models Afe^AooJs App/. <Sct. 9, 1247. Mamontov, Y. M., Willander, M . and Lewin, T. (1999) "Modelling of high-dimensional diffusion stochastic process with nonlinear coefficients for engineering applications—Part I: Approximations for expectation and variance of nonstationary process", Afat^n. Afode/s Afe^/tods App^. Set. 9, 1201. M a o , X. (1997) S^oenots^tc Dt//erenfta? EqKa?tons and f/tetr Appftcaftons, Horwood Publishing, Chichester. Martin, S., Archer III, V. D., Boulin, D. M., Frei, M . R., Ng, K. K. and Yan, R.-H. (1997) "Device noise in silicon R F technologies", PeM Laos Tec/t. J. 2(3), 30. Maruyama, T. (1977) Sfoc/tasftc ProoZetns :n PopuJatton Geneftcs, Springer-Verlag,

Berlin.

276

Bt5/:ograpAy

Matsuoka, H., Ichiguchi, T., Yoshimura, T. and Takeda, E. (1994) "Mesoscopic transport in Si metal-oxide-semiconductor field-effect transistors with a dual-gate structure", J. AppJ. PAys. 76, 5561. McNamee, J., Stenger, F. and Whitney, E. L. (1971) "Whittaker's cardinal function in retrospect", AfafA. Contp. 25, 141. Metropolis, N. and Ulam,S. (1949) "The Monte Carlo method", J. ATHer.Aa^. Assoc. 44, 335. Meyerson, B. S. (2000) "Silicon:germanium-based mixed-signal technology for optimization of wired and wireless telecommunications", ZBAf J. Res. Deue/op. 44, 391. Mihaila, M. (1986) "Mobility fluctuation 1//* noise in bipolar transistors", Reu%e RoMma:ne de PAystaue 31, 607. Min, H. S. (1981) "1//* noise in M O S transistors in ohmic region under strong inversion", in StxfA Znfernaftona/ Con/erence on Notse tn PAys:caJ Systems (NBS, Washington), pp. 221-225. Mityagin, B. S. (1970) "The homotopy structure of the linear group of a Banach space", Russian MafA. <SMrueys 25, 59. Mori, H. (1965) "Transport, collective motion, and Brownian motion", Progr. TAeor. PAys. 33, 423. Nagasawa, M. (1993) .ScArd'dtnger Eguaf tons and Dt/^HSton TAeory, BirkhauserVeriag, Basel. Natarajan, R. (1991) "Finite element applications on shared-memory multiprocessor: Algorithms and experimental results", J. Comput. PAys. 94, 352. Naveh, Y. and Likharev, K. K. (2000) "Shrinking limits of silicon MOSFET's: Numerical study of 10-nm-scale devices", SMperZaMtces ana! AftcrosfrucfMres 27, 111. Nayfeh, A. H. and Balachandran, B. (1995) App/ted JVonZtnearDyna/ntcs* Ana/y^tcaZ, CompMfmtiona^, and Expertfnenfa? Afe^Aods, John Wiley & Sons, N e w York. Nyquist, H. (1928) "Thermal agitation of electric charge in conductors", PAys. Reu. 32, 110. Ochi, M. K. (1990) Applet! Pro6a6t/:fy and <SfocAast:c Processes :n Engtneertng and PAystca/ Sctences, John Wiley & Sons, N e w York. Olaofe, G. O. and Mason, J. C. (1988) "Quadrature solution of ordinary and partial differential equations", AfafA?. ContpM?. AfoGM?:ng 11, 661. O'Malley, R. E., Jr. (1991) StngM/ar Per^Mroa^on Afe^Aods /or Ordinary D:^eren^:a/ Eoua^ons, Springer-Verlag, N e w York. Ord, G. N. (1996) "The Schrodinger and Dirac free particle equations without quantum mechanics", Annct/s o/PAystcs 250, 51. Ortega, J. M. and Rheinboldt, W . C. (1970) Reraftve 3o/Mf:on o/JVon^tnear Eguaftons tn Seuera/ Var:a&/es, Academic Press, N e w York. Ostrogradski, M. V. (1838) InAfent. Acad. Ret. S^. Pefers&Krg. Ser. 6. Set. Ma^A. PAys. e^ Na^ureHes 1, 35. Pardoux, E. (1993) "Stochastic partial differential equations: A review", BuH. <Sci. Ma^A. (2j 117, 29. Papoulis, A. (1991) Prooa&dtfy, Random Var:ao/es, ana* -StocAas^c Processes, McGrawHill, N e w York.

BtMtograp/ty

277

Pawiikiewicz, A. H., van der Ziel, A., Kousik, G. S. and Van Vliet, C. M. (1988) "Fundamental 1//noise in silicon bipolar transistors", Solid-State Electronics 31, 831. Pham, T. Q. and Garg, P. K. (1996) Mu^:^7-ea^ecf Programming HnYn IVm^ows N T , Prentice Hall PTR, Upper Saddle River (NJ, USA). Planck, M. (1917) "Uber einen Satz der statistischen Dynamik und seine Erweiterung in der Quantentheorie", in <St/zungs6er. Preuss. AAad. Wtss., Pnys. Afa^A. iH., 325. Pliss, V. A. (1966) Non/ocaZ Proo/ems o/^Ae TAeory o/'OsctMa^tons, Academic Press, N e w York. Pontryagin, L. S. (1962) Ordinary Dt^eren^taZ Fgua^tons, Addison-Wesley, Reading (MA, USA). Pope, S. B. (1983) "A Lagrangian two-time probability density function equation for inhomogeneous turbulent flows", PAys. F/Mtds 26, 3448. Pope, S. B. (1985) "PDF methods for turbulent reactive flows", Prog. .Energy Com^MS?. Set. 11, 119. Pope, S. B. (1994) "Lagrangian P D F methods for turbulent flows", in Annua/ Reuters o/F^:c! MecAantcs, Vo/. 26 (Annual Reviews, Palo Alto, CA, USA), pp. 23-63. Pozorski, J. and Minier, J.-P. (1999) "Probability density function modeling of dispersed two-phase turbulent flows", Pnys. Reu. F 59, 855. Preziosi, L., Teppati, G. and Bellomo, N. (1992) "Modeling and solution of stochastic inverse problems in mathematical physics", AfcfA/. Compuf. MbdeMtng 16, 37. Prohorov, Yu. V. and Rozanov, Yu. A. (1969) Pro6aMt%y TAeory; Paste Concepts, Ltmt? Theorems, Random Processes, Springer-Verlag, Berlin. Pugachev, V. S. (1995) "Stochastic differential equations in Banach spaces with a basis", DoAA AAaa*. MtuA 342, 592. Rack, A. J. (1938) "Effect of space charge and transit time on the shot noise in diodes", BeM System Teen. J. 17, 592. Rahmat, K., White, J. andAntoniadis, D. A. (1996) "Simulation of semiconductor devices using a Galerkin/spherical harmonic expansion approach to solving the coupled Poisson-Boltzmann system", ZFFF Trans. C A D /CAS 15, 1181. Reichl, L. E. (1980) A Modern Course tn ,Sfa?ts(tca/ PAystcs, Edward Arnold, London. Reif, F. (1985) Fundaments o/Sfaftsftca/ and Tnerma/ PAystcs, McGraw-Hill, Singapore. Resibois, P. and De Leener, M. (1977) CZasstca? Ktneftc TAeory o/*F/utds, John Wiley & Sons, N e w York. Ribar, J. L. (1993) F O R T R A N Programming/or IVtndo^s, McGraw-Hill, Berkeley (CA, USA). Ricciardi, L. M. (1977) Dt/^uston Processes and Re/afed Toptcs tn BtoJogy, SpringerVerlag, Berlin. Ridley, B. K. (1988) Quantum Processes tn Semtconduc^ors, Clarendon Press, Oxford. Risken, H. (1989) TAe FoAAer-PZancA Fgua^ton. Me^Aods o/So/ufton and App/tca^:ons, Springer-Verlag, Berlin. Roy, R. V. and Spanos, P. D. (1993) "Power spectral density of nonlinear system response: The recursion method", Trans. A S M F J. App/. AfecA. 60, 358. Roychowdhury, J., Long, D. and Feldmann, P. (1997) "Cyclostationary noise analysis

278

BtMtograpAy

of large R F circuits with multitone excitations", in Proc. o/ tAe 7 E E E 7997 CMS^o/H 7ntegrated CtrcMtts Con/erence (IEEE, N e w York), pp. 383-386. Rozovskii, B. L. (1990) StocAasttc EuoJutton Systems. Lmear TAeory a^d App/tcatttons fo Nbn/twear T^tJterwg, Kluwer, Dordrecht. Rubinstein, R. Y. (1981) StwtM^atton and tAe Mo^^e Carfo MetAod, John Wiley & Sons, N e w York. Sarpeshkar, R., Delbruck, T. and Mead, C. A. (1993) "White noise in M O S transistors and resistors", Ctrcutts & Deu:ces X (November), 23. Schauder, J., von, (1927a) "Zur Theorie stetiger Abbildungen in Funktionalraumen", MatA. Z 26, 47. Schauder, J., von, (1927b) "Bemerkungen zu meiner Arbeit 'Zur Theorie stetiger Abbildungen in Funktionalraumen'", AfatA. Z 26, 417. Scheidegger, A. E. (1958) "The random-walk model with auto-correlation of flow through porous media", Canad. J. PAys. 36, 649. Schein, O. and Denk, G. (1998) "Numerical solution of stochastic differential-algebraic equaitons with applications to transient noise simulation of microelectronic circuit", J. CowtpMt. App?. AfatA. 100, 77. Schottky, W . (1918) "Uber spontane Stromschwankungen in verschiedenen Elektrizitatsleitern", AK^ct/en o'er PAystA Mnd CAentte 57, 541. Schwarz, R. (1989) Analyse ntcAt/tnearer TVetzwerAe tm erwetterten ZMstandsraHm. R. Oldenburg, Verlag, 1989. Scott, S. L. (1999) http://www.epm.ornl.gov/-sscott/Projectspage.htm Sedov, L. 1. (1971-1972) A Course w CowttHMMm AfecAantcs, VoZs. 7-7V, Wolters-Noordhoff, Groningen. Seeger, K. (1973) Senttcona'Mctor PAystcs, Springer-Verlag, Wien. Serra, R., Andretta, M., Compiani, M . and Zanarini, G. (1986) /nfroducfMH ^o tAe PAystcs o/ Complex Systems. TAe Afesoscoptc ApproacA ^o ^YMC^HCt^toHS, JVonZ:near:fy a?M^ Se^OrgaKtza^o^t, Pergamon Press, Oxford. Shi, K. and Jastrzebski, A. K. (1996) "Improved state-space approach to microwave circuit simulation", E/ectrontcs LeM. 32, 1108. Shockley, W . (1949) "The theory of p-n junctions in semiconductors and p-n junction transistors", BeM System TecA. J. 28, 435. Singer, I. (1970) Bases tn BfmacA spaces. 7, Springer-Verlag, N e w York. Singer, I. (1981) Bases tn BanacA spaces. 77, Springer-Verlag, N e w York. Skorokhod, A. V. (1965) StMdtes tn tAe TAeory o/'Ray:<7om Processes, Addison-Wesley, Reading (MA, U S A ) Sobczyk, K. (1991) StocAasttc Dt//erentta/ Eguattons tuttA AppZtcattons to PAystcs ana* Ewgweertng, Kluwer, Dordrecht. Sobol', I. M . (1994) A Prtmer/br tAe Monte Car/o AfetAod, C R C Press, Boca Raton. Soize, C. (1988) "Steady-state solution of Fokker-Planck equation in higher dimension", ProoaMtsttc Engrg. AfecA. 3, 196. Soize, C. (1994) TAe EoA&er-P?ancA EguattoK /or <StocAast:c TJywaTHtcaJ Systems aKtf 7ts Exp/tc:t Steady-State So/uttons, World Scientific, Singapore. Spanier, J. and Oldham, K. B. (1987) A n At/as o/*7?Hnct:ons, Hemisphere, Washington.

BtM:ograpny

279

Spruill, C. (1977) "Determining the form of the mean of a stochastic process", J. AftJt:uartafe AnaJ.7, 278. Streit, L. (1996) "Stochastic differential equations. A pedagogical random walk", in Proc. o/f/te Eurocon/erence on JVonHnear R7e:K-GordoK and Scnrddtnger Systems; Tneory and App/tcat:ons, eds. L. Vazquez, L. Streit and V. M . Perez-Garcia (World Scientific, Singapore), pp 87-108. Strook, D. W . and Varadhan, S. R. Srinivasa (1979) AfM^:d:mensto?:a/ D://Ms:on Processes, Springer-Verlag, Berlin. Surdin, M . (1939) "Fluctuations de courant thermionique et le «Flicker Effect^", Le Journa/ de Pnystque et /e Radium, X (7° serie), Avril, 188. Sze, S. M . (1981) Pnystcs of Semiconductor Debtees, John Wiley & Sons, N e w York. Telichevesky, R., Kundert, K., Elfadel, I. and White, J. (1996) "Fast simulation algorithms for R F circuits", in Proc. o/'fne Z E E E i996 Custom integrated CtrcMt^s Con/erence (IEEE, N e w York), pp. 437-444. Ueyama, H. (1980) "Fluctuating hydrodynamic equations of mixed and of chemically reacting gases", J. Stctt. Pnys. 23, 463. Ueyama, H. (1981) "The Langevin-equation approach to dynamics of dense fluids", Progr. Tneor. Pnys. 66, 1926. Vaidyanathan, M . and Pulfrey, D. L. (1997) "Effects of quasi-ballistic base transport on the high-frequency characteristics of bipolar transistors", Z R R E Trans. Electron Deu:ces 44, 618. V a n Roosbroeck, W . (1950) "Theory of flow of electrons and holes in germanium and other semiconductors", Beit System Teen. J. 29, 560. V a n Vliet, K. M . (1971a) "Markov approach to density fluctuations due to transport and scattering. I. Mathematical formalism", J. Afatn. Pnys. 12, 1981. V a n Vliet, K. M . (1971b) "Markov approach to density fluctuations due to transport and scattering. II. Applications", J. Afatn. Pnys. 12, 1998. V a n Vliet, C. M . (1991) "A survey of results and future prospects on quantum 1//noise and l^f noise in general", ,SM:d-.S?a;e Electronics 34, 1. V a n Vliet, C. M . (1994) "Macroscopic and microscopic methods for noise in devices", 7 E E E Trans. .EHecfron Detnces 41, 1902. Vasil'eva, A. B. and Butuzov, V. F. (1985) "The method of boundary layer functions", Dt//erenttai EaKaftons 21, 1107. Vasil'eva, A. B., Butuzov, V. F. and Kalachev, L. V. (1995) Tne Boundary Function Afetnod /or Singular Perturbation ProMems, SLAM, Philadelphia. Vladimirov, V. S. (1984) Fqua^tons o/'Ma^neyna^tca/ Pnysics, Mir, Moscow. Weiss, G. H. (1994) Aspects and Applications of fne .Random IVaiA, North-Holland, Amsterdam. Weitz, D. A., Zhu, J. X., Durian, D. J., Gang, H. and Pine, D. J. (1993) "Diffusing-wave spectroscopy: The technique and some applications", Pnysica Scripta T49, 610. Wiener, N. (1923) "Differential space", J. Afatn. Pnys. 2, 131. Willander, M., Fu, Y., Karlsteen, M., Nur, O., Mamontov, Y., Patel, C. and Olsson, H. (1996) "Silicon based nanoelectronics", Proc. o/tne ^nd 7nf. Con/! on Afass:ue/y ParaiZeZ Computing Systems, 6-9 May, 1996, Ischia, Italy (IEEE Computer Soci-

280

J3:M:ograj3Ay

ety, Washington), p. 271. WiHander, M. and Mamontov, Y. V. (1999) "How to model stochastic noise in millimeterwave analog silicon-based single-chip systems," in Recent Researc/t Developments tn Aftcro^ctue T/teory & 7/eenntqHes, Vo?. i (Transworld Research Network, Trivandrum, India), pp. 101-131. WiHander, M. and Mamontov, Y. V. (2000) "The deterministic circuit model for noise influence on the averaged transient responses of large-scale nonlinear ICs analyzed with Ito's stochastic differential equations", in 7tn /K^erKa^:onat/ WorAsnop oM ContputotftonaJ EZec^roTtMs. Boo& ofAbstracts, 22-25 May 2000, Glasgow, Scotland (University of Glasgow, Glasgow, Scotland ), p. 89. Wilmott, P., Dewynne, J. and Howison, S. (1993) Optton Pr:ctng; Afatnentattca^ Afode/s and ContpM^a^oM, Oxford Financial Press, Oxford. Wong, E. and Hajek, B. (1985) Stocnasttc Processes tn Engineering Systems, SpringerVerlag, N e w York. Yano, K., Ferry, D. K. and Seki, K. (1992) "Molecular-dynamics study of single-electron phenomena—Impact of charge quantization on 1-100 n m Si-MOSFETs", in 7hfernaf:oHa/ Electron Deu:ce Afeettng. Tecnntca^ Dtgest, 13-16 December 1992 (IEEE, N e w York), pp. 557-560. Yates, R. D. and Goodman, D. J. (1999) Pro6aMi?y atMc^ SfocActsftc Processes; A ErtenaYy /M?roa!Mct:on /or E/ectrtcaZ a/td Computer E^gtKeers, John Wiley & Sons, N e w York (with a set of M A T L A B M-files). Yu, Z., Dutton, R. W . and Kiehl, R. A. (1998) "Circuit/device modeling at the quantum level", in ZWCE-6. Extended Abstracts of .Z998 S:xtn /ntemattoyta^ WbrAsnop on CompKtattona? E/ectrontcs, 19-21 October 1998, Osaka, Japan (IEEE), pp. 222229. Zhang, W . (1990) "The spectral density of the nonlinear damping model: Single D O F case", 7EEE Trans. AKtontatte Control 35, 1320. Zhang, W . (1994) "The spectral density of nonlinear damping model: Multi-DOF case", /FEE ?yans. Automate Control 39, 406.

Index

89, 207, 241 stochastic, 91 asymptotically stable quasi-neutral equilibrium point, see eqMi/t&rtuwt potnf autocorrelation function, see par^:cZe automatic-differentiation method, see nte^/too* autonomous I S O D E , see Zfo's s^ocAats(tc O D E average concentration, 183

absolute temperature, 133, 170, 192, 214, 216 value for open space, 184 absolutely continuous probability distribution, see pro&aMtfy cHsfrt&Kfton acceleration, see /^Mta* A D F O software, 264 A D I F O R software, 67 almost-certain validity, see aZ?nos(-SMre uaMity almost-everywhere existence, 14, 15 almost-periodic function, 32, 88 almost-sure validity, 5, 144, 148 almost surely continuous sample function, see sample /Mncf:on analytical method, see TnefAod analytical-numerical method, see metAoG! angular frequency, see /regency approximate co-ordinate, see co-ora*t-

Banach space, 6, 141, 142, 144-148, 157-159, 161, 170, 176 separable, 146 basis of Banach space, see coMM%ct6?e basts o/'BanotcA space beta function, the same as Eu/er m%egra% o/' (Ae /:rs^ ^twa! Bienayme-Chebyshev inequality, 100, 101, 209 biology, 60 bipolar transistor, 194, 195, 202 with thin base, 186 bit, 199 Boltzmann constant, 216 Boltzmann equation, 191, 253 linear stochastic, 144

?tafe

A R W , see asyfnme&*:c ra?tdom wa/^ assembly programming language, 264 A-stable method, see wtefAod asymmetric random walk, see random asymptotic stability in the large, 73,

281

282 nonlinear stochastic, 144, 191 Boltzmann-Enskog equation, 191 nonlinear stochastic, 144, 191 Borel set, 3, 4 sigma-algabra of, 4 boundary condition, 52, 145, 152, 158, 159, 166, 170, 172, 183, 186, 187, 197, 215, 248, 250 ignoring, 131 boundary-function method, see me^nod boundary of domain, see donm:?: boundary-value problem, 215 bounded domain, see Gfomatn Brownian motion, 43, 48, 53, 166, 167, 191, 225 BT, see 6tpo/ar franstsfor Bubnov—Galerkin method, see me^oe^ byte, 199 C compiler, 262, 263 C programming language, 262 capital gain, 29, 30 Cauchy matrix, 78, 102, 103, 242 Cauchy problem, 38, 47, 52, 54, 58, 59, 168, 205 centre-manifold method, see ntet/tod Chapman-Kolmogorov equation, 16,19, 24 characteristic linear size of domain, see domatn characteristic time, 49, 91, 92, 103, 111, 173, 175, 182, 199, 226 Chebyshev-Hermite polynomial, 146 Chebyshev-Laguerre polynomial, 146 chemical-reaction noise, see notse chemical-reaction phenomenon, 166, 247, 249 chemistry, 42, 60, 141 computational, 42 Choleski method, see m e ^ o d circuit, see seyn:co7M%Mc?or classical limit, 184 classical mechanics, see ntecnantcs classification of domains for fluid

transport, 165, 182, 185, 186 classification of types of fluid transport, 176, 177, 182, 183, 185, 186 coefficient of diffusion process, either dr:/i; uecfor or dt/yuszon ma^rtx coherence, 175 coherence function, 36, 49 coherent transport, see ^ranspor? collocation method, see /MefAoa! complex conjugate matrix, see ynafrtx complex system, 38 computer, 160, 164, 198 multiprocessor, 261-263 computer hardware, 198, 261 computer memory, 82, 137, 199 main, 49, 199 virtual, 199 Computer Physics Communications Program Library, 264 computing, 103-105, 150, 160, 199, 201 parallel, 76,106,136-138,160,161, 261-263 message-passing, 160, 262, 263 shared-memory, 160, 263 virtual, 263 concentration, 127, 131-133, 165, 167 -169, 177, 178, 182, 184, 189-191, 193, 215, 219, 221, 247, 248, 252 average, 183 of charged shallow donor atoms, see s/m^oHJ donor attorns conditional expectation, 39 conditional probability density, see prooaMtfy denst^y conduction-current surface density, see CMrrenf conductor, 171 continuous random variable, see rand o m uartaMe continuous-discrete random variable, see random uar:aMe continuum fluid mechanics, see ?necnantcs

index continuum mechanics, see HtecAfmtcs convective acceleration of fluid, see /Zutd conventional frequency, see /reguency convergence, 55, 64, 159 uniform, 55, 64, 65, 68, 69, 77, 78, 90, 91, 93, 98,112,113,124,145, 148, 207, 231, 233, 238, 246 convergent O D E , see or&Mary dt/?erenf:'c^ equafton convolution, 7, 206 cooperative phenomenon, 38 co-ordinate, 146, 148, 149, 168 approximate, 149, 151, 153 correlation coefficient of random variables,27 correlation time, 80 Coulomb law, 179 countable basis of Banach space, 144 -146, 148, 150, 151, 157-161 orthonormal, 146-148, 150, 159 countable set, 7 see also coKftfaMe 6as:s o/'BatnacTt space countable union, 4, 9 covariance, 30, 34, 35, 42, 49, 59, 61, 77, 106-109, 112, 114, 115, 117, 123, 125-127, 129, 132-133, 136, 137, 195, 204, 222, 225, 226, 229 C T compiler, 262, 263 C^* programming language, 262, 263 Cray X-MP/201, 203 cross spectral density, 36 cross variance, 27 C R phenomenon, see cAey?t:co^-reacf ton p/tenomenoyt C - D P D software, 42 C^*MesoDyn software, 164 current, 192, 209 displacement, surface density of, 179 conduction, surface density of, 179 total, surface density of, 179

283 damping function, 111, 112, 117, 122 damping matrix, 77, 81, 99, 111, 117, 118, 120, 121 Darcy velocity, see ue/octfy dB,seea*ectM D B , see defaced M a n c e D D description, see drt/Mt/^sion a*escrtp^ton, de Broglie wavelength, 176 decibel, 210, 211 degenerate semiconductor, see set%:delta-function, see Dtrac cMf ct-/M?:c?toM densiest package of hard spheres in the hard-sphere fluid, see /Zu:d design, 198, 209, 211 detailed balance, 56, 59, 92-95, 220 detailed-balance approximation, 95,97, 125 detectability bound, 211 deterministic quantity, see nonrana'o?M
284

Zndex

199 of velocity, 219 physical dimension of, 39 singular, 109 diffusion parameter, one half of diffusion matrix, see d*tj%sMM fnafrtx diffusion parameter of particle position see parf:c?e diffusion stochastic process, 37 ff, 44 -46, 48, 49, 51, 55, 60, 62, 63, 80, 85, 92, 99, 100-105, 141, 156, 163, 202, 221, 251, 253 high-dimensional, 49, 50, 60-62, 68, 83, 90, 92, 99,100,104-106,108, 114, 126, 134, 142, 156, 161, 192, 195, 204, 264 homogeneous, 40, 58, 219 invariant, 54, 60, 85, 91, 99, 101 -105, 143 stationary, 42, 57, 58, 61, 80, 100, 107-109,117,120,133,134,137, 222, 224, 225, 229 diffusion transport, see fransporf dimension, 1, 9, 25, 38 Dirac delta-function, 6, 7, 18, 19, 22, 24, 25, 30, 47, 206, 229 discontinuous stochastic process, see s?ocAasf:c process discrete model, 167, 169, 171 of quantum mechanics, 169, 182 discrete random variable, see random uartaMe discretization, 151, 152, 159, 160, 197, 198 regular with respect to countable basis of Banach space, 154 displacement-current surface density, see current dissipative phenomenon, 183, 187 dissipative transport, see fransporf dodecahedron (regular), 217 volume of, 217 domain, 9, 143, 153, 158, 166, 172, 177, 187, 197, 213, 247, 250, 251

boundary of, 9, 158, 159, 170, 213, 255 bounded, 47, 100, 127, 143, 167, 175, 176, 183, 247, 251, 255 characteristic linear size of, 172, 184 macroscale, 61, 164, 171, 173, 182, 187, 191 mesoscale, 61, 164, 171, 173, 182, 187, 195, 196 microscale, 138, 164, 171, 175, 182, 184, 187, 195, 196 unbounded, 47, 52, 127, 143, 167, 176, 247, 251 weakly-mesoscale, 138, 188, 191, 196 D O S operating system, 264 dot of fluid, 215 DQ, see cK/?ereMftct/
285

Zndex orthonormal, 257 eigenstructure, 183, 184, 186, 215 eigenvalue, 86, 89, 112, 116, 172, 173, 183, 255, 256, 260 electrodynamics, 170 electromagnetic field, 178 non-quasi-electrostatic, 195 quasi-electrostatic, 178 scalar potential of, 178, 180 vector potential of, 170, 178 electron-electron scattering, see scat^^ertng electron fluid, see /?Mtd electron-hole fluid, see /?Mt<^ electron lifetime, see par(tc?e electron momentum-relaxation time, see re/ctxaf:o?t f:?ne elementary charge, 178, 221 elementary events, 1, 4 space of, 1, 4, 5 emission, 193 empty set, 4, 5 energy, kinetic, seepmr^tc^e potential, see par^tc^e energy-dependent momentum-relaxation time, see re^axaftort ftme energy-independent momentum-relaxation time, see reZa%af:o?t f:?He energy-volume-density conservation equation, 178 engineering, 3, 29, 30, 37, 39, 42, 48 -50, 53, 55, 60-62, 88, 105, 106, 138, 141, 142, 156, 161, 164, 167, 169, 170, 192, 194-199, 201, 202, 207, 209-211, 261, 263, 264 equilibrium point, 33, 238, 240 neutral, 240 quasi-neutral, 240, 241 asymptotically stable, 73, 241 regular, 238 equilibrium probability density, see pro6a6t/:fy c?eHstt;y equivalent input noise, 211

error, 75, 96, 103, 202, 204, 227 projection, 149 truncation, 66, 149 Ethernet network, Euclidean space, 1, 4, 203 Euler integral of the first kind, 122 Euler method, see wtefAod expectation, 10, 12, 13, 25, 26, 28-30, 49, 60, 63, 64, 68, 71, 72, 76, 81, 85, 101, 104, 188, 204, 209-211, 220, 254 uniformly bounded, 29, 32, 86, 87, 107, 121, 136 explosion of solution, 45 external control variable, 199 fast motion, 92, 119, 173 Fermi-Dirac function, 218 Fermi velocity, see ue^octfy F D , see /mtfe dt/yerertce Fichera drift vector, see c^rt/? uecfor figure of merit, 48-50, 107, 204, 209, 211 final condition, 47 finite-difference method, see fnefAoG! finite domain, see domatw finite-element method, see ?nefAo<% finite-equation method, see me^Aoc^ first-order O D E , see ora!mBry c^t^eren?ta/ gHct^toy:

flicker effect, see /Ztc^er no:se flicker noise, see ftotse fluid, 166, 176, 221 acceleration of, 179, 184 convective, 179 instantaneous, 179 total, 179 dispersed, 132, 133 electron, 163, 177, 179, 184, 191, 218 pressure in, 178 electron-hole, 180, 247 hard-sphere, 61, 130, 217, 229 densiest package of hard spheres,

286 217 pressure in, 217 volume fraction of hard spheres, 217 in the densiest package, 217 hole, 180, 184, 191, 218 pressure in, inviscid, 61, 213 isothermal, 61, 213, 214 isotropic, 61, 168, 213, 214 non-uniform, 180 simple, 128,130, 132,133,138, 213, 215, 227, 229 uniform, 43, 61, 213, 247, 252 fluid dynamics, 161, 168, 180, 186, 187, 192 fluid physics, see pnystcs fluid transport, see ^rct/:spo7*^ Fokker-Planck equation, see Kb^mogcr<w /brH;ara! e(?Maf:on force, 43, 118 Lorentz, 178 formal adjoint differential operator, 51 F O R T R A N compiler, 262, 263 F O R T R A N programming language, 262, 264 Fourier coefficient, 146 Fourier series, 146 Fourier transform, 35, 131 inverse, 35 frequency, 131 angular, 34, 175 conventional, 34 effective angular, 172,175,182-184, 186 plasma, of fluid, 181 friction, 43, 219 linear, 213, 219 nonlinear, 127, 138, 190, 194, 213, 219, 228 Fubini theorem, 13, 14 fundamental solution, 52, 58, 59, 168

Zndex Galerkin-type method, see me^Aocf g a m m a function, 122, 228 Gauss formula, 236 Gaussian probability density, see prooaot/t(y denstty Gaussian random variable, see random uartaMe Gaussian white noise, see notse generalized function, 6 generalized random field, see random /teM generalized stochastic process, see sfocnasf:c process generation-recombination noise, see notse generation-recombination phenomenon, 166, 249 G R phenomenon, see^enera^ton-recom&cnaf:o?: phenomenon gradient, 256, 257 gradient mapping, 94, 135 Green formula, 235 Green-Kubo formula, 226 Hamilton operator, 46, 178 hard sphere, 217 radius of, 217 volume of, 217 hard-sphere fluid, see /ZKtd hardware, see computer nardmare harmonic, 73, 174 Hermitian matrix, see mct^rM hidden randomness, 182, 213, 247, 248, 252, 253 high-dimensional diffusion stochastic process, see GH/yHston sfocnas^tc process high-dimensional stochastic process, see s^ocnats^tc process Hi-Fi, see n:gn-/tde^t%y system high-fidelity system, 211 high-order O D E , see ordinary d:^*erenfta/
Zndex hole fluid, see /ZM:d homogeneous diffusion process, see cH/^ /MSMM s^oc/tos^tc process homogeneous Markov process, see Markov s^oc/tas^tc process homogeneous transition probability density, see pro6a6t/tfy c%enst?y HyperChem 6 software, 42 Hypercube, Inc., 42 IBM, see Mor&sfafton identity matrix, see wtafrtx imaginary unit, 34, 117, 183 impossible random event, see randowt euenf incoherent transport, see (rawsporf initial condition, 47, 70, 78, 80, 86, 101, 126, 242, 246, 247, 250, 251 for Markov process, 16, 20, 41, 52, 54, 58, 77, 81, 205, 206 initial probability density, see prooctMt(y dens:(y initial random variable, see random uartaMe initial time point, 21, 22, 104 initial value, 22 initial-value problem, 40, 41, 45, 66, 68, 70, 74, 80, 81, 85,103,104,123, 128, 129, 133, 137, 242, 245 instantaneous acceleration of fluid, see /ituta" insulator, 171 integral manifold, 88 interior of set, 8 internal phenomenon, 199 invariant diffusion process, see tft^uston sfocnasftc process invariant manifold, 88 invariant Markov process, see MarAou sfocAasftc process invariant probability density, see prooaMtfy deKs^y inviscid fluid, see /ZMta* ISDE, see Zfo's sfocActsfK: a*:/yereftfta?

287 equation isothermal fluid, see /!Mtd isotropic fluid, see /Zuto! I S O D E , see A6's sfocAas?:c O D E ISPDE, see 7t6's s(ocnas;;c P D E ISPIDE, see Ao's sfoc/tasftc P Z D E Ito's S D E , 23, 39, 41, 141-144 Ito's stochastic O D E , 40, 41, 43-46, 48, 49, 63, 68, 81, 91,101,141-144, 148, 155, 156, 158, 159, 161, 192, 194, 204, 219 autonomous, 41, 118, 120, 134, 210 linear in the narrow sense, 204 solution as diffusion process, 44 Ito's stochastic P D E , 16, 61, 141-143, 152, 159, 161, 163-165, 177, 197 Ito's stochastic PIDE, 16, 61,142-144, 148, 149, 152, 156, 159, 161, 164, 197 joint probability density, see prooato:/:ty denstfy junction field-effect transistor, 180 Karhunen-Loeve expansion, 157 K B E , see .Kb/nwgorou oac&MMtrd egMafton K F E , see Kb^ntogorou /orchard e
288 large-scale stochastic process, see A:gA-d:7nenstofKi/ s?oc7tasftc process Lebesgue-integrable function, 13, 14 Lebesgue-integrable probability density, see proAa6:Jt(y <^ews:^y Lebesgue integral, 5, 14 Legendre polynomial, 146 lifetime, see par?:c/e linear-growth condition, 45 linear operator, 170, 172, 176, 183, 235, 255 identity, 172, 186 non-negative definite, 256 self-adjoint, 256 Unpack benchmark, 203 logical address space, 198 long tail, see Mo?t-ex.poMe?t?ta^ asympfo?:c &eAautor o^parftc/e-ue^oc:fy co^ar:aKce Lorentz force, 178 volume density of, 178 Los Alamos National Laboratory, 262 low-signal-to-noise-ratio effect, 30, 211 L-stabe method, see me^Aoc^ Luzin criterion, 4 Lyapunov function, 89 macroscale domain, see a!oma:n macroscopic description, 43, 44, 164, 165, 167, 170, 171, 176, 177, 181, 185, 186, 196 macroscopic phenomenon, 171 macroscopic variable, 43 main m e m o r y of computer, see contpufer memory marginal probability density, see pro6
Zndex periodic, 32, 34 quasi-periodic, 32 right-continuous in probability distribution, 17 random-sign, 10 random-telegraph-signal, 10, 42 stationary, 30, 32, 34, 36, 210 mass, see pctr;:cZe mass-volume-density conservation equation, 178 matrix, 1, 154, 157, 159, 160 complex conjugate, 36 diagonal (principal) of, 222, 225 Hermitian, 36 identity, 117, 214 non-negative definite, 38, 74, 76, 242, 245 nonsingular, 75, 86, 105, 117, 118, 124, 135, 242 positive definite, 76, 79, 116, 117, 245 skew-symmetric, 36, 94, 95 sparse, 159, 160 symmetric, 36, 38, 74, 76, 86, 94, 96, 98, 102, 118 trace of, 29, 69 uniformly stable, 86—88 matrix norm, see norm Maxwellian probability density, see pro6a6:%:?y c!enst(y mean, see expec?a(:oM m e a n free path, 186 measure-theoretical treatment, 2, 3 mechanics, 60 classical, 184 continuum, 165, 167, 170, 175 fluid, 177, 181, 252 quantum, 163, 164, 168, 169, 175, 184, 196 statistical, 2, 53,127,131,132,156, 157,164,192, 194, 214, 221, 224, 225, 248, 252, 253 medicine, 60, 109 megabyte, 199

/ndex m e m o r y function, 16, 132 mesoscale domain, see dontatn mesoscopic description, 43, 167, 169, 175, 176, 180, 182 mesoscopic phenomena, 44, 176, 186, 196 message-passing parallel computing, see compMftng metal, 219 metal-oxide-semiconductor field-effect transistor, 138, 169, 186 method, 188, 192 analytical, 50, 59, 60, 62, 87, 105, 107,132,133,137,197, 201-203, 229 analytical-numerical, 50, 60, 62, 99, 105,106,125,134,136-138,160, 201, 202, 264 A-stable, 66 automatic-differentiation, 67 boundary-function, 92 Bubnov-Galerkin, 145, 159 centre-manifold, 88 Choleski, 76 collocation, 151 stochastic, 151, 153, 154 differential-quadrature, 153, 157, 159, 160 finite-difference, 67, 105, 159-161, 203 finite-element, 101, 160, 161, 203 finite-equation, 72, 86-88, 104 Galerkin-type, 145 Lagrangian probability-density-function, 190, 253 L-stable, 66 Monte Carlo, 48,102-104,136,137, 160, 264 Newton-Raphson, 87 numerical, 50, 91, 101, 132, 197, 201-203 semi-explicit Euler, 66, 74 small-parameter, 92, 108 stochastic adaptive interpolation,

289 61, 142, 153, 156-161, 197 stochastic-collocation, 151, 153, 154 microelectronics, 109 microplasma noise, see notse microscopic description, 44, 164, 165, 167, 170, 171, 177 microscopic phenomenon, 43, 44, 165, 166,171, 176, 177, 186, 195, 196 microscopic random walk, see random molecular-dynamics simulation, 131, 164, 184, 195 Molecular Simulation, Inc., 42, 164 momentum, 118, 185 momentum-relaxation time, see re^oucatton f:?ne m o m e n t u m volume density, 178 conservation equation of, 178, 214 Monte Carlo method, see mef/tod* Mori model, 16 M O S F E T , see Hte(a%-oxtcfe-sey?ttcona*MCfor /teM-e/^ecf franstsfor multidimensional Markov stochastic process, see MarAou s^oc/tots^tc process multidimensional stochastic process, see sfoc/tasf tc process multifold integral, 102, 160, 264 multiparticle system, 164-166, 175, 176, 186, 196, 197, 251 multiple analysis, 49, 50, 150, 197 multiprocessor computer, see compKfer nanometer, 184 neutral equilibrium point, see eqwZtortMTn potn? Newton (second) law, 214 Newton-Raphson method, see nte(7to<% n m , see ytanomefer noise, 192, 194, 209, 211 chemical-reaction, 247, 249, 252 flicker, 80, 111, 112, 134, 138, 192 -194 generation-recombination, 193,194,

290 247, 249, 252 in semiconductor, 61, 161, 204 microplasma, 194 ]/f, see /ZtcAer notse shot, 193, 194 spot, 37 thermal, 192-194 white (Gaussian), 25, 42, 190, 191, 225 noise factor, 211 noise figure, 211 noise-induced phenomena, 60, 64, 71, 161 noise source, 189-191, 195, 252, 254 noise temperature, 211 non-anticipative two-point stochastic process, see s^ocAcs^c process nondegenerate semiconductor, see sef7McondKC?or nondiffusion stochastic process, see sfocAasftc process non-exponential asymptotic behavior of particle-velocity covariance, 59, 61, 80, 131-133 non-Hermitian component, 176 non-Markov stochastic process, see s?ocActsftc process non-negative definite matrix, see ma^rtx nonparabohc potential energy, see energy nonperiodic function, 88 non-quasi-electrostatic electromagnetic field, see e/ecfrowtagneftc /teM nonrandom quantity, 7, 12, 22, 30, 45, 48, 156, 191, 193, 206, 208-210, 219, 229 nonsingular matrix, see mafrtx nonstationary K F E , see .KbJfnogorou /brMJara* e^uatton norm, 29, 86 of Banach space, 145, 146, 256 of matrix, 124 of vector, 6, 145-147, 256

/na'ex norm-coercive function, 86 normalized kinetic energy of random motion of particle, see par^:c/e n-type semiconductor, see semtconc?Mcfor number of particles, 167, 248, 249, 251 number volume density, the same as concenfrafton numerical method, see yne^o(f O D E , see orcHnctry a't/^renfta/ egHaftOM

one-point stochastic process, see sfoc/mst:c process open-space temperature, see a6sofufe fernperafKre operator, see /wear operator ordinary differential equation, 23, 40, 41, 43, 60, 61, 64, 69-73, 76, 78, 80 -82, 86-89, 99, 101, 204-205, 207, 239, 240-242, 244, 245 convergent, 86, 207 of the first order, 66, 70, 76, 126, 128, 129 of the high order, 60, 70, 71 of the second order, 60, 69, 70, 76, 173 Ornstein-Uhlenbeck stochastic process, 31, 117, 222 orthogonal eigenfunctions, see e:gen/Mncfton orthonormal basis of Banach space, see coun?a6Ze 6otsis o/*BanacA space orthonormal eigenfunctions, see e:gen/u?tcfton Ostrogradski formula, 236 parabolic potential energy, see energy Paradise software, 263 parallel computing, see compHttng Parallel Virual Machine software, 160, 262, 263 partial differential equation, 38, 41, 53, 91, 92, 97, 142, 166-168, 170,

Zndex 177, 179, 183, 184, 186, 196, 203, 204,215 partial integro-differential equation, 41 particle, 43, 118, 219 kinetic energy of purely random motion, 128, 227 normalized, 126, 128, 138, 188, 214, 215, 220, 222, 227, 228 lifetime of, 180 mass of, 118, 214 m o m e n t u m of, 43, 184, 185, 253 position of, 166, 185, 194, 214, 247, 251, 254 diffusion parameter of, 221, 225, 226 relaxation time of, 182, 185 potential energy of, 118, 122 nonparabolic, 120, 122 parabolic, 119, 120, 122 velocity, 253 velocity of purely random motion, 59, 81, 118, 119, 126, 127, 131, 134, 137, 188, 190, 213, 214, 222, 225, 254 autocorrelation function of, 226 particle-particle scattering, see scaMert"F particle-phonon scattering, see scaMertng particle-position probability density, see pro6aMtty deKst^y particle-wave duality, 168 PC, see personal contpHfer P D E , see parttctZ cH/y'ereMfta/ eqKaftoft Pentium II processor, 203 periodic function, 32, 88 permittivity of vacuum, 179 persistent random walk, the same as asymwe^rtc raHG?o7M HJa% personal computer (IBM compatible), 261-263 P H O E N I X software, 158 physics, 60, 61, 137, 141, 163, 169

291 fluid, 108 theoretical, 157, 176, 215, 216 Planck constant, 183, 218 plasma frequency of fluid, see /regue?tcy p-n semiconductor junction, 180, 197 Polyhedron Software, Ltd., 264 porous medium, 180 position of particle, see parf tc^e positive definite matrix, see matr:x potential barrier, 166, 193 potential energy of particle, see par(tcJe power spectrum, see spec?ra? (feMstfy preceding probability density, see pro6aMtZy deHstfy preceding time point, 11, 17, 26, 32, 47 pressure in fluid, 170, 178, 185, 216 see also /ZM:d for the pressures in specific fluids principal diagonal of matrix, see wm?r:x principal-eigenmode analysis, 173 probability, 5, 7, 101, 209, 210 reflection, 166, 169 transmission, 166, 169 probability current, seepro6aM:fy /Zux probability density, 2, 6-8, 10 ff., 46, 51-56, 58, 59, 63, 64, 68, 69, 82, 98, 156, 222, 252 conditional, 18 equilibrium, 220, 221, 224 Gaussian, 25, 68, 81, 99, 220, 221 initial, 22-24, 30, 39, 54, 58, 68, 81, 104, 205, 206, 224 invariant, 31, 51, 54-57, 59, 60, 64, 82, 83, 85, 90, 91, 96, 106, 107 joint, 8, 12, 25, 205 Lebesgue integrable, 6, 223 marginal, 16, 17, 253 Maxwellian, 221 of particle position, 166, 249, 250, 253 preceding, 16, 17, 19, 20, 27, 32

292

7ndex

stationary, 33, 51, 57-59, 100, 101, 120,134, 220, 221, 223, 224, 228, 229 steady-state, 55, 82 subsequent, 16, 17, 19, 21, 27, 28, 32, 206, 207 transition, 16, 18, 19, 20, 23, 24, 30, 31, 39, 40, 46, 51, 205-207 homogeneous, 33, 47, 57, 59,107, 108, 114, 125 unbounded, 59, 213, 228, 229 probability distribution, 3, 5 absolutely continuous, 5 singular, 5 probability flux, 53, 56, 91 probability theory, 1, 2, 209 probability velocity, 91, 97 progressive wave, see ?raueM:ng MMZue projection, 148, 149 projection error, see error p-type semiconductor, see senttconc^MCfor purely random motion, see potrfM^e P V M , see Parct/M Vtrfuct/ AfacAtne so/i!M7are Q H D , see
quasi-neutral semiconductor, see se??Mcon<%Mc(or quasi-periodic function, 32, 88 radius of hard sphere, see Aarcf spAere random event, 1, 3-5, 209 impossible, 5 of zero probability, 5 sure, 5 random event of zero probability, see rotyn^om euen? random field, 3, 8, 71, 214, 215, 253, 254 generalized, 190 random-number generator, 48, 264 random-sign stochastic process, see AfctrAtou sfoc/tctsftc process random variable, 1 ff., 11, 12 continuous, 6, 20 continuous-discrete, 7, 20 discrete, 7, 20 Gaussian, 25 initial, 21, 22, 205, 206 value of, 21, 22, 205, 206 random telegraph signal, see Afar^ou sfocAasftc process random walk, 10, 25, 166, 169 asymmetric, 165,167-170,182,187, 196 microscopic, 165 persistent, the same as ctsymMtefWc ra?M&)?H HJa%&

symmetric, 166 R A N L U X software, 264 R A N L U X A S M software, 264 R A N L U X I software, 264 rate of change in concentration due to C R / G R phenomena, 178 real-time signal processing, 199 recombination-generation phenomenon, 178, 247 rectangular parallelepiped, 4,100,103, 135, 172, 255 reflection probability, see pro6a6tZ:fy

Judex regular discretization, see a*:scre?tza?:on regular equilibrium point, see e
regular function with respect to Markov process, 27, 35, 115 regular function with respect to nonanticipative two-point process, 13 relative permittivity of semiconductor, 179 relaxation time, 175 m o m e n t u m , 118,128,129,132-134, 214, 215, 223, 224, 226 effective, 129, 168, 191, 226 electron, 178 energy-dependent, 127, 138, 190, 222, 226, 227 energy-independent, 190, 220 particle-particle, 215 of particle position, see pctrf:c^e resistance, 180 resistor, 193 resonance, 175 stochastic, see sfocAasftc resonance Ridley formula, 227, 228 right-continuous stochastic process, see Mctr&ou sfocAasf:c process R M S , see roof-7%ean-s
293 227 particle-phonon, 138 scattering act, 166, 169 scattering mechanism, 132, 133, 138, 215, 216 Schauder basis of Banach space, see coMn&zMe &ctsts o/BanacA space Schrodinger equation, 163, 164, 176, 183, 184 Scientific Computing Associates, 263 Scilab// software, 263 S D B , see stn:p/:^e^ a'etct^eo' &a/aftce S D E , see sfocAasftc dt/yereMftaJ egHa?:on second-order O D E , see ora*:nary dt/?erenftaJ (?Maf:on semiconductor, 42, 59, 164, 170, 177, 182, 186, 190-192, 195, 196, 247, 253 circuit, 42, 158, 164, 187, 194, 196, 204, 209 degenerate, 219 device, 138, 164, 180, 195, 196, 202 nondegenerate, 227 n-type, 165, 177, 178 p-type, 180 quasi-neutral, 181 semiconductor fluid, see e/ec?ron /?M:d or Ao/e /Zu:d or e/ecfron-Ao/e /Zntd semiconductor-fluid ISPDE, 187, 189 -196 semiconductor noise, see notse semiconductor system, 165, 187, 193 semiconductor theory, 185, 227 semi-explicit Euler method, see wte?Aod separable Banach space, see BanacA space SF-ISPDE, see sem:condMcfor-/ZM:d 7SPDE shallow donor atoms, 178 charged, concentration of, 179 shared-memory parallel computing, see cofnpM^tng

294 shot noise, see no:se sigma-algebra of Borel sets, see Bore/ sets signal-to-noise ratio, 29, 49, 72, 195, 209-211 silicon, 184 SimExel Corporation, 158 simplified detailed balance approximation, 90, 96-99, 106, 134, 135 simulator, 198 sine-function, 147 singular perturbation, 92 singular probability distribution, see prooot6:/tty d:sfr:oKf:on skew-symmetric matrix, see ywctfrtx slow motion, 119, 173 small-parameter method, see mef/Soc? small-size effect, 184 smoothness, 52, 53 S M P , see syntf?tefr:c yMu/ttprocesst?:^ sociology, 60, 109 S N R , see stgnct/-fo-Motse rcttto sound in fluid, 43; see also ue/octty o/ sound MMtues m /Zu:oJ space of elementary events, see e/entenfotry euents space-discretization regular with respect to countable basis of Banach space, see aKscretizctf:o?: sparse matrix, see watrtx spectral density, 30, 34-36, 42, 49, 59, 61, 81, 106-108, 117, 118, 121,125, 127, 133, 134, 136-138, 195, 222, 226 spherical function, 147 spiraling, 185 spot noise, see notse standard deviation, 25, 26, 129, 193, 209, 229 state space, 9, 82, 157, 204 state variable, 157 stationary diffusion process, see aK/^uston sfoc/msftc process stationary function, 32

ZncZex stationary Kolmogorov forward equation, see .Ko/rMOgorou /brM^are! eq*Mctf:on stationary Markov process, see MarAou sfoc/tasttc process stationary probability density, seeprooa6:/i?y denstfy statistical data processing, 49 statistical mechanics, see ntec/tantcs steady-state approximation, 85 steady-state probability density, see prooot6:/tty deMstfy steady-state solution of O D E , see K?M/brfn/y o o M n c M so/Mf:on o/ O D E stochastic adaptive interpolation m e thod, see ntef/toGf stochastic asymptotic stability in the large, see asyfnptottc sfaMtty :n tfte /arge stochastic Boltzmann equation, see Bo^tzmanyt eguattoK stochastic Boltzmann-Enskog equation, see BoZtzmaww-Ews^og e^Mattow stochastic chaos, 60, 64, 71, 73 stochastic-collocation method, see /netAooJ stochastic differential equation, 45, 63, 132, 251, 253 in the generalized sense, 190 stochastic linearization, 60, 64, 71, 73 stochastic O D E , 153 stochastic noise source, see notse source stochastic P D E , 191, 192 stochastic phase transition, 60, 64, 71, 73 stochastic process, 2, 3, 8, 9, 71, 73, 148, 209, 253, 254 discontinuous, 45 generalized, 25, 41, 42 high-dimensional, 3, 9 large-scale, the same as /MgA-cHnteHsiofm/ s^ocAastic process Markov, see Afar&ou stocAast:c pro-

Judex cess multidimensional, 9, 197, 209 nondiffusion, 45 non-Markov, 15, 16, 132, 142, 156, 161 one-point, 10 two-point, 10, 12 non-anticipative, 12-14 Wiener, see IVtener s?ocnasf:c process stochastic resonance, 60, 64, 71-73, 211 stochastic self-oscillations, 60, 64, 71 stochastic source, 166 stochastically dependent random variables, 8, 9, 11, 142 stochastically independent random variables, 1, 8, 10, 11, 25, 27 subsequent probability density, see pro&aMtty denstfy subsequent time point, 11, 17, 21, 26, 48 supercomputer, 203, 261-263 sure random event, see random euenf swap space on a hard drive, 199 symmetric matrix, see mafrM; symmetric multiprocessing, 263 symmetric random walk, see random HJa/&

synergetics, 38 Taylor formula, 94, 95 telegraph equation, the same as wauedt/^ston eaMafton temperature, see a&soZHfe temperature theoretical physics, see pAystcs thermal noise, see notse thermodynamic derivation, 193 time separation, 23, 35, 109, 112, 120 total acceleration of fluid, see /?Ktd total-current surface density, see current trace of matrix, see ma^rtx trajectory, 185

295 equation, 178, 179 transition probability density, see prooa6d:fy density transmission probability, see prooao:/:(y transport (of fluid), 169, 170, 176, 187 coherent, 174-176, 182, 187, 195 diffusion, 174, 182, 184, 187 dissipative, 183, 184 incoherent, 174, 175, 182, 187 quantum, 186, 187 quantum-like, 183 quasi-diffusion, 188 wave, 175, 182-185, 187, 196 wave-diffusion, 134, 173, 182, 187, 196 without scattering, 183 travelling wave, 176 truncated equation, 91, 92, 97 truncation error, see error two-point stochastic process, see s;oc/tctsf tc process U M approximation, see uncorrected ma^rtxes approximation U M condition, see uncorrected matrixes condition unbounded domain, see dontatn unbounded probability density, see pro6a6:^tfy density uncorrected random variables, 27, 79, 115, 116 uncorrelated-matrixes approximation, 77, 79, 81 uncorrelated-matrixes condition, 79, 108, 114-117, 123, 137 uniform convergence, see convergence uniformly bounded solution of O D E , 34, 86, 87, 89, 116, 207, 208 uniformly stable matrix, see mafrtx Universe, 176, 194 validity with probability 1, the same as a/mos^-sure ua/idt^y

296

/ndex

value of initial random variable, see rctntfoyn uotr:aMe variance, 26, 28, 29, 34, 49, 60, 63, 64, 76, 80, 81, 85, 101, 102, 104, 204, 209, 211, 221, 224, 245 uniformly bounded, 29, 32, 86, 88, 89, 99, 107, 116, 136, 192 vector norm, see 7tor?n vector potential of electromagnetic field, see eJecfromagneftc /:eM velocity, 168, 170, 190, 220 Darcy, 180, 185, 188, 190, 193 equation, 186 Fermi, 169, 218 of fluid, 178, 180, 185 of purely random motion of particle, see pa?itc/e of sound waves in fluid, 43, 133, 168, 188, 216 velocity-diffusion matrix, see dt/^uston ma^rtx velocity fluctuations, 189, 192-194, 254 velocity of purely random motion of particle, see parf tc/e virtual memory of computer, see ynemoTy virtual-shared-memory parallel computing, see cowpMfmg viscosity, 192, 214 voltage, 209 volume density of Lorentz's force, see Lorenfz /brce volume fraction of hard spheres in the hard-sphere fluid, see Ttarcf-spAere /ZH:d vortex, 186 V S M , see utrfKaJ-sAared-Tnernory parage/ compM^mg W A T C O M compiler, 262, 264 wave-diffusion equation, 167-171,173, 175, 177, 180, 182, 185-187 wave-diffusion transport, see frcftsporf

wave equation, 168, 171, 175, 183, 184 wave transport, see ?ransporf wave vector, 131, 176 weakly-mesoscale domain, see 6?omct:n white noise (Gaussian), see notse Whittaker cardinal function, 147 Wiener stochastic process, 23-25, 40, 48, 58, 142, 213 physical dimension of, 25 Windows 95, 42, 261-264 Windows 98, 263 Windows NT, 42, 263 Windows operating system, 261, 263, 264 Windows 2000, 42 workstation, 49, 198, 199 IBM, 42, 164 SGI, 42, 164 Corollary 1.1, 47, 48, 51 Corollary 1.2, 58, 59, 107, 220, 221, 223, 224 Corollary 2.1, 79 Definition 1.1, 10 Definition 1.2, 10, 11, 15 Definition 1.3, 12 Definition 1.4, 12, 14 Definition 1.5, 13, 14, 27, 109, 110 Definition 1.6, 14, 31 Definition 1.7, 18 Definition 1.8, 22, 30 Definition 1.9, 23, 39 Definition 1.10, 24, 40, 58 Definition 1.11, 30, 51, 54, 82, 85 Definition 1.12, 33, 51, 107 Definition 1.13, 37, 39, 46 Definition 3.1, 86, 89 Definition 5.1, 154 Definition 6.1, 172, 173, 175, 176 L e m m a 1.1, 27, 28, 31, 32, 67, 77, 78 L e m m a 1.2, 28, 31, 64, 231, 232, 234 L e m m a 1.3, 28, 31, 232, 234

JMex Lemma 1.4, Lemma 1.5, Lemma 1.6, Lemma 1.7, Lemma 1.8, Lemma 2.1, Lemma 4.1, Lemma 4.2,

31-33, 85 32, 33, 85 33, 34, 109, 244 33, 35, 109, 124 35 77, 78, 241 112, 113, 243, 244 114, 244

Remark 1.1, 7 Remark 1.2, 8, 11, 12 Remark 1.3, 11, 15 Remark 1.4, 13 Remark 1.5, 14, Remark 1.6, 15, 132, 156 Remark 1.7, 17 Remark 1.8, 22 Remark 1.9, 25 Remark 1.10, 28, 71 Remark 1.11, 35 Remark 1.12, 42 Remark 1.13, 43, 58, 59, 225 Remark 1.14, 43 Remark 1.15, 45 Remark 1.16, 53, 90 Remark 1.17, 55, 56, 59, 82 Remark 1.18, 53, 58, 220 Remark 2.1, 63 Remark 2.2, 66 Remark 2.3, 70 Remark 3.1, 86, 89 Remark 4.1, 134 Remark 4.2, 127, 128, 228 Remark 5.1, 149

297 Remark 5.2, Remark 5.3, Remark 5.4, Remark 5.5, Remark 6.1, Remark 6.2, Remark 6.3, Remark 6.4, Remark 6.5, Remark 6.6, Remark 6.7, Remark 6.8, Remark 6.9, Remark C.l, Remark C.2, Remark C.3,

149, 150 156 157 166, 167, 169, 171 172, 169, 181 183 185, 216, 223, 168

150

167, 170 169 175 175 176

195 228 224, 228

Theorem 1.1, 46, 47, 51 Theorem 1.2, 51, 53, 56, 57, 64, 65, 68, 238 Theorem 1.3, 57, 107, 109, 113, 114 Theorem 2.1, 63-66, 68, 77, 106, 231, 232, 234, 235, 237 Theorem 2.2, 63, 65, 66, 78, 106, 232 Theorem 2.3, 68-70, 237 Theorem 2.4, 77-81, 88, 93, 106, 115, 241, 244, 245 Theorem 3.1, 86, 88-90, 93, 98 Theorem 3.2, 98, 99, 106 Thoerem 4.1, 109, 111, 112, 124, 243, 244 Theorem 4.2, 112-115, 243 Theorem 4.3, 115, 117, 122, 123, 244 Theorem 4.4, 124-126, 132, 137, 245

Series on Advances in Mathematics for Appiied Sciences Editoria! Board N. B e H o m o &#for-/n-Cr]arge Department of Mathematics Politecnico di Torino Corso Duca degli Abruzzi 24 10129 Torino Italy E-mail: [email protected]

M. A. J. Chaptain Department of Mathematics University of Dundee Dundee DD1 4 H N Scotland C. M. Dafermos Lefschetz Center for Dynamical Systems Brown University Providence, Rl 02912 USA S. Kawashima Department of Applied Sciences Engineering Faculty Kyushu University 36 Fukuoka 812 Japan M. Lachowicz Department of Mathematics University of Warsaw Ul. Banacha 2 PL-02097 Warsaw Poland S. Lenhart Mathematics Department University of Tennessee KnoxviHe, T N 37996-1300 USA P. L. Lions University Paris Xl-Dauphine Place du Marechal de Lattre de Tassigny Paris Cedex 16 France

F. Brezzi R#for-/rt-Crtarcye Istituto di Analisi Numerica del C N R Via Abbiategrasso 209 1-27100 Pavia Italy E-mail: [email protected]

B. Perthame Laboratoire d'Analyse Numerique University Paris VI tour 55-65, 5ieme etage 4, place Jussieu 75252 Paris Cedex 5 France K. R. Rajagopal Department of Mechanical Engrg. Texas A & M University College Station, T X 77843-3123 USA R. Russo Dipartimento di Matematica Universita degli Studi Napoli II 81100 Caserta Italy V. A. Sotonnikov Institute of Academy of Sciences St. Petersburg Branch of V. A. Steklov Mathematical Fontanka 27 St. Petersburg Russia J. C. Willems Mathematics & Physics Faculty University of Groningen P. O. Box 800 9700 Av. Groningen The Netherlands

Series on Advances in Mathematics for Appiied Sciences A i m s and S c o p e This Series reports on n e w developments in mathematical research relating to methods, qualitative and numerical analysis, mathematical modeling in the applied and the technological sciences. Contributions related to constitutive theories, fluid dynamics, kinetic and transport theories, solid mechanics, system theory and mathematical methods for the applications are welcomed. This Series includes books, lecture notes, proceedings, collections of research papers. Monograph collections on specialized topics of current interest are particularly encouraged. Both the proceedings and monograph collections will generally be edited by a Guest editor. High quality, novelty of the content and potential for the applications to modern problems in applied science will be the guidelines for the selection of the content of this series.

instructions for Authors Submission of proposals should be addressed to the editors-in-charge or to any m e m b e r of the editorial board. In the latter, the authors should also notify the proposal to one of the editors-in-charge. Acceptance of books and lecture notes will generally be based on the description of the general content and scope of the book or lecture notes as well as on sample of the parts judged to be more significantly by the authors. Acceptance of proceedings wiil be based on relevance of the topics and of the lecturers contributing to the volume. Acceptance of monograph collections will be based on relevance of the subject and of the authors contributing to the volume. Authors are urged, in order to avoid re-typing, not to begin the final preparation of the text until they received the publisher's guidelines. They will receive from World Scientific the instructions for preparing camera-ready manuscript.

SER!ES O N A D V A N C E S !N M A T H E M A U C S F O R APPLIED S O E N C E S Vo). 17 The Fokker-Planck Equation for Stochastic Dynamical Systems and Its Explicit Steady State Solution by C. So/ze Vol. 18 Calculus of Variation, Homogenization and Continuum Mechanics eds. G. Boucn/ffe ef a/. Vol. 19 A Concise Guide to Semigroups and Evolution Equations by A Be//en;-Moranfe Vol. 20 Global Controllability and Stabilization of Nonlinear Systems by S. M M / n Vol. 21 High Accuracy Non-Centered Compact Difference Schemes for Fluid Dynamics Applications by A /. Tb/sfyMi Vol. 22 Advances in Kinetic Theory and Computing: Selected Papers ed. B. Peffhame Vol. 23 Waves and Stability in Continuous Media eds. S. R/onero and 7*. Rugger/ Vol. 24 Impulsive Differential Equations with a Small Parameter by D. Ba/noy and y. Coyacney Vol. 25 Mathematical Models and Methods of Localized Interaction Theory by A /. Bun/moy/cn and A M. Dub/ns/f// Vol. 26 Recent Advances in Elasticity, Viscoelasticity and Inelasticity ed. K. R. Ra/agopa/ Vol. 27 Nonstandard Methods for Stochastic Fluid Mechanics by M Cap/nsM and M. J. Cuf/and Vol. 28 Impulsive Differential Equations: Asymptotic Properties of the Solutions by D. Ba/noy and P. S/meonoy Vol. 29 The Method of Maximum Entropy by M. Gzy/ Vol. 30 Lectures on Probability and Second Order Random Fields byD. B. Hernandez Vol. 31 Parallel and Distributed Signal and Image Integration Problems eds. R. /V. Madan ef a/. Vol. 32 O n the W a y to Understanding The Time Phenomenon: The Constructions of Time in Natural Science - Part 1. Interdisciplinary Time Studies ed. A P. Ley/cn

SERtES O N A D V A N C E S !N M A T H E M A T t C S F O R APPLiED S C i E N C E S

Vol. 33

Lecture Notes on the Mathematical Theory of the Boltzmann Equation ed /V. Be//omo

Vol. 34

Singularly Perturbed Evolution Equations with Applications to Kinetic Theory by J. R. M/Aa and J. Banas/aA

Vol. 35

Mechanics of Mixtures by K. R. Ra/agopa/ and L 7ao

Vol. 36

Dynamical Mechanical Systems Under Random Impulses by R. /wanA/etv/cz

Vol. 37

Oscillations in Planar Dynamic Systems by R. E M/c/rens

Vol. 38

Mathematical Problems in Elasticity ed R. Russo

Vol. 39

O n the W a y to Understanding the Time Phenomenon: The Constructions of Time in Natural Science — Pan" 2. The "Active" Properties of Time According to N. A. Kozyrev ed A P. Lewcb

Vol. 40

Advanced Mathematical Tools in Metrology II eds. P. C/ar//n/ef a/.

Vol. 41

Mathematical Methods in Electromagnetism Linear Theory and Applications b y M . Cessenaf

Vol. 42

Constrained Control Problems of Discrete Processes by V. /V. Pbar

Vol. 43

Motor Vehicle Dynamics: Modeling and Simulation by G. Genfa

Vol. 44

Microscopic Theory of Condensation in Gases and Plasma by A L /Mn andf. G. Ko/esn/cbenAo

Vol. 45

Advanced Mathematical Tools in Metrology III eds. P. C/an//n/efa/.

Vol. 46

Mathematical Topics in Neutron Transport Theory — N e w Aspects by M Mo/fnfar-Kharroub/

Vol. 47 Theory of the Navier-Stokes Equations eds. J. G. Reywood ef a/. Vol. 48

Advances in Nonlinear Partial Differential Equations and Stochastics eds. S. Katvas/i/ma and 71 Yanag/satva

SERtES O N A D V A N C E S !N M A T H E M A U C S F O R APPLIED S O E N C E S

Vol. 49

Propagation and Reflection of Shock Waves by P. V. Sbtvgaes/ and /.. S. Sn^emen/ro

Vol. 50

Homogenization eds. S/. Berd/cne^s/ry, v*. J&o^ and G. Papan/co/aou

Vol. 51

Lecture Notes on the Mathematical Theory of Generalized Boltzmann Models 6y /V. Be//omo and M. Lo Scn/a w

Vol. 52

Plates, Laminates and Shells: Asymptotic Analysis and Homogenization by T. Lew/nsM and J. J. Te/ega

Vol. 53

Advanced Mathematical and Computational Tools in Metrology IV eds. P. C/an7n/ er a/.

Vol. 54

Differential Models and Neutral Systems for Controlling the Wealth of Nations b y E /v. Cnu/nvu

Vol. 55

Mesomechanical Constitutive Modeling by M. Ka%ra

Vol. 56

High-Dimensional Nonlinear Diffusion Stochastic Processes — Modelling for Engineering Applications by V. Mamonfoy and M. tV/7/ander

Vol. 57

Advanced Mathematical & Computational Tools in Metrology V eds. P. C/ar//n/era/.

SERiES ON ADVANCES !N MATHEMATtCS FOR APPLIED SOENCES Pub/zshed.' Mathematical Topics in Nonlinear Kinetic Theory II: The Enskog Equation by /V. Be//omo er a/. Discrete Models of Fluid Dynamics ed. A S. A y e s Fluid Dynamic Applications of the Discrete Boltzmann Equation by R. Monaco and L Prez/os/ Waves and Stability in Continuous Media ed. S. R/onero A Theory of Latticed Plates and Shells by G. /. Psnen/cnnot/ Recent Advances in Combustion Modelling ed. B. /.arroufurou Linear Kinetic Theory and Particle Transport in Stochastic Mixtures by G. C. Pomran/ng Local Stabilizability of Nonlinear Control Systems by A Bacc/oM Nonlinear Kinetic Theory and Mathematical Aspects of Hyperbolic Systems ed. K C. So/% Vol. 10

Nonlinear Evolution Equations. Kinetic Approach by /V. B. Mas/ova Mathematical Problems Relating to the Navier-Stokes Equation ed. G P Ga/d/

Vol. 12

Thermodynamics and Kinetic Theory eds. IV Kbs/ns/H er a/.

Vol. 13

Thermomechanics of Phase Transitions in Classical Field Theory by A R o m a n o

Vol. 14

Applications of Pade Approximation Theory in Fluid Dynamics by A Pozz/

Vol. 15

Advances in Mathematical Modelling of Composite Materials ed. K. Z Mad
Vol. 16

Advanced Mathematical Tools in Metrology eds. P C/anyn/era/.

tvnw. tw/dsc/M/'f/'c. com 4494 he