Random Processes in Physics and Finance

244 ANALYTICAL SOLUTION OF THE ELASTIC TRANSPORT EQUATION

A cumulant approximation for the particle density distribution is obtained from the exact expression we have a Gaussian shape

with a moving center located at

and the corresponding diffusion coefficients are given by

In contrast to Eqs. (14.18), (14.19) and (14.22)-(14.25), the results for N(r, t) are independent of gi for I > 2. Each distribution in Eq. (14.17) and Eq. (14.30) describes a particle "cloud" anisotropically spreading from a moving center, with time-dependent diffusion coefficients. At early time t —» 0, /(
IMPROVING CUMULANT SOLUTION OF THE TRANSPORT EQUATION245

FlG. 14.1. The moving center of photons, Rz, and the diffusion coefficients, Dzz and Dxx, as function of time, where g\ are calculated by Mie theory, assuming water drops with a/A = 1, with a the radius of droplet and A the wavelength of light, and the index of refraction m = 1.33. slows down, and the diffusion coefficients increase from zero. This stage of particle migration is often called a "snake-like mode". At large times, the distribution function tends to become isotropic. The particle density, at t » l^/v and r > / tr , tends towards the center-moved (1/tr) diffusion solution with the diffusive coefficient /tr/3. Therefore, our solution quantitatively describes how particles migrate from nearly ballistic motion to diffusive motion, as shown in Fig. 14.1. Figure 14.2 shows the light distribution as a function of time at different receiving angles in an infinite uniform medium, computed by the second cumulant solution, where the detector is located at 5/tr from the source in the incident direction of the source.

14.4

Improving cumulant solution of the transport equation

The analytical solution obtained, although it has exact center and half-width, is not satisfied in two aspects. First, at very early times, exp(—git) —>• I for all /, hence, one cannot ensure summation over / to be convergent. Second, particles at the front edge of the Gaussian distribution travel faster than speed v, thus violating causality.

246 ANALYTICAL SOLUTION OF THE ELASTIC TRANSPORT EQUATION

FIG. 14.2. The time-resolved profile of light at different angles measured on a detector 10 mm from the source in the incident direction. The parameters for this calculation are / t r = 2 mm, la = 300 mm, the phase function is computed using Mie theory for polystyrene spheres in water, with diameter d = 1.11 /im, and the wavelength of the laser source A = 625 nm, which gives the g-factor g = 0.926. Separating the ballistic component from the scattered component

In order to make the summation over I convergent, we separate the ballistic component from the total 7(r, s, t) and compute the cumulants for the scattered component 1^ (r, s, t). The ballistic component is the solution of the homogeneous Boltzmann transport equation, which is the transport equation, Eq. (14.1), without the "scattering in" term (the first term in right side of Eq. (14.1)). The solution of the ballistic component is given by

The moments of the ballistic component can be easily calculated. When SQ is along z, we have

and other moments related to

are zero.

IMPROVING CUMULANT SOLUTION OF THE TRANSPORT EQUATION247 The total distribution is the summation of the ballistic component and the scattered component:

hence, the moments of the scattered component can be obtained by subtracting the corresponding ballistic moments from the moments of I(r, s, t ) . For example, we have

Notice that

Substituting Eq. (14.38), (14.35) into Eq. (14.37), the corresponding cumulants for scattered component 1^ (r, s, t) can be easily obtained, which are the following replacements of Eqs. (14.4), (14.18), and (14.22):

The expressions of other components of the first and second cumulants are unchanged, provided all F(s, so,i) in G in Section 14.3 is replaced by F^ (s, SQ, t). Note that Eq. (14.38) actually is equal to zero at s ^ SQ, and there is no ballistic component at these directions. The replacement of equations in Section 14.3 by Eqs. (14.39)-(14.41) greatly improves calculation of cumulants at very early times. By subtraction introduced above, the terms for large I approaches to zero, and summation over / becomes convergent at very early times. Because g\ = ^s[l — ai/(2l + l)], which approaches

248 ANALYTICAL SOLUTION OF THE ELASTIC TRANSPORT EQUATION

FlG. 14.3. Time-resolved profile of the backscattered (180°) photon intensity inside a disk with center at r = 0, radius R = lltr, thickness dz = 0.1/tr and the received angle dcosO = 0.001, normalized to inject 1 photon. The Heyney-Greenstein phase function with g = 0.9 is used, and l// a = 0. The solid curve is for the second cumulant solution (Gaussian distribution), and dots are for the Monte Carlo simulation. The insert diagram is the same result drawn using a log scale for intensity. to /j,s for large /, f(gi — gi±i) ~ t, and E^ ~ t 2 /2 when t —> 0, which results in cancellation in the summand for large / at very early times. An example of successful use of this replacement is calculation of backscattering. When 9 = 180°, Pi(cos6) = 1 or —1, depending on I even or odd. The computed rcz at very early times using Eq. (14.18) oscillates with a cut-off of 1. But the computed TZ at very early times using Eq. (14.40) becomes stac(s) ble. Calculation shows that TV = 0 at any time for any phase function when 9 = 180°. Figure 14.3 shows the computed time profile of the backscattering intensity l(s\r, s, t) at a detector centering at r = 0 and detection angle 9 = 180°, comparing with the Monte Carlo simulation. The absolute value of intensity, as well as the shape of the time-resolved profile, computed using our analytical cumulant solution matches well with that of the Monte Carlo simulation. The insert diagram is the same result drawn using a log scale for intensity. Note, this result of backscattering, based on solution of the transport equation, is for a detector located near the source, different from other backscattering results based on the diffusion model,

IMPROVING CUMULANT SOLUTION OF THE TRANSPORT EQUATION249

which are only valid when the detector is located with a distance of several / tr from the source. Shape of the particle distribution

If cumulants with order n > 2 are assumed all zero, the distribution becomes Gaussian. The Gaussian distribution is accurate at long times. At early times, particles at the front edge of distribution travel faster than free speed of particles, thus violates causality, especially for particles move along near forward directions. In the following, two approaches are used for overcoming this fault: (A) including higher cumulants; and (B) introducing a reshaped distribution. A. Calculation including high order cumulants We have performed calculation including the higher order cumulants to obtain more accurate shape of the distribution. Codes for calculation is designed based on the formula in Section 14.2. Figure 14.4 shows /(r, s, t) with a detector located at z = 6/tr front of source and received direction along (9 = 0, computed using the analytical cumulant solution up to tenth order of cumulants (solid curve), to the second order cumulants (dotted curve), the diffusion approximation (thick dotted curve), and the Monte Cairo simulation (discrete dots). The figure shows that the tenth order cumulant solution is located in the middle of the data obtained by the Monte Carlo simulation, and /(r, s, t) K- 0 before the ballistic time t\, = Ql^/v. The second order cumulant solution has nonzero /(r, s, t) before tf,, which violates causality. The computed JV(r, t)/4vr using the diffusion model has a large discrepancy with the Monte Carlo simulation, and the diffusion solution has more nonzero components before fy,, which violates causality. Using the second order cumulant solution, the distribution function can be computed very fast. The associated Legendre functions can be quickly computed using recurrence relations with accuracy limited by the computer machine error. It takes a minute to produce 105 data of /(r, s, t) on a personal computer. On the other hand, in order to reduce the statistical fluctuation to a level shown in Fig. 14.4, 109 events are counted in the Monte Carlo simulation, which takes tens of hours computation time on a personal computer. Computation of high order cumulants also is a cumbersome task, because the number of involved terms rapidly grows with increase of order n. Also, It has been proved that as long as there are some nonzero cumulants higher than 2, all cumulants up to infinite order must be nonzero (See Section 8.3). Therefore, no matter how a cut-off at a finite order n > 2 is taken, the cumulant solution of the Boltzmann transport equation cannot be regarded to be exact. B. Reshaping the particle distribution For practical applications, we use a semiphenomenological model. The Gaussian distribution is replaced by a new

250 ANALYTICAL SOLUTION OF THE ELASTIC TRANSPORT EQUATION

FlG. 14.4. Time-resolved profile of transmission light in an infinite uniform medium, computed using the tenth order of cumulants solution (solid curve), the second cumulant solution (dotted curve), and the diffusion approximation (thick dots curve), comparing with that of the Monte Carlo simulation (discrete dots). The detector is located at z = 6^r from source along the incident direction, and the received direction is 0 = 0. The Heyney-Greenstein phase function with g = 0.9 is used, and the absorption coefficient l/la = 0. shaped form, which maintains the correct center position and the correct half-width of the distribution. The new distribution satisfies causality, namely, /(r, s, t) = 0 outside the ballistic limit, vt. There are an infinite number of choices of shape of the distribution under the above conditions. We choose a simple analytical form as discussed later. At long times, the half-width of the distribution a ~ (4B) 1 / 2 , with B shown in Eq. (14.21), spreads with t 1 / 2 , hence, a
IMPROVING CUMULANT SOLUTION OF THE TRANSPORT EQUATION251

FlG. 14.5. The ID spatial photon density at time t = 2ltT/v, obtained by the reshaped form Eq. (14.43) (solid curve) and the Gaussian form (dashed curve), comparing with that of the Monte Carlo simulation (dots). The Heyney-Greenstein phase function with g = 0.9 is used, and l/la = 0. In the figure, the unit in z axis is ltr', Rc is the center position of distribution computed by the cumulant solution; zc is the distance between the origin of the new coordinates and the source. where Rcz and Dzz in Eqs. (14.31), (14.32). As shown in Fig. 14.5, although the ID Gaussian spatial distribution (the dashed curve) at time t = 2ltT/v, Eq. (14.42), has the correct center and half-width, the curve deviates from the distribution computed by the Monte Carlo simulation (dots), and a remarkable part of the distribution appears outside the ballistic limit vt = 2/ tr - At early times the spatial distribution is not symmetric to the center Rc. While Rc moves from the source toward the forward side, causality prohibits particles appearing beyond vt. This requires the particles in the forward side being squeezed in a narrow region between Rc and vt. For a balance of the parts of distribution in the forward and backward sides of Rc, the peak of the distribution should move to a point at the forward side and the height of the peak should increase. Based on this observation we propose the following analytical expression: (1) to move the peak position of the distribution from Rcz to zc, while the parameter zc will be determined later; (2) to take this point as the origin of the new coordinates; and (3) to use the following form of the shape of ID density in new coordinates:

252 ANALYTICAL SOLUTION OF THE ELASTIC TRANSPORT EQUATION

where

At the ballistic limit z = z±, N(z) reduces to zero, and N(z) = 0 when z is outside of z±. The parameter b in Eq. (14.43) can be determined by normalization; the parameters (a, zc) can be determined by fitting the center and half-width of the distribution. This fit requires

The integrals in Eqs. (14.45), (14.46), and (14.47) can be analytically performed, related to the standard error function:

The solid curve in Fig. 14.5 shows the reshaped spatial distribution, Eq. (14.43), of the ID density at time t = 21^ /v, using the Heyney-Greenstein phase function with g = 0.9, which satisfies causality and matches the Monte Carlo result much better than the Gaussian distribution. For nonlinear fitting a difficulty is how to quickly find a global minimum. The optimization codes require setting a good initial value of the parameters, so the obtained local minimum is the true global minimum. The following procedure is used to quickly obtain a global minimum. At the long time limit, zc « Rcz, and c? Ki (4Dzzvt)~l, the distribution approaches the original Gaussian distribution.

IMPROVING CUMULANT SOLUTION OF THE TRANSPORT EQUATION253

We set these value of parameters at a long time tm, and take them as initial values, using a nonlinear fitting, to determine the parameters at tm-i = tm — At, where At is a small time interval. Then, we use parameters at t m _i as initial values to determine parameters at t m _2- Step by step, the parameters in a whole time period can be computed. SD-density In this case the ballistic limit is represented by a sphere with center located at the source position and radius vt. We move the peak position of the distribution from Rcz to zc along the SQ = z direction, take this point as the origin of the new coordinates, and use the following form of the shape of 3D density as a function of the position in the new coordinates, f:

where N(r) = 0 when r > r*, and x is the polar angle of r in the new coordinates, f* is the distance between the new origin and the point by extrapolating f to the surface of the ballistic sohere:

In Eq. (14.52) a(x) is defined by

The parameters b can be determined by normalization; the parameters (o^, aj_, zc) are determined by fitting the center and half-width of the distribution. This fit requires

In the above integral (fir = 2'rrf'2dfdcos(x), integration over f can be analytically performed, integration over x is performed numerically. Figure 14.6 shows the computed time profile of the 3D density N(r, t), with the source at the origin and the detector is located at r = (0, 0, 3/ tr ), using the HeyneyGreenstein phase function with g = 0.9. The solid curves is for the reshaped form Eq. (14.52). The dashed curve is for the Gaussian form, and the dots are for the Monte Carlo simulation. The results clearly demonstrate an improvement by use of the reshaped form than that by use of the Gaussian form. The nonzero intensity

254 ANALYTICAL SOLUTION OF THE ELASTIC TRANSPORT EQUATION

FIG. 14.6. Time-resolved profile of 3D photon density, where the detector is located at z = 3/tr from source along the incident direction, obtained by the reshaped form Eq. (14.52) (solid curve) and the Gaussian form (dashed curve), comparing with that of the Monte Carlo simulation (dots). The Heyney-Greenstein phase function with g = 0.9 is used, and the absorption coefficient l/la = Q. before £& = 3ltr/v has been completely removed in the reshaped form, while the Gaussian distribution has nonzero components before t\,. The reshaped time profile matches with result of the Monte Carlo simulation in most time periods, but the peak values is about 20% lower. The errors are much smaller than that of the Gaussian distribution. By integration over time, the density for the steady state can be obtained. The difference in the steady state density between the reshaped analytical model and the Monte Carlo simulation is about 3%. Distribution function 1^ (r, s, t) When the detector is located less than 8/tr from the source in a medium with large (/-factor, the distribution function /( s )(r, s, t) is highly anisotropic, and the intensity received strongly depends on the angle. One needs to use the photon distribution function /^(r, s,t) instead of the photon density N(r, t). In this case the center position rc, as a function of (s, t), is not located on the axis at incident direction SQ. Without loss of generality, we set the scattering plane (s, SQ) as the x-o-z plane. The center position now is located at rc = (r£, 0, rcz}. The orientations and lengths of axes of the ellipsoid, which characterize the halfwidth of spread of the distribution, can be computed as follows. The nonzero

IMPROVING CUMULANT SOLUTION OF THE TRANSPORT EQUATION255

FIG. 14.7. Schematic diagram describing the geometry of the particle spatial distribution for scattering along a direction s ^ SQ. At certain time t, the center of the distribution is located at rc. The half-width of the spread is characterized by a ellipsoid (the gray area). The large sphere represents the ballistic limit. The origin of new coordinates is set by extending from \rc to zc. f * is a point by extrapolating a position f (in the new coordinates) to the surface of the ballistic sphere, and the length f* is determined by Eq. (14.53). components for the second cumulant now are (Bxx,Bxz,Bzz,Byy), expressed in Eq. (14.21). Byy represents the length of one axis of the ellipsoid, perpendicular to the scattering plane. By diagonalizing the matrix

the lengths and directions of other two axes of the ellipsoid on the scattering plane can be obtained. In fact, calculation shows that the direction of rc is also the direction of one axis of the ellipsoid, since at a certain time t the direction rc can replace s as the unique special direction in the scattering plane. In order to reshape the distribution we choose a new z axis along the rc direction, and move the peak position of the distribution from \rc\ to zc, and take this point as the origin of the new coordinates (x, y = y, z), as shown schematically in Fig. 14.7. In the new coordinates we use a shaped form similar to that of the 3D density Eqs. (14.52), while a(x) in Eq. (14.52) is

where x and (p are, separately, the polar angle and the azimuthal angle of a position r in new coordinates. The parameters (ax, ay,az, zc) are determined by fitting the

256 ANALYTICAL SOLUTION OF THE ELASTIC TRANSPORT EQUATION

center rc and lengths of three axes of the ellipsoid characterizing the half-width of the distribution. In many cases, the ellipsoid can be approximately treated as an ellipsoid of revolution, with the length of the axis of the ellipsoid along the x direction approximately equal to that along the y direction then the computation can be simplified. The new shaped distribution function 7^(r, s, i) for a certain direction s is normalized to F^ (s, SQ, t). Figure 14.8 shows the computed time profile of distribution function /( s )(r, s, £), when the detector is located at 4/ tr in front of the source, using the Heyney-Greenstein phase function with g = 0.9. Fig. 14.8(a), (b) are, separately, for different directions of light s: 9 = 0 and 30°. The solid curves are for the reshaped form Eq. (14.52) and the dashed curve is for the Gaussian form. The dots are for the Monte Carlo simulation. Anisotropic distribution is shown by comparing with Fig. 14.8(a) and Fig. 14.8(b). The reshaped distribution removes the intensity before tb = 4/ tr /t>, which appears in the Gaussian distribution. The reshaped distribution matches the Monte Carlo result much better than the Gaussian distribution. While causality, together with the correct center and half-width of the distribution, are major controlling factors in determining the shape and the range of the particle distribution, the detail shapes are, to some extent, different by use of the different models. For s at the near backscattering direction, the Gaussian distribution can be a good approximation as shown in Fig. 14.3, because most particles suffer many scattering events to transfer from the forward direction to the backward direction. Our calculation shows that the center position rc is close to the source for 0 KI 180° and far from the ballistic limit, hence, reshape has little effect on the backscattering case. Beside improving convergence, separating the ballistic component from the scattered component also provides a more proper time-resolved profile for transmission. In the time-resolved transmission profile the ballistic component is described by a sharp jump exactly at time vt, separated from later scattered component. The intensity of the ballistic component, comparing to the scattered component, strongly depends on the ^-factor. For g = 0, l^ = ls, the ballistic component decays to exp(—1) = 0.368 at distance 11^. But for g = 0.9 it decays to exp(—10) = 4.54 x 10~5 at l/tr» because l^ = 10/s. The jump of the ballistic component can be seen in experiments of transmission of light for medium of small sized scatterers (small ^-factor), but is difficult to be observed for medium of large sized scatterers (large (/-factor). Our formula provide a proper estimation for both small and large ^-factors by explicitly separating these two components. Using the obtained analytical expressions, the distribution I(r, s, t) can be computed very quickly. The cumulant solution has been extended to the polarized photon distribution (Cai, Lax, and Alfano 2000c), and extended to semi-infinite and slab geometries (Xu, Cai, and Alfano 2002, Cai, Xu, and Alfano 2003). Using

IMPROVING CUMULANT SOLUTION OF THE TRANSPORT EQUATION257

FlG. 14.8. Time-resolved profile of photon distribution function, for light directions (a) 9 = 0, (b) 6 = 30°, where the detector is located at z = 4/tr from the source along the incident direction, obtained by the reshaped form Eq. (14.52) (solid curves) and the Gaussian form (dashed curve), comparing with that of the Monte Carlo simulation (dots). The Heyney-Greenstein phase function with g = 0.9 is used, with the absorption coefficient l/la = 0. a perturbative approach, the distribution in a weak hetergeneous medium can also be calculated based on the cumulant solution (Cai, Xu, and Alfano 2003). The nonlinear effect for strong heterogeneous objects inside the medium can also be calculated using a correction of a "self-energy" diagram. (Xu, Cai, and Alfano 2004).

15 Signal extraction in presence of smoothing and noise

15.1

How to deal with ill-posed problems

The importance of ill-posed problems The problem of extracting a signal from a distorted output (i.e., the solution of an inverse problem) in the presence of noise is a ubiquitous one. It occurs in the analysis of spectral data (Jansson 1970), in geophysical problems requiring the inverse of potential theory (Bullard and Cooper 1948), or of heat conduction (John 1955), and in the inversion of information from optical instruments, electron spectroscopy, radioastronomy (Kaplan 1959), medical image, etc. A related ill-posed problem is that of pattern analysis or recognition (Benjamin 1980). Also of interest is the use of lasers to probe atmospheric temperature distributions (Hillary et al. 1965). The most recent example is the analysis and repair of distorted signals from the Hubble telescope. The nature of ill-posed problems

In what sense is the typical inversion problem an ill-posed problem? The signal (e.g., the line shape) s(x) we are trying to determine is acted on by some apparatus K, and the output is contaminated by noise n. In a simple linear case, the measured output m(x) is given by

If there were no noise, and if m(x) were measured with infinite precision, one could invert the integral equation to write

where

i.e., K

1

is the kernel inverse to K(x, y).

SOLUTION CONCEPTS

259

The difficulty is that all kernels K(x,y) perform some smoothing on their input. Thus one could add to the solution s(x) a high frequency term Asmujx such that for any small e by choosing u> sufficiently large - for any A, even an intense A. Thus, if our measured result is precise only to within a noise n(x) of order e, many different solutions are possible such that

One may object, however, that the solution s(x) should be smooth and not contain a superposition of high frequency components. The answer is that the problem is, in general, not well-posed (that is with a unique solution) until the nature of the smoothness is specified. Unfortunately, the smoothness of the solution may not be known in advance. One is attempting to determine this from the measured results! However, if one makes no specification of smoothness, it is difficult to tell which of the frequencies in the measurement m(x) are properties of the signal, and which are spurious. It is our opinion, and that of a number of others, whose work will be discussed shortly, that this issue can only be resolved by making a separate measurement of the spectrum associated with the noise n(x) (or its autocorrelation (n(x)n(x + x')}). Not only the shape of the noise spectrum, but also its intensity is relevant, since Fourier components in a particular measurement much below the corresponding noise intensity can not be regarded as significant, and should be excluded from the estimated signal s(x). Thus we see that the correct procedure for computing an estimate s(x) from m(x) must be a nonlinear one. 15.2

Solution concepts

We shall use as a guide to the literature the paper by Price (1982) and the extensive review of statistical methods presented by Turchin, Koslov and Malkevich (1971). All of the methods, to be discussed below, reduce the continuum problem to a discrete one. In the simplest case, the observation points used are at Xi and the solution s(y) is to be evaluated at yj. Equation (15.1) then reduces to a set of coupled linear equations An alternative procedure is to expand s(y) in some set of orthonormal functions (f>v(y) and m(x) in the same set <j>fj,(x). The result is that Eq. (15.6) is replaced by an equation of identical form, with subscripts i and j replaced by /j, and v. Of course, m^ and sv are the expansion coefficients in the ^ basis.

260 SIGNAL EXTRACTION IN PRESENCE OF SMOOTHING AND NOISE

In terms of either discrete representation, the ill-posed nature of the problem manifests itself in the matrix KIJ (or K^v) being an ill-conditioned matrix. See for example, Dahlquist and Bjorck (1974). Filtering If noise is ignored and the kernel possesses translational invariance

then a formal solution for s ( x ) can immediately be obtained using Fourier transform techniques:

where m(p) and K(p) are the Fourier transforms of m(x) and K(x), respectively. The ill-posed nature of the problem can be made evident by considering an instrument K with Gaussian line shape:

In this case, its Fourier transform is

The use of Eq. (15.10) in Eq. (15.8) clearly produces a large enhancement in any high frequency components in m(p). If there were no noise, m(p) would vanish as p —>• oo more rapidly than K(p) so that a well-defined expression would result for the signal s(x). But the added noise n(x) can be white noise, meaning that n(p) and hence rh(p), do not fall off, but remain constant as p —> oo. The most elementary way to avoid this difficulty is to set

where F(p) is a filter factor that falls off rapidly as p —» oo, chosen to impose the desired smoothness on s(x). The problem is the arbitrariness involved in specifying F(p).

METHODS OF SOLUTION

261

Regularization

A second class of methods, known as regularization methods, replaces the problem of minimizing | \Ks — m\ |2 by a well-posed problem of the form

where D is a linear operator that measures the degree of nonsmoothness, say a second derivative, and a is a parameter that determines the amount of nonsmoothness allowed.

15.3

Methods of solution

Relation of regularization to filtering For a general D, we claim, with Price (1982), that the regularization method is equivalent to using the filter factor

since Eq. (15.12) yields

from which we deduce that

or

In Fourier space, the factor in brackets is the filter factor of Eq. (15.13), but the matrices in Eq. (15.16) can taken in any basis set 4>^(x). Perhaps the earliest suggestion for regularization was made by Phillips (1962) who proposed that to keep the solution smooth one should for fixed \\Ks — m\\

262 SIGNAL EXTRACTION IN PRESENCE OF SMOOTHING AND NOISE minimize

and he replaced s"(x) by its discrete counterpart

But then so that Thus the Phillips choice is a special case of smoothing with |D(fc)| 2 = 16sin 4 (/c/2). The chief purpose is to have \D(k)\2 increase with k to be more effective in eliminating the high Fourier components. If Fourier transform methods are used, it would be simpler to use which is simply the statement that the spectrum of s"(x) is k4 times that of s(x). A more general discussion of regularization is given by Tikhonov (1977). Iteration

One of the earliest deconvolution schemes is the iteration scheme of van Cittert (1931). In this scheme one starts with

and passes from the /j,th to the (/j, + l)th iterate according to

which is, in effect, a Jacobi iterative solution of the simultaneous equations. (The prime on the sum omits the diagonal j = i term.) Jansson (1970) proposed an overrelaxation scheme of the form

where K need not equal unity. In this case, all components of s are updated simultaneously. If, in updating any component, the updated values of earlier components are used, we get a generalization (for K ^ 1) of the Gauss-Seidel iteration

METHODS OF SOLUTION

263

procedure:

Jansson quotes the convergence condition for this scheme as

However, the matrix Kij will always be ill-conditioned, Eq. (15.26) will be disobeyed, and convergence will never occur. Jansson succeeds, however, for a different reason. At the start, and after each iteration, he applies a smoothing procedure of the form

where the smoothing formula involves 21 + 1 terms. This smoothing procedure is equivalent to the choice of a particular value of D(k). Optimum choice of the relaxation parameter K is discussed by Fadeeva (1959). Jansson was able to resolve adjacent peaks in infra-red spectra whose separating valley was eliminated by noise. His method, however, "converged only to within a certain value of the root-mean-square difference then diverged". Although Jansson attributes this divergent behavior to noise in the data, in our opinion, it may be associated with round-off error generated by multiple matrix processes. If his procedure were replaced by an equivalent D(k), then one fast Fourier transform, a filtered product, and an inverse fast Fourier transform would yield his best result without iteration. However, his scheme, like that of Phillips, and of Twomey's (1963) improvement of Phillips can all be criticized for the arbitrariness in their choice of D(k). Nontranslationally invariant kernels

Noble (1977) provides a simple, numerical example:

with a = 0, b = 1, s(y) = 1, and m(x) computed from this integral with a kernel that arises from potential theory:

264 SIGNAL EXTRACTION IN PRESENCE OF SMOOTHING AND NOISE

He converts the integral equation to a set of simultaneous equations using Simpson's rule with n points. He solves for s(0) [exact s(0) = 1] with the results:

"Contrary to what we might expect at first sight, the larger the number of points, the worse the results are; the smoother the kernel, the worse the results are". 15.4

Well-posed stochastic extensions of ill-posed processes

Franklin's method Our section heading is borrowed from the title of a lucid contribution by Franklin (1970). Our description of Franklin's work follows that of Shaw (1972), whose improvement will be detailed in the next section. In our notation, Franklin considers the solution of the problem

where the noise n has given Gaussian statistics, and the signal s also has given statistics. The statistics of m is derived from the corresponding statistics of s and n. Presumably, the statistics of the noise n can be obtained by measurements in the absence of a signal. The aim is to construct a linear operator L such that an estimate s of s can be constructed from m via

To minimize the mean square error, we vary L in

To obtain this result, we expanded in an arbitrary basis fa:

so that the second term takes the form

WELL-POSED STOCHASTIC EXTENSIONS OF ILL-POSED PROCESSES265 where we have taken the ensemble average and used

If we vary L* in Eq. (15.32), and use the cyclic property of the trace we get

from which we obtain in agreement with Shaw's (3.15). Note, however, that the statistics of m are determined by those of n and s according to Eq. (15.30): The scalar product with s yields

or To evaluate all the components of Rmm, we need:

with the help of the expanded form

The explicit result can be written as a matrix relation by suppressing the subscripts:

Thus

In the usual case, in which noise is uncorrelated with the signal, Rsn = Rns = 0, L reduces to: This is the result used by Franklin. We can verify that when the noise is neglected

the correct, but useless value because K is inevitably an ill-conditioned matrix.

266 SIGNAL EXTRACTION IN PRESENCE OF SMOOTHING AND NOISE 15.5

Shaw's improvement of Franklin's algorithm

Franklin's procedure had one significant defect: He assumed that the signal s was drawn from a space in which the mean value {s} = 0. In addition, one often wishes to use a number M of measured values rrij significantly larger than the number TV of signal values Sj to be estimated. Franklin's procedure requires the solution of M simultaneous equations. Shaw has produced an algorithm that requires inversion only of a smaller N x N matrix as one does in a least squares calculation. In addition, he makes an initial estimate s^ of the signal and interates assuming (s) = s^ in computing §(n+1\ We first note that a simple least squares algorithm which requires the minimization of

leads to the equations

If we multiply the original relation Eq. (15.30) between m and s by K^ we obtain

Here K^K has the reduced N x N size. This problem now has the same form as Franklin's original problem with the replacements Thus Eq. (15.35) for Rsm is replaced by a reduced noise matrix

The reduced signal-measurement correlation is

Thus we obtain the matrix relation:

Similarly, the reduced measurement-measurement correlation is given by:

The estimate based on Franklin's procedure, but using the reduced matrices, takes the form:

SHAW'S IMPROVEMENT OF F R A N K L I N ' S ALGORITHM

267

or in expanded form:

If cross-correlations are neglected,

If, in addition, white noise is assumed

now, Rmrn in Eq. (15.56) has a factor K^K on the left, so that its reciprocal takes the form

where / is a unit matrix in the ji space. Then the estimated signal, Eq. (15.56), can be rewritten as:

Since the matrix RSSK^K commutes with itself as well as with the unit matrix /, we can move it through to the right to obtain:

When a nonzero signal is present:

our aim is to determine (s} = h from the measured data. Franklin's procedure, Eqs. (15.314) and (15.37), is modified to

in its original form. In the reduced form Eq. (15.56) is replaced by:

In the case of white noise we can return to Eq. (15.60) and replace s by s — h, and m by m — h. After the subtraction involving the h terms is performed, we get the simplified result:

in agreement with Shaw (4.19).

268 SIGNAL EXTRACTION IN PRESENCE OF SMOOTHING AND NOISE

In his calculations Shaw also treats the signal s as having white noise

In this case, Eq. (15.64) simplifies further to:

Since s should agree with h, Shaw adopts the iterative procedure

Shaw then provides a starting estimate s^ from the least square equation

by assuming that K^K is sufficiently sharp that Sj can be replaced (on the left) by Sj with the result

If we set s(n+1) = s^ = s in Eq. (15.68) and attempt to solve directly for s, we obtain which is just the least squares solution. Thus the iterative procedure should eventually become unstable!

15.6

Statistical regularization

What is the meaning of the statistical fluctuations in the signal characterized by the correlation function Rss These fluctuations may represent actual noise that contaminates the signal. However, even when the signal is not contaminated by noise, but only added later, the Franklin and Shaw procedures would break down (the problem became illposed) if Rss were set equal to zero. Another interpretation is that we impose on the problem an a priori distribution -P([s]) of possible signals. This distribution, for example, should give weight to our prejudice that the Sj = s(xj) arise from

STATISTICAL REGULARIZATION

269

a smooth function s(x). The Phillips's regularization procedure emphasized this point by adding a term in the minimization procedure

which becomes large if s(x) becomes highly oscillatory. These ideas can be cast in the language of statistical decision theory. Let

represent the probability of a signal s and a measurement m. (For simplicity, think of s as the set of numbers s$ = s(xj) and m as the set nij = TO(XJ).) Then the Bayes' theorem solution of the problem

represents the a posteriori probability of s, after having made the measurement m in terms of the a priori estimate P(s). Since the noise n is Gaussian and additive,

If we do not know the a priori probability in detail but only its correlation matrix

then the distribution with maximum entropy where

subject to the constraint of having a correlation matrix C is given by the Gaussian

In summary, although the regularization methods appear to be distinct, they all convert an ill-posed problem to a well-posed one by adding a requirement of smoothness. This is done most explicitly by specifying an a priori probability associated with the signal. (Iteration procedures, such as Eq. (15.67), due to Shaw and Franklin give the signal an a priori spread in Eq. (15.65).) The disadvantage of all the above procedures, is that the predicted signal depends linearly on the measured signal. Regularization is undoubtedly necessary, but the best estimate of the signal should depend nonlinearly on the measured value and noise.

270 SIGNAL EXTRACTION IN PRESENCE OF SMOOTHING AND NOISE 15.7

Image restoration

Nonlinear methods have been introduced in connection with the problem of image restoration. These methods recognize that an image is likely to have sharp edges. The methods introduced consist in a mixture of a regularized solution with the unregularized result with the degree of admixture varied in a local manner that is sensitive to the gradient of the measured signal. This procedure reduces the amount of undesirable smoothing that occurs in the vicinity of an edge. But no investigation has been made of the stability of these new procedures. The work of Abramatic and Silverman (1982) is based upon a procedure introduced in geophysics by Backus and Gilbert (1970) and on the work of Frieden (1975). The idea of Abramatic and Silverman is to allow the regularization parameter which controls smoothness of the solution to adapt to the local characteristics of the image (a flat field or an edge). This was done by taking into account the masking effect of the human eye. The eye is quite sensitive to a small amount of noise in a flat field, but is able to tolerate a large amount of noise in the surroundings of an edge. In their procedure, the masking function is estimated in the form of

from the noisy image where g(i,j) is the gradient of the image at the pixel ( i , j ) and do is at the order of the typical size of an edge. The amount of regularization at each pixel (i, j ) is scaled by a visibility function, f(M(i, j)), a monotonic decreasing function from 1 to 0 as M goes from 0 to oo. Abramatic and Silverman used the visibility function

where a > 0 is a tuning parameter. Via the visibility function, stronger regularization is then applied to the flat field where M is small and weaker regularization near an edge where M is large. In short, nonlinear image restoration is much harder than linear restoration. An excellent summary of image restoration can be found in G. Demoment (1989).

16 Stochastic methods in investment decision

Derivative securities, with which we shall be concerned, have a value related to the underlying security or asset. When a derivative such as a put, or a forward, or future sale, is not priced correctly, one can, by making compensating purchases, make a risk free profit. This is sometimes referred to as arbitrage. The purpose of this chapter is to develop the relation between the price of the underlying asset, and a derivative, and to include the effect of noise, or random fluctuations on this relation. See Hull (1989, 2001) for a detailed discussion of "Options, Futures, and Other Derivative Securities". A more mathematically oriented description of this subject is contained in Nielsen (1999). There is a controversy between the mathematical economists and financiers, who use methods based on the Ito's (1951) calculus lemma, and physical scientists, for example van Kampen (1992), whose applications have been made to chemistry problems, and Lax (1966IV). An attempt will be made to compare the two techniques to avoid pitfalls that sometimes occur in the use of the Ito method. 16.1

Forward contracts

A forward contract is an agreement by one person to sell at a time T (in years) for K dollars (at delivery) an asset whose value at the current time t (in years) is S. The forward price F is that delivery price K chosen to make the value of the contract zero. If the interest rate r in risk free money were zero, we would have F = S. However, if the delivery price is K, one only needs cash equal to K exp[—r(T — t)\ at the present time to be able to pay K at the time T — t later. If we assign / to be the value of the forward contract, then

since the first two items are equivalent to owning the third. The forward price F is then the value of K that makes / = 0.

As an example, from the Wall Street Journal on Friday, May 22, 1998 we take the price in dollars for 100 yen (table 16.1).

272

STOCHASTIC METHODS IN INVESTMENT DECISION

TABLE 16.1. Price in dollars for 100 yen Time Price Interest Spot 1.1675 30 day 1.1715 2.05% 90 day 1.1808 2.26% 180 day 1.1948 2.31% Except for commission, that we neglect, the ratio of the 30 day, 90 day and 180 day forward prices describe the interest rate factor exp[r(T — t)] for the three different periods. In the third column we list the rate r, consistent with the above data. It would appear that Eq. (16.2) contains a hidden assumption that the price S (at the initial time t) will be equal to the final price ST at the settlement time T. But this is not the case! An arbitrageur can buy the asset at the spot price S and take the short (seller) side of the forward contract. To do this, he must borrow S dollars at a total cost of 5exp[r(T — t)]. When he sells the asset, he receives ST for a gain (possibly negative) of

From the forward contract, he receives F, but then must supply an asset of value ST leading to a gain of In the combined gain

the value of ST disappears. This result agrees with Eq. (16.2). Thus if F > 5exp[r(T — t)] he makes a risk free gain. If F < S l exp[r(T — £)] , an arbitrage in the opposite direction also yields a risk free gain. This expression, Eq. (16.2), for the value of a forward contract is independent of the final value ST of the asset. 16.2

Futures contracts

Futures contracts, like forward contracts, are an agreement now to buy or sell an asset in the future at an agreed upon delivery price. However, future contracts are handled on an exchange such as the Chicago Board of Trade. To buy a future contract through a broker, a deposit, the initial margin, must be supplied to guarantee delivery. This could be 20% of the value of the contract.

A VARIETY OF FUTURES

273

Thus to buy 100 ounces of gold at $400/ounce, a contract of $40000 might require an $8000 deposit. If the price of gold goes up by $10 the buyer of the future contract finds that his margin account has gone up by 100 x 10 = 1000. However, any balance above the initial margin can be withdrawn. Even if not withdrawn, additional interest is earned. Conversely, if the price goes down, the value of the margin account declines a corresponding amount, and the interest earned declines. If the margin falls below a maintenance level, the investor will receive a margin call to make up the difference. If not received, the broker sells the contract thus closing out the position. In Appendix 2A of Hull (1989), or 3B of Hull (2001) he establishes that if the interest rate is the same for all maturities, futures contracts and forward contracts should have identical prices. However, if interest rates change, particularly if they change in a way that is correlated with price S charges, the equivalence no longer holds. We shall ignore these fine points in the discussion that follows. Typically, European contracts involve action only at the closing date. But American puts and calls can be exercised at any intermediate date, or held to the close. This introduces a need for a strategy as to when to take action, and can also cause a modification of the price of the put or call. 16.3

A variety of futures

The general behavior of future contracts were described in the previous section, but there are differences that depend on the nature of the assets. If we are dealing with stock index futures, where the stock has a dividend yield q, the forward price, Eq. (16.2), is modified to Stock: since the underlying asset has a dividend yield q that partly compensates for the interest rate r. For future contracts involving currencies, if the local currency has interest rate r, and the foreign currency has interest rate rt we get since the yield in the foreign currency plays the role of a dividend. Table 16.2 shows an decrease of price with maturity corresponding to the fact that interest rates in the US are less than those in Canada. For gold and silver, the forward price is Gold: or

Gold:

274

STOCHASTIC METHODS IN INVESTMENT DECISION

TABLE 16,2. Price of Canadian dollars in America Time Price Spot 0.7404 30 day 0.7396 90 day 0.7383 180 day 0.7308 where U is the present value of all storage costs over the life of the contract. If the storage cost is proportional to the value of the gold, with the storage cost u per year per dollar of value, this is equivalent to a total storage cost

This results in Eq. (16.9) with n, acting as if it were a negative dividend, or an added interest charge. 16.4

A model for stock prices

Black and Scholes (1973) have developed a procedure for estimating the appropriate price for puts and calls which are forward contracts where the asset involved is a stock. A similar contribution was made by Merton (1973) at the same time. (The Nobel prize was shared for this work.) For this purpose, they need a model for how the price S of a stock varies with time. If the change in S is proportional to S, the growth is necessarily exponential. In the absence of fluctuations then, the model assumes

where the growth rate /j,, in the stock price, presumably can be estimated from the growth rate in earnings of the stock. In the lowest order, one might expect the stock to execute a Brownian motion with fluctuation parameter anS. But this choice has two disadvantages. The first as remarked by Hull (1989), Section 3.3 and Hull (2001), Section 10.3 is that investors expect to derive a return as a percentage of the stock value, independent of the price. Thus they classify stocks by their growth rate. To add Brownian motion, Eq. (16.11) is rewritten in the form Hull where (3.7) refers to Hull (1989) and (10.6) refers to Hull (2001). Hull, of course, doesn't use the subscript H in his work. Here dz is the differential of a Wiener

A MODEL FOR STOCK PRICES

275

process of pure Brownian motion, namely one whose mean remains zero, and whose standard deviation is (At) 1 / 2 . If 4>(m, S) describes a normal process with mean m and standard deviation E, then the distribution of Ax = A5/5 is described correctly by

to allow for the growth of the mean value m = //Ai and the standard deviation o-H(At) 1 / 2 with time. This correct result, stated as Eq. (3.8) in Hull (1989) and Eq. (10.9) in Hull (2001), also avoids the second disadvantage. If 5* itself were described as a Wiener process, the price S could reach unacceptable negative values. In the accepted model, all that can happen is for In S to go negative. We therefore advocate the use of Eq. (16.12) as the fundamental description of a model for stock prices. The model, Eq. (16.12), defines // as the slope of the ensemble average of x(t) if an ensemble of measurements can be made. If not, one takes a logarithm of one sample price series and asks for the slope of the best linear fit to that line of price logarithms. Now we ask what is the Ito's stochastic differential equation (ISDE) for stock 5? Using the Ito's calculus lemma, Eq. (10.41), and dS/dx = S and d2S/dx2 = S, this leads to ISDE for S,

A model called geometric Brownian motion is also often used in financial quantitative analysis, which is written as Then, by <

and

Ito's lemma leads to

The fact /z / v indicates that Ito's calculus \emmsiprohibits simply multiplying by S on both sides of Eq. (16.12), which would produce

This simplest example shows the puzzle in performing Ito's calculus lemma. But this kind of manipulation has appeared in some well known books in the financial area. For example, in Hull's Eq. (10.6), with the word "or", Eq. (16.12) is rewritten as Equations (16.12) and (16.16) appear to be are regarded as equivalent by Hull and the finance community. But we believe the second choice is not equivalent to the first choice.

276

STOCHASTIC METHODS IN I N V E S T M E N T DECISION

Before further analysis we must note an annoying but unimportant difference in notation. The standard Wiener notation is equivalent to the correlation

This leads to a Brownian motion in which

whereas the customary physics notation would have a factor of 2 placed on the right hand side of these equations. In this chapter we follow Hull's convention common in the economics world and and set

where f ( t ) d t is our definition for standard fluctuation, as defined in Chapters 9 and 10. In this book,CTHis an abbreviation for <JH U H> and cr is an abbreviation for ^Lax-

Our approach is developed in Sections 8.2-8.3, and Sections 10.1-10.3, which uses the ordinary calculus rule for the variable transform in the Langevin stochastic differential equation (LSDE). By Eq. (16.19), Eq. (16.12) can be written in the form of our Langevin equation

Thr LSDE approach allows us to simply multiply S on Eq. (16.12), and it is written as Equation (16.21) seems similar to Eq. (16.16), however, using our approach the average of the product in the second term is not zero. Thus Eq. (16.21) should not be regarded as an Ito's equation. The custom in the economic field, under definition of Ito's integral, is to replace Eq. (16.16) by an equation Hull: where tc = t — e is a slightly earlier time than t. This guarantees that the average of the second term vanishes. This happens because in an integration from t to t + At the time tc is not included in the region of integration. Equation (16.22) is then a true Ito's equation, but it is not equivalent to Eq. (16.21). Do these models, Eq. (16.12) and Eq. (16.16), or Eq. (16.21) and Eq. (16.22), yield different results?

A MODEL FOR STOCK PRICES

277

Equation (16.21) has been solved for a well behaved f ( t ) (finite, not 5 functions) stochastic variable such as one with a Gaussian correlation in time (see Section 10.2). The average of the product in the second term is not zero in a finite time interval, and remains nonzero as one approaches the white noise limit by letting the correlation time approach zero when At —» 0. By specializing to the delta-correlated case, Eq. (10.27) can be written in the form Lax :

This is equivalent to the statement that Lax :

i.e., that In ST has a normal distribution with mean In S + ^(T - t) and variance a^(T - t). For a small time interval, At, Eq. (16.24) reduces to Eq. (16.13) or Eqs. (3.8)10.9) in Hull (1989)2001). However, Eqs. (4.6) 11.1) in Hull (1989)2001) Hull: is in disagreement with our result, and also with Eqs. (3.8110.9) in Hull (1989)2001). How did this discrepancy of (l/2)
which is in agreement with Eq. (16.14). For the average, S on the right hand side of Eq. (16.26) is replaced by (S}. In our Langevin expression, the first term represents motion driven by finite number of driving forces to the system, hence, is a deterministic function of time. The second term represents the fluctuation driven by many unknown random forces. Under transform of variables, the ordinary calculus rule can be applied, separately, on both terms. Hence, the meanings of drift and fluctuation are clearly kept separate in the first and second terms after transform of variables. The average of the second term is generally nonzero when an(S) is not a constant.

278

16.5

STOCHASTIC METHODS IN INVESTMENT DECISION

The Ito's stochastic differential equation

Because mathematicians concentrate on Brownian motion, which is rather singular in behavior, it is not clear how to define the integral of a product of a random variable and a random force. In particular, following our notation of Section 10.1 it is not clear how to convert the differential equation, Eq. (10.1),

to an integral. The Riemann sum of calculus would be

where The Riemann integral exists if the sum approaches a limit independent of the placement of tj in the interval in Eq. (16.29). The Riemann integral exists, according to Jeffries and Jeffreys (1950), when the integrand is bounded over the interval of integration and for any positive ui and 17, the interval of integration can be divided into a finite set of intervals such that those with hops (jump discontinuities) > (jj have a total length < 17. Our point is that Brownian motion violates these condition. See Ito (1951), and Doob (1953). Ito (1951) avoids the difficulty by evaluating a (a) at the beginning of the interval, and evaluating the integral over /(£) as a Stieltjes integral

However, even the Stieltjes sum does not converge to a unique integral, and the evaluation at the beginning of the interval is an arbitrary choice. The effect of this choice, since /(s) is independent of a(t) for t < s, is that the average of the second term in Eq. (16.27) vanishes, so that the Ito drift vector is in contrast to our result, Eq. (10.14). Stratonovich (1963) makes the arbitrary but fortuitous choice of using

the average of the values at the two end-points of each interval.

THE ITO'S STOCHASTIC DIFFERENTIAL EQUATION

279

It is intuitively clear that an average of the end-points is better than using either one. But is this procedure always correct? It is just as ad hoc as Ito's procedure. In Lax (1966IV), and in Section 10.1, Lax uses an iterative procedure that yields all results accurate to order At in two steps. The result, Eq. (10.14) and Lax (1966IV) Eq. (3. 10) yields

where D = cr(a) 2 is the diffusion coefficient. This result is in agreement with that found in Stratonovich, The justification for our procedure is that physical processes are described by noise that is only approximately white. For the physical process, one can use the ordinary methods of calculus. The iteration is necessary to retain terms that will retain a finite value in the limit as the correlation of the noise approaches a delta function. Direct use of the Ito choice, Eq. (16.32), and starting from Eq. (16.16), leads to Eqs. (4.6|11.1) in Hull (1989(2001), the result quoted in our Eq. (16.25). We simply claim that this result is not the answer to the original model, namely that the logarithm of the price obeys the standard Brownian motion. A direct proof of this remark can be made without using stochastic integrals. Noting that the Gaussian distribution of P(x, t) satisfies the Fokker-Planck equation

which contains the constant diffusion term D = (l/2)<7jj, and the constant drift term A = p,. One can obtain the equation for S from the equation for P(x, t), by introducing the relation x = ln[S/S(0)]:

The result after some labor is

The Ito equation corresponding to this Fokker-Planck equation is simply

written in our notation, where tc = t — e guarantees that the last term averages to zero. The same equation written in Ito notation looks like:

280

STOCHASTIC METHODS IN INVESTMENT DECISION

The confusion in the financial literature arises because Eqs. (3.7J10.6) in Hull (1989)2001) states that his model is Lax :

but by occasionally multiplying this equation by S (without the strict use of the Ito's calculus lemma) to obtain or

Hull :

In summary, we do not claim that the Ito definition is wrong, but requires extreme care to obtain correct results. It tends to mislead smart people into obtaining an incorrect answer. The proper intuitive view for the Black-Scholes model should be that x, the logarithm of the price, obeys a standard Brownian motion. In other words, Eq. (16.12) is the correct model regardless of which calculus is used. When one makes a change of variable from the logarithm x, to the actual price, S, the appropriate stochastic differential equation that obeys the Ito rules will be Eq. (16.39). The differences between Hull's results and mine (Lax) (as well as with some of his own results) is due primarily to the use of two different models. The Ito notation just obscures this point. It could be used with great care to obtain correct results. The models based on Ito's lemma should be avoided because it is counterintuitive for physical reasons. On the other hand, the procedure used in Section 10.3 would reduce the number of errors and avoid the use of Stieltjes and Lebesgue integration. The main disadvantage of our proposal is that it will reduce the number of jobs needed for mathematicians to teach measure theory. The discussion of models for stock prices and market behavior can then be devoted more heavily to real world questions and less heavily to formalism. For volatile stocks, the difference between the two possible market models, Eq. (16.12) and Eq. (16.16) can be appreciable. In a completely rational world, the growth parameter would be determined completely by the growth rate in earnings per share. Since this is not the case in the real world, the parameter /z is obtained by fitting against stock prices. Depending on how this is done, the fitting procedure might cancel the error in the use of the model Eq. (16.16) instead of Eq. (16.12). In our work on laser line-widths discussed in Chapter 11, the growth and decay rate can be separately determined, and there is no flexibility in our choice. The excellent agreement for the laser line-widths with experiment, as shown in Fig. 11.3, supports the iterative procedure used in Chapter. 10 for relating the Langevin to the Fokker-Planck pictures. It is supposed that the mathematical techniques developed for the study of random processes in physical systems can be applied in future for the economic and financial worlds.

VALUE OF A FORWARD CONTRACT ON A STOCK

16.6

281

Value of a forward contract on a stock

As the simplest application of Ito's lemma, Hull (Example 4.1) considers the value of a forward contract on a nondividend paying stock. We already found in Eq. (16.2) that the forward price should be

However, the dynamics of S implies an associated dynamics of F. To get dF/dt when Ito's lemma is used, Hull's Eq. (4.16) which applies to any function of S and t is

The extra term involving d2F/dS2 due to Ito's lemma vanishes for the choice of Eq. (16.41), since F is linear in S. Thus F obeys a dynamics, including the noise, similar to that of the stock price S, but with the growth rate v reduced by the riskfree interest rate r. However, in Eq. (16.43) v = p, + (l/2)<7^ when the physical stock model is used in Ito's formula, which leads to [p, — r + (l/2)a^l]Fdt in the first term. Now, we use our Langevin approach described in Section 10.3, where the ordinary calculus can be used. From Eq. (16.12), we have for S

where For F, we have

in Eq. (16.19) is used.

In order to obtain the average d(F}, using

we have and for the conditional average, (F} on the right hand side is replaced by F, which is in agreement with result of Ito's approach.

282

STOCHASTIC METHODS IN INVESTMENT DECISION

Our Langevin approach is easier than that using Ito's lemma. Instead of using Ito's calculus lemma at each steps of transform from x to S, and to F, the ordinary calculus rule can be used in each step in our approach, and d(F}/dt is then determined using Eq. (16.46) at last step. 16.7

Black-Scholes differential equation

We showed in Section 16.2 that the risk of owning an asset could be canceled out by also having a forward contract to sell the asset. And the combination is risk neutral provided that an appropriate value F is set as the forward contract. Can this scheme be extended to the case when the asset is a stock that has a growth rate ^5 and is subject to noise proportional to the value of the stock? It is assumed that derivative asset (say a put) has a value g = g(S, t) for one derivative security. Using the Ito's lemma, g(S, t) obeys the stochastic differential equation

where, by the Ito convention, the second term has a vanishing average value. To obtain a risk free portfolio, we must have a combination of assets in which the term related to the change rate of stock v vanishes. This can be accomplished by combining a put the equivalent of —1 shares which takes the value —g, with dg/dS shares of the stock. The combined value is

where the subscript t reminds us that the number of shares does not change during time evolution, except when it is adjusted by investor. The ratio of these two components was chosen to cancel the v contribution from each other. This cancellation occurs if one follows Hull and writes

Using Eq. (16.48) for Ag and AS = vS + aHSdz, the result is

Assuming the validity of this result, the value of II must grow at the interest rate r , or arbitrageurs could make a risk free profit:

DISCUSSION

283

The result is the differential equation

Thus we obtain the Black-Scholes equation:

In the above Ito's equations, z/ = // + (1/2)0^. However, v has been canceled out in the Black-Scholes equation. Now, we use our Langevin approach described in Section 10.3 for the combined asset IT in Eq. (16.49). Using the ordinary calculus, the Langevin equation for II is given by

where The contribution to the average or the conditional average with II (t) = IT, from the second term in Eq. (16.55), is the average of

The result on average of A(II) is same as Eq. (16.51). If the definition of Ito's integral is applied, the second term in Eq. (16.55) should be zero, because a(g) is estimated at time t, hence, the fluctuation term is completely canceled. However, according to our point of view, through a very short period of time, the change of a(g) leads to the nonzero average of the second term Eq. (16.56), and it is impossible for one to adjust his shares in such a fast way. 16.8

Discussion

Stock model

The motivation to separate the right hand side of the stochastic differential equation into two terms and to assign a standard fluctuation force f ( t ) in the second term is not from mathematics, but from modeling of the real world. In physics, using physical modeling of a system of finite parameters (for example, an oscillator, or a system of coupled oscillators) and using the physical

284

STOCHASTIC METHODS IN INVESTMENT DECISION

rules (for example, the Newton's law), the deterministic function of drift velocity B(a,t) of a system can be determined, and it should be a smooth function of time t. On the other hand, the fluctuation force comes from connection of this system with the "heating reservoir", which has infinite dimensions. See Sections 7.6 and 7.7 for detailed discussion. Concerning the price of a stock, in a completely rational world, the growth parameter of the price of a stock would be determined by the growth rate in earnings per share and other known conditions. Since this is not the case in the real world; the stock price fluctuates because of many unknown causes. The fluctuation force introduces an irregular, nondeterministic, and very fast-varying oscillatory motion of S with time. LSDE builds a model to represent effects of two different original forces on the system. Of cause, there are some stochastic processes (for example, scattering in a turbid medium in physics), where fluctuation plays the dominate role, and separating into two terms in LSDE is not proper. Here, we limit, however, to the cases that the model in LSDE is suitable, as well as in ISDE. The equation of motion described by a linear model of a stock, Eq. (16.12) or Eq. (16.15), is only the first perturbative approximation of a nonlinear model, and is only valid over a very short period from the spot time. For a longer period of time, the first term in this equation means that the price of S will increase (or decrease) exponentially, which certainly does not reflect real development of S. Also, the second term of this equation means that the width of fluctuation of S will increase with time, and there is no force to limit the amplitude of fluctuation. We suggest the following model for a stock S. First, a model for underlying value of a stock So(t) can be built based on fundamental analysis, which is a deterministic and slow-varying function of time. One may also build a linear model of So(t) by use of an extrapolation of previous data of S to obtain So(t) = n(t — to) So (to), where So (to) is the extension of previous value of SQ up to the spot time to, and n is a rate of increasing (or decreasing) of the stock. The So(t) provides an "operating point" for stock S as a random variable. We define S(t) = S(t) — So(t), and build a stochastic differential equation for S(t~). The stock value S'(t) oscillates around So(t), hence, B(S,t~) represents a model of drift velocity of S. The simplest model for this purpose is a harmonic oscillator, where a constant recovering force F = —aS makes S(t) oscillating around S = 0 with a certain amplitude. The amplitude of S(t) limits the value of S(t) within a certain range. When a fluctuation force is added, the value of S(t) will not diffuse to infinity with time. However, the velocity of a harmonic motion is too fast when S is near zero, and too slow when S is near its amplitude value. The real situation is that S(t) slowly changes when its value is near S'o(t), but it rapidly changes when its value

DISCUSSION

285

largely deviates from So(t). This suggests that a nonharmonic model is needed, for example, the recovering force can be chosen as F — —aS — (3S3. Recently, a form of B(S, t) = —aS is often used, which originates from Uhlenbeck's model. Under this linear model S(t) will exponentially approach to So(t) from its spot value. The advantage of this model is easy to be manipulated in calculation. However, one may ask why S(t) only approaches to So(t) from one side, but cannot cross through So(t) to the other side of So(t). If S is limited to oscillate in a certain range, cr(S) in the second term of SDE should not dramatically change, and the extra term in Ito's lemma (or the contribution to expectation from the second term in LSDE) may be relatively small and can be treated using some approximation, that reduce the difficulty in solving the equation of motion of the expectation value of a derivative security with time. Conditional expectation

We emphasize that the stochastic processes in finance, similar to that in physics science, are natural processes. People who have some prior information and knowledge may build the more proper model to match development of the natural stochastic process of a marketed asset, but, in general, does not change this natural process. The state of a random variable a at time t is best described by its probability distribution P(a,t), not its spot value realized at time t, because the spot value of a is undetermined and it jumps up and dawn in a very fast way. Based on this view point, we would like to discuss the concept of the conditional expectation. Mathematicians denote the conditional expectation by the symbol E(Xt> \J-~t), t' > t, where Xf is a stochastic process, and J-± is a a-field known as "natural filtration". For example, estimation of the price of an option (or future) H(T — t) with maturity time T is determined based the spot value of its underlying stock S at current time t. Speaking differently, one already has information that the closing price today of S(t) is S. This information provides a filtration under which the the expectation of S(t') in future can be calculated. However, one may ask a question whether the the price H(T - t} determined based on S at 4:00 PM, or that based on 5" at 3:50 PM is more reasonable, since the spot value of S may be remarkably different by irregular jumps during the last ten minutes before closing. In our opinion, the expectation based on probability distribution P(S, t) at spot time t could provide a more reasonable estimation of H(T -1}, not the conditional expectation, because the probability P(S, t) can not dramatically change during a small interval of time, but its realization value can jump up or dawn during a very short period of time. In physics, we use "ensemble" to describe all possible states of a system under a certain probability, and do not regard a sample in an ensemble as a meaningful quantity. In Monte Carlo simulations we do not regard a single

286

STOCHASTIC METHODS IN I N V E S T M E N T DECISION

path essential; similarly, the point by point value in the time path of a stock is not essential. Discrete processes

Discrete random processes, for example, the early process after a suddenly big jump of a stock, is beyond the LSDE description and also beyond ISDE. In discrete random processes, the nth order diffusion coefficients Dn are nonzero, up to infinite order of n, and Eq. (10.4) should be replaced by a series of equations:

In general, it is a difficult problem since it needs an infinite number of parameters to describe the detailed shape of the probability distribution function. In some special cases, for example, the Poisson process, the generation-recombination process, and the Rice shot noise processes, the parameters describing fluctuation can be reduced to a finite number. Chapters 8 and 9 provide a theoretical approach where cumulants higher than second order is involved, so that the distribution will be nonGaussian. In this case, a generalized Fokker-Planck equation, instead of the ordinary Fokker-Planck equation, should be used, which is discussed in Section 8.4. A similar example in physics is that particles are injected in a scattering medium with certain velocity, hence, the process is not a pure Brownian motion. At early time, particles move as "ballistic". By multiple collision with scatterers in the medium, the particle distribution changes with time from ballistic-like to a Gaussian-like. This process is quantitatively studied in Chapters 13 and 14. Similar phenomena often appear in the stock market. 16.9

Summary

Our viewpoint expressed in Lax (1966IV), Section 16.4 is that mathematicians have concentrated too exclusively on the Brownian motion white noise process which has delta correlation functions. Real processes can have a sharp correlation time of finite width. Thus their spectrum is flat, but not up to infinite frequency. Thus, for real processes, the Riemann sums do converge, and no ambiguity exists. After, the integration is performed, the correlation time can be allowed to go to zero, that is, one can then approach the white noise limit. Thus the ambiguity is removed by approaching the integration limit, and the white noise limit in the correct order. In summary, there are two kinds of stochastic differential equations: (a) those whose random term need not average to zero, as used in Eq. (10.33), and (b) those used by Ito in Eq. (16.32) in which the average of the random term vanishes

SUMMARY

287

by Ito's definition of the stochastic integral, together with Ito's calculus lemma. Both can be used correctly, but Ito's choice requires more care because the usually permissible way in which we handle equations is no longer valid. An able analysis of this situation, with references to earlier discussion, is presented by van Kampen (1992). Hull (1989, 2001) book is an extremely well-written text on Options, Futures, and other Derivative Securities. The Ito-Stratonovich controversy applies to physics, chemistry and other fields. We have analyzed Hull's treatment in detail because of his use of the Black-Scholes (1973) work. Although the Ito choice can be dangerous, as shown above, we trust that by now the Ito choice is used consistently in practice. However, there may be more serious problems, since real options may have statistics based on more wildly fluctuating processes than Brownian motion, such as Levy process and fractal processes. This has recently been emphasized by Peters (1994), by Mandelbrot (1997), and by Bouchard and Sornette (1994). For an excellent review of Brownian and fractal walks see Shlesinger (1997). Gashghaie et al (1996) have also found a parallelism between prices in foreign exchange markets and turbulent fluid flow. Thus the conventional Brownian motion approach will be invalid in such markets.

17

Spectral analysis of economic time series

17.1

Overview

Nature of the problem

Most of our discussion of random variables referred to variables existing over a continuous time. The phrase "time series" emphasizes variables known only at a discrete set of times. The phenomenon may actually be continuous in time, but experimental measurements may have been made only at discrete times. Closing (and/or opening) stock market prices occur only one each day, although prices may be available minute by minute. Extrapolation, Interpolation and Smoothing of Stationary Time Series, the title of a book by Wiener (1949a), indicates the problems to be solved. In Section 17.10 we present a sampling theorem that demonstrated that the value of a band-limited function sampled at the Nyquist rate, 1/(2W), where W is the bandwidth, can be deduced at all points from all the sample values on a discrete lattice. In this chapter, however, we face the more difficult problem of answering the same question when data is available only over a finite number of points. Even more difficult is to do the smoothing, interpolation or extrapolation when we deal not with a deterministic function, but with a random variable. Norbert Wiener (1930) laid the foundations of this subject. In his Generalized Harmonic Analysis, he also developed prediction methods used in gun control that were first made public in Wiener (1949b). A brief summary of Wiener's work leading to a derivation of a Wiener-Hopf type of integral equation is given by Levinson (1947). The smoothing, or filtering part of Wiener's work is an attempt to extract signal from contaminating noise, and has much in common with our work in Chapter 15 on "Signal Extraction in Presence of smoothing and Noise". However, the word smoothing is used in two different senses. In Chapter 15, the smoothing is that which occurs in any measurement process, and one tries to invert this process, to obtain a sharp signal from a blurred one. In Wiener's case, one merely wishes to smooth away the effects of added noise to get at what the signal would be if no noise were present. Statisticians are typically more at home in the time domain associated with "time series" whereas communication engineers prefer the frequency domain. These two are united by the Wiener-Khinchine theorem that establishes the (time)

OVERVIEW

289

autocorrelation as the Fourier transform of the noise spectrum. Wold's theorem is the corresponding relationship for a discrete time series. Whether the problem is discrete or continuous, it is clear that a principal problem is to determine the spectrum of frequencies contained in the data. Presumably, if one can determine the spectrum of the nondeterministic part of the motion, one can smooth out, or attempt to eliminate the random part of the motion, and to better predict the nonrandom part. Actually, one usually may proceed in the opposite direction to eliminate any secular part of the motion, and any sharp (periodic) frequency components. This last part includes making seasonal corrections to the data. Then a better estimate of the residual spectrum can be made. Presumably, better results can be obtained by using a combined procedure, or by iterating the trend removal and spectral analysis procedures. Why is the problem so hard Why is the spectral analysis of time series so hard that it has spawned dozens of books and hundreds of papers, often recommending different procedures. After all, a direct Fourier transform of a time series can be performed using the FFT (fast Fourier transform) which is normally a reliable accurate procedure. One problem is that for continuous time data, a Fourier integral is only given correctly by a discrete sum if the function is bandlimited. See the discussion of the sampling theorem in Section 17.10. This difficulty is compounded by the fact that even if the trend or periodic components in the data are band-limited, the noise present in the data is not frequency limited. The result is that the spectral calculation, is an ill-posed problem, whose answer, like all the problems discussed in Chapter 15, is nonunique until the problem is regularized by some smoothing procedure. The customary smoothing procedure in the time series case is referred to as windowing or tapering. Harris (1967) compares 23 different classes of windows. Priestley (1981) gives details of 11 windows, and Percival and Waldon (1993) uses four windows: Bartlett, Daniell, Parzen, and Papoulis, in examples, and describes Manning and Hamming windows. The problems in this chapter are more difficult than those discussed in our previous chapters because here we are dealing directly with unaveraged data. In our earlier work, we started from some physical model and obtained equations, such as Fokker-Planck equations obeyed by the ensemble average. Here the aim is to deduce the model from the data. A compromise procedure involves introducing a model described by a small number of parameters, and using the data to deduce these parameters. Such parametric models are discussed in many texts. See, for example, Brockwell and Davis (1991). The rectangular window, a direct transform, is equivalent to what we have called the engineering definition of noise in Section 4.1. The unbiased form of this window is called a periodogram, and dates back to Schuster (1898). However,

290

SPECTRAL ANALYSIS OF ECONOMIC TIME SERIES

Thomson (1982) has shown that although the method converges as JV —» oo, in a typical physical example, reasonable results were only obtained for N > 106. There is a biased form of the periodogram that uses a factor 1 — r\/N that adds bias but reduces variance, with a net improvement. An optimum discrete window was described by Eberhard (1973). The window coefficients are given by a discrete prolate spheroidal sequence (DPSS). The fundamental importance of discrete prolate spheroidal functions was explained in terms of the uncertainty principle in a series of papers by Slepian (1964, 1965, 1978) with the initial paper by Slepian and Pollak (1961). Slepian and Pollak (1961) recognized that the uncertainty principle imposes limitations on the attempt to fit a nonbandwidth limited function, known only over a finite interval —T/2 < t < T/2 by a function band-limited over the frequency band — W < f < W. One can obtain an extremely accurate fit at the expense of wild oscillations from components outside the bandwidth. Slepian and Pollak (1961) proposed getting a least squares fit inside the band — W < f < W, while maintaining a fixed energy outside the bandwidth. Their result leads to a variational principle in which the fraction of the total energy inside the bandwidth is maximized. This turns out to be the equivalent of maximizing the fraction of energy within the time interval —T/2 < t < T/2. The variational expression leads to an integral equation whose solutions are recognized to be prolate spheroidal wavefunctions. These eigenfunctions of the integral equation constitute an orthogonal basis set. Thomson (1982) arrives at the same integral equations in a completely different manner by seeking a direct inversion of the relation between spatial and frequency amplitudes to obtain the frequency amplitude rather than just the power spectrum (or absolute squared amplitude). In this way, phase information is retained. Thomson's contributions My (Lax) interest in the field of time series has been greatly stimulated by personal contact with David J. Thomson who has spent a lifetime career covering all aspects of time series. We can describe the present chapter as our attempt to learn enough about time series to be able to read Thomson's work. We shall therefore record a subset of his publications to indicate the breadth of topics covered. His work started, appropriately for Bell Laboratories, with the analysis of time series in waveguides used to transmit information in the telephone network. See Thomson (1977). This work was expanded in Thomson (1982) already referred to. His 1982 paper constitutes the foundation of much of his later work. In Kleiner, Martin and Thomson (1979) Thomson shows how to apply Tukey's ideas of robustness to spectral estimation. Tukey's book on Exploratory Data Analysis shows how to deal with real data. When is an apparently deviating point an outlier to be discarded? See Tukey (1977) and Thomson (1982). His work was also applied to the

THE WIENER-KHINCHINE AND WOLD THEOREMS

291

global warming problem in Kuo, Lindberg, Craig, and Thomson (1990). This first serious paper on 'recent' climate has been cited by Al Gore! Thomson (1990a) on analysis of the earth's climate extended to a period of 20,000 years, and correlated CO2 data with tree ring data. His next work, Thomson (1990b), extended time series over 600,000 years and established a sensitivity of the results to the small time differences between the siderial year, the equatorial year, and the solar year. Thomson and Chave (1991) also adapted jack-knife procedures to deal with nonnormal variables and confidence limits. The current picture of global heating is discussed in Thomson (1995). Thomson, MacLennan, and Lanzerotti (1995) use time series techniques to analyze the propagation of solar oscillations through the interplanetary medium. Perhaps Thomson's most important work on global heating filters the precession signal out of the data and establishes a strong correlation between global worming and the CO2 concentration. See Thomson (1997). An application is made to the financial world by studying stock and commodity data over a 40 year period; see Ramsey and Thomson (1999). A comprehensive review was given by Thomson (1998) of his work on "Multitaper Analysis of Nonstationary and Nonlinear Series Data", presented at the Isaac Newton Institute.

17.2

The Wiener-Khinchine and Wold theorems

The purely random part of a time series is usually described as a superposition in frequency space of various spectral contributions. The inverse of Eq. (4.48) in our notation [with a(t) replaced by x(t) but a(uj) replaced by x(/)] is

It is consistent to think of x(t) as a realization or sample of the random variable X(t). In statistics books the above equation is written in Stieltjes notation as

where a comparison of these notations suggests

The Stieltjes form is needed for mathematical rigor, when the spectrum of X contains Brownian motion (white noise), or a delta function of time autocorrelation. In the real world, in which the noise can be approximately white, and spectral lines are narrow, but not infinitely so, Stieltjes integrals are unnecessary, and our simpler

292

SPECTRAL ANALYSIS OF ECONOMIC TIME SERIES

notation can be followed. Even when white noise is present, the second moments are defined via Eq. (4.50) and the relation

The first and last terms follow the notation of the time series texts by Percival and Walden (1993) and Priestley (1981), and the middle terms follow the notation in this book. The Wiener-Khinchine theorem (for continuous time), Eq. (4.11), takes the form

For the discrete time case, with x(t) known only at the integers t = 0,±1,±2,..., the same theorem applies but the limits of integration in Eq. (17.5) extend only from — 1/2 to 1/2. That is because exp(j2vr/i) is indistinguishable from exp[j2vr(/ + l)t] since t is an integer. Thus all frequencies outside the basic bandwidth from —1/2 to 1/2 are folded into that basic bandwidth. This process is referred to as aliasing, and is well known to anyone who has watched rotating wheels in the movies (at l/24th of a second) and found that they can appear to be rotating backward. In the study of crystals, there is a similar folding of all wave vectors into the first Brillouin zone. Since the spectrum S(f) is, by definition positive, the normalized p(f) = S(f)/ j S(f)df is a probability density. Correspondingly, we defined Priestley (1981) describes the Wold (1938) theorem, Eq. (17.7), below, as the necessary and sufficient condition that the set of numbers p(±n) can be an autocorrelation. The Wold condition is

that is, the autocorrelation p(r) must be the Fourier transform of a probability density p(f). Assuming that we are dealing with a stationary random process, the first objective in dealing with a time series is to obtain its spectrum. The simplest procedure, for a single discrete sample of x(t] for t = 1 , 2 , 3 , . . . , AT is to use the estimate

This is referred to as the Schuster (1898) periodogram in the statistics literature. It is equivalent to what we have called our standard engineering definition of noise,

MEANS, CORRELATIONS AND THE K A R H U N E N - L O E V E THEOREM 293

Eq. (4.3), except that the latter definition takes an ensemble average and a limit as N —> oo. In time series analysis the ensemble and time averages are not available since we have only one finite set of data. A closely related estimate of the periodogram can be obtained by using the Wiener-Khinchine theorem to get the spectrum as the Fourier transform of the autocorrelation and the estimate the latter from the sample data. For times, t, separated by At instead of unity, the spectrum is given in terms the correlations by

where ST is given by

with the Nyquist sampling frequency. If one makes the common choice of units such that At = 1, one obtains Eq. (17.8). Equation (17.10) is biased. If all the summands were unity, the result would be 1 — r\/N. The correlator associated with this periodogram can be modified by replacing 1/N by 1/N' in Eq. (17.10), with Nf is given the value N — \T\, and the "periodogram" is said to be unbiased. However, due to variance errors, the biased (original) periodogram is often superior. We here follow the notation and approach of Percival and Weldon (1993) including the assumption that the process mean is zero, which is easily eliminated by subtracting the mean from each variable. By rearranging the order of summation, the inverse of Eq. (17.10) then permits the spectrum to be estimated by

Thomson and Chave (1991) however claims that the use of autocorrelations and spectral densities throws away phase information. He therefore follows a procedure outlined in Section 17.4. 17.3

Means, correlations and the Karhunen-Loeve theorem

Means and correlations The mean of the set of random variables (RV) X± at all times t is given by

294

SPECTRAL ANALYSIS OF ECONOMIC TIME SERIES

where the last step is valid if the process is restricted to be stationary. An unbiased estimate of the mean is given by

since E(X) = //. Note that X is still a random variable. No ensemble average has been performed. The variance of Xt is denned by where, again, the last expression is valid only in the stationary case. The covariance of two RV is defined by is linear in both X and Y. The correlation of two random variables is defined by The variance in X, the estimate of the mean can be written

If one sums first along the diagonal (at fixed r = t — s), Eq. (17.18) can be rewritten as

where we note that TV — |r| is the length of the relevant diagonal and r| < N. Since the sum converges, var(X) approaches 0 as N —» oo, and the estimate X is unbiased. The Karhunen-Loeve theorem

Our previous discussion describes the expansion of our series variables Xt = X(t) in a Fourier series. Are there advantages in expanding in another set of orthogonal functions (£)? For simplicity, let us assume that mean values have been subtracted off so that The Karhunen-Loeve theorem, (karhunen 1947, see also Kac and Siegert 1947) states that if the orthogonal functions are chosen to be eigenfunctions of the correlation function R(t, s), the expansion coefficients will be uncorrelated random variables.

SLEPIAN FUNCTIONS

295

Proof Start with the expansion

Then the correlation of the expansion coefficients is given by

with and the eigenvalue equation

Eq. (17.22) reduces to and the theorem is proved. If one assumes stationarity, then

In the case of discrete time, the integral over time is replaced by a sum over time. But the theorem remains valid. 17.4

Slepian functions

Although Slepian's argument that spheroidal functions should be used as a basis to localize signal as much as possible in space and time is fairly compelling, it does not establish that this procedure yields the best spectrum. Thomson (1982) starts, instead, from Cramer's spectral representation of the discrete signal x(t) where t = l,2,...,n,...

where for convenience, times are measured from the center of the series. The limits ±1/2 are appropriate for the spacing At = 1 since that corresponds to the Nyquist

296

SPECTRAL ANALYSIS OF ECONOMIC TIME SERIES

sampling rate, and frequencies outside the first zone can be shifted in to the first zone by a change of an integer which has no effect on the exponential. The Stieltjes integral can be replaced by a Riemann integral for a continuous spectrum by making the replacement as was done to get the second form of Eq. (17.27). It is necessary to remember that x(n) is the discrete time series, and that dZ(v) and x(v) are in frequency space. A zeroth approximation to the frequency amplitude is given by taking the discrete FFT:

Since Eq. (17.29) provides a Fourier series representation of y(f) with period 1, the Fourier coefficients are given by

If one now inserts Eq. (17.27) for x(n) into Eq. (17.29) one obtains

where KN is simply a geometric sum

This kernel reduces to sinc(Arvr/) = sin(A^7r/)/(vr/) in the limit of continuous time, the same function as found in Eq. (17.71) for the case of continuous variables. The continuous kernel had appeared very early in the classic work by Wannier (1937). Equation (17.31) can be regarded as the integral equation for the signal x(v) given a measured output y(f). This is exactly analogous to the integral equation relating the signal s(x) to the measurement m(x) in Chapter 15 on extraction of signals from noise in an ill-posed problem. As in that case, the nature of the difficulty can be understood by performing the inversion in a representation in which the measuring device K has been diagonalized. In that representation the inversion simply multiplies each component by the inverse of the corresponding eigenvalue.

SLEPIAN FUNCTIONS

297

The difficulty occurs because the high-order eigenfunctions, which are rapidly oscillating, have small eigenvalues. Thus any rapidly oscillating noise is greatly amplified. Hence, like all ill-posed problems the solution is not unique and cannot be made so without a regularization procedure that rejects rapid oscillation. In the time series problem, the time-honored procedure is to filter the inversion procedure through a low-pass "window". Thomson's (1982) procedure is to generalize to a multiwindow ("multitapered") procedure. The above arguments indicate that it is appropriate as well as convenient to solve Eq. (17.31) by expanding the solution in terms of the eigenfunctions of this Dirichlet kernel, £>(/) = KN(f)/N,

In the presence of band limitation, W will be less than 1/2. These eigenfunctions are referred to as the Slepian functions, in honor of Slepian who recognized their importance in signal representation. Each eigenfunction depends on N and W as parameters, so a complete labeling of the Slepian eigenfunctions is Uk(N, W, /). These are also referred to as Discrete Prolate Spheroidal Functions (DPSF). To obtain spectral amplitudes, we must solve Eq. (17.31) for x(v) in terms of j/(/), the preliminary direct spectrum (where v is also a frequency). By expanding the known y(f) in terms of the eigenvectors

we can solve for x(f)

or in final form

The properties of the Slepian (1978) functions are analyzed in great detail in his paper. The eigenvalues are found to be nondegenerate and monotonic decreasing: The eigenfunctions are found, surprisingly, to be orthogonal over the interval of [—1/2,1/2] as well as over [-W, W]. If they are normalized over the larger region

298

SPECTRAL ANALYSIS OF ECONOMIC TIME SERIES

(where no complex conjugate is taken because the eigenfunctions are real), the corresponding orthogonality condition over the limited bandwidth is

Comparison between Eqs. (17.38) and (17.39) shows that the eigenvalue A& represents the ratio of the energy in the inner region [-W, W] to that in the full region [—1/2,1/2]. This guarantees that all the eigenvalues are less than one, and in view of Eq. (17.37), the shape most concentrated in frequency is associated with the k = 0 mode. Indeed, that is why Eberhard (1973) advocated using that mode, alone, as the appropriate window. 17.5

The discrete prolate spheroidal sequence

DPSF Uk(f) as functions with a period of unity in frequency space, can be represented as a Fourier series,

This series from n = no to n = no + N — 1 is centered at the midpoint. The (k] sequence vn is called the discrete prolate spheroidal sequence (DPSS). Slepian (1978) considers this series with an arbitrary choice of no, whereas Thomson (1982) specializes to the case of no = 0. We shall follow Thomson's choice here. The phase factor, otherwise arbitrary, is chosen to be

to conform to Slepian's notation. The sequence obeys the matrix eigenvalue equation

where

The eigenvalues are shown to be monotonic decreasing

In principle, the eigenvectors and eigenvalues can be obtained by standard subroutines such as those included in EISPACK, or modernizations of the same.

THE DISCRETE PROLATE SPHEROIDAL SEQUENCE

299

Unfortunately, this is not useful because the first 2NM eigenvalues are close to unity. Table 17.1 displays the first four eigenvalues, A/t, for k = 0, 1, 2, 3, listed as \k(N, W), for N = 31 and three bandwidths W = 6/31, 7/31, 8/31 obtained in this way. TABLE 17.1. Eigenvalues, \k(N, W), of the discrete prolate spherical series (after Percival and Walden 1993). k A fc (31,6/31)_ A fe (31,7/31)_ Afc(31,8/31) 0 0.9999999999999997 1.000000000000007 1.000000000000002 1 0.9999999999999769 0.9999999999999933 1.000000000000001 2 0.9999999999978725 0.9999999999999921 0.9999999999999945 3 0.9999999998764069 0.9999999999998924 0.9999999999999908 The eigenvalues are so close, include some values above unity, which are clearly wrong. This inaccuracy means that the associated eigenvectors, using these values, will also be inaccurate. In the case of continuous time problems Slepian and Pollak (1961) found that eigenfunctions that minimize the amount of energy outside some time boundary [—1/2, 1/2] and obey an integral equation, can also be found to be solution of a second order differential equation. For the discrete time problem, The solutions of an integral equation of the form of Eq. (17.33) can also be obtained as solutions of a second order difference equation

with the boundary conditions

This matrix equation has eigenvectors t/fc) and eigenvalues 9k, as functions of N, and W. The eigenvectors v^ (N, W) are identical to those in the original integral equation. However, the eigenvalues Ok(N, W} are now well separated. As a result, it is easy to calculate the eigenvectors vk accurately, without requiring quadruple precision, and then A& can be determined by

where the matrix Anm was denned in Eq. (17.42). The matrix in Eq. (17.44) is a symmetric, tridiagonal matrix. For such matrices there are subroutines in most libraries that will perform the diagonalization, using

300

SPECTRAL ANALYSIS OF ECONOMIC TIME SERIES

computation time of order N, rather than the time of order N3 needed for the general matrices. The necessary subroutines appear in the IMSL, NAG, and the PORT Bell Laboratories Mathematical Subroutine Library by Fox, et al. (1978a) as well as among the subroutines included in the Numerical Recipes book by Press et al. (1992). 17.6

Overview of Thomson's procedure

The procedure used by Thomson (1982) to analyze time series, in 41 IEEE pages, is sufficiently complicated that we would like to give a road-map. (1) The first (optional) step is to replace the original input data by a new set that is prewhitened. An example of such a procedure is outlined by Percival and Walden (1993). Convolute Xt with gu to get

The spectrum associated with g is given by

If the estimated spectrum, 5y (/), of Y is obtained by methods discussed below, that of §x (/) can be obtained from

If one can choose
Thomson (1982) recommends that this smoothing window be constructed using methods due to Papoulis (1973). The subscript D above implies that a data window

HIGH RESOLUTION RESULTS

301

Dn has already been used. However, the post smoothing window depends on the choice of data window. (3) The principal element of the Thomson (1982) method is to window the original, or prewhitened data by the discrete prolate spheroidal sequence (DPSS), Dn = (k] Vn , which proportional to the discrete Fourier transform of the discrete prolate spheroidal function (DPSF), [/&(/). Here, /, is a frequency, n is an integer, for the discrete time t, and k is an integer that refers to the fcth mode, £/&(/). Both Vn and Uk(f) are functions of N, the number of data points (times) and W, the bandwidth. 17.7

High resolution results

The first preliminary estimate of the spectral amplitude is y(f), the discrete Fourier transform of x(n), given in Eq. (17.29). But this covers the full frequency range, [—1/2, 1/2] with a single formula. Thomson (1982) in his Eq. (3.1) suggests that a Fourier transform truncated to the interval [/ — W, f + W] would provide a better resolution using the formula

This proposal is an excellent one, since one can prove that if S(f) were flat over this smaller interval, it would yield the correct spectral density at /. Presumably because Eq. (17.51) cannot be expressed directly in terms of observed x(n), Thomson introduces another formula for quantity yk(f), related to Zk(f) by Eq. (17.58) in the next section,

which also covers the truncated frequency region. Since y(f) is expressible i terms of the data x(n) via Eq. (17.31), he then obtains

after using the integral equation (17.33) for the Slepian functions. The advantage of this new form is that it is directly expressible in terms of the data

The disadvantage of this estimate is that it is no longer local but has contributions from the entire [—1/2,1/2] domain.

302

SPECTRAL ANALYSIS OF ECONOMIC TIME SERIES

Thomson then uses yk(fo) to obtain an estimate of the spectral density S(f; /o) at any / in the interval /o — W < f < /o + W . He then averages this result over the interval to obtain the average result

where each can be regarded as an individual spectral estimate, with the fcth data window

17.8

Adaptive weighting

Thomson (1982) attempts to obtain an estimate closer to the genuinely local estimate of Eq. (17.51) by using a weighted average

The weights are chosen to minimize the sum of least squares of namely the differences between the ideal estimate of the amplitude and the weighted estimate dk(f)yk(f)- The result is that the weights are estimated from

This procedure depends on S(f) and so must be self-consistent. The result is an iterative solution of

Here £?&(/) is referred to by Thomson as the broadband bias

The integral in Eq. (17.61) is over the cut region

The evaluation of the estimate B(f)is discussed in Thomson (1982), Section 5.

TREND REMOVAL AND SEASONAL ADJUSTMENT

17.9

303

Trend removal and seasonal adjustment

For an elementary discussion of the removal of periodicity and trend see Williams (1997). For a broad perspective on spectral analysis see Tukey (1961). A good general reference is Handbook of Statistics III, by Brillinger and Krishnaiah (1983), as is Harris (1967) and Tukey (1967). A more detailed description of the techniques used in what is now called "complex demodulation" is given in Hassan (1963). The methods described in this chapter are nonparametric, in the sense that one doesn't have a model with a small number of parameters which are fitted by analyzing experimental time series data. Fourier techniques have been used. In a sense, parameters are involved. But there are so many that no assumptions are really made that pertain to a model, except possibly the assumption that one is dealing with a stationary process. A recent comparison of the relative merits of parametric and Fourier transform methods has been given by Gardiner (1992). He also suggests methods of combining parametric and Fourier methods. For an excellent elementary presentation of methods of dealing with nonstationary time series see Cohen (1995) who discusses the effects of many of the popular filters. 17.10

Appendix A: The sampling theorem

The Poisson sum formula A useful lemma, see Lax (1974), that leads to a special Poisson sum formula is

This theorem can be established by taking the left hand sum from —NtoN, and then taking the limit. A simpler approach is to note that the right hand side g(x) regarded as a function of x, is periodic

The Fourier series representation over the domain 0 < x < a leads to the relation

where

304

SPECTRAL ANALYSIS OF ECONOMIC TIME SERIES

In order for Eq. (17.65) to be true, we must have

In order that g(x) be periodic, A(x — x') must possess the same delta function in each interval from na to (n + l)a. In other words,

If this result is equated to the definition, Eq. (17.66) we have the special Poisson formula, Eq. (17.63). If we integrate Eq. (17.63) over an arbitrary function f ( x ) , we obtain the usual Poisson sum formula, see Titchmarch (1948),

where f ( x ) is the Fourier transform of F(k),

A generalization of the Poisson sum formula to three-dimensional (possibly nonorthogonal) lattices was given in Lax (1974), Chapter 6. The sampling theorem

and B = tr/a, makes the remarkable statement that f ( x ) is determined everywhere by its sample values f ( n a ) provided that its Fourier transform, F(k), in Eq. (17.70) is band-limited to the region

Proof Since F ( k ) = 0 unless \k\< B = vr/a, we can represent F(k) by a Fourier series over this finite region:

APPENDIX A: THE SAMPLING THEOREM

305

where the Fourier coefficients are determined by

where the second form makes use of the definition, Eq. (17.70). If Eq. (17.74) is inserted into Eq. (17.74) we obtain

Thus F(k) is uniquely determined by the sample values f(na). Inserting Eq. (17.75) into Eq. (17.70), but restricting the integration region to the band from —TT/O, to TT/O we immediately obtain Eq. (17.71), the sampling theorem. An alternate view of the sampling theorem can be phrased as follows. Suppose one has a set of values f(na) at the lattice points na and we wish to interpolate to obtain the values /(x) at other points. The sampling theorem, in the form of Eq. (17.71) provides the smoothest interpolation in the sense that any other interpolation will not be band-limited, and hence will involve higher Fourier components.

This page intentionally left blank

Bibliography Abramatic, J. F. and Silverman, L. M. (1982). Nonlinear restoration of noisy images. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAM14, No. 2, 141-148. Alexander, H. W. (1961). Elements of Mathematical Statistics. John Wiley and Sons, New York. Alfano, R. R. ed. (1994). OSA Proceedings on Advances in Optical Imaging and photon Migration. Optical Soceity of America, Washington D.C. Anis, A. A. and Lloyd, E. H. (1976). The expected value of the adjusted rescaled hurst range of independent normal summands. Biometrika, 63, pi 11-116. Arfken, G. B. and Weber, H. J. (1995). Mathematical Methods for Physicists. Academic Press, San Diego. Backus, G. and Gilbert, F. (1970). Uniqueness of the inversion of inaccurate gross earth data. Philosophical Transactions of the Royal Society, 256, 123—192. Bayes, T. (1763). Essay towards solving a problem in the doctrine of chance. Philosophical Transactions of the Royal Society, Essays LII, 370—418. Bell, D. A. (1960). Electrical Noise. Van Nostrand. London. Bellman, R. E. (1964). Invariant Embedding and Time-independent Transport Processes. American Elsevier Pub. Co., New York. Bellman R. and Wing G. (1976). An Introduction to Invariant Embedding. Wiley, New York. Bendat, J. S. and Piersol, A. G. (1971). Random Data: Analysis and Measurement Process. Wiley, New York. Benjamin, R. (1980). Generalization of maximum-entropy pattern analysis. IEEE Proceedings, 127 pt F, 341-353. Bernstein, P. L. (1998). Against the Gods, the Remarkable Study of Risk. Wiley, New York. Black, F. and Scholes, M. (1973). The pricing of options and corporate liabilities. Journal of Political Economy, 81, 637-654. Bouchard, J. and Sornette, D. (1994). The Black-Scholes option pricing problem in mathematical finance: generalization and extensions for a large class of stochastic processes. Journal de Physique I (Paris), 4, 863-881. Brillinger, D. R. and Krishnaiah, P. R. (1983). Handbook of Statistics III. North Holland, Amsterdam.

308

BIBLIOGRAPHY

Brockwell, P. J. and Davis R. A. (1991). Time Series Theory and Methods. Springer Verlag, New York. Brown, R. (1828). A brief account of microscopical observations made in the months of June, July, and August 1827, on the particles contained in the pollen of plants; and on the general existence of active molecules in organic and inorganic bodies. The Philosophical Magazine, 4, 161-173. Bullard, B. C. and Cooper, R. J. B. (1948). The determination of the masses necessary to produce a given gravitational field. Proceedings of the Royal Society A (London), 194, 332-347. Burgess, R. E. (1965). Fluctuation Phenomena in Solids. Academic Press, New York. Cai, W., Lax, M., and Alfano, R. R. (2000a). Cumulant solution of the elastic Boltzmann transport equation in an infinite uniform medium. Physcial Review E, 61, 3871-3876. Cai, W., Lax, M., and Alfano, R. R. (2000b). Analytical solution of the elastic Boltzmann transport equation in an infinite uniform medium using cumulant expansion. Journal of Physical Chemistry B, 104, 3996-4000. Cai, W., Lax, M., and Alfano, R. R. (2000c). Analytical solution of the polarized photon transport equation in an infinite uniform medium using cumulant expansion. Physical Review E, 63, 016606. Cai, W., Xu, M., Lax, M., and Alfano, R. R. (2002). Diffusion coefficient depends on time not on absorption. Optics Letters, 27, 731-733. Cai, W., Xu, M., and Alfano, R. R. (2003). Three-dimensional radiative transfer tomography for turbid media. IEEE Selected Topics in Quantum Electronics, 9, 189-198. Cai, W., Xu, M., and Alfano, R. R. (2005). Analytical form of the particle distribution based on the cumulant solution of the elastic Boltzmann transport equation. Physical Review E, 71, 041202. Callen, H. B. and Welton, T. A. (1951). Irreversibility and generalized noise. Physical Review, 83, 34-40. Callen, H. B. and Greene, R. F. (1952). Theorem of irreversible thermodynamics. Physical Review, 86, 704-710. Cameron, R. H. and Martin, W. T. (1944). The Wiener measure of Hilbert neighborhoods in the space of continuous functions. Journal Mathematical Physics, 23, 195-209. Cameron, R. H. and Martin, W. T. (1945). Transformations of Wiener integrals under a general class of linear transformations. Transactions of the American Mathematical Society, 58, 184-219. Campbell, N. (1909). The study of discontinuous phenomena. Proceedings of the

BIBLIOGRAPHY

309

Cambridge Philosophical Society, 15, 117-136. Carley, A. E. and Joyner, R. W. (1979). The application of deconvolution methods in electron spectroscopy - a review. Journal of Electron Spectroscopy and Related Phenomena, 16, 1-23. Casimir, H. B. and Polder, D. (1948). The influence of retardation on the Londonvan der Waals forces. Physical Review, 73, 360-372. Cercignani, C. (1988). The Boltzmann Equation and its Applications. Series in Applied Mathematical Sciences, Springer-Verlag, New York. Chan, H. B., Aksyuk, V. A., Kleiman, R. N., Bishop, D. J., and Capasso, F. (2001). Quantum mechanical actuation of microelectromechanical systems by the Casimir force. Science, 291, 1941-1944. Chandrasekhar, S. (1943). Stochastic problems in physics and astronomy. Reviews of Modern Physics, 15, 1-89. Chandrasekhar, S. (1960). Radiative Transfer. Dover, New York. Chung, K. L. (1968). A Course in Probability. Harcourt Brace and World, New York, van Cittert, P. H. (1931). Zum einfluss der spaltbreite auf die intensitatsverteilung in spektrallinien. Zeitschrift fiir Physik, 69, 298-308. Cohen, L. (1995). Time-Frequency Analysis. Prentice Hall, New Jersey. Dahlquist, G. and Bjorck, A. (1974). Numerical Methods. Prentice Hall, New Jersey. Demoment G. (1989). Image reconstruction and restoration: Overview of common estimation structures and problems. IEEE Transactions on Acoustics, Speech, and Signal Processing, 37, 2024-2036. Deryagin, B. V. and Abrikosova, I. I. (1956). Direct measurement of molecular attraction of solid bodies. I. Statement of the problem and methods of measurement of forces using negative feedback. Zhurnal Eksperimentalnoi i Teoreticheskoi Fiziki, 30, 993-1006. Deryagin, B. V., Abrikosova, I. L, and Lifshitz, E. M. (1956). Direct measurement of molecular attraction between solids separated by a narrow gap. Quarterly Reviews - Chemical Society (London), 10, 295-329. Deutch, R. (1962). Nonlinear Transformations of Random Processes. Prentice Hall. New Jersey. Dirac, P. A. M. (1935). Principles of Quantum Mechanics. Oxford, New York. Doob, J. L. (1942). The Brownian motion and stochastic equations. Annals of Mathematics, 43, 351-369. Doob, J. L. (1953). Stochastic Processes. Wiley, New York.

310

BIBLIOGRAPHY

Durduran, T., Yodh, A. G., Chance, B., and Boas, D. A. (1997). Does the photondiffusion coefficient depend on absorption? Journal of the Optical Society of America A, 14, 3358-3365. Dyson, F. (1949). The S matrix in quantum electrodynamics. Physical Review, 75, 1736-1755. Dyson, F. (1949). The radiation theories of Tomonaga, Schwinger and Feynman. Physical Review, 75, 486-502. Eberhard, A. (1973). An optimum discrete window for the calculation of power spectra. IEEE Transactions on Audio andElectroacoustics, 21, 37-43. Eberly, J. H. and W'odkiewicz, K. (1977). The time-dependent physical spectrum of light. Journal of the Optical Society of America, 67, 1252-1261. Einstein, A. (1905). On the movement of small particles suspended in a stationary liquid demanded by the molecular-kinetic theory of heat. Annalen der Physik, 17, 549-560. Einstein, A. (1906). On the theory of the Brownian movement. Annalen der Physik, 19, 371-381. Einstein, A. (1956). Investigations on the Theory of the Brownian Movement. Dover, New York. Ekstein, H. and Rostoker, N. (1955). Quantum theory of fluctuations. Physical Review, 100, 1023-1029. Fadeeva, V. N. (1959) Computational Methods of Linear Algebra. Dover, New York. Feller, W. (1957, 1971). An Introduction to Probability Theory and its Applications. Volume I and II. Wiley, New York. Feynman, R. P. (1948). Space-time approach to non-relativistic quantum mechanics. Reviews of Modern Physics, 20, 367-398. Feynman, R. P. (1951). An operator calculus having applications in quantum electrodynamics. Physical Review, 84, 108-128. Ford, G. W. and O'Connell, R. F. (1996). There is no quantum regression theorem. Physical Review Letters, 77, 798-801. Ford, G. W. and O'Connell, R. F. (2000). Driven system and the Lax formula. Optics Communications, 179, 451—461. Fox, P. A., Hall, A. D., and Schryer, N. L. (1978a). The PORT Mathematical Subroutine Library. ACM Transactions on Mathematical Software, 4, 104-126. Fox, P. A., Hall, A. D., and Schryer, N. L. (1978b). Algorithm 528: framework for a portable library. ACM Transactions on Mathematical Software, 4, 177-188. Franklin, J. N. (1970). Well-posed stochastic extensions of ill-posed linear problems. Journal of Mathematical Analysis and Applications, 31, 682—716.

BIBLIOGRAPHY

311

Freeman, J. J. (1952). On the relation between the conductance and the noise power of certain electronic streams. Journal of Applied Physics, 23, 1223-1225. Frieden, B. R. (1975). Image enhancement and restoration in Topics in Applied Physics, ed. Huang, T. S. Springer-Verlag, New York. Fry, T. C. (1925). The theory of the schroteffekt. Journal of the Franklin Institute, 199, 203-220. Fry, T. C. (1928). Probability and its Engineering Uses. Van Nostrand, London. Furth, R. (1920). Die brownsche bewegung bei beriicksichtigung einer persistenz der bewegungsrichtung mit anwendungen auf die bewegung lebender infusorien. Zeitschrift fur Physik A, 2, 244-256. Furth, R. (1922). Die bestimmung deer elektronenladung aus dem schroteffekt an gluhkathodenrohren. Physikalische Zeitschrift, 23, 354-362. Gandjbakhche, A. H. ed. (1999). Proceedings of Inter-Institute Workshop on invivo Optical Imaging at the NIH, Optical Soceity of America, Washington D.C. Gardiner, W. A. (1992). Fundamental comparison of Fourier transformation and model fitting methods of spectral analysis. Imaging Systems and Technology, 4, 109-121. Ghashghaie, S., Breymann, W., Peinke, J., Talkner, P., and Dodge, Y. (1996). Turbulent cascades in foreign exchange markets. Nature, 381, 767-770. Glauber, R. J. (1963a). Photon correlations. Physical Review Letters, 10, 84-86. Glauber, R. J. (1963b). The quantum theory of optical coherence. Physical Review, 130, 2529-2539. Glauber, R. J. (1963c). Coherent and incoherent states of the radiation field. Physical Review, 131, 2766-2788. Glauber, R. J. (1965). Optical coherence and photon statistics in Quantum Optics and Electrons, eds. DeWitt, C., Blandin, A., Cohen-Tannoudji, C., Gordon and Breach, New York. Goldstein, H. (1980). Classical Mechanics. Addison-Wesley, New York. Goursat, E. (1917). Differential Equations. Ginn and Co., Boston. Greene, R. F. and Callen, H. B. (1952). Theorem of irreversible thermodynamics II. Physical Review, 88, 1387-1391. Grobner, W. and Hofreiter, N. (1950). Integraltafel, Bestimmte Integrale. Springer Verlag, Berlin. Hamming, R. W. (1991). The Art of Probability. Addison Wesley, New York. Harris, F. J. (1978). On the use of windows for harmonic analysis with discrete Fourier transform. Transactions of IEEE, 66, 51-53. Hartmann, C. A. (1921). The determination of the elementary electric charge by means of the "shot effect". Annalen der Physik, 65, 51-78.

312

BIBLIOGRAPHY

Harris, B. (1967). Spectral Analysis of Time Series, John Wiley, New York. Hassan, T. (1983). Complex demodulation: some theory and applications in Handbook of Statistics III, eds. D. Brillinger and P. Krishnaiah. North Holland, Amsterdam. Hashitsume, N. (1956). A statistical theory of linear dissipative systems, II. Progress of Theoretical Physics, 15, 369—413. Hauge, E. H. (1974). What can we learn from Lorentz models? in Transport phenomena, 337-367, Springer-Verlag, Berlin. Hempstead, R. D. and Lax, M. (1967CVI). Classical noise VI: noise in selfsustained oscillators near threshold. Physical Review, 161, 350-366. Hill, J. E. and van Vliet, K. M. (1958). Ambipolar transport of carrier density fluctuations in germanium. Physica, 24, 709-720. Hillary, D. J., Wark, D. Q., and James D. G. (1965). An experimental determination of the atmospheric temperature profile by indirect means. Nature, 205, 489-491. Hull, A. W. and Williams, N. H. (1925). Determination of elementary charge e from measurements of shot-effect. Physical Review, 25, 147-173. Hull, J. (1989, 2001). Options, Futures and other Derivative Securities. Prentice Hill, New Jersey. Hurst, H. E. (1951). Long term storage capacity of reservoirs. Transactions of the American Society of Civil Engineers, 116, 770-799. IMSL Inc. (1987). IMSL Math/Library Users Manual. IMSL, Houston. Ishimaru, A. (1978). Wave Propagation and Scattering in Random Media, Volume I and II. Academic, New York. Ito, K. (1951). On stochastic differential equations. Memoirs of the American Mathematical Society, 4, 1-51. Ito, K. and McKean, H. P. (1965). Diffusion Processes and their Sample Paths. Academic Press, New York. Jackson, J. D. (1975). Classical Electrodynamics. Wiley, New York. Jansson, P. A., Hunt, R. H. and Plyler, E. K. (1970). Resolution enhancement of spectra. Journal of the Optical Society of America, 60, 596-599. Jansson, P. A. (1970). Method for determining the response function of a highresolution infra-red spectrometer. Journal of the Optical Society of America, 60, 184-191. Jansson, P. A., Hunt, R. H. and Plyler, E. K. (1970). Resolution enhancement of spectra. Journal of the Optical Society of America, 60, 596-599. Jaynes, E. T. (1958). Probability Theory in Science and Engineering. Socony Mobil Oil Co., Dallas.

BIBLIOGRAPHY

313

Jeffries, H. and Jeffreys, B. S. (1950). Methods of Mathematical Physics. Cambridge University Press, London. Jeffreys, H. (1957). Scientific Inference. Cambridge University Press, London. John, F. (1955). Numerical solution of the equation of heat conduction for proceeding times. Annali di Matematica Pura edApplicata, 4, 129-142. Johnson, J. B. (1928). Thermal agitation of electricity in conductors. Physical Review, 32, 97-109. Kac, M. and Siegert, A. J. F. (1947). On the theory of noise in radio receivers with square law detectors. Journal of Applied Physics, 18, 383-400. Kahaner, D., Moler, C., and Nash, S. (1989). Numerical Methods and Software, Prentice Hall, New Jersey. van Kampen N. (1992). Stochastic Processes in Physics and Chemistry, North Holland, Amsterdam. Kaplan, L. D. (1959). Inferences of atmospheric structures from satellite remote radiation measurements. Journal of the Optical Society of America, 49, 10041014. Karhunen, K. (1946). Zur spektraltheorie stochastischer prozesse. Annales Academiae Scientiarum Fennicae, 37. Kelley, P. L. and Kleiner, W. V. (1964). Theory of electromagnetic field measurement and photoelectron counting. Physical Review, 136, A316-A334. Kendall, M. G. (1999). Kendall's Advanced Theory of Statistics. Oxford University Press, London. Kendall, M. G. and Stuart, A. (1969). The Advanced Theory of Statistics. Volume I. Hafner, New York. Khinchine, A. (1934). Korrelationstheorie de stationaren stochastichen prozesse. Mathematische Annalen, 109, 604-615. Khinchine, A. (1938). The theory of stationary chance processes. Rossiiskaya Akademiya Nauk, 5. Kitchener, J. A. and Prosser, A. P. (1957). Direct Measurement of the long-range van der Waals forces. Proceedings of the Royal Society of London. A, 242, 403409. Kittel, C. (1958). Elementary Statistical Physics. John Wiley, New York. Kleiner, B., Martin, R. D., and Thomson, D. J. (1979). Robust estimation of power spectra. Journal of the Royal Statistical Society B, 41, 313-351. Kolmogorov, A. N. (1950). Foundations of the Theory of Probability, Chelsea, New York (a translation of the 1933 Russian version). Kuo, C., Lindberg, C. R., and Thomson, D. J. (1990). Coherence established between atmospheric carbon dioxide and global temperature. Nature, 343,

314

BIBLIOGRAPHY

709-714.

Kubo, R. (1957). Statistical mechanical theory of irreversible processes I. Journal of the Physical Society of Japan, 12. 570-586. Kubo, R. (1962). Generalized cumulant expansion method. Journal of the Physical Society of Japan, 17, 1100-1120. Kyburg, H. E. (1969). Probability Theory. Prentice Hall, New Jersey. Lampard, D. G. (1954). Generalization of the Wiener-Khintchine theorem to nonstationary processes. Journal of Applied Physics, 25, 802-803. Langevin, M. P. (1908). Sur la thAl'orie du mouvement brownien. ComptesRendus de VAcadAl'mie des Sciences (Paris), 146, 530-533. Lawson, J. L. and Uhlenbeck, G. E. (1950). Threshold Signals, MIT Radiation Lab Series, 24, McGraw Hill, New York. Lax, M. and Phillips, J. C. (1958). One dimensional impurity bands. Physical Review, 110, 41-49. Lax, M. (1958QI). Generalized mobility theory. Physical Review, 109, 1921-1926. Lax, M. and Mengert, P. (1960). Influence of trapping, diffusion and recombination on carrier concentration fluctuations. Journal of Physics and Chemstry of Solids, 14, 248-267. Lax, M. (19601). Fluctuations from the nonequilibrium steady state. Reviews of Modem Physics, 32, 25-64. Lax, M. (1963QII). Formal theory of quantum fluctuations from a driven state. Physical Review, 129, 2342-2348. Lax, M. (1964QIII). Quantum relaxation, the shape of lattice absorption and inelastic neutron scattering lines. Journal of Physics and Chemistry of Solids, 25, 487-503. Lax, M. (1966III). Classical noise III: nonlinear via Markoff processes. Reviews of Modern Physics, 38, 359-379. Lax, M. (1966IV). Classical noise IV: Langevin methods. Reviews of Modern Physics, 38, 541-566. Lax, M. (1966QIV). Quantum noise IV: quantum theory of noise sources.Physical Review, 145, 110-129. Lax, M. (1967). Quantum theory of noise in masers and lasers, in 1966 Tokyo Summer Lecture in Theoretical Physics, Part 1: Dynamical Processes in Solid State Optics, eds. Kubo, R. and Kamimura, H., Benjamin, New York, 195-245. Lax, M. (1967V). Classical noise V: noise in self sustained oscillators. Physical Review, 160, 290-307. Lax, M. (1968). Fluctuations and coherence phenomena in classical and quantum physics in 7966 Brandeis Summer Lecture Series, Statistical Physics, volume

BIBLIOGRAPHY

315

2, eds. Chretien, M., Gross, E. P., and Deser, S., Gordon and Breach Science Publishers, New York, 270-478. Lax, M. (1968QXI). Quantum noise XI: multitime correspondence between quantum and classical stochastic processes. Physical Review, 72, 350-361. Lax, M. and Yuen, H. (1968). Quantum noise XIII: six classical variable description of quantum laser fluctuations. Physical Review, 172. 362-371. Lax, M. and Zwanziger, M. (1973). Exact photocount statistics: lasers near threshold. Physical Review A, 7, 750-771. Lax, M. (1974). Symmetry Principles in Solid State and Molecular Physics. John Wiley and Sons, New York. Lax M. (1997). Stochastic processes. Encyclopedia of Applied Physics, 20, 19-60. Lax, M. (2000). The Lax-Onsger regression 'theorem' revisited. Optics Communications, 179, 463-476. Levinson, N. (1947). A heuristic derivation of Wiener's mathematical theory of prediction and filtering. Journal of Mathematics and Physics, 26, 110-119. Louisell, W H. (1973). Quantum Statistical Properties of Radiation. Wiley, New York. Lukacs, E. (1960). Characteristic Functions. Griffith, London. MacDonald, D. K. C. (1962). Noise and Fluctuations. Wiley, New York. Mandelbrot, B. B. (1983). The Fractal Geometry of Nature. Freeman, San Francisco. Mandelbrot, B. B. (1997). Fractals and Scaling in Finance. Springer, Berlin. Merton, R. C. (1973). Theory of rational pricing. Bell Journal of Economics and Management Science, 4, 141-183. Middleton, D. (1960). Introduction to Statistical Communication Theory. McGraw-Hill, New York. von Mises, R. (1937). Probability Statistics and Truth, 2nd edn. Macmillan, New York. Montroll, E. W. (1952). Markoff chains, Wiener integrals, and quantum theory. Communications in Pure and Applied Mathematics, 5, 415-453. Montroll, E. and Shlesinger, M. (1983). A wonderful world of random walks, in CCNYPhysics Symposium in Celebration ofMelvin Lax's Sixtieth Birthday, City College of New York, New York. Moran, P. A. P. (1968). An Introduction to Probability Theory. Clarendon, Oxford. Morse, P. and Feshbach H. (1953). Methods of Theoretical Physics. McGraw-Hill, New York. Moullin, E. B. (1938). Spontaneous Fluctuation of Voltage. Oxford, New York.

316

BIBLIOGRAPHY

Moyal, J. E. (1949). Quantum mechanics as a statistical theory. Proceedings of the Cambridge Philosophy Society, 45, 99-124. Murdoch, J. B. (1970). Network Theory. McGraw-Hill, New York. NAG Fortran Laboratory. Numerical Analysis Group. von Nageli, K. (1879). Sitzber, Kgl. Bayerische Akad. wiss. Munchen, MathPhysik Kl 9, 389-453. Nielsen, L. T. (1999). Pricing and Hedging of Derivative Securities. Oxford University Press, London. Noble, B. (1977). The Numerical solution of IBs. in The State of the Art in Numerical Analysis, ed. D. Jacobs, Academic Press, London. North, D. O. (1940). Fluctuations in space charge limited currents at moderately high frequencies, part II: diodes and negative grid triodes. RCA Review, 4, 441. Nyquist, H. (1927). Minutes of the New York meeting, February 25-26, 1927. Joint meeting with the Optical Society of American. Physical Review, 29, 614. Nyquist, H. (1928). Thermal agitation of electric charge in conductors. Physical Review, 32, 110. Onsager, L. (1931a). Reciprocal relations in irreversible processess. I. Physical Review, 37, 405-426. Onsager, L. (193 Ib). Reciprocal relations in irreversible processess. II. Physical Review, 38, 2265-2279. Page, C. H. (1952). Instantaneous power spectra. Journal of Applied Physics, 23, 103-106. Papoulis, A. (1973). Minimum bias windows for high resolution spectrum estimates. IEEE Transactions on Information Theory, 19, 9-12. Parzen, E. (1960). Modern Probability Theory and Its Applications. John Wiley, New York. Pawula, R. F. (1967). Approximation of the linear Boltzmann equation by the Fokker-Planck equation. Physical Review, 162, 186-188. Pearson, K. (1900). On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. Philosophical Magazine B, 50, 7. Percival, D, B. and Walden, A. T. (1993). Spectral Analysis for Physical Applications. Cambridge University Press, London. Peters, E. E. (1994). Fractal Market Analysis. Wiley, New York. Phillips, D. L. (1962). A technique for the numerical solution of certain integral equations of the first kind. Journal of Association for Computing Machinery, 9, 84-97.

BIBLIOGRAPHY

317

de-Picciotto, R., Reznikov, M., Heilblum, M., Umansky, U., Bunun, G., and Mahalu, D. (1997). Direct observation of a fractional charge. Nature, 389, 162. de-Picciotto, R. (1998). Shot noise of non-interacting composite Fermions. Preprint Weizman Institute of Science, Talk at Bell Laboratories, 03/02/1998. PORT (1984). PORT Mathematical Subroutine Library. Bell Labs, New Jersey. Power, E. A. (1964). Introductory Quantum Electrodynamics. Longmans, Green and Co., London. Press, W. H., Teukolsky, S. A., Vetterling, W. T, and Flannery, B. P. (1992). Numerical Recipes in Fortran, 2nd edn. Cambridge University Press, London. Price, G. L. (1982). Isolation of instability in the Fredholm integral equation of the first kind: application to the deconvolution of noisy spectra. Journal of Applied Physics, 53, 4571-457. Priestley, M. B. (1981). Spectral Analysis and Time series. Academic Press, London. Rack, A. J. (1938). Effect of space charge and transit time on shot noise in diodes. Bell System Technical Journal, 17, 592. Ramsey, J. B. and Thomson, D. J. (1999). A reanalysis of the spectral properties of some economic and financial time series, in Nonlinear Time Series Analysis of Economic and Financial Data, ed. Rothman, P., Kluwer Academic Publishers, New York. Reggiani, L., Lugli, P., and Mitin, V. (1988). Generalization of Nyquist-Einstein relationship to conditions far from equilibrium in nondegenerate semiconductors. Physical Review Letters, 60, 736. Rice, S. O. (1944). Mathematical analysis of random noise, part I: shot noise. Bell System Technical Journal, 23, 282-310. Rice, S. O. (1945). Mathematical analysis of random noise, part II: power spectra and correlation functions. Bell System Technical Journal, 24, 46-70. Rice, S. O. (1948a). Mathematical analysis of random noise, part III: statistical properties of random noise currents. Bell System Technical Journal, 27, 109. Rice, S. O. (1948b). Mathematical analysis of random noise, part IV: noise through nonlinear devices. Bell System Technical Journal, 27, 115. Risken, H. and Vollmer, H. D. (1967). Correlation of the amplitude and of the intensity fluctuation near threshold. Zeitschrift fur Physik, 181, 301-312. Robinson, F. N. H. (1962). Noise in Electric Circuits. Clarendon, Oxford. Robinson, F. N. H. (1974). Noise and Fluctuations in Electronic Devices and Circuits. Clarendon, Oxford. van Roosebroeck, W. (1953). Transport of added current carriers in a homogenous semiconductors. Physical Review, 91, 282-289.

318

BIBLIOGRAPHY

Schawlow, A. L. and Townes, C. H. (1958). Infrared and optical masers. Physical Review, 112, 1940. Scher, H. and Lax, M. (1973). Stochastic transport in a disordered solid. I. theory. Physical Review B, 7, 4491-4502. Schottky, W. (1918). Uber spontane Stromschwankungen in verschiednen Elektrizitatsleitern. Annalen der Physik, 57, 541-567. Schuster, A. (1898). On the investigation of hidden periodicities with application to a supposed 26 day period of meteorological phenomena. Terestrial Magnetism, 3, 13-41. Scully, M. O. and Lamb, W. E. (1967). Quantum theory of an optical meser. I. general theory. Physical Review, 159, 208-226. Scully, M. O. and Lamb, W. E. (1969). Quantum theory of an optical maser. III. theory of photoelectron counting statistics. Physical Review, 179, 368. Scully, M. O. and Zubairy, M. S. (1995). Quantum Optics. Cambridge University Press., London. Shaw, C. B. Jr. (1972). Improvement of the numerical resolution of an instrument by numerical solution of the integral equation. Journal of Mathematical Analysis and Applications, 37, 83-112. Shlesinger, M. F. (1996). Random processes. Encyclopedia of Applied Physics, 16, 45-70. Shockley, W. (1938). Currents to conductors induced by a moving point charge. Journal of Applied Physics, 9, 635-636. Shockley, W. (1950). Electrons and Holes in Semiconductors, Van Nostrand, London. Shockley, W., Copeland, J. A., and James, R. P. (1966). The impedance field method of noise calculation in active semiconcuctor devices, in Quantum Theory of Atoms, Molecules and the Solid State, ed. Lowdin, R O., Academic Press, New York, 537-563. Slepian, D. and Pollak, H. (1961). Prolate spheroidal wave functions, Fourier analysis and uncertainty. Bell System Techcal Journal, 40, 43-64. Slepian, D. (1964). Prolate spheroidal wave functions, Fourier analysis and uncertainty IV. Bell System Technical Journal, 43, 3009-3057. Slepian, D. (1965). Some asymptotic expansions for prolate spheroidal wave functions. Journal of Mathmatical Physics, 44, 99-140. Slepian, D. (1978). Prolate spheroidal wave functions, Fourier analysis and uncertainty V: the discrete case. Bell System Technical Journal, 57, 1371-1429. Smith, W. A. (1974). Laser intensity fluctuations when the photon number at threshold is small. Optics Communications, 12, 236-239.

BIBLIOGRAPHY

319

Smoluchowski, M. V. (1916). Drei Vortrage uber Diffusion, Brownische Bewegung und Koagulation von Kolloidteilchen. Physik Zeitschrift, 17, 557, 585. Stratonovich, R. L. (1963). Topics in the Theory of Random Noise, Vol. I. Gordon and Breach, New York. Thiele, T. N. (1903). Theory of Observations, Dayton, Londons. Reprinted in Annals of Mathematical Statistics, 2, (1931), 165-308. Thomson, D. J. (1977). Spectrum estimation techniques for characterization and development of WT4 waveguide. Bell System Technical Journal, 56, Part I, 17691815; Part II, 1983-2005. Thomson, D. J. (1982). Spectrum analysis and harmonic analysis. Proceedings IEEE (special issue on spectrum estimation), 70, 1055-1096. Thomson, D. J. (1990a). Time series analysis of holocene climate data. Philosophical Transactions of the Royal Society of London Series A, 330, 601-616. Thomson, D. J. (1990b). Quadratic-inverse spectrum estimates; applications to paleoclimatology. Philosophical Transactions of the Royal Society of London Series A, 332, 539-597. Thomson, D. J. and Chave, A. D. (1991). Jackknifed error estimates for spectra, coherences, and transfer functions in Advances in Spectrum Estimation, ed. Haykin S., Prentice Hall, New Jersey, 58-113. Thomson, D. J. (1995). The seasons, global temperature, and precession. Science, 268, 59-68. Thomson, D. J., MacLennan, C. G., and Lanzerotti, L. J. (1995). Propagation of solar oscillations through the interplanetary medium. Nature, 376, 139-144. Thomson, D. J. (1997). Dependence of global temperatures on atmospheric CC>2 and solar irradiance. Proceedings of the National Academy of Sciences of the United States of America, 94, 8370-8377. Thomson, D. J. (2001). Multitaper analysis of nonstationary and nonlinear time series data. Nonlinear and Nonstationary Signal Processing, eds. Fitzgerald, W., Smith, R., Walden, A., and Young, P., Cambridge University Press, London, 317394. Thornber, K. K. (1974). Treatment of microscopic fluctuations in noise theory. Bell System Technical Journal, 53, 1041-1078. Tikhonov, A. N. and Arsenin, V. A., (1977). Solution of Ill-Posed Problems, Winston & Sons, Washington. Titchmarch, E. C. (1948). Introduction to the Theory of Fourier Integrals. Oxford, London. Transistor Teacher's Summer School (1952). Experimental verification of the relation between diffusion constant and mobility of electrons and holes. Physical Review, 88, 1368-1369.

320

BIBLIOGRAPHY

Tukey, J. W. (1961). Discussion, emphasising the connection between analysis of variance and spectral analysis. Technometrics, 3, 191-219; also in The Collected Works of John W. Tukey, ed. Brillinger, D. R., Wadsworth Advanced Books and Software, Belmont, California. Tukey, J. W. (1968). An introduction to calculations of numerical spectral analysis, in Spectral Analysis of Time Series, ed. Harris, B., Wiley, New York, 25-46. Tukey, J. W. (1977). Exploratory Data Analysis. Addison-Wesley, New York. Turchin, V. R, Koslov, V. P., and Malkevich, M. S. (1971). The use of mathematical-statistics methods in the solution of of incorrectly posed problems. Soviet Physics USPEKHI, 13, 681-840, Twomey, S. (1963). On the numerical solution of Fredholm integral equations of the first kind by the inversion of the linear system produced by quadrature. Journal of Association for Computing Machinary, 10, 97-101. Uhlenbeck, G. E. and Ornstein, L. S. (1930). On the theory of Brownian motion. Physical Review, 36, 823-841. Uspensky, J. V. (1937). Introduction to Mathematical Probability. McGraw-Hill, New York. Valley, G. E. Jr. and Wallman, H. (1948). Vacuum Tube Amplifiers. MIT Radiation Lab Series, 13, McGraw-Hill, New York. Wang, L. and Jacques, S. L. (1994). Error estimation of measuring total interaction coefficients of turbid media using collimated light transmission. Physics in Medicine and Biology, 39, 2349. Wang, M. C. and Uhlenbeck, G. E. (1945). On the theory of Brownian motion II. Reviews of Modern Physics, 17, 323-342. Wannier, G. (1937). The structure of electronic excitation levels in insulating crystals. Physical Review, 52, 191. Wax, N. (1954). Noise and Stochastic Processes. Dover, New York. Welsh, D. (1988). Codes and Cryptography. Clarendon Press, Oxford, 41. Welton, T. A. (1948). Some observable effects of the quantum-mechnical fluctuations of the electromagnetic field. Physical Review, 74, 1157. Wiener, N. (1926). Harmonic analysis of irregular motion, Journal of Mathematical Physics, 5, 99. Wiener, N. (1930). Generalized harmonic analysis. Acta Mathematica, 55, 117258. Wiener, N. (1949a). Extrapolation, Interpolation, and Smoothing of Stationary Time Series. Wiley, New York, and MIT Press, Cambridge. Wiener, N. (1949b). Time Series. MIT Press, Cambridge. Wigner, E. (1932). Quantum corrections for thermodynamic equilibrium. Physical

BIBLIOGRAPHY

321

Review, 40, 749-760. Williams E. C. (1937). Thermal fluctuations in complex networks. Journal of Electrical Engineering, 81, 751. Williams, E. C. (1936). Fluctuation voltage in diodes and in multi-electrode valves. Journal of the Institution of Electrical Engineers, 79, 349-360. Williams, G. P. (1997). Chaos Theory Tamed. Joseph Henry Press, Washington. Williams, N. H. and Vincent, H. B. (1926). Determination of electronic charge from measurements of shot-effect in aperiodic circuits. Physical Review, 28, 12501264. Williams, N. H. and Huxford, W. S. (1929). Determination of Charge of positive thermions from measurements of shot effect. Physical Review, 33, 773-778. Wold, H. O. A. (1938). A Study in the Analysis of Stationary Time Series. Almquist and Wiksell, Uppsala. Wolf, P. E. and Maret, G. (1985). Weak localization and coherent backscattering of photons in disordered media. Physical Review Letters, 55, 2696-2699. Xu, M., Cai, W., and Alfano, R. R. (2002). Photon migration in turbid media using a cumulant approximation to radiative transfer. Physical Review E, 65, 066609. Xu, M., Cai, W, and Alfano, R. R. (2004). Multiple passages of light through an absorption inhomogeneity in optical imaging of turbid media. Optical Letters, 29, 1757-1759. Xu, M., Cai, W, Lax, M., and Alfano, R. R. (2004). Stochastic view of photon migration in turbid media. arXiv.cond-mat, 0401409. Xu, M. and Alfano, R. R. (2005). Random walk of polarized light in turbid media. Physical Review Letters, 95, 213905. Xu, M. and Alfano, R. R. (2005). Circular polarization memory of light. Physical Review £,72,065601. Yodh, A. G., Tromberg, B., Sevick-Muraca, E., and Pine, D. (1997). Diffusing photons in turbid media. Journal of the Optical Society of America A, 14, 136-342.

This page intentionally left blank

INDEX X2 distribution, 27 o--field, 136,285 absorption rate, 238 acceptors, 215 adaptive weighting, 302 adiabatic elimination, 63 after-effect function, 215, 224 aliasing, 292 American put and calls, 273 amplitude fluctuation, 195, 200, 205 angular momentum theory, 241 anti-Stokes, 121 arbitrage, 271, 272 arbitrageur, 272, 282 asset, 271 associated Legendre function, 243 autocorrelation, 69, 70, 169, 292 function, 141 autonomous, 194 backward equation, 152, 153 ballistic light, 228 ballistic motion, 244 ballistic sphere, 253 band-limited function, 288 barring operation, 115 Bayes' theorem, 17 biased, 293 biased form, 290 binomial distribution, 20 bipolar drift velcity, 224 birth and death process, 139 black box, 168 Black-Scholes differential equation, 282, 283 model, 280 Boltzmann approximation, 214 Boltzmann transport equation, 237 Boltzmann's constant, 114, 129 Brillouin zone, 212, 292 broadband bias, 302 Brownian motion, 132, 135, 151, 155, 188,275,286 Bust theorem, 137 Campbell's theorem, 95, 96 generalized, 111 canonical form, 199

Cartesian coordinates, 240 Cauchy-Schwarz inequality, 137 causality, 245, 249 central limit theorem, 238 Chaos, 64 Chapman-Kolmogorov condition, 46, 132, 148,239 equation, 132 generalized, 152 relation, 134 characteristic function, 7, 143, 151, 154 generalized, 175 Chebychev's inequality, 8 Chi-square, 26 Chicago Board of Trade, 272 Clebsch-Gordan coefficients, 241 close price, 285 commission, 272 commutation rule, 126, 128, 130 complex demodulation, 303 compressibility, 158 concentration, 215 fluctuation, 158,218 conditional average, 136, 187, 188, 191 conditional correlation, 173 conditional expectation, 135, 285 conduction band, 212 edge, 212 conductivity fluctuation, 215 correlations, 293 covariance, 294 Cramer's spectral representation, 295 cumulant, 10, 237 expansion, 239 expansion theorem, 202 first cumulant, 238 kurtosis, 11 second cumulan, 238 skewness, 11 cumulant solution, 256 currencies, 273 current-current noise, 116 cusp, 161 data window, 300, 301 decay parameter, 141 delivery, 271 delivery price, 271 delta correlation, 169, 176, 184

324 delta function, 35, 169, 184, 193, 239 multi-dimensional, 39 density matrix, 117 of states (DOS), 211 operator, 117, 118 depolarization, 236 derivative, 271,282 detailed balance, 219 deterministic function, 277, 284 diffusion, 54 coefficient, 141, 144 constant, 130 conditional, 132 diffusion and mobility, 58 diffusion coefficient, 52, 234 diffusion constant, 61, 63 frequency dependent, 90 diffusion equation, 237 diffusion length, 226 diffusive light, 228 dilemma, 161 Dirichlet kernel, 297 discontinuity, 161 discrete process, 286 discrete prolate spheroidal function (DPSF), 290, 297, 301 sequence (DPSS), 290, 298, 301 disjoint events, 14 dispersion, 8 dissipation, 113, 115, 121 dissipative response, 136 distribution function, 4 dividend, 273 donors, 215 Doob's theorem, 162 drift vector, 133, 136, 184, 190, 201 drift velocity, 182 effective mass, 212, 213 Einstein relation, 55, 91, 129, 136, 141, 157, 163, 170, 181,220 generalized, 159 ellipsoid, 212, 254 energy gap, 215 ensemble, 285 ensemble average, 185, 239, 241 equilibrium density operator, 114 ensemble, 114 theorem, 121 equipartition, 129 equipartition theorem, 84 error function, 252 Europen contract, 273 exercise, 273

INDEX expectation value, 5 extrapolation, 288 factorization approximation, 138 factorization of probabilities, 147 fast Fourier transform (FFT), 289 Fermi energy, 213 Fermi function, 213 Fermi-Dirac statistics, 217 filtering process, 130 filtration, 285 financial area, 187, 189 financial quantitative analysis, 135, 275 fluctuation, 113, 115, 129, 133 Casimir effect, 89 fluctuation-dissipation theorem, 58, 87, 114, 115, 117, 118, 121, 123, 124, 128, 181 Fokker-Planck operator, 147 equation, 129, 130, 138, 164, 195, 197, 201, 207, 208, 279 generalized, 137, 141, 154,286 ordinary, 137, 286 process, 168, 182, 184, 188, 201, 205 generalized, 143 foreign currency, 273 forward, 271 contract, 271-273, 281 price, 271 fractal process, 287 fundamental analysis, 284 future, 271 contract, 272, 273 gambling, 32 first law, 32 Gambler's ruin, 42, 52, 64 second law, 33 Gaussian distribution, 237 random force, 195 random variable, 152 generalized characteristic functional, 146 generating function, 7 generation, 179 generation-recombination noise, 130 process, 139, 286 geometric Brownian motion, 275 geometric sum, 296 global minimum, 252 global warming problem, 291 harmonic oscillator, 123, 284 heating reservoir, 284 Heaviside unit function, 169 Heyney-Greenstein phase function, 248, 252

INDEX ill-posed problems, 258, 289, 296 filtering, 260, 261 Franklin's method, 264 image restoration, 270 kernels, 263 regularization, 261 Shaw, 266 statistical regularization, 268 inertia, 180 inertial system, 161 instability, 194 intelligent quasilinear approximation, 207 interaction picture, 119 interpolation, 288 invariant embedding, 153 Ito's calculus lemma, 135, 185, 188, 271, 275, 282, 287 integral, 136, 276 stochastic differential equation (ISDE), 275, 278 jack-knife procedures, 291 Johnson noise, 116, 168 joint events, 12 Karhunen-Loeve theorem, 293 Langevin approach, 57, 60, 172, 175 equation, 168, 184, 186, 189, 207, 276 force, 125, 127 form, 136 problem, 200 process, 168, 184 stochastic differential equation (LSDE), 276 treatment, 182 law of chemical equilibrium, 215 law of mass action, 215 Lebesgue integration, 280 Levy process, 287 line-width, 195, 198 linear damping, 149, 159, 171 linked-average, 159 linked-moment, 144 Liouville theorem, 166 Lorentzian spectrum, 203 low-pass window, 297 margin, 272 call, 273 Markovian process, 129, 132, 133, 153, 168, 169, 195 martingale measure, 136 master equation, 140 maturity, 273 time, 285 mean, 293 mean-square fluctuations, 137

325

measure theory, 280 metronome-like driving source, 194, 200 mobility, 130,215 monotonic decreasing, 297 Monte Carlo simulation, 237, 248 multidimensional form, 153 multiple scattering, 237 multitapered, 297 multitime characteristic function, 159 multitime correlations, 179 multivariate normal distribution, 29 natural filtration, 136, 285 noise, 69 autocorrelation, 70 evenness, 75 filters, 73 homogeneous, 149, 159, 171 Johnson noise, 82, 85 nonstationary, 77 Nyquist noise, 90 Nyquist's theorem, 87 shot noise, 93, 104, 139, 141, 145, 151, 155, 168, 204, 225 generalized, 177 process, 155, 168 Rice shot noise process, 286 standard engineering definition, 69 thermal, 82 white, 130 Wiener-Khinchine theorem, 71 noise anticommutator, 128 noise spectrum, 155, 163 nondegenerate, 213, 297 nonequilibrium steady state, 218 nonharmonic model, 285 nonlinearity, 195 normal distribution, 277 Nyquist rate, 288 sampling rate, 296 theorem, 87, 205 operating point, 136, 141, 163, 171, 196, 199,200, 206, 284 option, 271,287 ordinary calculus rule, 187, 191, 276, 282 orthogonality relation, 241 overlapping events, 14 paradox, 161 partial differential equation (PDE), 164 particle distribution, 237 partition sum, 119 path integral, 151 average, 146

326 Pauli principle, 218 periodogram, 289 phase fluctuation, 195, 200, 205 phase function, 228, 237, 238 phase variable, 194 photon migration, 228 Planck' constant, 114 Poisson bracket, 114, 119, 122 Poisson distribution, 23 generalized, 228, 232 Poisson process, 140, 142, 286 Poisson sum, 303 portfolio, 282 position-position correlations, 124 prewhitened, 300 data, 301 primitive cell, 212 probability, 1 conditional probability, 16, 44, 131, 162 frequency ratio, 1 mathematical probability, 2 subjective probability, 3 put, 271 quasi-Fermi level, 219 quasi-Markovian limit, 128 quasilinear, 130 approximation, 136, 142, 171 case, 168 treatment, 195, 206 radiative transfer equation, 237 random process, see stochastic process random variable, see stochastic variable, 293 random walk Brownian motion, 56 diffusion, see diffusion one-dimensional, 50, 54 photon migration, 228 reciprocal lattice, 212 recombination, 179 rectangular window, 289 recurrence relation, 240, 249 regression, 163 regression theorem, 129-131, 142 regularization procedure, 297 reservoir, 126 reservoir oscillator, 126 reshaping, 249 residual spectrum, 289 resistive nonlinearity, 195 resonant circuit, 195 response function, 114 Riemann integral, 278 Riemann sum, 278, 286 risk

INDEX free, 272, 281 free profit, 271 neutral, 282 rotating wave approximation, 126, 131, 195, 196 oscillator, 199 van del Pol oscillator (RWVP), 164, 194, 197, 200 sampling theorem, 289, 303, 304 scattering rate, 238 Schuster periodogram, 292 security, 271 self-energy diagram, 257 self-sustained oscillator, 195 self-sustained oscillator, 130, 164 semi-invariants, 145 semiphenomenological model, 249 short, 272 Slepian function, 295, 297, 301 smoothing, 288, 289 smoothing window, 300, 301 snake-like mode, 245 spectral analysis, 289 spherical harmonics, 239 spot price, 272 time, 284 value, 285 standard deviation, 275 stationary, 156 stationary state, 136 Stieltjes integral, 278 stochastic differential equation, 136, 188, 283 stochastic integral, 188 stochastic model, 130 stochastic process, 44, 136 Gaussian, 45 Markovian, 45, 228 nonstationary, 77 Poisson, 48 spectrum, 69 stationary, 45 stochastic variable, 5 Gaussian, 66 sum, 19 stock, 187 stock index futures, 273 Stokes, 121 Stokes-anti-Stokes ratio, 126, 130 storage cost, 274 tapering, 289 Taylor expansion, 134, 141, 152 telegrapher's equation, 237 thermodynamic treatment, 216 threshold, 204, 210 time autocorrelation, 291

INDEX time reversal, 115, 122, 142, 160, 180 time series, 288, 289 time shift, 194, 200 time-dependent Green's function, 240 time-ordered multiplication, 240 transfer matrix, 116 transition probability, 131, 132, 134, 147, 218 rate, 140 translation invariance, 238 transport mean free path, 239 turbid medium, 227, 237 two-time correlation, 129 Uhlenbeck's model, 285 Uhlenbeck-Ornstein process, 151 unbiased, 293 form, 289 uncorrelated, 294 underlying asset, 271

value, 284 valence band, 214 van del Pol equation, 199 velocity-velocity noise, 116 volume fluctuation, 158 white noise, 184, 197, 277, 286 Wiener process, 184, 188, 275 Wiener-Khinchine form, 113 relation, 80, 173 theorem, 71, 74, 202, 203, 288, 291 window factor, 225 windowing, 289 Wold theorem, 291,292 zero-point contribution, 116, 203 zero-point effect, 125 zero-point oscillations, 116 zero-point energy, 89

327