Selected Proceedings of the Symposium on Inference for Stochastic Processes

0 and λ is bounded and bounded away from 0. Typically λ is primarily a function of the direction of x rather than its magnitude. Optimally choosing λ is equivalent to optimally choosing Tj0stheim's m (see below). Furthermore, the geometric drift condition can be expressed in terms of the drift of the logarithm of V(Xt) (the log-drift condition). For the following theorems let Λ be the class of all nonnegative measurable functions on W that are bounded and bounded away from 0. Theorem 1 Assume {Xt} is an aperiodic, φ-irreducible T-chain in W such that E (||ΛΊ||7(1 + ||z|| r ) I XQ = x) is bounded for some r > 0. The following are equivalent conditions, each sufficient for {Xt} to be geometrically ergodic.

(i) ί m s u p | M | _ , o o g ( Λ ( ^ ) ) | | ^ J | r I Xo = x) < 1 for some λ e A, r > 0.

(it) l i m s u p H s i i ^ E ( i j p ί | X°

= x

) < l for

some

r>0,n>m>0.

(in) l i m s u P | N K o o E (log (δ + ^ j j ^ l " ) | *o = x) < 0 for some δ > 0, λeΛ. (iv) limsupiiaju^oo E (log {δ + l^\\χ^) m > 0.

Xo = x) < 0 for some

δ>0,n>

NONLINEAR TS STABILITY

159

Proof The equivalences follow from Cline and Pu (1999a, Lena. 4.1, Lem. 4.2) and the sufficiency for geometric ergodicity from condition (10) with test function V(x) = 1 + λ(z)||z|| r . D If the noise terms in a FCAR(p) model do not dominate when the time series is very large then they may be at least partly ignored, as the following theorem suggests. Theorem 2 Assume {Xt} is an aperiodic, φ-irreducible T-chain defined by r (1) and (6) such that αo,...,α p are bounded, sup|j a .|| < M 2?(|c(ei;£)| ) < oo r r for some r > 0 and all M < oo, and lim||x||_>ooi£ (|c(ei;x)| /||α:|| ) = 0. Let a be given by (8) and θ{x) — (a(x),xι,... ,xp-ι)/(l + ||a?||) for x = (xι,..., xp). The following are equivalent conditions, each sufficient for {Xt} to be geometrically ergodic. (i) U m s u P | W K o o £ ( ^

I Xo = x) \\θ(x)\\r < 1 for some r > 0, λ G Λ.

(it) l i m s u p | N K o o E (j[nj=m ||0(-Xj )IΓ \XQ = X) 0, n > m > 0: (in) l i m s u p | N K o o E (log (δ + ^ 1 | | 0 ( * ) | | ) | Xo = x) < 0 for some δ > 0, λΛ λ(EΛ. (iv) U m s u p | W K o o S ( Σ j = m log ( ί + 1 1 0 ( ^ ) 1 1 ) \Xo = x) 0, n > m > 0. Proof This essentially is Theorem 3.2 in Cline and Pu (1999a).

D

Similar theorems express equivalent conditions for transience of a Markov chain (e.g., Cline and Pu (2001, Thm. 2.2)). Example 6. We refer to conditions (iii,iv) in Theorem 1 and (iii,iv) in Theorem 2 as "log-drift" conditions. A model which illustrates the benefits of using a log-drift condition is the Periodic FCAR(l) model,

where the coefficient function a\(x) is periodic with period r. This is an unusual model but there is a suggestion of periodicity in the coefficient functions fitted by Chen and Tsay (1993a) for the sunspot number data, and it was at the urging of Prof. Chen that we studied the stability of this particular model. At any rate, it is a useful example. With n = m = 1, the condition in Theorem 2(iv) can be reexpressed as limsupE (log(ί + K(ξi)|) ξo = x) < 0 for some δ > 0.

(13)

160

CLINE AND PU

(This can also be determined from the 2-step condition with V(x) — 1 + |x|.) The function E (log(£ + |αχ(£i)|) | ξo = %) is close to being periodic in |αi(x)x|. Thus the left-hand side of (13) is only a limsup, not a limit, which has the unfortunate consequence that condition (13) is not sharp. The solution is to choose n = m — 2 instead. Assume a\(x) is continuously differentiate with a derivative that is 0 on a set of measure 0. Then the function E (log(ί + |αi(ξ2)|) I ξo = x) does in fact have a limit and therefore does lead to a sharp condition, namely if

Γ log(|αi(u)|)*ι<0

(14)

JO

then {ξt} is geometrically ergodic and if f£ log(\aι(u)\)du > 0 then {ξt} is transient (Cline and Pu (1999a, Thm. 3.4; 2001, Thm. 3.2)). Note that the skeleton process, xt = ao(xt-ι) + aι{xt-i)zt-i is geometrically stable if and only if sup|αi(α:)| < 1. Therefore, this is an example where stability of the time series does not coincide with stability of its skeleton. For a specific example, suppose a\{x) = c + dcos(x) with \c\ + \d\ > 1, M < |d| < 2. Then the time series is geometrically ergodic but the skeleton is not stable. Although condition (14) does not explicitly refer to the noise distribution, the noise does play a major role in determining the condition by causing the drift to be averaged. Example 7. The directional method, on the other hand, seems to work well with many threshold models of order 1, including ones that employ a delay. Consider, for example, the TAR(l) time series with delay d given by ξt = ao(Xt-ι) + aι(Xt-i)ξt-i + eu

Xt-ι = (ξt-u

>6-d

ao(x) bounded and a\{x) depends only on (sgn(xχ),... ,s x = (xi,..., Xd). There are thus 2d regions i?χ,..., R2d each corresponding to a coefficient: a\j = 01(1), x G Rj. Notice that the thresholds — boundaries of the regions — are the axial hyperplanes. As long as the time series remains large (which is all that is of concern for stability), ξt avoids the thresholds and hence Xt cycles among some subset of the regions. There may be several such subsets possible, depending on the signs of the αij's, and it is the drift of the "worst case" cycle that is critical. Thus the geometric drift condition for stability is exactly (Cline and Pu (1999b, Cor. 2.4))

max

161

NONLINEAR TS STABILITY

where the maximum is taken over the possible cycles described above. This corresponds precisely to the geometric stability of the skeleton. The test function for establishing this condition takes the directional form, V(x) = 1 4- A(rr)||a;||r, and the optimal choice for X(x) is constant on each of the 2d regions. 6. The Piggyback Method. To this point we have presented examples studied with what one might call the traditional methods of drift analysis. Not all models yield to this analysis, however, including some surprisingly simple models. In this section we present a new approach, as yet somewhat informal, that employs a much more sophisticated ra-step test function. We call it the piggyback method because it relies on finding a stable Markov chain similar to a process embedded in the one of interest and building a new test function on top of one that works for the known stable chain. We will first present a sketch of the piggyback method and then provide three examples. The sketch, however, is quite rough because in fact the method is applied somewhat differently for each of the examples. (See the papers referenced below.) Indeed, the concept is elegant but its application is messy and as yet we do not know how generally useful it may prove to be. The time series {ξt} is embedded in a Markov chain {Xt} At the same time we consider another Markov chain {Yt} similar to a simpler process embedded in {Xt}> The chain {Yt} is assumed to be geometrically ergodic and, in particular, to satisfy the geometric drift condition with test function V\{y). Its stationary distribution we denote G. If there is a function H(y) which somehow exemplifies (or bounds) the relative change in magnitude of X\ when XQ is large and YQ — y then, intuitively, the stationary value of H(Yt) will measure the geometric drift of {Xt}- Thus, a log-drift condition for geometric stability would be exp ( ίlog(ί + H(y))G(dy)\ < 1 for some δ > 0.

(15)

To obtain a test function that will yield such a condition, we first define

and let y(x) identify the "embedding" of Yt into Xt An integer m is chosen suitably large, a "correction" function c(x) constructed and the ultimate test function is (something like) l/m

V(x)=c(x)[l[E(h(Yj)\Y0 =

162

CLINE AND PU

The key point for this paper is that the piggyback method and the resulting condition for ergodicity capture the implicit stochastic behavior of {F t }, not the behavior of a deterministic skeleton of {Xt} or {£*}. Even if such a skeleton can be identified, its stability properties will not coincide with those of {&}. Alternatively, one may think of {Xt} as having a sort of stochastic skeleton which must be analyzed for stability. Example 8. Our first example is a bivariate threshold model (Cline and Pu (1999a, Ex. 3.2; 2001, Ex. 3.2)). Indeed it is the simplest such model that is not just two independent univariate models joined together. Suppose

where ai(xχ) = α ϋ l X l < o + α^lxjXb i = 1?2. Note that the nonlinearity of the second component Xt^ is driven by the univariate TAR(l) process {Xt,ι} The latter is our "embedded" process and is stable when max(αii,αi2,αiiαi2) < 1.

(16)

Let G be its stationary distribution. The function |α2(#i)| plays the role of H(y) so that the resulting (sharp) stability condition is, in addition to (16),

which represents the stationary value of the relative change in magnitude of Xti2 when it is very large. This condition neither implies nor is implied by the stability condition for the corresponding skeleton process: (16) plus

Example 9. The second example (cf. Cline and Pu (1999c)) is the threshold ARMA(l,ςr) model (TARMA) with a delay d:

ξt = oo(Xt-i) + aι(Xt-i)ξt-i

+et + &i(-ϊt-i)et-i + • +

bq(Xt-ι)et-q,

where Xt-ι = {ξt-i,..., ξt-d), «o3 &i, , bq are bounded and aχ{x) is (asymptotically) piecewise constant. We further assume the thresholds are affine which implies the regions on which a\(x) is constant are cones in Md. This is the simplest interesting example of a TARMA process. See also Brockwell, Liu and Tweedie (1992) and Liu and Susko (1992). In the case q > 0, the time series must be embedded in the Markov chain {(Xt,Ut)} where Xt is as above and Ut = ( e t , . . . , e t _ g +i). Threshold ARMA models have not seen a lot of study, perhaps in part because the moving average terms can affect the irreducibility and periodicity

NONLINEAR TS STABILITY

163

properties of the chain in complicated ways as yet not well understood. (See, for example, Cline and Pu (1999c)). This by itself is a major role played by the noise but we will pass by it here. Let the regions i?χ,..., Rm be the partition of W such that a\(x) is constant on each region, with a n , . . . , a i m being the corresponding constants. There are basically two types of situations that arise in these models when ξt is very large: cyclical and noncyclical. For the cyclical situation, {Xt} essentially cycles close to certain rays having the form

(Uϊ:lalji,nPiZΪalji,...,l)xu

(17)

if all are in the interior of the conical regions. Noise plays no role in determining the stability in this situation since Xt avoids the thresholds; all that matters is the product of coefficients realized by moving through the cycle. For a model which is purely cyclical the stability condition is based on the "worst case" cycle, it is deterministic and corresponds to that of the skeleton process, and it is very much like that of the TAR(l) process with delay discussed in section 5. The model may also have, however, situations where one or more of the rays of type (17) actually lie on a threshold. In such a case, Xt can fall on either side of the threshold, and thus into one of two possible regions, at random but depending on both the present error et and the past errors e t _i,..., et-q. If Jt denotes the region that Xt is in then {(Jt, Ut)} behaves something like a Markov chain where the first component is one of a finite number of states and the second component is stationary. (If q = 0 then {Jt} itself is like a finite state Markov chain.) We relate {(Jt, Ut)} to such a Markov chain denoted, say, {(JtiUt)}. This chain is not necessarily irreducible or aperiodic but clearly every invariant measure is finite. Indeed it may be decomposed into a finite number of uniformly ergodic subprocesses. The coefficients |αχj| play the role of H(y) in this model. Now let G be any stationary distribution for {(Jt, Ut)} and define ΈJ = IRQ G(j,du). If (condition (15))

regardless of the choice of G then {(XuUt)} is geometrically ergodic aiίid, again, the condition is sharp. Because at least one ray lies on a threshold, the noncyclical models are special cases, but the stability condition for a noncyclical process can be quite different from that of nearby purely cyclical processes. See the parameter spaces for the TARMA(1,1) with delay 2 in Cline and Pu (1999c).

164

CLINE AND PU

Example 10. The third example combines the nonlinearity of piecewise continuous coefficient functions with a piecewise conditional heteroscedascity, a model called the threshold AR-ARCH time series: ξt = a(Xt-i) + b(Xt-i)et + c(et; X t -i), where a{x) and b(x) are piecewise linear, {c(ei a )} is uniformly integrable and Xt-i = (ξt-i, ,ζt-p) We further suppose a(x) and b(x) are homogeneous, b{x) is locally bounded away from zero except at x = 0, the thresholds are subspaces containing the origin and the regions of constant behavior are cones. Note that these assumptions need only hold asymptotically (in an appropiate sense) as x gets large. Once again, the Markov chain under study is {Xt} The basic idea on which we piggyback is that the process {Xt} collapsed to the unit sphere behaves very much like a Markov chain. The compactness of the unit sphere serves to make this chain stable and then the stability condition for the original chain can be computed. More specifically, define

S = α ( X t _ 0 + &(*_!)*,

X; = (ξl...,ξ*t-p+ι)

and θ*t = X*/\\X*t\\.

Then, due to the homogeneity of a(x) and 6(x), {0£ } is a Markov chain on the unit sphere and is uniformly ergodic with stationary distribution G, say. By the piggyback method, therefore, Xt has geometrically stable drift if

E (flog(\a(θ) + b(θ)e1\/\θι\)G(dθ)yj < 0. For a simple demonstration, suppose p = 1, a(x) = (αil x o)x, b{ Φ 0, i = 1,2. Then {#£} is a Markov chain on {—1,1} with transition probabilitiesp_i5i = P(a\+b\e\ < 0) = 1—p-^-i and pi,_i = P{a2 + b2eλ < 0) = 1 - p M . If +

+

P

p^y +P

log

(|α2

)

then {ξt} is geometrically ergodic. This example and generalizations of it will be considered fully in a forthcoming paper (Pu and Cline (2001)). Example 11. A very simple model which has not been analyzed fully is the ordinary TAR(2) model with additive noise, ξt = αi(X t -i)ξt-i + a2(Xt-ι)ξt-2

+ eu

where Xt-ι = (ξt-i^t-2) &nd αi(#) and a2(x) are piecewise constant. The precise stability condition is not known even when there is but one affine threshold. The results of this section, however, suggest that the key will be to identify an appropriate stochastic skeleton process to study.

165

NONLINEAR TS STABILITY

7. The Role of the Noise Distribution Tails. Spieksma and Tweedie (1994) pointed out how, with appropriate assumptions on the error distribution tails, an ordinary drift condition (such as (9) with V(x) = 1 + \x\) can be boosted to ensure geometric ergodicity of the process. We generalize the result as follows. Theorem 3 Assume {Xt} is an aperiodic, φ-irreducible T-chain in W and V : W —> [l,oo) is locally bounded. Suppose there exists a random variable W(x) for each x such that V{X\) < W(x) whenever XQ = x, {\W(x) — V(x)\ + er(w(x^~v(x^} is uniformly integrable for some r > 0 and limsupE (W(x) - V(x)) < 0. Then there exist s > 0 and V\(x) = esV^ ergodic (and hence geometrically ergodic).

(18)

such that {Xt} is V\-uniformly

Proof This follows directly from the drift condition for ^-uniform ergodicity (cf. Meyn and Tweedie (1993, Thm. 16.0.1)) and uniform convergence (cf. Cline and Pu (1999a, Lem. 4.2)). (See also the proof to Theorem 4.) D Essentially, this is the log-drift condition in another guise: if the test function in (10), for example, is replaced with V\{x) = esV^ with some sufficiently small s > 0 then (18) is a log-drift version of the condition. As a bonus, if V(x) is norm-like, satisfying ||x|| < V(x) < M +jfif||a;||, one gets strong laws and central limit theorems for all the sample moments (Meyn and Tweedie (1992), Chan (1993a,b)) and exponentially damping tails in the stationary distribution (Tweedie (1983a,b)). Example 12. For example, consider the FCAR(p) model discussed in section 3. If the noise term c(et;Xt-ι) is such that sup x E(er^eux^) < oo for some r > 0 then it frequently is possible to satisfy the requirements of Theorem 3 with a norm-like V(x). To illustrate how this can work, consider the FCAR(l) process, Xt = ξt = αi(6-i) + Φ t ; 6 - i ) with

-L < a\{x) < a\\x + αoi Ίix < -L,

, *

L > a\(x) > a\2X + αo2 if x > L, where a\\a\i — 1, a\\ < 0 and L < oo. We assume here that E(c(e\\ x)) = 0 for all x G R and sup x E(er^eux^) < oo for some r > 0. For the special case of equality on the right in (19) (the SETAR(l) model of Example 4), Chan et al (1985) showed {ξt} is ergodic if and only if 7 = 011^02 + «oi < 0. We thus assume 7 < 0. Let λi = λ^"1 = \J—a\\ and choose δ{ > 1, i = 1,2 so that -λiα O 2 + δχ- δ2 = X2OΌI + h - δ\ = λ 2 7/2. Define

V{x) = (\ι\x\ + δλ)lx<0

+ (X2\x\

166

CLINE AND PU

Then it is a simple computation to show that for some e > 0 and K < oo, V(Xι) - V(x) < (λ 2 lχ< 0 - λil x >o)c(ei; x) + X2j/2 + K\c(eγ,x)\l\c{ei]x)\>€\x\ χ ιs when |Xo| — \ \ sufficiently large, which satisfies the conditions of Theorem 3 with the limit in (18) being λ27/2. The time series is thus geometrically ergodic. On the other hand its skeleton, while stable, is not geometrically stable since a\\aγι = 1. In fact we would say both have only a linear drift. Tanikawa (1999) studied this example and Cline and Pu (1999b) looked at similar first order threshold-like models, but with a possible delay. Using a similar approach but with stronger stability conditions, Diebolt and Guegan (1993) studied multivariate examples and An and Chen (1997) investigated FCAR(p) models withp > 1. One of the drawbacks to a log-drift condition such as the one in Theorem l(iii) is that it guarantees geometric ergodicity only with test functions of the form V(x) = l-hλ(α;)||x|| r where r may be arbitrarily small and therefore it fails to imply needed limit theorems for sample moments. To be able to conclude Vί-uniform ergodicity with an exponential-like Vί, the condition must again be boosted and then the desired limit theorems will hold. Theorem 4 Assume {Xt} is an aperiodic, φ-irreducible T-chain in W and V : W —> [l,oo) is locally bounded and V(x) -> oo as \\x\\ -> oo. Suppose there exists a random variable W(x) for each x such that V(X\) < W(x) whenever Xo = x, {\ log{W{x)/V{x))\ + e ( ^ W ) r - ( ^ ) ) Γ } is uniformly integrable for some r > 0 and limsup£ (log{W(x)/V(x))) < 0.

(20)

IMI->oo

Then there exist s > 0 and V\{x) = e^v^s such that {Xt} is V\-uniformly ergodic (and hence geometrically ergodic). Proof For v > w > 1 and 0 < s < r, we have i {ew3~vS - 1) < ^ ( ^ - l) and log(w/v) < j fe - l j < 0. By the uniform integrability of {log(W(x)/V(x))}, truncation and uniform convergence (as s I 0),

Λi ((ii ((e("W-(v(*))« ("W(v(*)) __ Λ

< limsupE (\og(W(x)/V(x))lw{x) 0 and 5 > 0 small enough.

Λ

+ e < -e,

(21)

167

NONLINEAR TS STABILITY For w > v > 1 and 0 < s < r/2, we have 0 < \og(w/υ) < - (ewS-υ° - l) < - (ewT~vT

-

and if wΓ-υr < K, v > M > 1 then ± (β 10 '-"* - 1) < g £ £ . By the uniform integrability of {e^ί*))'"^**))'}, truncation and V{x) -> oo as ||z|| ->• oo,

0 <

\imsupE(log(W(x)/V(x))lw{x)>v(x})

\\x\\-κx>

)M^)) _ Λ i

) <6,

(22)

for e > 0 and s > 0 small enough. From (20)-(22), therefore, we conclude there exists s > 0 small enough that limsupE (- (eWi»MV(*)) _ Λ IX o = \ Also, sup|| x ||< M J5 M v ( X l ) ) s

χ0 —xλ < oo for all M < oo, and hence geo-

metric ergodicity is assured with test function V\.

D

Example 13. We again consider an FCAR(l) model, ξt = «i(6-i) + c{et\it-ι), satisfying (19) but now we assume a n < 0 < anau < 1 and |c(ei;z)| < ci|a;|^|ei| where cx > 0, 0 < β < 1 and E(eη^) < oo for some η > 0. Let λi = y/-an, λ 2 = V~ α i2 and F(x) = 1 + (λil x o)|^|. Then for \XQ\ = \x\ sufficiently large and some e > 0 and K < oo, V{X\) < (1 - e)V(x) + ίί|x|^|ei|, which satisfies Theorem 4 with r < 1 - β. See also Diebolt and Guegan (1993) and Guegan and Diebolt (1994) for related results. When the errors are bounded, an otherwise unstable model can sometimes be stable. See Chan and Tong (1994) for an example. References An, H.Z. and Chen, S.G. (1997). A note on the ergodicity of non-linear autoregressive model, Stat. Prob. Letters 34, 365-372. An, H.Z. and Huang, F.C. (1996). The geometrical ergodicity of nonlinear autoregressive models, Stat Sinica 6, 943-956. Athreya, K.B. and Pantula, S.G. (1986). Mixing properties of Harris chains and autoregressive processes, J. Appl. Probab. 23, 880-892. Bhaskara Rao, M., Subba Rao, T. and Walker, A.M. (1983). On the existence of some bilinear time series models, J. Time Series Anal 4, 95-110. Borkovec, M. (2000). Extremal behavior of the autoregressive process with ARCH(l) errors, Stock. Proc. Appl 85, 189-207.

168

CLINE AND PU

Brockwell, P.J., Liu, J. and Tweedie, R.L. (1992). On the existence of stationary threshold autoregressive moving-average processes. J. Time Series Anal 13, 95-107. Chan, K.-S. (1989). A note on the geometric ergodicity of a Markov chain, Adv. Appl. Probab. 21, 702-704. Chan, K.-S. (1990). Deterministic stability, stochastic stability, and ergodicity, Appendix 1 in Non-linear Time Series Analysis: A Dynamical System Approach, by H. Tong, Oxford University Press (London). Chan, K.-S. (1993a). A review of some limit theorems of Markov chains and their applications, Dimensions, Estimation and Models, ed. by H. Tong, World Scientific (Singapore), 108-135. Chan, K.-S. (1993b). On the central limit theorem for an ergodic Markov chain, Stock. Proc. Appl 47, 113-117. Chan, K.-S., Petruccelli, J.D., Tong, H. and Woolford, S.W. (1985). A multiple threshold AR(1) model, J. Appl Probab. 22, 267-279. Chan, K.-S. and Tong, H. (1985). On the use of the deterministic Lyapunov function for the ergodicity of stochastic difference equations, Adv. Appl Probab. 17, 666-678. Chan, K.-S. and Tong, H. (1986). On estimating thresholds in autoregressive models, J. Time Series Anal 7, 179-190. Chan, K.-S. and Tong, H. (1994). A note on noisy chaos, J. Royal Stat. Soc. 56, 301-311. Chen, R. and Hardle, W. (1995). Nonparametric time series analysis, a selective review with examples, Bulletin of the International Statistical Institute, 50th session of ISI, August, 1995, Beijing, China. Chen, R. and Tsay, R.S. (1991). On the ergodicity of TAR(l) process, Ann. Appl Probab. 1, 613-634. Chen, R. and Tsay, R.S. (1993a). Functional-coefficient autogressive models, J. Amer. Stat Assoc. 88, 298-308. Chen, R. and Tsay, R.S. (1993b). Nonlinear additive ARX models, J. Amer. Stat Assoc. 88, 955-967. Cline, D.B.H. and Pu, H.H. (1998). Verifying irreducibility and continuity of a nonlinear time series, Stat & Prob. Letters 40, 139-148. Cline, D.B.H. and Pu, H.H. (1999a). Geometric ergodicity of nonlinear time series, Stat Sinica 9, 1103-1118. Cline, D.B.H. and Pu, H.H. (1999b). Stability of nonlinear AR(1) time series with delay, Stock. Proc. Appl 82, 307-333. Cline, D.B.H. and Pu, H.H. (1999c). Stability of threshold-like ARMA time series, Statistics Dept., Texas A&M Univ. Cline, D.B.H. and Pu, H.H. (2001). Geometric transience of nonlinear time series, Stat Sinica 11, 273-287. Collomb, G. and Hardle, W. (1986). Strong uniform convergence rates in robust nonparametric time series analysis and prediction: kernel regression estimation from dependent observations, Stock. Proc. Appl 23, 77-89. Diebolt, J. and Guegan, D. (1993). Tail behaviour of the stationary density of general non-linear autoregressive processes of order 1, J. Appl. Probab. 30, 315-329.

NONLINEAR TS STABILITY

169

Foster, F.G. (1953). On the stochastic matrices associated with certain queueing processes. Ann. Math. Stat. 24, 355-360. Guegan, D. (1987). Different representations for bilinear models, J. Time Series Anal. 8, 389-408. Guegan, D. and Diebolt, J. (1994). Probabilistic properties of the β-ARCH model, Stat. Sinica 4, 71-87. Guo, M. and Petruccelli, J.D. (1991). On the null-recurrence and transience of a first order SETAR model, J. Appl. Probab. 28, 584-592. Hardle, W., Lϋtkepohl, H. and Chen, R. (1997). A review of nonparametric time series analysis, Internal. Stat Review 65, 49-72. Hardle, W. and Vieu, P. (1992). Kernel regression smoothing of time series, J. Time Series Anal 13, 209-232. La Salle, J.P. (1976). The Stability of Dynamical Systems, CMBS 25, Society for Industrial and Applied Mathematics (Philadelphia). Lim, K.S. (1992). On the stability of a threshold AR(1) without intercepts, J. Time Series Anal. 13, 119-132. Liu, J. (1992). On stationarity and asymptotic inference of bilinear time series models, Stat. Sinica 2, 479-494. Liu, J. and Brockwell, P.J. (1988). On the general bilinear time series models, J. Appl. Probab. 25, 553-564. Liu, J., Li, W.K. and Li, C.W. (1997). On a threshold autoregression with conditional heteroscedastic variances, J. Stat. Plan. Inference 62, 279300. Liu, J. and Susko, E. (1992). On strict stationarity and ergodicity of a nonlinear ARMA model, J. Appl. Probab. 29, 363-373. Lu, Z. (1996). A note on the geometric ergodicity of autoregressive conditional heteroscedasticity (ARCH) model, Stat. Prob. Letters 30, 305-311. Lu, Z. (1998a). Geometric ergodicity of a general ARCH type model with applications to some typical models, in Advances in Operations Research and Systems Engineering, J. Gu, G. Fan and S. Wang, eds., Global-Link Informatics Ltd., 76-86. Lu, Z. (1998b). On the geometric ergodicity of a non-linear autoregressive model with an autoregressive conditional heteroscedastic term, Stat. Sinica 8, 1205-1217. Masry, E. and Tj0stheim, D. (1995). Nonparametric estimation and identification of nonlinear ARCH time series: strong convergence and asymptotic normality, Econometric Theory 11, 258-289. Meyn, S.P. and Tweedie, R.L. (1992). Stability of Markovian processes I: Criteria for discrete-time chains, Adv. Appl. Probab. 24, 542-574. Meyn, S.P. and Tweedie, R.L. (1993). Markov Chains and Stochastic Stability, Springer-Verlag (London). Nummelin, E. (1984). General Irreducible Markov Chains and Non-negative Operators. Cambridge University Press, Cambridge. Nummelin, E. and Tuominen, P. (1982). Geometric ergodicity of Harris recurrent Markov chains with applications to renewal theory, Stoch. Proc. Appl. 12, 187-202. Petruccelli, J.D. and Woolford, S.W. (1984). A threshold AR(1) model, J. Appl. Probab. 21, 270-286.

170

CLINE AND PU

Pham D.T. (1985). Bilinear Markovian representation and bilinear models, Stock. Proc. Appl. 20, 295-306. Pham, D.T. (1986). The mixing property of bilinear and generalised random coefficient autoregressive models, Stock. Proc. Appl. 23, 291-300. Pham, D.T. (1993). Bilinear times series models, Dimensions, Estimation and Models, ed. by H. Tong, World Scientific Publishing (Singapore), 191-223. Popov, N. (1977). Conditions for geometric ergodicity of countable Markov chains, Soviet Math. Dokl. 18, 676-679. Priestley, M.B. (1980). State-dependent models: A general approach to non-linear time series analysis, J. Time Series Anal. 1, 47-71. Pu, H.H. and Cline, D.B.H. (2001). Stability of threshold AR-ARCH models, tech. rpt., Dept. Stat., Texas A&M University (forthcoming). Quinn, B.G. (1982). A note on the existence of strictly stationary solutions to bilinear equations, J. Time Series Anal. 3, 249-252. Spieksma, F.M. and Tweedie, R.L. (1994). Strengthening ergodicity to geometric ergodicity for Markov chains, Stock. Models 10, 45-74. Tanikawa, A. (1999). Geometric ergodicity of nonlinear first order autoregressive models, Stock. Models 15, 227-245. Tj0stheim, D. (1990). Non-linear time series and Markov chains, Adv. Appl. Probab. 22, 587-611. Tj0stheim, D. (1994). Non-linear time series: a selective review, Scand. J. Stat. 21, 97-130. Tj0stheim, D. and Auestad, B.H. (1994a). Nonparametric identification of nonlinear time series: projections, J. Amer. Stat. Assoc. 89, 1398-1409. Tj0stheim, D. and Auestad, B.H. (1994b). Nonparametric identification of nonlinear time series: selecting significant lags, J. Amer. Stat. Assoc. 89, 1410-1419. Tong, H. (1981). A note on a Markov bilinear stochastic process in discrete time, J. Time Series Anal. 2, 279-284. Tong, H. (1990). Non-linear Time Series Analysis: A Dynamical System Approach, Oxford University Press (London). Tuominen, P. and Tweedie, R.L., (1979). Markov chains with continuous components, Proc. London Math. Soc. (3) 38, 89-114. Tweedie, R.L. (1975). Sufficient conditions for ergodicity and recurrence of Markov chains on a general state space. Stock. Proc. Appl. 3, 385-403. Tweedie, R.L. (1976). Criteria for classifying general Markov chains. Adv. Appl. Probab. 24, 542-574. Tweedie, R.L. (1983a). Criteria for rates of convergence of Markov chains with application to queueing and storage theory, in Probability, Statistics and Analysis, London Math. Society Lecture Note Series, ed. by J.F.C. Kingman and G.E.H. Reuter, Cambridge Univ. Press (Cambridge). Tweedie, R.L. (1983b). The existence of moments for stationary Markov chains, Adv. Appl. Probab. 20, 191-196.

173

Institute of Mathematical Statistics LECTURE NOTES — MONOGRAPH SERIES

TESTING NEUTRALITY OF mtDNA USING MULTIGENERATION CYTONUCLEAR DATA Susmita Datta Department of Mathematics and Statistics Georgia State University Abstract The neutrality theory of evolutionary genetics assumes that DNA markers distinguishing individuals and species are neutral and have little effect on individual fitness (Kimura, 1983). Under this hypothesis, the action of genetic drift or genetic drift in combination with mutation or migration can be used to describe the evolution of most DNA markers. In recent years, scientists have set up experiments to collect cytonuclear data over several generations to test whether the empirical evidence is consistent with this theory. In this paper, we review the existing statistical tests for neutrality based on such data and propose a new test that we believe is vastly superior. The new test arises from the likelihood theory after embedding the neutral model in a larger class of selection models, where the selection effect takes place due to a difference in fertility of various gametes. A power study based on Monte Carlo simulation is presented to demonstrate the superior performance of the new test.

1

Introduction

A major debate amongst evolutionary geneticists in recent years is whether most DNA markers distinguishing individuals and species are neutral and have little effect on individual fitness (Kimura, 1983). As a profound application of this theory, DNA sequence differences between extant species have been used to reconstruct the history of life. The classical theoretical developments of random genetic drift is built around this assumption. Under this hypothesis, the action of genetic drift or genetic drift in combination with mutation or migration can be used to describe the evolution of most DNA markers. The recent attacks on the neutrality theory are twofold. Firstly, it has been pointed out that in some cases non-neutral models can also explain behavior consistent with empirical evidence. For example, Gillespie (1979) showed that his model of selection in a random environment has the same

174

DATTA

stationary distribution as the infinite allele neutral model. Therefore, the agreement between observations and that predicted by the infinite allelic model noted by Fuerst et al. (1977) can be used with equal strength to support Gillespie's model of natural selection. In another context, Rothman and Templeton (1980) showed that, under some departure from the model assumptions, a neutral model (Watterson, 1977; Ewens, 1972) can yield frequency spectra and homozygosity similar to those expected from heterosis. In addition to the above results, a number of recent experiments suggest apparent non-neutral behaviors of mtDNA markers (Clark and Lyckegaard, 1988; MacRae and Anderson 1988; Fos et al., 1990; Nigro and Prout, 1990; Pollak, 1991; Arnason, 1991; Kambhampati et al., 1992; Scribner and Avise, 1994a, b; Hutter and Rand, 1995; etc.). Singh and Hale (1990) suggested that the apparent "non-neutral" behavior may also be caused by mating preference and that any attempt to understand the role of selection on mtDNA variants should first begin with simpler conspecific variants rather than with interspecific variants; however see MacRae and Anderson (1990), Jenkins et al. (1996). Multi-locus empirical comparisons have been undertaken by Karl and Avise (1992; also see McDonald, 1996), Berry and Kreitman (1993), McDonald (1994). In view of these recent experimental developments it is important to test whether the apparent non-neutral behavior of the markers are indeed statistically significant. Consequently it is more important than ever to devise appropriate statistical tests for testing the neutrality of a mtDNA marker. As we will see in Section 3, the existing statistical tests are often too limited to take full advantage of the multi-generation cytonuclear data that are now available. As a result, a new test based on the recent works by Datta (1999, 2001) is proposed. This test is based on an approximate likelihood for the full available data constructed from a broad parametric selection model and is therefore expected to perform well in practice. The data collection scheme and the underlying model of random drift for genetic evolution is introduced in the next section. This neutral model serves as the null model for the statistical tests which are introduced in Sections 3 and 4. A numerical power study based on Monte Carlo simulation is reported in Section 5. The paper ends with some concluding remarks in Section 6.

2

Data Collection Scheme and the Random Drift Model

In recent kitty-pool experiments, there are two potential sources of variation in cytonuclear frequencies, namely, genetic sampling variation and statistical sampling variation (Weir, 1990). Genetic sampling variation arises

NEUTRALITY OF mtDNA

175

from genetic drift, the sampling of gametes from a finite breeding pool of individuals in nature to constitute the next generation. Statistical sampling variation arises from sampling individuals from a population and using the genotypic frequencies from the sample in subsequent calculation. In Datta et al. (1996), test statistics based on cytonuclear disequilibria were constructed which can account for both sources of variation. The sampling scheme is described below. Such sampling schemes were introduced by Fisher and Ford (1947) and subsequently considered by Schaffer et al. (1977). Kiperskey (1995) also collected data on the fruit fly Drosophila melanogaster following such a scheme. We feel that these types of sampling schemes will become increasingly important in prospective tests for selection (White et al., 1998) using molecular markers in which a cytoplasmic marker is included as a control. Consider a population propagating through discrete non-overlapping generations. Although this is a simplifying and restrictive assumption, it can be achieved for an experimental population with specially selected species, such as Gambusia and fruit flies. At each generation, a portion of the adult population is collected by simple random sampling and sent for genotyping after they form the next generation; eggs by random mating. The eggs are then collected and placed in a cage to form the next generation. Thus, in this case, only the sample genotypic relative frequencies are available and are therefore subject to the additional source of sampling variation. We let g denote the number of consecutive generations from which samples were drawn. Throughout the rest of the paper, we will simultaneously concentrate on a nuclear site with possible alleles A and a and a cytoplasmic site with possible alleles Cand c. The various relative frequencies at the genotypic and the gametic levels are indicated in Tables 1 and 2, respectively. Note that since the cytoplasmic marker is only maternally inherited, its representation remains the same at both levels. Also, if needed, we will denote the generation number (i.e., time) in parenthesis and the corresponding quantities at the sample level will be indicated by the hat notation. Table 1 Genotypic frequencies Nuclear Allele Cytoplasm AA Aa aa Total C V2 P3 q Pi c P4 P5 Pe 1-q V u w 1 Total

DATTA

176

Table 2 Gametic frequencies

Cytoplasm C

Nuclear Allele A a Total eι

c

Total

q

e4 P

l-q 1

Under the action of genetic drift alone, the evolution of the population through generations can be modeled by the following Markov chain. Under the RUZ (random union of zygotes) model (Watterson, 1970), the probability of observing an offspring which received gametic types / and m, respectively, from the two parents is e/e m . Thus, the probability distribution of the counts X(t + 1) = (Xn(t +1), , Xu(t + 1)) in generation t + 1,given Ww> the gametic combination counts up to time ί, is multinomial and is given by (1)

(2) where JVi+i = Σ/,m x /m(^+l) is the size of the ί+1 generation. Finally note that this in turn determines the distribution of the genotypic and the gametic proportions p(t + 1) and e(t + 1), since they are just linear combinations of the x(t + l);viz, pk(t) = {Σf,m<*fmkXfm{t)}/Nt and φ) = ΣkβikPk(t) The coefficients a and β are given in Tables 3 and 4, respectively. For example, since the mtDNA is only maternally transmitted, the genotype AA/C can be formed by either A/C from the father and A/G from the mother, or by A/c from the father and A/C from the mother leading to = {xn(t)+X2i{t)}/Nt.

NEUTRALITY OF mtDNA

177

Table 3 The coefficients aifmk

f

m

1

2

k 3

4

5

6

1 2 3 4

1 0 0 0

0 0 1 0

0 0 0 0

0 1 0 0

0 0 0 1

0 0 0 0

1 2 3 4

0 0 0 0

1 0 0 0

0 0 1 0

0 0 0 0

0 1 0 0

0 0 0 1

lor 2

3or4

Table 4 The coefficients i k 1 2 3 4

5 6

3

1 1 1/2 0 0 0 0

2 0 0 0 1 1/2 0

3 0 1/2 1 0 0 0

4 0 0 0 0 1/2 1

Existing Neutrality Tests Based on Multigeneration Data

Here we review two existing tests of neutrality based on multigeneration data. The first one, due to Schaffer et al. (1977), compares the relative frequencies at a single locus with that expected under random drift over the generations. The test due to Datta et al. (1996) takes advantage of simultaneous data collected at a nuclear site and a cytoplasmic site over generations and compares the pathways of association measures, called the cytonuclear disequilibria, with their expected values over time under random drift. Other existing tests for neutrality generally compare various empirical characteristics based on the very last generation data with the corresponding

178

DATTA

asymptotic value reached at the equilibrium distribution under random drift. The obvious criticisms of such methods are (i) they don't take full advantage of the multigeneration data and (ii) the theoretical basis is questionable unless the population has been in existence for a long time.

3.1 The Schaffer-Yardley-Anderson

tests: This test is a modifica-

tion of a classical test due to Fisher and Ford (1947). They considered the variance stabilizing angular transformation ( 2 sin~ ^relative frequency ) of the proportions at a single locus and compared them with their constant expected value under the action of genetic drift leading to the asymptotically chi-squared distributed test statistic:

where

^ _ l'W~ιY ~ 1W-1Γ Y is the vector of transformed relative gene frequencies, 1 is the vector of ones and W is the matrix whose elements are given by μ

z-1

,

k-l/N

k=l

i y k

j=ϊ \

,*

i <j . ±y

3 J

In an effort to improve the power properties of their test, Schaffer et al. (1977) also proposed an alternative test which effectively tests for linear trend in the transformed frequencies. Although such a selection model may be hard to justify biologically, this approach does lead to a usable test statistic.

3.2 The disequilibria test due to Datta et al.: The Schaffer et al. tests do not make full use of cytonuclear data because they were constructed for tracking the information at a single locus. Datta et al. (1996) proposed testing the dynamics of the sample cytonuclear disequilibria coefficients (see Arnold 1993) with those expected under a drift model. This resulted in a test statistic of the form

T2 = (D-μYΣ^φ-μ) which has an approximate chi-squared distribution under the null hypothesis of random drift. Here D is the vector of sample cytonuclear disequilibria, μ is its expectation under the neutral model of random drift and Σ is its estimated variance-covariance matrix. The above test is somewhat difficult to implement in the sense that the formulas for Σ are complicated. Moreover its power properties may be poor due to its omnibus nature.

NEUTRALITY

4

179

OF mtDNA

A New Test Based on a Selection Model

In order to take full advantage of the multigeneration cytonuclear data, very recently Datta (2000) considered a fairly broad selection model that includes the neutrality model of random drift as a special case. The selection effect takes place because of a difference in the fertility of various gametes. We propose the resulting likelihood based score test for testing the neutrality hypothesis. Not only does it arise very naturally, it incorporates the entire cytonuclear information present in the data; furthermore since it is derived from embedding the null hypothesis of random drift into a fairly rich parametric alternative selection model, it should enjoy reasonable power properties at least when the selection model holds. A simulation based power comparison study reported in the next section shows that this is indeed the case. Consider the following selection model as an alternative to the random drift model describing the evolution of the population. The probability of observing an offspring which received garnetic types / andra, respectively, w e from the two parents is ef(w)em(w)where eι(w,t) = Wieι(t)/ yΣ j j{t))\ here W{ denote the relative fertility of gametic type i, 1 < i < 4. Therefore, the distribution of the counts x(t + 1) given the t-th generation is given by (1) with the product ef(t)em(t) replaced by ej{w, t)em(w, t). Note that when W{ = 1/4, one has the random drift model. Therefore, a test of neutrality can be based on the score statistic s(wo), with wo = (1/4, , 1/4). Since s, the derivative of the log-likelihood based on the population gametic relative frequencies, is not computable from the observed data, Datta (2001) suggested using an approximate version of it obtained by replacing them with the corresponding sample versions. This results in additional terms for the variance-covariance matrix, all of which can be consistently estimated from the observed data. One can show that the approximate log likelihood has a simple closed form expression given by g-i

6

Άw) = Σ Σ Nt+ιPk(t + 1) log(ifc(ti;,i)) t=l

k=l

Lι(w,t) = e\(w,t) +

eι(w,t)e2{w,t),

L2{w,t) = 2e1(ϊi;,ί)e3(iί;,t) + e2(w,t)es{w,t) + eι(w,t)e4(w,t) Ls(w, t) = e\(w, t) + e3(w, t)e4(tu, t),

180

DATTA

t) = ej(w,t) +es(w,t)eA(w,t)

1

and βi{w,t) = ei{t)wi/ ί Σej{t)wj J , 1 < i < 4. See Datta (2001) for the details of the algebraic calculations. Of course, the approximate score is defined as s(w) ^ (d/dw) I (w) where we interpret it as a function of w = (wι,W2, ws) ( i.e., replace w^ by I — wι — w2 — w^). Datta (2001) was able to calculate the estimated asymptotic variancecovariance matrix of s given by Σ y = (d/dw)(s(w))\w=g

+ C f d + CξC2 + ••• +

CjCg,

where

ds{w;p{l), -,p(g)) dP(t)

-1I/2

\{diag{p(t))-p(t)p(t)T)/nt]

1 < < ^ 9 Therefore, a test statistic for the neutrality hypothesis is given by T = S {WQ)Σ^S{W ) with w0 = (1/4,1/4,1/4). The neutrality hypothesis would be rejected if T exceeds Xχ_Q(3). T

0

5

Power Studies

We now report the results of a simulation study where we compare the power of Datta's (2001) test with those of earlier tests by Schaffer et al. (1977). The experimental setup is as follows. The w are parametrized by a single parameter μ so that w\ = w2 = 1/2 - μ; ^3 = W4 = μ. We simulated 2000 multigeneration samples each of g = 5 generations. The (constant) population size Nt equaled 1000 and (constant) sample size nt equaled 100. The initial population frequencies were given by pi — Pz — PA = P6 = 0; P2 = 0.5, p$ = 0.5. The counts x at the population level of successive generations are generated recursively using the multinomial model described in the second paragraph of Section 4. Next the genotypic and the gametic proportions are obtained by the formulas pk = Σ/,m afmkXfmlN and βi = Σk βikPk Finally, at every generation given the population p^ the sample pk are generated by multinomial (n; pi, • ,pβ) sampling. We simulated the powers of three different tests; (i) the omnibus test by Schaffer et al. for the mitochondrial locus, (ii) the linear trend test by Schaffer et al. and (iii) the new approximate score test by Datta described in the previous section. A nominal level of α = 0.05 was used for each case.

NEUTRALITY OF mtDNA

P _

181

*

0.8

1

•

Scor

s

SYA

1-

Unw

0.4

5-

0.2

s

ie

e

.

»

0.0

Λ

0220

0225

0230

0235

0.240

0.245

0.250

mu

Figure 5.1: Power of 5% tests at selected values of μ Figure 5.1 describes the findings. To reduce computational time involved we computed the power at only four values at and near the null hypothesis value of μ = 0.25. The figure clearly illustrates the superior performance of Datta's approximate score test over the older tests. Whereas the power of this test reaches nearly one for μ = 0.22, the power for the other tests remain flat in the entire range of μ values under consideration.

6

Concluding Remarks

Testing the neutrality of DNA markers is an important problem in evolutionary biology. In recent years, experiments have been designed in a controlled setting where a population with discrete generation can be allowed to propagate at the same time random samples from each generation are collected. Often genotyping is done at two loci, say one nuclear and one cytoplasmic, simultaneously. Utilization of such multigeneration cytonuclear data in a set up where both genetic and statistical variabilities are present is a challenging problem. We present some recent work by Datta (2001) in this direction, where a neutrality test is constructed by correctly identifying an approximate likelihood for such a setup. A numerical comparison of power with earlier tests show great promise. It will be interesting to investigate the

182

DATTA

power properties more extensively covering a broad range of non-neutrality models, possibly going beyond the selection model considered here. Such a study is underway and will be reported elsewhere.

References

Arnold, J., 1993: Cytonuclear disequilibria in hybrid zones. Annu. Rev. Ecol. Syst. 24, 521-554. Arnason, E., 1991: Perturbation-reperturbation test of selection vs. hitchhiking of the two major alleles of Esterase-5 in Drosophila pseudoobscura. Genetics 129, 145-168. Berry, A. J. and Kreitman, M., 1993: Molecular analysis of an allozyme cline: alcohol dehydrogenase in Drosophila melanogaster on the east cost of North America. Genetics 134, 869-893. Clark, A. G. and Lyckegaard, E. M. S., 1988: Natural selection with nuclear and cytoplasmic transmission. III. Joint analysis of segregation and mtDNA in Drosophila melanogaster. Genetics 118, 471-481. Datta, S., 2001. Estimation of Selection Parameters using Multi-generation Cytonuclear Data, Biometrical Journal 43, 219-233. Datta, S., 1999: Hypotheses testing for different selection models using multigeneration cytonuclear data. In Proceedings of American Statistical Association, Biometrics Section, 157-161, Alexandria, USA. Datta, S., Kiparsky, M., Rand, D. M. and Arnold, J., 1996: A statistical test of a neutral model using the dynamics of cytonuclear disequilibria. Genetics 144, 1985-1992. Ewens, W. J., 1972: The sampling theory of selectively neutral alleles. Theor. Popul. Biol. 3, 87-112. Fisher, R. A. and Ford, E. B., 1947: The spread of a gene in natural conditions in a colony of the moth Panaxia dominula L. Heredity 1, 143-174. Fos, M., Dominguez, M. A., LaTorre, A. and Moya, A., 1990: Mitochondrial DNA evolution in experimental populations of Drosophila pseudoobscura. Proc. Natl. Acad. Sci. USA 87, 4198-4201. Fuerst, P. A., Chakraborty, R. and Nei, M., 1977: Statistical studies on protein polymorphism in natural populations. I. Distribution of single locus heterozygosity. Genetics 86, 455-483.

NEUTRALITY OF mtDNA

183

Gillespie, J. H., 1979: Molecular evolution and polymorphism in a random environment. Genetics 74, 175-195. Hutter, C. M. and Rand, D. M., 1995: Competition between mitochondrial haplotypes in distinct nuclear genetic environments : Drosophila pseudoobscura vs Drosophila persimilis. Genetics 140, 537-548. Jenkins, T. M., Babcook, C, Geiser, D. M. and Anderson, W. W., 1996: Cytoplasmic incompatibility and mating preference in Colombian Drosophila pseudoobscura. Genetics 142, 189-194. Karl, S. A. and Avise, J. C, 1992: Balancing selection at allozyme loci in oysters: implications from nuclear RFLPs. Science 256, 100-102. Kambhampati, S., Rai, K. S. and Verleye, D. M., 1992: Frequencies of mitochondrial DNA haplotypes in laboratory cage populations of the mosquito Aedes albopictus. Genetics 132, 205-209. Kimura, M., 1983: The Neutral Theory of Molecular Evolution. Cambridge University Press, New York. Kiparsky, M., 1995: Cytonuclear genetics of experimental Drosophila melanogaster population. Unpublished Honor's Thesis, Department of Ecology and Evolutionary Biology, Brown University. MaCrae, A. and Anderson, W. W., 1988: Evidence of non-neutrality of mitochondrial DNA haplotypes in Drosophila pseudoobscura. Genetics 120, 485-494. MaCrae, A. and Anderson, W. W., 1990: Can mating preference explain changes in mtDNA haplotype frequency? Genetics 124, 999-1001. McDonald, J. H., 1994: Detecting natural selection by comparing geographic variation in protein and DNA polymorphisms. In Non-Neutral Evolution, B. Golding, Ed., 88-100, Chapman and Hall, New York. Mcdonald, J. H., 1996: Lack of geographic variation in anonymous polymorphisms in the American oyster Crassostrea virginica. Mol. Biol. Evol. 13, 1114-1118. Nigro, L. and Prout, T., 1990: Is there selection on RFLP differences in mitochondrial DNA? Genetics 125, 551-555. Pollak, P. E., 1991: Cytoplasmic effects on components of fitness in tobacco hybrids. Evolution 45, 785-790. Rothman, E. D. and Templeton, A. R., 1980: A class of models of selectively neutral alleles. Theor. Popul. Biol. 18, 135-150. Schaffer, H. E., Yardley, D. and Anderson, W. W., 1977: Drift or selection: test of gene frequency variation over generations. Genetics 87, 371-379.

184

DATTA

Scribner, K. T. and Avise, J. C , 1994: Population cage experiments with a vertebrate: genetics of hybridization in Gambusia fishes. Evolution 48, 155-171. Singh, R. S. and Hale, L. R., 1990: Are mitochondrial DNA variants selectively non-neutral? Genetics 124, 995-997. Watterson, G. A., 1970: The effect of linkage in finite random-mating population. Theor. Popul. Biol. 1, 72-87. Watterson, G. A., 1977: Heterosis or neutrality? Genetics 85, 789-814. Weir, B. S., 1990: Genetic Data Analysis. Sinauer Assoc, Sunderland. White, T., Marr, K. A. and Bowden, R. A., 1998: Clinical, cellular, and molecular factors that contribute to antifung drug resistance. Clinical Microbiology Reviews 11, 382-402.

185

Institute of Mathematical Statistics LECTURE NOTES — MONOGRAPH SERIES

INFERENCE ON RANDOM COEFFICIENT MODELS FOR HAPLOTYPE EFFECTS IN DYNAMIC MUTATIONS USING MCMC. Richard M. Huggins1, Guoqi Qian1 Sz Danuta Z. Loesch2 department of Statistical Sciences, La Trobe University, Bundoora, 3083, Australia. 2 School of Psychological Science, La Trobe University, Bundoora, 3083, Australia. Abstract Modern genetic research involves complex stochastic processes and difficult inference problems and research in this area is necessarily a collaboration with geneticists and biologists. The major difficulty is in defining stochastic processes which are biologically meaningful yet amenable to analysis. To illustrate this we examine a class of random effects models for dynamic mutations. Dynamic mutations characterize several inherited disorders in humans. In these disorders a mutated segment of the gene typically increases in size as it is transmitted from generation to generation until the gene fails. Biological interest is in the effect of various genetic markers on the rate of expansion of the mutated segment and to what extent these markers describe alternate pathways. We concentrate on the widely studied fragile X disorder as there are data sets available for statistical analysis. We use heirarchical Bayes models fitted via MCMC methods to examine some data and hence determine a class of random coefficients, branching, time series models which have applications in genetical research.

1

Introduction

The realistic modelling of the stochastic processes occurring in biology quickly leads to analytically intractable models. However, these models may be of enormous practical importance. Here we examine a model for the recently discovered phenomenon of dynamic mutations, that lead to a number of genetic disorders. In these disorders rather than an on-off allele being transmitted from a parent to their offspring the mutation itself changes upon transmission. The mutation arises through expansion at a region of the gene, i.e. the insertion of extra genetic material in the form of trinucleotide

186

HUGGINS, QIAN AND LOESCH

repeats at that site. This induces instability at that site with the size of the expansion usually increasing from generation to generation. Ultimately the expansion becomes large enough to affect the functioning of the gene resulting in phenotypic expression of the disorder. A feature of these dynamic mutations is that they are clinically undetectable until the expansion crosses a threshold and phenotypically abnormal individuals arise. The fragile X Syndrome is the most widely studied dynamic mutation. It is a common X-linked genetic disorder associated with intellectual disability. The fragile X syndrome arises as the result of progressive CGG trinucleotide repeat expansion in the FMR1 gene, which culminates in the failure of gene expression. The mechanism for the expansion of the repeat size has not been determined. The fragile X syndrome is classified according to the number of CGG repeats. The premutation (55-200 repeats) and full mutation (200+ repeats) categories are unstable and some instability has also been reported within grey zone (35-55 repeats) alleles. Previously the rate of expansion of CGG repeats on transmission has usually been modelled as a function of the size of CGG repeat sequence on the maternal X chromosome. However, some molecular characteristics of the gene other than the size of the CGG repeat in this chromosome have recently been shown to affect the rate of expansion. These genetic covariates are certain combinations of microsatellite markers (haplotypes) flanking the repeat sequence, and the number and position of the AGG triplets which normally interrupt CGG repeats at intervals of 9-10 units. Eichler et al. (1996) have recently hypothesised that at least two different mutational pathways leading to the fragile X syndrome related to those genetic covariates are in action in the population. However, the data obtained by Dana et al (2000) from an Afro-American population showed that this hypothesis should be viewed from a broader evolutionary perspective. Several mathematical models for the transmission of fragile X have been proposed. These include comprehensive transmission models (Ashley and Sherman 1995, Morton & Macpherson 1992, Morris et al 1995), iterated branching models (Gawel and Kimmel 1996) and Bernoulli counting process models (Bat et al 1997). The model of Ashley and Sherman (1995) accounts for the dynamics of fragile X mutation by assuming the multi-step expansion of CGG repeats (meiotic expansion in either sex, and mitotic expansion restricted to somatic alleles of maternal origin). They also included population assumptions, such as selection against full mutation, as well as molecular mechanisms, such as the loss of AGG interspersions, which in their model constitutes the initial mutation in the FMR1 gene. These previous models were biologically meaningful but were difficult to analyse statistically and tended to become outdated as more data on molecular characteristics of the gene came to hand.

RANDOM COEFFICIENT MODELS

187

Huggins, Loesch & Sherman (1998) introduced a multi-step non-linear time series models for the transmission of dynamic mutations which could be analysed using standard statistical methods. They used a random intercepts model to examine the effects of haplotypes or genetic markers on the transmission of the mutation. Here we extend that model in several directions, with an emphasis on the effect of the genetic markers on the rate of transmission. Firstly, we use cubic splines to model the relationship between the length of the repeat sequence in offspring X chromosomes to that in the parent X chromosome. Secondly we allow the rate of transmission to be random, rather than just a random intercept. Available data consists of observations on several generations of affected families as, due to the lengths of human generations, it is not possible to collect DNA to construct longer chains. However, one purpose of our modelling is to be able to model distantly related affected individuals who are known to have a common ancestor. We illustrate the inference method using published data of Murray et al (1997) who report a variety of haplotypes which may be related to the rate of expansion in fragile X. The biological meaning of our results shall be discussed elsewhere, as will formal tests of hypotheses.

2

Notation

We label the observed haplotypes by h = 1,2,..., H. We suppose there are Nh families with haplotype /ι, labelled /ι/, / = 1,..., iV^. Let n^f denote the number of parents that produce observed offspring in family /&/, labelled /ι/fc, k = 1,..., rihf. Also, let Whfk be the number of offspring of individual hfk. The offspring of individual hfk are denoted by hfkl, I = 1,..., Whfk Note that individuals may be labeled both as parents and as offspring of their parents. This notation aids in constructing the conditional densities required below. In this paper, inference is conducted conditional on the initiating individual in each family, the observed family structure and the observed haplotypes. Further, the probands, or the individuals initially detected with the disorder in each family, have been omitted. This was done on order to reduce the ascertainment bias as the probands were usually detected by their having an extreme phenotype and hence genotype. Let Y be the vector of observations of the CGG triplet repeat lengths on all offspring. Namely, Y = {Yhfki : (Λ,/,M) e S], where S is defined by S = {(Λ,/,M): h = 1,2,-- ,£Γ; /|Λ = 1,2,--.,ΛΓΛ; k\hf = 1,2, -- , n Λ / ; l\hfk = 1,2, ,whfk.}. Here f\h is understood as the value of / given h.

188

3

HUGGINS, QIAN AND LOESCH

The Hierarchical Model

Our full model for the length of the CGG triplet repeat sequence in offspring I of individual k is of the general form { h)

(βh + β f )IF(hfk)}

+ δFIFζhfkl

+ σεh}ku

(3.1)

h = 1,... ,iϊ, f\h = 1,... ,ΛΓΛ, k\hf = 1,..., n Λ /, Z|/ι/fc = 1,..., wh}k, where sa n the C/ι/Jfc/' d tfifkis are independent normal random variables with 0 means and variance 1, Xhfk is a design matrix and IF(hfk) takes the value 1 if individual hfk is female and zero otherwise. The response g(Yhfkι) is taken m to be log log Yhfki this paper as this gave errors that appeared normally distributed, although other choices are possible. We suppose βh follows a common probability distribution for all individuals with haplotype h and βi ' follows a common distribution for all individuals in family / within haplotype h. The introduction of the sex effect allows us to mimic the Ashley-Sherman (1995) model and the subsequent model of Huggins et al (1998) which were two step models with an initial step common to transmissions from males and females, and a second step that only occurs in transmissions from females. This is difficult to directly model in a linear framework so instead we allow different rates in transmissions from males and females. Available data, see the plots of Huggins & Loesch (1998), suggest there is little difference between the haplotypes or families in transmissions from males and a single vector β is used to model this transition. The conditional mean Xhfk{β + (βh + βf^I^hfk)} of g(Yhfkι) as a function of the parameters and log Yhfk was modelled using cubic splines (Dierckx 1993). The corresponding B-splines are contained in Xhfk- As the present work is exploratory ten degrees of freedom were used in the cubic splines to allow flexible modelling. This gave similar results to models with different degrees of freedom (6, 8 and 14). Let m denote the number of columns of Xhfk- The main interest is in the posterior distributions of βh

and βf\

We will use the following prior distributions for the parameters /?, βh, βf \σ2 and tip:

2. βh - b2(βh\Σho) = DMVN(0, ΣΛO), 3. iβJ Λ) ~6i(i9j Λ) |Σ / o) = MyΛΓ(O1Σ/o), 4. σ 2 ~ 6 4 ( σ > , λ ) = = IGa(i//2,i/λ/2), 5. 4 ~ 6 5 ( 4 1 ^ , λ F ) = IGa(ι/F/2, I/

h = 1,..., £Γ, f\h = 1,... ,JVΛ, Λ = l , . . . ,

189

RANDOM COEFFICIENT MODELS

which are all assumed to be independent. Here MVN denotes the multivariate normal distribution, IGa the inverse Gamma distribution (e.g. Robert 1994, p. 153). The values of v, λ, up and Xp do not have any significant effect on the analysis so will be specified in the study (see Appendix A). The hyperprior distributions for the hyper-parameters in the above prior distributions are taken to be: 1. βo ~ Wo|&,Σ) = DMVN{b,Σ), 2. Σ o - W - H ξ o + m + l^oiϊo),

ξo>m

ι

3. Σfco ~ W- (ξh0 + m + l,ξhoRhθ), ι

4. Σ/o - W- (ξf0

+ m + l,ξ/OΛ/o),

ξhx> > m, ξ/o > m,

which are also assumed to be independent. Here W7^1(ξ. + m + l.ξ.R.) denotes an inverted Whishart distribution for an m x m random matrix Σ. with degrees of freedom £ + m + 1 and positive definite matrix R. It can be shown that ^ ( Σ " 1 ) = R~ι and E(Σm) = {(£ - m - ljξ.β.}" 1 if these exist (Muirhead 1981, p.113). The values of &,Σ,fθj-Ro?fΛθ?Λ/ιθjf/o a n d Rfo will be specified in Appendix A. We denote the last three densities by wo(Σo|ξo,i?o), u>hθ(Zho\ξhθ,Rhθ) and^ /o (Σ/o|ξ/o,Λ/o) respectively. In order to simplify the presentation, we use Θ and 7 respectively to denote the parameters and hyper-parameters in the hierarchical model defined here. Namely, Θ = ( / ^ / ^ / ^ σ 2 , ^ ) * and 7 = (/3b,Σo,ΣΛO,Σ/o), where Ph — vPi?

7 P/f J 5 P/ — vP/

5

? P/

a n α

P/

— vPi

/ι = 1, , H. Note that Θ is a vector of dimension (1 + H + Σ Λ ^ I Nh)m + 2 and 7 is an m x (1 + 3m) matrix. The equation (3.1) together with the prior distribution of Θ and hyperprior distribution of 7 comprise a hierarchical random effects model for the CGG triplet repeat sequence. In order to make inference about Θ and 7, and in particular βh and βj ' from the data, we need the posterior distributions of βh and βί '. Because of their mathematical complexity we will not be able to make inferences directly from these posterior distributions. Rather we will first generate a sample from these distributions and then make inferences from the sample. The Markov chain Monte Carlo (MCMC) technique has shown to be powerful in simulating a sample from those probability distributions which are analytically intractable. Recall that the basic idea of MCMC is to use a relatively simple transition probability kernel to generate a Markov chain in such a way that its invariant distribution is that from which we want to generate a sample. After a sufficient number of generations of the chain have been simulated, subsequent generations can be regarded as a sample from the target distribution. Two most basic algorithms used in MCMC methods are Gibbs sampling and Metropolis-Hastings

190

HUGGINS, QIAN AND LOESCH

algorithm. We use a mixture of these two algorithms which is called MCMC block-at-a-time algorithm (Chib and Greenberg, 1995). Several methods are available (Robert 1998) for monitoring the convergence to stationarity of the resulting Markov chain. We employ the GelmanRubin statistic (Gelman and Rubin 1992) which measures the correlation between the within- and between-chain variations of the simulated multiple Markov chains. When the Gelman-Rubin statistic becomes very close to 1 (usually less than 1.2 or 1.1 is enough in practice), the chains can be regarded as having achieved convergence. Once a sample has been generated from the joint posterior distribution, it can be examined to detect the random effects of haplotypes and families. The sample can also be used to simulate the posterior predictive distribution for the response variable CGG repeats which, after comparison with the true CGG observations, allows determination of the goodness of fit of the model. The various joint and conditional densities and the MCMC procedure used to fit the model are described in the appendices.

4

Results

The data of Murray et al (1997) contain observations on CGG repeat sequences for 124 individuals and their parents after the probands were omitted. The remaining 124 individuals are the offspring of 86 observed parents who come from 57 different families, nested in 18 different haplotypes. In addition, 10 of the 86 observed parents are fathers who between them have 18 offspring. When applying the hierarchical model of section 3, the parameter vector Θ contains 762 components and the hyper-parameter 7 is a 10 x 31 matrix. For each of these parameters we generated 3 Markov chains of length 10,000 from which the posterior samples were formed. We first check the convergence of the simulated Markov chains. It was found that for the current model at most 2400 transitions would be sufficient for the simulated Markov chain to be stationary. In the simulations the slowest convergence occurred at the Markov chain of Θ(88) = /?s (8) (component 8 of haplotype 724) among all parameters and at 7(6,17) = E^o^β] (the 6th diagonal element of Σ^o) among all hyper-parameters. Figure 1 displays the sub-chain of a simulated Markov chain (by filter [fc/50], i.e., taking every 50th value) for each of /?s(8) and Σ^oίθjθ) and their Gelman-Rubin statistic value sequences based on the 3 parallel chains. To form a posterior sample for each parameter or hyper-parameter, we discard the first half of each of the 3 generated chains and combined the rest into one sequence; then we filter this sequence by the operator [Λ /50] and take the resulting sequence of length 300. Shown in Figure 2 are histogram densities of the posterior samples for the 10 components of the haplotype

RANDOM COEFFICIENT MODELS

191

tt> 6

I G-RvaluβsforSigr ω

^

y-1.2 " V-1-1 40

100

160

Figure 1: Row 1 gives Markov chains generated for β$(8) and Έho(6,6). Row 2 gives the corresponding Gelman-Rubin statistic values based on the three parallel chains.

8 effect 0 8 (the 81st-90th elements of Θ, i.e. Θ[81 : 90] and the 10 components of the family effects β[2) = Θ[261 : 270] (effects of the first family in the second haplotype which are the 261-270th components of Θ). These histograms are also typical of other haplotype and family effects. The haplotype effects can be analysed by examining whether or not there is any heterogeneity among the marginal posterior distributions of {βh(ϊ), h = 1, , 18} (i = 1, , 10). Shown in the first column of Figure 3 are the Xbar-chart and S-chart with 95% confidence level for posterior samples of {0i(8), ,0is(8)}, and the histogram density and the QQ-plot for the corresponding posterior means. The other two columns of Figure 3 are for {0i(6), ,/?i8 (6)} and {0i (3), ,0is(3)}. Figure 3 gives evidence of haplotype effects in the expansion process. The family effects given the haplotype effects can be similarly analysed based on the marginal posterior distributions of βf(i) (i — 1, , 10). The three columns in Figure 4 give the results of these effects from the posterior samples of 0/(8) 's, 0/(4)'s and 0/(1) 's respectively. Again family effects are evident in Figure 4. The goodness of fit of the model can be assessed by comparing the response observations g(Y) to their posterior predictive distributions (Gelman et al 1995, sec. 6.3). For each response observation g(Yhfki) a sample of size 300, regarded as the replicates of g(Yhfki) which could have been observed,

HUGGINS, QIAN AND LOESCH

20

20 40 60

0

0

40 60

192

1 A3 A bθta_8(3)

s

Jh J L

°

•10 bita.Vf

s

1

°

"

s

5 -10 -5 0 5 10 bβta_8(7)

•10

0

1

jjj^H

-5bβ°te_8δ)10

1 S

s

L

3 d

H

^

—

-

0-5 0 5 10 bβta_8(10)

S

J

o

5 bθta_1Λ{(2)}(5)

bθta_1Λ{(2)}(3)

bβta_1A{(2)}(2)

i

b 5 βtL8?8) 1 °

s

w -5

bθta_8(5)

S

l riL -10

bθta_8(4)

s

A!

jL

A (2)

Figure 2: Histograms for all the 20 components of βs and β\

II

I„

»^umber beyond

V\/ \/\

- Λ

ii

-o Λ Λ beyond ϊmits=0

Λ. ^

Number beyond id bmi(s=0 bmi(s=0

s Chart for beta.h.8

\ /°

A

V-

π

7\

' \i i&>

_

/

\ / X<

Number beyond lirn"ite=2 °

s Chart for beta.h.3

s Chart rorbeta.h.6 "*

effects.

/\

.•"'„••

\/

Number beyond limils=2

"

I:

*

o

V

'

/

"

•

%

-

.

Nurnber bβvond lirr |s=2

11

Figure 3: Effects of haplotype factors evaluated at components 8, 6 and= 3.

RANDOM COEFFICIENT MODELS

1101

^.. ^..^.. . X Al

Number beyond Km ls=2

Number beyond limits=4

s Chart lor bθta.f.8

I: " ^ I3

193

1 4 7 10 13 I t in

s Chart for bela.f.4

ΐ

s Chart (orbβta.1.1

g oil "juj'jj ib^'jis,"

111

πv*ϊ

κ

N U j mber beyond Bmits=0

π Figure 4: Effects of family factors evaluated at components 8, 4 and 1.

was generated from its posterior predictive distribution. Then the Bayes pvalue was calculated for each sample, which is the proportion of the replicates that are greater than g{Yhfki) The histogram density for the replicates of log log Yg, which is typical for all 124 response observations, is given in Figure 5. A histogram of the 124 Bayes p-values is also given in Figure 5. Since most of the Bayes p-values are very close to 0.5 (all but one are between 0.4 and 0.563), it implies that the response observations are typical under the posterior predictive distributions. In addition, we provide in Figure 5 the plots of log log Ynew IY — the sample means of the posterior predictive distributions against= log log Y, and £?(loglog Y|Θ p o 5 ) — the fitted response values using the posterior means of the distributions of the /?'s against log log Y. These plots show the reasonable fit of the proposed model to the data which is also supported by the values of the goodness of fit statistic computed from (2) below. Actually it was found that || loglog Y-loglog Yne™|Y||2 = 14.10 and 11 log log Y - E(loglog Y | Θ p o 5 ) | | 2 = 7.26. However, the histogram density in Figure 5 shows that the posterior predictive distribution of Y has a quite wide range and a large variance, particularly considering the nature of a loglog( ) transformation for the response Y. This implies that the proposed model has a large variation in its predictability although it is not the main interest of this paper. One could possibly obtain better predictions if another transformation g(Y) or a discrete model such as a Poisson distribution for Y were used. Finally we propose to evaluate the goodness of fit of the model by the

HUGGINS, QIAN AND LOESCH

194

summary(bayesPIO) Min. 1st Qu. Median Mean 3rd Qu. Max. 0.317 0.477 0.497 0.497 0.517 0.563

-15-11-7-3 1 5 9 13 17 Histogram density of log(log(Y_9)) from a

0.25 0.31 0.37 0.43 0.49 0.55 0.61 Histogram for θayes p-vakies with individual

v obs. parent is father t-obs. parent is mother l

••

+ obs. parent is father +obs. parent is mother

k>g(log(Y))

Figure 5: Model assessment based on posterior predictive distributions.

following statistic N

N

i=l hfkl

Var(Yhfkl\θi)

(4.1)

where θ i , , θ # is a posterior sample of θ . Although no rigorous justification is available, it sounds sensible to approximate T by a χf 24 distribution without being too erroneous. Based on the posterior θ sample generated in our study we found T = 87.26, which corresponds to 0.5-percentile of a χ{24 and thus indicates that the proposed model gives an adequate fit.

5

Discussion

Many of the advances in inference for stochastic processes over the past few decades have arisen from branching processes and time series models. A population model for the transmission of dynamic mutations combines time series models with branching processes, for, as well as (3.1) holding the number of offspring Whfk of individual hfk has a distribution that depends on both the sex of individual hfk and the value of Yhfk- The outstanding problem is to find conditions under which the population model has a stationary distribution. We have concentrated on developing and fitting time series type models for the transmission of a dynamic mutation given the observed haplotypes

RANDOM COEFFICIENT MODELS

195

and family structures. The model can be viewed as an extension of the simpler bifurcating autoregressive models for cell lineages. The work on cell lineage studies starting with Cowan (1984), Cowan &: Staudte (1986) and others considers the bifurcating autoregressive process for cell lineage studies. Cell lineage trees arise from the binary splitting of cells and the processes of interest are time series, typically low order autoregresive models, down lines of descent with correlated errors for sister cells and in later models (Huggins & Basawa 1999) correlations between individuals in the same generation. Bui & Huggins (1999) considered random effects models for the bifurcating autoregressive(l) model. The processes of interest here have a random number of offspring rather than just the two of the bifurcating autoregressive process and the models down each line of descent are more sophisticated than the ARMA type processes. Moreover, individuals with a given identified haplotype may differ at other gene sites, which naturally gives rise to the random effects models. Hence we take the transition rates to be random, which are common for individuals in the same families and there is communality between unrelated individuals with the same haplotype. Markov models are typically used to model the transmission of a characteristic from parent to offspring. That is, the genetic make up of the offspring only depends on that of the parents. The indiscriminant use of such Markov models may be unrealistic in the branching situations that occur in family studies. For example, in related work on cell lineage studies, Staudte et al (1996) determined that the correlation between cousins in the data examined by them was larger than what was predicted by AR(1) type Markov models. In Markovian AR(1) models for cell lineages, correlations drop off as powers of the mother-daughter correlation θ. That is, mother-daughter correlation is 0, the= sister-sister correlation is θ2 (plus environment effects) the cousin-cousin correlation is 04 etc. Huggins & Basawa (1999) proposed several models which allow higher correlations between cousins and other more distant relatives including an AR(2) type model as well as random effects models of generation effects. This situation may also occur in dynamic mutations unless genetic covariates that affect the expansion rates are taken into account. This motivated our hierarchical random effects model. The models examined here are based on transmissions over one or two generations and may not reflect the full complexity of the dynamic mutations. Moreover, the processes are sparsely observed and as the process is unobserved in its initial stages, there is little information concerning the mutation rate currently available. The nested random effects, which depend on haplotypes that are themselves evolving according to some stochastic process, add another level of complexity. However, the haplotypes occur on a portion of the gene which does not appear to code for a protein hence they should not be exposed to selection so that their evolution should be depen-

196

HUGGINS, QIAN AND LOESCH

dent only on random mutations and rare recombinations and be simpler than that of the the FMR1 gene itself. Future work will involve the collection and modelling of data on distantly related affected individuals, where the evolution of the haplotypes will need to be considered. The strength of the relationship between individuals will be determined either through family trees or through genetic markers with known mutation rates. It is not necessary to use MCMC to estimate random effects although it was convenient in the present example. For example, Park & Basawa (2000) have developed an optimal estimating function approach. However, they have not yet fully developed the inferential procedures. The use of cubic splines has advantages in that the models are linear and may to some extent be regarded as non-parametric. Moreover, it was not necessary to introduce a threshold as in Huggins et al (1998). However, the non-linear models of Huggins et al (1998) did have some biological advantages in that they more naturally modelled the expansions in the meiotic and mitotic phases. In the non-linear case it was possible to let the parameters in the model depend on the size of the parents repeat sequence which is more difficult in linear models. The increasing interest in molecular genetics will result in new and complex stochastic processes. However, as in cell lineage studies and the example considered here, these new processes are combinations of familiar simpler stochastic processes with perhaps extended error structures. Nevertheless, it appears certain that many new problems in statistical inference and applied probability will arise in this area. Acknowledgements This research was supported by NICHID grant # HD36071 and an Australian Research Council small grant.

References [1] Ashley, E.A. & Sherman, S.L (1995) Population dynamics of a meiotic/mitotic expansion model for the fragile X syndrome. Am. J. Hum. Genet 57: 1414-1425. [2] Bui, Q.M. & Huggins, R.M. (1999) The random coefficient bifurcating autoregression model in cell lineage studies. J. Stat. Plan. & Inf. 81; 253-262. [3] Bat O, Kimmel M, & Axelrod DE (1997) Computer simulation of expansions of DNA triplet repeats in the fragile X syndrome and Huntington 92s disease. J.Theor.Biol. 188: 53-67

RANDOM COEFFICIENT MODELS

197

[4] Chib, Siddhartha and Greenberg, Edward (1995) Understanding the Metropolis-Hastings Algorithm American Statistician V. 49, No. 4 327335. [5] Cowan, R. (1984). Statistical concepts in the analysis of cell lineage data. Proceedings of the 1983 Workshop on Cell Growth and Division., 18-22, LaTrobe University. [6] Cowan, R. and Staudte, R.G. (1986). The bifurcating autoregression model in cell lineage studies. Biometrics. 42, 769-783. [7] Dana C. Crawford, Charles E. Schwartz, Kellen L. Meadows, James L. Newman, Lisa F. Taft, Chris Gunter, W. Ted Brown, Nancy J. Carpenter, Patricia N. Howard-Peebles, Kristin G. Monaghan, Sarah L. Nolin, Allan L. Reiss, Gerald L. Feldman, Elizabeth M. Rohlfs, Stephen T. Warren, and Stephanie L. Sherman (2000) Survey of the Fragile X Syndrome CGG Repeat and the Short-Tandem-Repeat and SingleNucleotide-Polymorphism Haplotypes in an African American Population. Am. J. Hum. Genet. 66 480-493. [8] Dierckx, P. (1993). Curve and surface fitting with splines, Clarendon: New York. [9] Eichler, E.E., Macpherson, J.N., Murray, A. Jacobs, P.A., Chakravarti, A. and Nelson, D.L. (1996) Haplotype and interspersion analysis of the FMR1 CGG repeat identifies two different mutational pathways for the origin of the fragile X syndrome. Human Molecular Genetics. 5 319-330. [10] Gawel B & Kimmel M(1996) The iterated Galton-Watson process. J.Appl.Prob. 33: 949-959. [11] Gelman, A. and Rubin, D.B. (1992). Inference from iterative simulation using multiple sequences (with discussion). Statistical Science 7,457511. [12] Gelman, A., Carlin, J.B., Stern, H.S. and Rubin, D.B. (1995). Bayesian Data Analysis, Chapman & Hall: London. [13] Huggins, R.M. & Loesch, D.Z. (1998) On the analysis of mixed longitudinal growth data. Biometrics [14] Huggins, R.M., Loesch, D.Z., & Sherman, S.L. (1998) A branching nonlinear autoregressive model for the transmission of the fragile X dynamic repeat mutation. Ann. Hum. Genet. 62 337-347. [15] Huggins, R.M. & Basawa, I.V. (1999) Extensions of the bifurcating autoregressive model for cell lineage studies. J.Appl.Prob. 36 1225-1233.

198

HUGGINS, QIAN AND LOESCH

[16] Morris, A., Morton, N.E., Collins, A., Macpherson, J., Nelson, D. and Sherman, S. (1995) An n-allele model for progressive amplification in the FMR1 locus. Proc. Natl. Acad. Sci. USA 92 4833-4837. [17] Morton, N.E. and Macpherson, J.N. (1992) Population genetics of the fragile X syndrome: multiallelic model for the FMR1 locus. Proc. Natl. Acad. Sci. USA 89 4215-4217. [18] Muirhead (1981).Aspects of multivariate statistical theory, John Wiley: New York. [19] Murray, M.A., Macpherson, J.N., Pound, M.C., Sharrock, A., Youings, S.A., Dennis, N.R., McKechnie, N., Lineham, P., Morton, N.E., Jacobs, P.A.(1997). The role of size, sequence and haplotype in the stability of FRAXA and FRAXE alleles during transmission. Hum. Mol. Genet. 6:173-84. [20] Park, Jeong-gun &; Basawa, I.V. (2000) Optimal estimating equations for mixed effects models with dependent observations. Technical report 2000-16, Department of Statistics, University of Georgia. [21] Robert, C.P. (1994) The Bayesian Choice. New York: Springer-Verlag [22] Robert, C.P. (1998). Discretization and MCMC Convergence Assessment, Springer-Verlag: New York. [23] Staudte, R.G., Zhang, J., Huggins, R.M. and Cowan, R. (1996). A reexamination of the cell lineage data of E. O. Powell. Biometrics., 52, 1214-1222.

199

RANDOM COEFFICIENT MODELS

Appendices In the appendices we derive the joint and conditional densities required to estimate the model parameters using MCMC, and then give the MCMC algorithm.

A

Joint Densities

Using the notations introduced in the paper, it follows that the joint conditional density of g(Y) given Θ, 7 and the observed ancestor of each family is H

{g(Y)\θ,Ί}

P

Nh

nh=fhfk

ί>{5(Y)|ΘJQΠ Π h=lf=l nhf

H Nh

ΐ[p{9(Yhfki)\β,βh,βf\σ2,δ2F}

k=l 1=1

h=l /=1 k=l

{g(Yhfki)-Xhfk[β 2(PpIF(hfk)

+ σ*) (A.1)

The joint prior density of the parameter Θ given 7 is H Nh

π(θ|7)

H

Σ

Π Π Wrl /o) Π W Λ o ) h=lf=l

h=l

H Nh /ι=l/=l H X h=l

{

1

ff

Λ=l/=1

200

HUGGINS, QIAN AND LOESCH

x|Σ Λ 0 Γ* / 2 exp {-^ x|ΣoΓ 1/2 exp {—tr&^iβ

- βo)(β - βoΫ}]

(A.2) The joint prior density of the hyper-parameter 7 is τr(7)

= ω/0(Σ;o|ξ/θιΛ/o) ωω(Σ!fto|ξfco,ΛAθ) ^o(Σo|ξo,i?o) • bo{βo\b,Σ)

x(2π)-"/ 2 |ΣΓ 1 / 2 exp{-i(ft,-6)'Σ- 1 (A-6)}

(A.3)

where Γ m ( ) is the multivariate gamma function (see Muirhead 1981, pp. 61-63). In the simulations, the value of b was taken to be the estimate of β in the spline regression model g(Yhfki) = Xhfkβ We also used the following values in the simulation in such a way that they should have very weak effects on the simulation results: v = λ = up = \ F = 10,= Σ = RQ = Rho — Rfo — 10^10 with Jio being the 10 x 10 identity matrix, and £o = 6ιθ = ζfo = 50.

B

Posterior Conditional Densities

In order to simulate the posterior distributions using MCMC methods we need a number of posterior conditional densities. The joint posterior density of Θ, 7 given the observed ancestor of each family is

The conditional posterior density of θ given 7 and the observed ancestor of each family is p{g(Y)|Θ,γ}7r(Θ|Ύ)

fR

.

201

RANDOM COEFFICIENT MODELS

The conditional posterior density of 7 given Θ and the observed ancestor of each family is

C

Simulating the Posterior Densities by MCMC

To simulate the posterior distributions of the β^s and βί s, we first generate a random sample for (©,7) from the posterior density π(θ,7|Y). Then we extract those values of βh and βf in the sample, which clearly comprises a 1 random sample from the posterior distributions of (/?£, βj) . So the question is how to generate a random sample from the posterior density π(θ,7|Y). To answer this question we apply an Markov chain Monte Carlo method (MCMC) which uses a Metropolis-Hastings acceptance-rejection algorithm alternately to the two blocks of the random quantity (©,7). In other word, this is the so-called MCMC block-at-a-time algorithm (refer to Chib and Greenberg, 1995). We list the algorithm in the following: Obtain the initial values (7(0), 1. Generate a matrix 7^°) from the prior π(7). 2. Generate a vector θ<°) from the prior 7r(e Update ( 7 ϋ), θ ^ ) to {^j+ι\ θ ^ + 1 ) ) . Repeat for j = 0,1,

, N - 1.

1. Use an M-H acceptance-rejection algorithm to generate a matrix 7^ + 1 ) from the conditional posterior density π ( 7 | Y , θ ^ ) : - Generate an initial matrix 70 from τr(7). - Repeat for i = 0,1, , / — 1. - Generate a matrix 7' from π(7) and u from the uniform distribution ZY(/, 00). - If u < Oij(ji ,V), set 7^\ = 7 ; . The acceptance probability is defined by

a 7(7ί ( τ ω τ7 V mini )

' ~

π ( θ | 7 )

U(Θ(%?Y

l

- Finally set 2. U s e a n M - H a c c e p t a n c e - r e j e c t i o n a l g o r i t h m t o g e n e r a t e a v e c t o r QU+ι) f r o m t h e c o n d i t i o n a l p o s t e r i o r d e n s i t y π ( ^ ^

202

HUGGINS, QIAN AND LOESCH

- Generate an initial vector ΘQ fr°m 7r(θ|7^ + 1 )). - Repeat for q = 0,1, , Q — 1. - Generate a matrix θ ' from π(θ|7^ + 1 ^) and u from the uniform distribution Z//(/, oo). - If u < OLQ(QΨ,&), bility is defined by

set θ ^ i = ©'. The acceptance proba-

- If u > aθ(θφ,&), set Θ% = θ[j). - Finally set θ^ + 1 ) = θ § } . Return the sequence {(7<°\ β ^ ) , ( 7 ( 1 ) , θ W ) , • • •, (7 ( Λ Γ ) , Θ(JV))}. As in any MCMC method, the sequence generated by the above algorithm is actually a Markov chain with τr(θ,7|Y) a s i*s invariance density. So by ignoring the first iVo values the sequence can be approximately regarded as a random sample from the posterior density τr(θ, 7|Y), provided that both NQ and N are sufficiently large. In our simulations we found that when taking / = Q = 50 the simulated Markov chain became convergent after No > 48 runs.

205

Institute of Mathematical Statistics LECTURE NOTES — MONOGRAPH SERIES

SEMIPARAMETRIC INFERENCE FOR SYNCHRONIZATION OF POPULATION CYCLES RE. Greenwood Dept. of Mathematics, Arizona State University, Tempe, Arizona and Dept. of Mathematics, University of British Columbia, Canada D.T. Haydon Centre for Tropical Veterinary Medicine Easter Bush, Roslin, Midlothian, U.K.

1

Introduction

We consider a dynamic random field. On each of a discrete array of sites is located a hyperbolic dynamical system perturbed by noise. We assume that the dynamics are identical, with a unimodal limit cycle and that the perturbations are independent, centered, and with the same distribution. In addition the individual processes are coupled with one another in a homogeneous pattern. The coupling may be global, in which case we are thinking of a mean-field type of system. Or the coupling may be local, the coupling strength between each site and its neighbors attenuating with distance. The application we have in mind is to cycling populations of animals, where the log of each local population increases roughly linearly to an apparent critical point from which it falls precipitously to a minimum. The data is discrete in space and time, being based on periodic reports from catchment or reporting regions. In a previous study [1] we focussed on data from Canadian lynx populations. Lynx population cycles are known to follow those of snowshoe hare. Previous analyses of this data have been concerned with inferring the length and regularity of the evident population cycles. We consider in [1] the very different challenge of estimating a parameter identified as strength of coupling among populations. That paper is primarily addressed to data analysis and interpretation. The emphasis in this report is on the steps involved in arriving at a suitable model and estimator. We explore some of the difficulties posed by this rather unusual problem. In forthcoming studies we, with colleagues, will apply the method described here to cycling population data from Canadian muskrat and mink, and from the greysided vole of Hokkaido.

206

2

GREENWOOD AND HAYDON

Development of a Coupled Evolving Phase Field Model

To begin, let us introduce a general definition of synchronization applicable to random fields. Let X = {Xa,i € I,t = 0,1,2,...} denote the values at site i and time t of a dynamic random field. Let S = {£(•)*, t > 0} denote a real-valued functional or collection of functionals of {Xis, i € / , s < *}. Note that we allow S evaluated at time t to depend on data up to and including time t. Define S'-synchronization to mean {SX)t = 0, almost surely, for t in some time-set T. This gives us a flexible definition of exact synchronization. But a random field will not be exactly synchronized. In order to formulate a statistically useful definition we need to allow for and to measure departures from synchronization. For example, we probably want the mean and variance of S applied to random data to be small. We are led to the following: Definition. A dynamic random field X departs from 5-synchronization by no more than e over the time interval T if E(SX)Ί < e for all t eT. One might use other norms, e.g. Lι or Kullbach-Leibler instead of L 2 . However this definition has a familiar form and is computationally convenient. At each site i in an array of sites / is located a "skeleton" process which, in the absence of any noise or coupling can be written -ΪM+i = HXith

i e /, t = 0,1,2,....

(2.1)

In this treatment we write the dynamics as of first order in discrete time. We assume there is a unimodal limit cycle t(t),t = 0, . . . , p , ^(0) = t{p), where, since / is the same for each i, the form of the cycle, ί^ and its period, p, are the same for each site i. For definiteness let ^(0) be the minimum point of the unimodal cycle. If we picture the deterministic field (1) running in equilibrium, we see the periodic cycle ί{ ) being executed at each site, the only difference between sites being the phase, which will differ if the initial points X^o were not identical. Now suppose we watch, instead, the stochastic field, X^t+ι=f(Xit)

+ eiu

t 6 / , t = 0,l,2,...

(2.2)

where en represents a small, identically distributed, centered noise. A theorem of deterministic dynamics called the "shadowing lemma" implies that the stochastic paths of (2) shadow the paths of (1), where "shadow" means that they remain in a distributional neighborhood of the deterministic path up to a time shift. The differences among the paths at the various sites, then, in addition to the small width of this neighborhood, are the phases, which change in a random way. The neighborhood width depends on the

207

INFERENCE FOR SYNCHRONIZATION

noise variance. This picture motivates a model for coupling based on the randomly evolving phases of the components. We now move our attention from the evolving random field of population levels, (2), to a corresponding evolving random field of phases. Going back to the deterministic model (1) we can unambiguously define the phase φa at site 2, time £, to be the time fraction of the current cycle, ^(0),... ,^(p), which has been accomplished at time t. Then each phase φu is in the interval [0,1). The arithmetic for phases is modi. We think of the set of points {Φit,i G /} as a set on the circle of circumference 1. As time advances the points progress around the circle. Since in fact we wish to consider the phase field associated with the stochastic field (2), where the "limit cycle" is a stochastic perturbation of the deterministic one, we may not see a unique minimum for each cycle, and the definition of φu may be ambiguous. We will assume that the noise en is small enough so that the ambiguity can be resolved by a device described in the next section. For the moment let us ignore this problem and assume that in the stochastic model the phase φn is defined as the fraction of the current orbit which has been traversed at time t by each path X{ of the stochastic field (2). We describe the structure of the phase field φa, i G /, t = 0,1,2,..., by writing Φiί+\ = Φit + 9it + tit

mod 1,

(2.3)

where gu is the fraction of the current orbit traversed at site i at time t. Now we introduce a hypothetical coupling force into the phase field which will shift the phase at each site i and time t in the direction of the "mean phase". For this purpose we need to devise an approp