State of the Art in Probability and Statistics: Festschrift for William R. Van Zwet (Lecture Notes - Monograph Series, Volume 36)

(2-8)

(n-q)-1 £ i=9+l

Extremal REACT Fits

77

Consistency of σ2H is assured when this bias tends to zero as n tends to infinity. Economy of UE makes the bias small when q exceeds the number of basis vectors needed to approximate η well. The estimator σ2H is particularly useful when n = p, the one-way layout with one observation per factor level. Having devised a consistent estimator σ2 of σ 2 , we estimate ζ2 by z2 — σ2 and the risk p(/, ξ 2 , σ 2 ) by p(f) = ave[σ 2 / 2 + (z2 - σ 2 )(l - /)*].

(2.9)

Tacit in the construction of p is the supposition that the law of large numbers will make ave[(l - f)2(z2 - σ2)] consistent for ave[(l - f)2ξ2)]. Because p(/) can sometimes be negative, we will consider as well the risk estimator p+(/) = max{σ 2 ave(/ 2 ),p(/)}. The uniform consistency of p(/) and p+(/) over suitable T is treated by Beran and Dύmbgen (1998).

2.3

Adaptation

It is natural to use p(/) as a surrogate for the risk p(/,ξ 2 , σ 2 ) in identifying the best candidate estimator. This strategy generates the fully data-based estimator ήjr = UEdidLg(f)U'E, where (2.10)

/ = argminj&(/).

Apart from the details of the variance estimator σ 2 , this construction of / amounts to minimizing the Mallows (1973) Cx criterion or minimizing the Stein (1981) unbiased estimator of risk. Successful adaptation, meaning that the risks of ήjr and ή? converge, requires restrictions on the richness of T. Beran and Dύmbgen (1998) developed sufficient conditions on the covering number of T to ensure success p of adaptation. The global class T = [0, ΐ\ is too large for adaptation. Two smaller but very useful shrinkage classes for which adaptation works are: p a) Monotone class Tw This is the closed convex set {/ G [0, l ] : /i > /2 > • > /p} It makes sense to damp down the higher order components of z in constructing fz precisely because UE is an economical basis. The value / M that minimizes estimated risk p(f) over all / G TM is unique and can be computed by algorithms for weighted isotonic regression, as detailed in Beran and Dύmbgen (1998). The corresponding estimator of η is ήM = UEdia,g{fM)U'Ey. b) Nested selection class TNS- This subset of TM is defined as follows. For 0 < k < p, let e(k) denote the p dimensional column vector whose i-th component is 1 if 1 < i < k and is 0 otherwise. Then TNS is the union of the vectors {e(fe): 0 < k < p}. Nested model-selection is the idea behind whose convex hull is TM- Computation of SNS-, the value that

78

Rudolf Beran

minimizes p(f) over all / G FNS, is straightforward. The corresponding estimator of η is ήws = UEdi&g(fNs)UfEy. In the original parametrization, the candidate estimators have the form UEdia,g(f)U'Ey. They are thus symmetric linear smoothers, in the sense of Buja, Hastie and Tibshirani (1989), whose eigenvectors are given by the columns of UE This observation explains why REACT estimators can act like adaptive locally linear smoothers. 2.4

Case studies with polynomial contrasts

The underlying scatterplot in Figure 1 exhibits log-income versus age of the individual sampled. This Canadian earnings data was introduced by Ullah (1985) and treated further by Chu and Marron (1991). Conditioning on the observed ages, we fit an unbalanced one-way layout to the n = 205 observed log-incomes, the factor levels being the p = 45 distinct ages from 21 to 65, taken in numerical order. The top row of Figure 1 exhibits the polynomial contrast estimators ήπs and T)M, using the least squares estimator of variance to compute risks of candidate REACT estimators. In both plots, the components of the estimator have been interpolated linearly. The visual impression created by cubic spline interpolation is similar. Such interpolation is more than a visual device if we consider mean log(income) to be a continuous function of age. Using p( ) to estimate the risks of Ϊ]LS, VNS and Ί]M yields, respectively, PLS — σ\s — .295, PNS — —-029 and pM — —.037. The negative values are inconvenient here. Using instead p+( ) yields the risk estimates p+,ΛΓ5 = -039 and p+,M = .036. Both of the REACT estimators have fax smaller estimated risk than the least squares estimator f]LS- If the latter is plotted with linear interpolation, the resulting curve is jagged, especially at the higher ages. The two interpolated REACT fits are fully data-based once the basis Uβ is selected, with no tuning parameter requiring attention, and resemble fits obtained for this data by locally linear smoothers. Fits to this data by Nadaraya-Watson kernel smoothers are biased upwards near ages 21 and 65 but otherwise resemble the REACT fits (see Chu and Marron (1991)). Because ages are equally spaced, the two polynomial contrast REACT estimators actually fit polynomials at the earnings data-points, though not in between. The estimator ήjsrs fits a polynomial of degree 5 (which coincides with the degree 5 least squares fit) while J]M fits a polynomial of degree 14 (which differs considerably from the degree 14 least squares fit). All coefficients of the fitted REACT polynomials beyond the term of degree 2 are very small. Indeed, a classical F-test at level .10 does not find evidence of nonzero coefficients beyond degree 2. Because rejection, not acceptance, is important in testing, this result only indicates that we should not use a fit of degree less than 2. The parabolic least squares fit to the earnings data

Extremal REACT Fits

79

completely misses the econometrically interesting dip between ages 40 and 50 and has notably larger estimated risk (namely .106) than either of the two REACT fits. Estimating the mean vector with small risk is an enterprise that differs from seeking overwhelming test evidence that certain regression coefficients are non-zero. In Figure 2, the (1,1) cell plots the signed square root of Zi versus i. The square-root transformation makes more visible the values of Z{ that are close to zero in value. This diagnostic plot supports the hypothesis that the polynomial contrast basis is economical for the earnings data, a plausible finding because the levels of the age factor are ordered in time. The (1,2) cell exhibits the components of first five orthonormal polynomial contrasts, with linear interpolation to aid visibility. The underlying scatterplot in Figure 3 exhibits, for each row in a vineyard near Lake Erie, the total grape harvest over three years. Simonoff (1996) used this vineyard data as a case study for smoothing methods and provided further background. Conditioning on the p = 52 row numbers, we fit a balanced one-way layout to the n = 52 observed three-year harvests. The top row of Figure 3 exhibits, with linear interpolation, the polynomial contrast estimators ή^s and f\w The high-component variance estimator with q = 15 served to compute estimated risks. The estimated risks of 7)2,5, ήπs and Ύ]M are, respectively, PLS = σ2H — 6.08, pπs = 1-24 and pM = 1-02. Row numbers being equally spaced, the estimator ήπs fits a polynomial of degree 10 to the points in the vineyard data scatterplot (but not in between). This fit coincides with the degree 10 least squares fit to the same points. The estimator J]M fits a polynomial of degree 17 to the points in the scatterplot (but not in between). This fit differs substantially from the degree 17 least squares fit to these same points. Linear interpolation between fitted points avoids the wiggliness inherent in polynomial curves. Both REACT fits resemble a locally linear nonparametric regression fit to this data exhibited in Fig. 5.13 of Simonoff (1996). The diagnostic plots in the top row of Figure 4 and the relatively small estimated risks of both REACT fits support the supposition that the polynomial contrast basis is economical for the vineyard data. This conclusion is plausible because the levels of the row-number factor are spatially ordered.

3

Confidence Sets and Saturation

We begin by applying the confidence set idea sketched at the end of Stein (1981). For T equal to either TM or JJVS, consider the root (3.1)

tτ = pl/2\p-1\ήτ-η\2-pCf)}.

The right side of (3.1) compares the normalized quadratic loss of ή? with an estimate of its expectation, which is the estimated risk. A confidence

Rudolf Beran

80

Best M Fit using Polynomial Contrasts

Best NS Fit using Polynomial Contrasts

S

20

30

40

50

20

60

30

40

Age

Most Unsaturated 95% Confident M Fit

'• S

•sK

1*

20

-:

30

•.litr:

.

40

50

Ξ

"" ! .

i .

;'. ••• .

60

20

30

40

I 1

ll

:M i'/.

a

^\

••

I m

•

f 20

60

15

.m

• •

14

•

50

Most Saturated 95% Confident M Fit

nco me)

•

a

.:

Age

• •

•

•

2

Most Saturated 95% Confident NS Fit

:l

• - ' I -

S l

Age

CD

60

Age

Most Unsaturated 95% Confident NS Fit

:

50

CO

.: .ί: "\W

•yi

.

•

•

1

N".

•

•

•.. ι

• 1 CsJ •

I

•

30

40 Age

50

•

.

60

20

•

30

40

50

Age

Figure 1. REACT fits and extremal confident fits to the Canadian earnings data.

•

60

81

Extremal REACT Fits

set for η is obtained by referring t? to the α-th quantile of its estimated distribution. For reasons detailed in Section 4, the variance estimator σ 2 that enters into the definition of ί^ strongly affects the next step in the construction. We therefore write tjr^ and i^2 to distinguish the two cases. Least squares variance estimator. When σ2 = σ 2 5 , the distribution of tjri for large p and n — p is approximately iV(0, fj x ) with (3 2)

^

=

2

^ a v e [ ( 2 / - I) 2 ] + 2[p/(n - p)]σis[ave(2/ - I)] 2 + [ 4 £ [ ( ^ | ) ( l / ) 2 ] ]

Accordingly, a confidence set of approximate coverage probability a for η is (3.3)

Cyfi(α) = { θ e

M ( X ) :

\ ή ? - θ\2 < p p Q )

1

/

2

1

In this expression, Φ " 1 is the quantile function of the standard normal distribution. When T consists of the single vector e(p), defined in Section 2, the corresponding REACT estimator ήp is just 7)^5. The classical confidence set for η based on the F-distribution is {θ G M(X): \ήf-θ\2 < pσ2LSF~^_p{a)}. If p and n—p both tend to infinity in such a way that p/(n—p) converges to a finite constant, then the classical confidence set is asymptotically equivalent to the confidence set given by (3.3) when T = {e(p)}. High-component variance estimator. The preceding confidence set is not available for one-way layouts with one observation per factor level. Suppose that n = p and σ 2 = σ\. Let Λi = 2/ - 1 + \p/(p - 9)][ave(l - 2/)](l - e(q)) h2

= f - l + \p/(p - g)][ave(l

ι 2

2

The distribution of ijr2 for large p and p — q and small p~ l Σf = ς + 1 ξ is approximately j ^ (3.5) The (3.6) 3.1

>

corresponding confidence of approximate coverage probability a for η is C y | 2 ( α ) = { θ e M ( X ) : \ ή τ - θ\2 < pp(f)

1

/

2

ι

Saturated and unsaturated fits

Visualizing either of the confidence sets Cjr^a) or CJ:^{OL) as subsets of n the regression space M(X) C R is difficult at best. One useful way of interpreting such confidence sets centered at ήj? is to ask:

82

Rudolf Beran

First 5 Polynomial Contrasts

0.2

Earnings z for Polynomial Contrasts

8

0.2

0.0

CD 3

0

10

20

30

40

0

50

100

150

200

Component

Observation

Shrinkage Factors for the NS Fits

Shrinkage Factors for the M Fits

1 CO

CO

d

d

ω CO

I2

I2 1

q d 10

20

q d 30

40

10

Component

Residuals after Best NS Fit

Residuals after Best M Fit

•

• •

• 1a

2• S

•

α>

•

1 •

•

1

•

:

esi dual

O

o -

s

i !i':. .!:•••••;•-. " • " • .

I"-

ιγ-t

20

40 Age

50

"

•

60

*

T -

Cvl

30

1

•

•

.

•

• m m

•»-

•

• CVJ

40

30

Component

•

Λ

20

•

.

20

30

40 Age

Figure 2. Diagnostic plots for REACT fits to the Canadian earnings data.

50

•

60

Extremal REACT Fits

83

a) Which features in the estimator ηjr are not necessarily present in η once sampling error is taken into account? b) Which features of η might have been smoothed out by the estimator ήj: because of sampling error? We will construct extremal members of the confidence sets Cjr^(α) that throw light on both of these questions. For the bases considered in this paper, these extremal elements amount to "smoothest" and "roughest" perturbations of the REACT fit that lie on the boundary of the confidence set. A shrinkage vector / G [0, l]p is said to be saturated up to order k if /i = . . . = fk = 1. It is said to be unsaturated down to order k if fk = .. = fP = 0. Let (k) = {/ G FM' / is unsaturated down to order k} ^(k)

= {/ G TM' f is saturated up to order k}.

Define (3.8)

/M,U(A)

= argmin p(/), / ^ M

fhf,s(k) = argmin p(/). f { k )

Among the shrinkage vectors {fM,u{k):k > 1} such that the vector UE<&&g{fM,u{k))U'Ey lies in C ^ ( α ) , let ]M,U be the one for which k is smallest. We say that Ϊ)M,U — UEdidLg(fM,u)UfEy is the most unsaturated a-confident M fit for η. In the other direction, among the shrinkage vectors lies i n {fM,s(k):k ^ !} s u c h t h a t uEdiaLg(fM,s(k))uΈy C>,z(α), let fM,s be the one for which k is largest. We say that r)M,s = UE(li3ig(fM,s)UfEy is the most saturated a-confident M fit for η. Suppose that a is close to 1. Comparing T\M,U with T)M indicates which features of the the estimator f)M need not be present in η once allowance is made sampling error. This addresses question (a) above. On the other hand, comparing Ϊ)M,S with Ύ]M indicates which features of η may have been smoothed out by the estimator T)M because of sampling error. This addresses question (b) above. The strategy just described for identifying interesting extremal members of confidence sets centered at f)M extends readily to confidence sets centered at ήNS In the preceding discussion, the word "smooth" tacitly assumes that the column vectors in UE are are of decreasing smoothness as column index increases. This is the case for the bases mentioned in Section 2.4. More generally, "smooth" should be be replaced by a description of the key feature that is increasingly captured as we move through the basis.

3.2

Further analysis of the case studies

The second row of Figure 1 exhibits the most unsaturated NS and M fits that lie within the 95% confidence balls for η centered, respectively, at

84

Rudolf Beran

Best M Fit using Polynomial Contrasts

Best NS Fit using Polynomial Contrasts °

0

•

•

.

•

•

"

en

3 •5

8 •

1

3

Z

\

^^^' " \

I

E 0

1

.

10

20

30

40

1 8z

^

^

\ •••

•••••• \

1

.

10

50

20

30

40

Row

Row

Most Unsaturated 95% Confident NS Fit

Most Unsaturated 95% Confident M Fit

a

3

I

3

50

Ml

z

10

3

0

/ /

20

30

40

§

50

10

20

30

40

Row

Row

Most Saturated 95% Confident NS Fit

Most Saturated 95% Confident M Fit

50

σ>

8

Φ -Q

I

3

10

20

30

Row

40

50

0

10

20

30

Row

Figure 3. REACT fits and extremal confident fits to the vineyard harvest data.

40

50

Extremal REACT Fits

85

and f]M- That both extreme unsaturated fits retain the middle-aged dip in income makes it safe to conclude that mean log(income) η has this feature. Possible reasons for the dip include mid-life changes in career, re-entry of women into the work-force after child-rearing, down-sizing of middlemanagement positions, and so forth. The third row in Figure 1 exhibits the most saturated NS and M fits that lie within the respective 95% confidence balls for η. These extreme saturated fits both point to two features that Ύ]M and ή^s might have smoothed out: a small dip in mean log(income) around age 30 and greater variability in mean log(income) at ages 60 to 65. Such informed conjectures rest on the hypothesis of homoscedastic errors. Heteroscedastic errors would offer another possible explanation of the variation in observed log(income) between ages 60 to 65. The (2,2) cell of Figure 2 compares the REACT shrinkage vector fM (solid) line with the most unsaturated shrinkage vector }M,U (dotted line) and the most saturated shrinkage vector /M, S (dashed line). The (2,1) cell of the same figure makes the analogous comparisons of shrinkage vectors for the various NS fits. The second and third rows of Figure 3 present the most unsaturated and most saturated M and NS fits to the vineyard data. We conclude from the most unsaturated fits that the "valley" in mean harvest around row number 35 cannot be discounted. The most saturated fits indicate additional possible local patterns in how mean harvest depends on row number. More polynomial contrasts are needed for the analysis of the vineyard data than for the earnings data (see the middle row in Figure 4). 4

Technical Matters

The methodology described in the preceding two sections has a firm foundation in asymptotic theory and in computational algorithms for isotonic weighted least squares. This section outlines the most salient points. 4.1

Asymptotics for confidence sets

Underpinning the two confidence sets described in Section 3 are the following theorems that determine the asymptotic distribution of t ^ and establish the asymptotic coverage probability and asymptotic loss of C^^α). Let d be any metric for weak convergence of probability measures on the real line and let C(tp7i) denote the distribution of ifj under the model. Theorem 4.1 Suppose that T is either TNS or TM For i = 1, assume 2 2 that σ = σ\s, m = min(p,n — p), and lim m H > o o p/(n - p) = η < oo. For 2 i = 2, assume that σ = σ%, m = min(p,p-ς), limm_>oop/(p--g) = β2 < oo,

Rudolf Beran

86

First 5 Polynomial Contrasts

Vineyard z for Polynomial Contrasts in

(Γ

oτ .

CM

~

d

m o in

•

•

c

(7)

O

Iue

N

. •

•

.

:.... ....-^.:..

•

10

20

30

40

ct >

q d

•

0

50

10

20

30

40

50

Component

Observation

Shrinkage Factors for the NS Fits

Shrinkage Factors for the M Fits

00

d

C/)

I

p d )

10

20

30

40

50

10

20

Component

30

esidual

Residuals after Best M Fit CD

-

CM

-

•

•

• • >

•

cr.

1

-2

.

-4 10

20

30 Row

50

Component

Residuals after Best NS Fit

0

40

40

50

0

10

"• " a

•

20

30 Row

Figure 4. Diagnostic plots for REACT fits to the vineyard harvest data.

• •

40

50

87

Extremal REACT Fits

and limm-toop-1/2 ΣLς+i $ = ° Then, for every r > 0 and every σ2 > 0, (4.1)

sup

JSJO^

2

d [ £ f c ) , Λ Γ ( 0 , r ^ ) ] = 0,

where 2σ 4 ave[(2/ - I) 2 ] + 2[p/(n -p)]σ 4 [ave(2/ - I ) ] 2 f4 2)

2

2

2

+ 4σ ave[£ (l-/) ]

and τ£ > 2 = 2σ4ave(Λ2) + 4σ2ave(Λ2.£2)

(4.3) with (4 4)

Λ! = 2/ - 1 + b/(P - 9)][ave(l - 2/)](l - e(q)) h ~fl + \/(- g )][ave(l -

The variance r^ ^ depends on p, ξ 2 , σ 2 , and on n — p or p — q according to i. The estimators r ^ defined in (3.2) and (3.5) substitute z2 — σ2 for ξ 2 , / for /, and σ 2 for σ 2 , constraining the estimator of the last term on the right of (4.2) and (4.3) to be non-negative and using the appropriate definition of σ 2 . The next theorem establishes that the α-th quantile of the 7V(0,rj^) distribution is consistently estimated by τj?jΦ~ι(α). This leads to the definitions of the confidence sets Cjr^α) in (3.3) and (3.6). Let r2:Fi — p(f) + p~1/2f> j iΦ~ 1 (α). Of interest are two properties of Cjr^α): the coverage probability P(η E Cjr^α)) and the geometrical quadratic loss (4.5)

L(CTti(α),η)

=

sup

p~ι\θ - η\2 = [ p " 1 / 2 | ^ - η\ + f ^ ] 2 .

Treating Cj:^{α) as a set-valued estimator of 77, this geometrical loss measures how poorly elements of the confidence set can estimate η. Theorem 4.2 every σ2 > 0,

Under the hypotheses of Theorem 4.1, for every r > 0 and

lim m->oo, K^oo

sup ave

(4.6)

(ξ2)<σ2Γ

lim

P[\HCrti(α),η) sup

m->oo, K-+00 a V e(ξ2)<

For every e > 0, (4.7)

2

2

ι 2

- 4p(/,ξ ,σ )| > Kp~ l ] = 0

88

Rudolf Beran

Moreover, (4.8) V

liminf

inf 2

'

r ^ >0 2

m-+oo a ve(ξ )<σ r

'

and (4.9)

lim m

"

>

sup o

o

(

2

|P(τ; E CTi(α))

- α\ = 0.

) 2

Theorems 3.1 and 3.2 in Beran and Dumbgen (1998) imply the two theorems above for the case i = 1. The results for i = 2 are proved bystraightforward modification of the argument, the salient difference being that σ\ - σ2 = \p/(p - g)]{ave[δ(g) Wi] + 2ave[e(g)W2]

(4.10)

with e{q) = 1 - e(g), Wi = (z - ξ)2 - σ2 and W2 = ξ{z - ξ). According to Theorem 4.2, the asymptotic geometrical loss of each confidence set is four times the asymptotic risk of the estimator at the center of the confidence. This is a compelling reason for using confidence sets centered at superefficient REACT estimators in place of the classical confidence set centered at the least squares estimator. 4.2

Computing saturated and unsaturated fits

Computation of fws &nd of the unsaturated and saturated nested selection shrinkage vectors fNS,u(k) and fNS,s(k) is accomplished by finite search. To compute / M , let g = (z2 — σ2)/z2 and observe that (4.11)

fM = argminp(/) = argminave[(/ -

g)2z2].

Let B = {b G Rp: b\ > 62 > . . . > bp}. A further argument given in Beran and Dumbgen (1998) shows that (4.12)

fM = /+

with

/ = argminave[(6 -

g)2z2}.

beB

Algorithms for isotonic weighted least squares yield /. When / is unsaturated down to order A;,

(4.13)

p(f) = p- 1 Σ

W + k 2 " * 2 )(1 - h)2] +P'1 Σ ( ^ ? - * 2 )

»=1

i=k

On the other hand, when / is saturated up to order fe,

(4.14)

p(f) = p-'kσ2 +P- 1 J2 [σ2ff + {zf - σ2)(l - 0). i=k+l

Thus, as in the preceding paragraph, algorithms for isotonic weighted least squares suffice to compute fM,u(k) and /M,s(fc)

Extremal REACT Fits

89

Acknowledgements Professor J. S. Marron introduced the author to the Canadian earnings data. The stimulus provided by participants in a Weekend Seminar at Oberflockenbach in May 1998 is gratefully noted.

REFERENCES Beran, R., (1996). Confidence sets centered at Cp estimators. Annals of the Institute of Statistical Mathematics 48, 1-15. Beran, R., (2000). REACT scatterplot smoothers: supereίϊiciency through basis economy. Journal of the American Statistical Society 95, in press. Beran, R., and Dϋmbgen, L., (1998). Modulation of estimators and confidence sets. Annals of Statistics 26, 1826-1856. Buja, A., Hastie, T., and Tibshirani, R., (1989). Linear smoothers and additive models (with discussion). Annals of Statistics 17, 453-555. Chu, C.-K., and Marron, J.S., (1991). Choosing a kernel regression estimator. Statistical Science 6, 404-436. Donoho, D.L., and Johnstone, I.M., (1994). Ideal spatial adaptation by wavelet shrinkage. Biometrika 81, 425-455. Donoho, D.L., and Johnstone, I.M., (1995). Adapting to unknown smoothness via wavelet shrinkage. Journal of the American Statistical Association 90, 1200-1224. James, W., and Stein, C , (1961). Estimation with quadratic loss. Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability 1 (J. Neyman, ed.), 361-380. University of California Press, Berkeley. Kneip, A., (1994). Ordered linear smoothers. Annals of Statistics 22, 835866. Li, K.-C, (1987). Asymptotic optimality for C p , CL and generalized crossvalidation: discrete index set. Annals of Statistics 15, 958-976. Mallows, C.L., (1973). Some comments on Cp. Technometrics 15, 661-676. Pinsker, M.S., (1980). Optimal filtration of square-integrable signals in Gaussian noise. Problems of Information Transmission 16, 120-133. Simonoff, J.S., (1996). Smoothing Methods in Statistics. Springer, New York. Stein, C , (1956). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability 1 (J. Neyman, ed.), 197-206. University of California Press, Berkeley.

90

Rudolf Beran

Stein, C , (1966). An approach to the recovery of inter-block information in balanced incomplete block designs. Research Papers in Statistics: Festschrift for Jerzy Neyman (F.N. David, ed.), 351-366. Wiley, London. Stein, C , (1981). Estimation of the mean of a multivariate normal distribution. Annals of Statistics 9, 1135-1151. Ullah, A., (1985). Specification analysis of econometric models. Journal of Quantitative Economics 2, 187-209. DEPARTMENT OF STATISTICS UNIVERSITY OF CALIFORNIA, BERKELEY BERKELEY, CA 94720-3860

USA beran@stat. berkeley. edu

THE BOOTSTRAP IN HYPOTHESIS TESTING

1

PETER J. BICKEL AND JIAN-JIAN REN

2

University of California at Berkeley and Tulane University We propose a unifying principle which identifies a very broad class of hypotheses and statistics for which a suitable application of the n out of n bootstrap yields asymptotically correct critical values and power for contiguous alternatives. We also show that this attractive principle can fail in situations which the m out of n bootstrap can deal with (Bickel, Gδtze and van Zwet, 1997)(BGvZ). We formalize the m out of n bootstrap theory for testing and show that under mild conditions, it provides correct significance level, asymptotic power under contiguous alternatives, and consistency. We conclude with simulation results supporting the asymptotics. AMS subject classifications: 62G09, 62G10.

Keywords and phrases: Censored data, change points, empirical processes, goodness of fit, Hadamard differentiability, manifolds, U statistics.

1

Introduction

It is logically clear but not always noted that the usual nonparametric bootstrap (the n out of n bootstrap) fails in setting critical values for test statistics in hypothesis testing. The problem is that hypothesis restrictions are not reflected adequately by the empirical distribution when one is resampling as many observations as one has in the sample. For example, Preedman (1981) points out that in setting confidence intervals for the usual slope estimate for regression through the origin, one must resample not the residuals but the residuals centered at their mean. If one considers setting confidence bands as the dual of hypothesis testing, a moment's thought will show that not centering the residuals is tantamount to not imposing the model requirement for the hypothesis tests that the expectation of the error is 0. For more recent examples, see Hardle and Mammen (1993), Mammen (1992) and Bickel, Gδtze and van Zwet (1997) (BGvZ). BGvZ note that the m out of n bootstrap, m —> oo, ™ —> 0, is in principle usable. In particular, Bickel and Ren (1996) study the following situation: testing for goodness of fit with doubly censored data where the usual bootstrap as usual fails and finding a distribution approximating the truth under Ho is difficult. They propose using the m out of n bootstrap to set the critical value of the test Research partially supported by NSF Grant DMS 9504955. Research partially supported by NSF Grants DMS 9510376 and 9626535/9796229.

2

92

Peter J. Bickel and Jian-Jian Ren

and show that the proposed testing procedure is asymptotically consistent and has correct power against \/n-alternatives. There have been a number of papers in the literature detailing modifications of the bootstrap for correct use in testing, see Beran (1986), Beran and Millar (1987), Hinkley (1987, 1988), Romano (1988, 1989), among others. In particular, Hinkley indicated quite generally that bootstrapping from a distribution obeying the constraints of the hypothesis which is closest in some metric to the empirical distribution should give asymptotically correct critical values. Unfortunately, this requires an exercise in ingenuity in most cases, and as has been frequently noted, say, Shao and Tu (1995) for example, that it may in practice be very difficult to construct such a distribution. Romano showed that in an interesting class of situations, including testing goodness of fit to parametric composite hypothesis and independence, there was a natural definition of a distribution in the null hypothesis HQ closest to the empirical, and that, for natural test statistics, bootstrapping from this distribution would yield asymptotically appropriate critical values. In a prescient paper, Beran (1986) gave two general principles for construction of tests of abstract hypotheses in the presence of abstract nuisance parameters and estimation of the power functions of such tests. In Section 2, we propose a unifying principle which identifies a very broad class of hypotheses and statistics including all those considered by Romano (1988) for which a suitable application of the n out of n bootstrap yields asymptotically correct critical values, power for contiguous alternatives, and consistency under mild conditions. We state a general theorem and apply it in eight examples including all those of Romano, those of Bickel and Ren (1996), a test for change-point (Matthews, Farewell and Pyke, 1985) with censored data, and a number of others. This result, Theorem 2.1, applies only to test statistics which are regular in the sense of stabilizing on the n" 1 / 2 scale under the hypothesis. We then in Theorem 2.2 extend Theorem 2.1 to a broader class of statistics based on estimates of irregular parameters such as densities. Moreover, we show that our proposed unifying principle can fail in situations which the m out of n bootstrap can deal with. Our unifying principle, though not our point of view, can be viewed as a particular case of one of Beran's two approaches, even as Hinkley's work corresponds to the other. However, that part of Beran's formulation which is relevant to the principle we state emphasized construction of tests from confidence region for abstract parameters in the presence of nuisance parameters rather than the setting of critical values for natural test statistics. Perhaps for this reason, the abstract point of view which obscured the rather simple geometrically based special case we focus on and the general conditions whose checking is usually the heart of the matter, the broad applicability of his argument was not appreciated (even by us until a referee

The Bootstrap in Hypothesis Testing

93

brought his paper to our attention). We focus here on checkable conditions and examples. In Section 3, we state and prove a theorem showing that the m out of n bootstrap is an approach that generally provides correct significance level, asymptotic power under contiguous alternatives, and consistency. This is essentially a formalization of the discussion of BGvZ. We close with simulations and a brief appendix indicating where the regularity conditions for the examples can be found.

2

A general approach to defining semiparametric hypotheses

For simplicity, we start this section with the case where the data Λ Ί , . . . , Xn are independently and identically distributed (i.i.d.) taking values in A', usually Rk, with an unknown distribution function (d.f.) F E f . However, it should be apparent from our discussion that our approach is more generally applicable. Suppose that we want to test (2.1)

Ho:FeTovs.

HuFφFo.

We begin with considering the case that X takes on k+1 values #o? #i? only. Thus T is parametrized by

eiP=lpeRk',pj>o,

P

5

#fc

i<j
A hypothesis To is then described by, say, {h(θ) E JP; θ E Rq, q < &}, where θ -> h(θ) is 1-1. If h(θ) is continuously differentiate and (dhi(θ)/dθj)kxq is of full rank, then h is an embedding of Rq in Rk (Vaisman, 1984, page 11, 13 and 15). This means that for any p0 E To = h(Rq) Π JP, there exist open sets Upo and UQ in Rk and a differentiate function ηQ : UQ -¥ Rk~q, such that pQ e Upo and UPo Π T* = {p E Uo \ ηo{p) = 0}. The map ("atlas") η0 can in many cases of interest be pieced together consistently to a single η0 such that (2.2) To = {P € P I 77o(p) = 0}. 3 Thus, if the random sample X i , . . . , X n is from some p E P, which is in a neighborhood Upo of some p0 E To, and if ή = ηo(P), where p = (ΛΓi/n, Λ^/n,... ,Nk/n)τ is the empirical distribution (vector of frequencies) of XL, . . . ,X n , typical tests are of the form (or asymptotically equivalent to tests of the form): Reject if τ{y/nή) is large, where the function 3

Even if the "atlas" can not be reduced to a single function, we still can naturally base a test on ηo(p) for p as above and p 0 as the member of To closest to p.

94

Peter J. Bickel and Jian-Jian Ren

τ : Rk~q -> R+ is continuous with τ(t) = 0, iff t = 0. Typically, τ is equivalent to a norm on Rk~q. For instance, the usual Wald test is to use nηTTi~ιη^ where Σ is an estimate of the covariance matrix Σ(p) of r). This is equivalent to using τ(x) = X ΣQ X. In this situation, we can bootstrap parametrically in one of two ways: T

1

(a) Estimate θ by θo, the maximum likelihood estimator (MLE) under Ho : ηo = 0, then use the appropriate percentile of the distribution of τ{y/nff) as the critical value, where X*,... ,X* are i.i.d. p(0o); (b) Note that y/n(ή — r/0) and y/nή have the same distributions under HQ and use the appropriate percentile of the distribution of τ(\/n(ή* — ή)) as the critical value, but where now X*,..., X* may be obtained from the 'nonparametric' bootstrap, i.e., i.i.d. p. If ΘQ is uniformly consistent on Θ, it follows from, for example, a theorem of Rao (1973, page 360-362) that these bootstraps are both valid. (Note that p(0o) can be used instead of p in case (b).) If X does not have finite support, the corresponding conditions for characterization of an embedding in Hubert space are more involved. Nonetheless, as we shall see by example below, the equivalence (2.2) holds quite broadly. Sufficient conditions for use of bootstrap (b) are easily given. Suppose that for hypothesis (2.1), there exists Γ : T —> T, where T is a Banach space, possibly Rp but often a function space such as D[RP], such that (2.3)

To = {F; T(F) = 0}.

It is often convenient to think of both T and T as subsets of spaces of finite signed measures defined on spaces of bounded functions Hx^Hy on X and another space y and identify F as a member of loo{Ήχ),T(F) = G as a member of loo(Hy) via,

(2.4)

F(h) = Jh(x)dF(x), G(r) = Jr(y)dG(y).

We shall throughout assume that measurability technicalities are dealt with by the Hoffman-J0rgensen approach — see van der Vaart and Wellner (1996). Let Fn denote the empirical distribution of A Ί , . . . , Xn and r :T -> R+ be continuous with τ(t) = 0 iff t = 0. Tests for (2.1) are naturally based on rejecting Ho for large τ(y/nT{Fn)) (provided that y/n(T(Fn) - T(F)) is well behaved). In analogy to the multinomial situation, it seems natural to use either

The Bootstrap in Hypothesis Testing

95

(a) The quantiles of the distribution of τ(y/nT(F*)), where X*,... ,X* are i.i.d. from Fo G To, which is a 'uniformly consistent' estimate of F under HQ; or (b) The quantiles of the nonparametric n out of n bootstrap distribution of τ(y/fi(T(F*) - T(Fn))), where F* is the empirical distribution of X*,..., Jf *, a sample from Fn; as critical values for τ(y/nT{Fn)). In the framework of Beran (1986), this can be viewed as using the test: Accept iff 0 G C(F n ), where C(Fn) is the asymptotic 1 — α confidence region {ί; r(y/n(T(Fn) — ί)) < dn(Fn)} and dn{F) is the 1 - α quantile of the distribution of τ(y/n(T(Fn) - T(F))). We shall give sufficient conditions for the validity of alternative (b) in this abstract framework below, but before doing so we give some examples where (2.3) applies. Example 1 Goodness of fit to α single hypothesis. Here To = {Fo}, T is all distributions and we can clearly take T = loo(X), finite signed measures on X andT(F) = F-Fo. Possible r's are τ(μ) = ||/i||oo? where |H|oo = sup{|μ(/ι)| : h e Ux) for X suitable Ux. For example, X = R, Ux = {l(_oo,ί)5 * G R} gives the Kolmogorov-Smirnov test. Another possibility is weighted averages of μ2(h) over Wx. Thus, τ(μ) = f(μ(—oo,x))2dFo(x) leads to the Cramervon Mises test. Option (b) corresponds to using the bootstrap distribution of r(^/n(F* — Fn)), while (a) leads to simulating from FQ. D Example 2 Goodness of fit to α composite hypothesis. Here JF0 = {FQ; θ G

Θ}, θ G Rd, say, and To is a regular parametric model. Suppose that θ(Fn) G Θ is a regular estimate of θ in the sense of Bickel, Klaassen, Ritov and Wellner (1993) (BKRW) where θ : T -* Θ is a parameter. For instance Θ(F) = argmin||F — Fβlloo may be a possibility. Again we can take T C ioo(Wx) and T(F) = F - FΘ{F).

Note that we could take Θ(F) as any parameter defined on T such that Θ(FΘ) = θ.

This example figures prominently in Romano (1988). There he considered To describable by To = {F; F = j(F)} and recommended scheme (a), resampling from 7(F n ) for statistic \\Fn — 7(^)11 a n d Ί ( F ) = ^0(F) O U Γ scheme simply rewrites F = j(F) as F - *y(F) = 0. However we prescribe bootstrapping from the empirical for statistic y/n\\Fn - 7(^n) -F + That is we use the bootstrap distribution of ^ll^t (K) Example 3 Tests of location. Suppose X = Rq and T is a location parameter (2.5)

T(F( - θo)) = T{F) + ΘQ

96

Peter J. Bickel and Jian-Jian Ren

for all 0o e Rq Let JF0 = {F; T(F) = 0}. Thus if T(F) = fxdF, this is the hypothesis that the population mean of F is 0. If T(F) = F~ι (g), then this is the hypothesis that the population median is 0. We can similarly consider trimmed means etc. In fact our discussion applies to scale parameters and more generally transformation parameters — see BKRW (1993) — but we do not pursue this. In this case our prescription is to use the bootstrap distribution of y/n(T(F*) — T(F ? )). Here prescriptions (a) and (b) coincide since the distribution of y/n{T{F*) - T(Fn)) under Ho is by (2.5) the same as that of y/nT(F*) where F* is the empirical distribution of a sample from Fn(- — T(Fn)). Equivalently say in the case of the mean the bootstrap distribution of X* — Xn is the same as the distribution of Xn, the mean of a resample from the residuals X\ — X,..., Xn — X. The latter (a) form is the prescription of Preedman (1981) and Romano (1988), and the special case of the mean is Example 2 of Beran (1986). α We now turn to some simple results. Suppose T is a subset of a space of finite signed measures with T viewed as a subset of the Banach space loo(Ήy) as above. Suppose T is extendable to T and, (Al) T is Hadamard differentiate at all Fo G To as a map from (^*, || ||oo) t° (T, || I loo) with derivative T : T —» looiUy), a continuous linear transformation, and T a closed linear space containing T. That is (2.6) sup{||T(F0 + λΔ) - T(FQ) - λΓ(F 0 )Δ||;Δ G K} = o(λ) where K is any compact subset of Zoo(7ΐx) a n d λ —> 0. (A2) y/n{Fn — FQ) => Zp0 in the sense of weak convergence for probabilities on loo(Hx) given by Hoffman-J0rgensen and P{ZF G F} = 1 for all 0

Fo G TQ.

Theorem 2.1 Under (Al) and (A2), for all Fo G T^ (2.7)

Vϊι(T(Fn) - T(F0)) =• f

(F0)ZFQ

and with probability 1, (2.8)

v^(T(Fn*) - T(Fn)) =• Γ(F o )Z F o .

Proof By Gine and Zinn (1990), (A2) implies that

(2.9)

yft(K - Fn) =ϊ ZFo

with probability 1. Now apply a standard argument. By Hadamard differentiability

(2.10)

MT(Fn)

- T(F0)) = r(F 0 )Vn(F n - Fo) + op(l)

The Bootstrap in Hypothesis Testing

(2.11)

v^(T(Fn*) - Γ(Fo)) = t(F0)Vϊι(K

97

- Fo) + op(l)

(2.10) yields (2.7) and subtracting (2.10) from (2.11) yields (2.8). • Now letting Co be the distribution of corollary.

T(T(FQ)ZFΌ),

we have the following

Corollary 2.1 Under the assumptions of Theorem 2.1, if Co is continuous, and respectively, Ca and C% are the (1 — a)-quantiles of τ(y/n[T(F*) — T(Fn)]) and Co, then as n -> oo, (2.12)

P{τ(VET(Fn))

> C W I HΌ} -> a.

In fact, as n —>• oo,

(2.13)

P{[τ(VET(Fn))

> C*a^]A[τ(V^T(Fn)) > C°a] \ HQ} -> 0.

If {Fn} is a sequence of alternatives contiguous to Fo E T$, then (2.13) continues to hold with P replaced by Pn corresponding to Fn, and hence the power functions for the tests using Ca and Ca are the same. If (Al) and (A2) hold for all F e T not just T§ and τ(t) -> oo, as ||*||oo —>• oo, then the test based on Ca

is consistent for all F £ T§.

Proof (2.12) and (2.13) follow from C* ( r ι ) 4 c £ 0 ) for all Fo G JF0, an immediate consequence of the theorem and Polya's theorem. Contiguity preserves convergence in probability to constants so that equivalence of the power functions follows. Finally consistency follows since under the assumption Ca converges in probability under F to the (1 — α)-quantile of CF(T(T(F)ZF)). But,

4 oo since the first term in the norm is tight while the second term has norm of the order y/n since T(F) ^ 0 . • The examples 1-3 cited above all satisfy our assumptions essentially under the mild regularity conditions needed to justify that the test statistics in question have a limit law under Ho. We discuss the conditions briefly in the appendix. Now we turn to some further examples and a mild extension. Our next example falls outside of the Romano domain. Example 4 Goodness of fit test of a lifetime distribution under censoring. Suppose that for a desired observation T^, there are right censoring variable Ci and left censoring variable B{ such that T{ is independent from (B^Ci) and that the available observations are in the form X{ = (Yί,£i), where in

98

Peter J. Bickel and Jian-Jian Ren

the right censored sample case, we have Y{ = min{Ti,Ci},δi = I{T{ < C^}, and in the doubly censored sample case (Turnbull, 1974), we have Y{ =

max{mm{TuCi},Bi},δi

= I{B{ < T{ < d} + 2I{Ti > d} + 3/{Ti < J3;}

with P{B{ < Ci} — 1. Let G be the distribution function of T;, then in this frame work the goodness of fit test HQ: G = Go, for a given Go, is important. We write F as the distribution of X = (Y,δ). Then if G is identifiable, we have G = φ{F) with Gn = φ(Fn) to be the nonparametric maximum likelihood estimate (NPMLE) for G (see Bickel and Ren, 1996). Thus, we can take T(F) = φ(F) - G o = G - G o . Although Γ( ) is not Hadamard differentiate here, prescription (b) says to use the bootstrap distribution of r(y/n(G^ - Gn)). As Bickel and Ren (1996) point out, it is difficult to fulfill prescription (a) in this case for doubly censored data since it is not clear what to use as the member of T§ from which we should resample. We will return to this example subsequently in Section 4. α Example 5 U statistics. A natural generalization of Example 3 is testing Ho : T(F) = 0 where T(F) = EFφ{Xu... ,X fc ), k > 1. The statistic we would be led to is the V statistic

Typically, however, one considers the equivalent (2.15)

un =

71 i

where T(F,s) = Π t i ' Γ i i j

/••• / Xl<

...

dF x

Ψ{xu...,xt)ΠU ( i'>

B!aA

<Xk

T(F, 0) = T(F) if F is continuous. This example is not quite covered by our theory on two grounds. (i) F -> T(F) is not Hadamard differentiate with respect to any of the usual metrics unless φ is of bounded variation. (ii) T(Fn) is not the statistic Un one wants to consider. Both are covered by noting that all we need to do for (ii) is to replace T(F) by T(F,s), 0 < 5 < 1 and T by T x [0,1], following a suggestion of Reeds (1976). For (i) we note that (2.7) and (2.8) can be established directly for such statistics, (Arcones and Gine, 1993). So again, bootstrapping n \T (P*, l/nj

— T ί F n , 1/nJ j gives the correct answer.

The Bootstrap in Hypothesis Testing

99

Another interesting possibility suggested by this example is f(F) = 0 in which case the limit law CQ is point mass at 0. We need to renormalize. It is easy to see (Bretagnolle, 1981) that in this case, (2.7) holds with n replacing yjΰ and a suitable limit, but (2.8) fails. It is possible to bring this example also into our framework obtaining a solution proposed by Arcones and Gine (1993), but the hypothesis implicitly tested, Ho : T(F) = 0, t{F) = 0 is somewhat artificial. D Next is a complex example illustrating the broad applicability of our approach to semiparametric hypotheses. Example 6 Test of change-point (Matthews, Farewell and Pyke, 1985). Consider a parametric problem where F has the following hazard rate function: λ

'

ifθθ

where 0 < ξ < 1 and λ > 0 are unknown, and θ > 0 is the unknown changepoint parameter for the hazard rate which changes from λ to (1 — £)λ at time θ. If θ is confined to a finite interval [#i, #2], the following test statistic was proposed on maximum likelihood grounds by Matthews, Farewell and Pyke (1985) for the irregular hypothesis Ho : ξ = 0 vs. H1 : ξ φ 0 (2.17)

Tn=

sup \Zn(θ)\

Θ1<θ<θ2

where λn = \(Fn) = [JxdFn(x))

and

(2.18) Zn(θ) = (1- e-^Θ)-ιl2{ne^θ)1'2

Γ((x - θ)\n - 1) dFn(x). Jθ

Under Ho we have F(x) = F\(x) = 1 - e~ λ x , thus by integration by parts, Zn can be expressed as

(2.19)Zn(0) = -(l-e-^y^e^^hn

1^ U(x,Fn)dx -

where U(x, Fn) = Fn{x)-Fχ{Pn)(x). The limiting distribution of Tn in (2.19) is studied by Matthews, Farewell and Pyke (1985). Now suppose that we have right censored data or doubly censored data as in Example 4. Then the analogous test statistic is obtained by making Fn in U( ,Fn) and Xn = λ(Fn) be the NPMLE (the Kaplan-Meier estimate for right censored data) of F for doubly censored data. We modify Tn slightly

100

Peter J. Bickel and Jian-Jian Ren

to avoid the usual technicalities in censoring replacing Zn by Zff given by

,n f

U(x,Fn)dx-U{θ,Fn)\

Jθ

J

(2.20) where M > 02- Under HQ and some regularity conditions, λ n converges to λ in probability and y/nU(',Fn) converges weakly to a centered Gaussian process G\ on [0χ,M]. Therefore we know that under HQ, Z^(θ) weakly converges to (2.21)

and (2.22)

ZM(Θ) = - ( 1 - e-λ*)-i/2eλ*/2 J λ /

=

sup \Z^f(i9)| -^ sup θι<θ<θ2

Write Z™(•) = y/nT{Fn),

θι<θ<θ2

Gx{x)dx - Gχ(θ) S

\z(θ)\, M

as n --> 00.

where

I

Λ

(2.23) To obtain critical values for T n , again we simply need to use the bootstrap distribution of sup{y/n\T(F*)(θ) - T(F n )(0)|; 0i < 0 < θ2}. • Our next example illustrates another extension of the paradigm beyond Theorem 2.1. Example 7 Goodness of fit test using kernel density estimates. Consider the problem of Bickel and Rosenblatt (1973). We have observations on X = R and wish to test HQ : F = FQ where F' = f exists and, in fact, H/Ίloo < Mo < oo for all F G f , and inf{/(x); \x\ < M} > e > 0 for some constant M > 0. A natural test statistic is sup{|/n(x) — EQfn(x)\; \x\ < M}, where

(2-24)

fn(x) =

Jκhn(x-y)dFn(y)

and Kh{y) = h~1K(y/h), where K is at least twice differentiate, has compact support, is symmetric about 0, J^°oQK(y)dy = 1, and hn = n~^, 0 < β < 1 for some β. This test is consistent against F φ FQ concentrating on \—M,M\ and the natural Tn(F) = Tn(ΊF) here is

=f

-y)d(F(y)

-F 0 (y)).

The Bootstrap in Hypothesis Testing Of course, y/nTn(Fn) (2.25)

101

diverges, but ^hn(Tn(xJn)-Tn(x,F))^λί(0,σ2(x,F)),

Un(x) =

where σ2(x,F) = f(x)fK2(y)dy, and for x φ y, Un(x) and Un(y) are asymptotically independent. If our prescription is valid, the distribution of the test statistic (2.26) Tn = sup{\Un(x)\;\x\<M} should be approximable by the bootstrap distribution of

(2.27)

f:

/

^

J

J

This is valid, under the conditions of Bickel and Rosenblatt4 (1973), by applying the strong approximation to the empirical process and extreme value theory used in Bickel and Rosenblatt (1973) to satisfy the conditions of Theorem 2.2 below. Suppose T n (F n ), as in Theorem 2.1, are such that (Al'): (2.28)

||Γ n (F n ) - Tn(F) - fn(F)(Fn

- F ) | U = op{n~ιl2)

for all F G f o Suppose further that X is such that a strong approximation theorem of the following form applies: (A2'): There exists a probability space on which we can construct /oo(^χ) valued random elements y/n(Fn—F) having the same distribution as y/n(Fn— F) and also loo(Ήχ) valued Gaussian random elements Zp(') with mean 0 and the same covaxiance structure as y/n(Fn — F) for which both

(2.29)

\\y/ϊiφn - F)( ) -

and

(2.30)

||Vn(F n - F n )( ) -

For such results see Csόrgό and Revesz (1983), Massart (1989), and Einmahl (1989). Theorem 2.2 Under (Al') and (A2') suppose that (2.31) 4

\\Tn{F)\\o0 = 0[rC)

Bickel and Rosenblatt (1973) did not use the Komlos-Maior-Tusnady (1976) strong approximation, so their results and the bootstrap extension are weaker than they need be, which is why the optimal bandwidth hn = rf1^ is used in our simulation studies in Section 4.

102

Peter J. Bickel and Jian-Jian Ren

and T is a seminorm, τ{ct) = cr(t) for c>0, and r is subadditive. Suppose also that there exist {an}, {bn} scalar possibly depending on F such that an T{tn{F)n-l'2ZF)

(2.32) and an = o(nιl2).

+

bn^CF

Then, anτ{Tn(Fn))+bn^CF

and in probability an τ{Tn{F*) - Tn{Fn)) + bn =» £F. Proof By our previous argument and (Al') (2.33)

Tn(Fn*) - Tn(Fn) = T n (F)(F* - Fn) + o^n

and the corresponding statement holds for Tn(Fn) — Tn(F). Under (A27) and since r is a seminorm, anτ(Tn(F)(Fn-F))

+ bn

= =

anτ(Tn(F)n-V2ZF) +&n + O p ( α n | | T n ( F ) | | 0 0 n an τ(fn(F)ZFn-^2) +bn + o p (l) =• CF.

The last identity uses (2.31) and an = o(nιl2). anτ(Tn(Fn))

+ bn

=

But, under ifo,

anτ(Tn(Fn)-Tn(F))+bn

so that an r(Tn(Fn))+bn =*• Cp The same argument applies to an τ(Tn(F*) — Tn(Fn)) + bn and the theorem follows. • We close this section with an old example in which although our formalism applies, the conditions (Al) or (Al') of our theorems fail and our solution is incorrect. Example 8 Test of distribution support Suppose T — {F; F has support on [0,6] with unknown 6, continuous density / and f(b—) > 0}. Then, as is well known, if X^ < . . . < X{n) are the ordered -XVs, n(b — X(n)) has a limiting distribution (f(b—))~1Exp(l), where Exp(μ) denotes the exponential distribution with mean μ. Thus the natural test statistic for HQ : b = bo is Tn = n(X ( n ) - 6Q) If we let T(F) = F " 1 ^ ) - b 0 , we have put the hypothesis in our framework and have noted that under i ? -Tn = -nT(Fn) =

The Bootstrap in Hypothesis Testing

103

However, the bootstrap distribution of n(T(F*) —T(Fn)) does not converge as was already noted by Bickel and Preedman (1981) — see also BGvZ. Although Putter and van Zwet (1996) gave a method for repairing bootstrap inconsistency for a similar case in their Example 3.2, there is a much more general solution for this problem discussed in BGvZ, which we recapitulate and discuss briefly in the next section. D

3

The m out of n bootstrap hypothesis tests

This method, presented generally in Bickel, Gόtze and van Zwet (1997) (BGvZ) and in an alternative form by Politis and Romano (1996), is based on the assumption that under HQ : F G To, the test statistic, Tn — Tn(Fn) is such that (3.1) Tn =ϊ CF which is nondegenerate. The m out of n bootstrap prescribes that the appropriate quantile of the bootstrap distribution of T m ( F ^ ) be used, that is of the distribution of the statistic based on m observations resampled from Xi,... ,X n . The history of this approach which goes back to Bickel and Preedman (1981) and Bretagnolle (1981) is partially reviewed in BGvZ. If m —> oo and ™ —> 0 the prescription succeeds in giving an asymptotically correct level under very mild conditions which we detail below. Politis and Romano (1996) argue that by resampling without replacement this conclusion holds with no conditions. Bickel and Ren (1996) checked the regularity condition for the applicability of this method in Example 4 when data are doubly censored. BGvZ shows its applicability in Example 8. Here is a formal theorem. Let Tn be as above, T^ = Tm(F^) and C* be given by (3.2)

Pn{T^ > C*α} = α.

Let n = {h : M -+ 1R; \h(x) - h(y)\ <\x-

y|, \\h\\ < 1}, and for h G U,

θm(F) = EF{h(Tm{Xi,..., where Tm{Xu ...,Xm;F) Jb Furthermore, with X^

= Tm(Xu...

X

, X m ) - μ m ( F ) and/i m (F) = 0 if F E

= (X*,..., X<)iχj,

Theorem 3.1 Let m = o[n) with m -ϊ oo, as n -> oo and let C* be given by (3.2). Assume forO
104

Peter J. Bickel and Jian-Jian Ren

(α) Tn(Xu..., Xn] F) =* WF where WF ~ CF for all F e T and CF is continuous for F G To(b) μ>n(F) —> oo, μm(F) — μn(F) —> —oo uniformly on bounded Lipschitz compacts contained in T§ if m -* oo, ™ -» 0. (c) swphen

\θm9n(F) - θm(F)\ = o(l), for all FeT.

Then, (i) l i m ^ o o P ^ > C* I Ho} = P{WF

>C% \ F

(ii) For alternatives Hn : F = Fn such that for some F o G To, {Fn} are contiguous to Fo, we have that under Hnj C* -> C°, as n -> oo and Λence the tests based on the critical values C* and C% have the same asymptotic power functions; (Hi) For a fixed alternative Hi : F = JF\ £ ^b,^{T n > C* | i ί j -> 1, as n —>> oo. Remark. Assumption (c) essentially says that Tm is not really perturbed by o(\/n) ties in its arguments — see BGvZ. Proof Theorem 2 in BGvZ shows that (c), m —>• oo, m/n -> 0 implies that,

(3.3)

4

where WF ~ £F. But bounded Lipschitz convergence is equivalent to weak convergence. Thus, noting that under Ho-, the identity T n ( X i , . . . , Xn\ F) = T n ( X i , . . . , Xn), (3.3) and Polya's theorem imply that C* 4 C% and another application of Polya's theorem yields (i). Assertion (ii) follows from the definition of contiguity. To argue for (iii), note that Theorem 2 of BGvZ implies that for all F (3.4) T^-μm(Fn)^CF. Therefore under F ^ To, (3.5)

C*a-μm(Fn)

= Op(l).

But Tn - μm(Fn) =Tn- μn(Fn) + {μn{Fn) - μm(Fn)) 4 oo. Thereto reject iff Tn > C* is equivalent to reject iff T n - μm{Fn) > C* - μm{Fn). The result follows from (3.5). • The proof shows that C* -> oo if F £ To and thus we expect that the power of this test is less than that of the tests proposed in Section 2 where

The Bootstrap in Hypothesis Testing

105

these are valid. We give some simulations to show that this is indeed the case. The question naturally presents itself: Is there a way of correcting the m out of n bootstrap to give results comparable to those we obtain by simulating the tests of Section 2? A systematic answer is given in Bickel and Sakov (1999) (in preparation) and the 1998 thesis of Sakov. We note that the m out of n bootstrap has the additional advantage of computational savings, see Bickel and Yahav (1988) for instance. In fact the computational savings can be garnered in the context of Section 2 also. Specifically it is clear that the conclusions of Theorem 2.1 continue to hold if the bootstrap distribution of τ(y/n(T(F*)— T(Fn))) is replaced in calculating the critical value by that oΐτ(y/fn(T(F^ι) — T(Fn))) as long as m -> oo. It is intuitively clear that m « n may give poor critical values. But, in practice, the effect as long as m is moderate seems small in the simulations we have conducted. Further investigation is necessary. 4

Simulations

In this section we present some simulation results exhibiting the success of the method given in Theorem 2.1 and Corollary 2.1 by a number of our examples and the inferior behavior of the m out of n bootstrap in all cases but Example 8. We give simulations for Example 3 — the median test, Example 4 — the goodness of fit test with doubly censored data, Example 7 and Example 8. In our studies, the following power curves are compared: P0(θ) = P{Tn > C£O|0},

PQ :

Pn(θ) = P{Tn > C*αW\θ},

Pn:

where α = 0.05, Tn is the test statistic, Cα is the true critical value obtained by the Monte Carlo method, Cα is the critical value based on the adjusted n out of n bootstrap as in Corollary 2.1, C* is the critical value based on the m out of n bootstrap as in Theorem 3.1, and θ is the parameter used to compute the power of the test. For each simulation run, Cα and Cα are based on 400 bootstrap samples, and Po{θ),Pn(θ),Pm(θ) are computed based on 400 random samples for each θ. (I). In Example 3, we consider test Ho : θ = 0 vs. Hi : θ > 0, where θ is the median of the distribution F from which X\,..., Xn is drawn. Figure 1 compares the power curves Po,Pn and P m , where n = 400, F is the normal distribution with mean θ and variance 25, and all power curves are the average of 500 simulation runs.

106

Peter J. Bickel and Jian-Jiaπ Ren

-0.4

-0.2

0

0.2 0.4 0.6 0.8 1 Median of Normal Distribution with Variance 25

Figure 1. Power curves of median test with complete sample of size 400.

Here m — \/n = 20 is used since for the median an Edgworth expansion to terms of order -4= is valid if the density F has a finite derivative F' and the Edgworth expansion for the m out of n bootstrap is valid for m = O(y/n) under the same conditions with the same leading term and error terms -7= and y/ψ (Sakov and Bickel, 1998). The "optimal" rate of m then balances \/f a n d ^ t o Sive m = y/n. (II). In Example 4, we consider the goodness of fit test HQ : G = Go vs. Hi : G φ GQ for doubly censored data using the Cramer-von Mises test statistic: 2

= nJ(Gn-Go) dGo. Denoting Exp(0) as the exponential distribution with mean 0, for n = 200,m = y/ή,Gv = Exp(l),C = Exp(3),£ = §C - 2.5 (which, under iί 0 , gives 55.7% uncensored, 25.2% right censored and 19.1% left censored observations), Figure 2 compares the power curves Po,Pn and P m , which are the average of 100 simulation runs. (III). In Example 7, we consider test Ho : F = Fo vs. Hi : F φ F o , with Fo = Exp(l). For test statistic Tn given by (2.26) with n = 400, hn = n~ι^,M = 3, K = U(—1,1) and θ as the mean of the exponential distribution, Figure 3 compares the power curves Po? Pn and P m , which are the average of 100 simulation runs. Here for m = y/n, the power curve P m by the m out of n bootstrap uses the critical value based on f^ =

The Bootstrap in Hypothesis Testing

0.6

0.8

107

1 1.2 Mean of Exponential Distribution

Figure 2. Power Curves of GOF Test with Doubly Censored Sample of Size 200.

m — / n |; \x\ < M}, which coincides with T* given by (2.27) if m — n. (IV). In Example 8, we consider test HQ : b = 1 vs. Hi : b > 1, with F = [7(0,6). For n = 400, m = n 1 / 3 and θ = 6, Figure 4 compares the power curves Po,Pn and P m , which are the average of 1000 simulation runs. In this case, the power function Pn{β), when the adjusted n out of n bootstrap is used, is a total breakdown under HQ. One should note that in Figure 4, the power Pn{θ) under Ho, i.e., when θ = 1, is always 0, while α = 0.05, although it seems that Pn(θ) and Po{θ) are quite close overall. Here the heuristics based on the asymptotic expansion the distribution of the maximum whose first error term is of order ^ and heuristics discussed in BGvZ suggest that an appropriate order of m = n 1 / 3 , in this case m = 7. 5

Appendix

We give brief arguments for the validity of the application of Theorem 2.1 in our examples. Example 1 Taking %x = Hy = indicators of rays for R and r corresponding to the Kolmogorov, Smirnov and Cramer - von Mises tests are covered by Corollary 2.1 as are the analogous tests when one takes Wx to be a universal Donsker class in higher dimensions (van der Vaart and Wellner (1996). Example 2 Suppose the model T is regular and θ(Fn) is a regular estimate in the sense of BKRW. Suppose also that θ : T -> Rd is Hadamard differen-

Peter J. Bickel and Jian-Jian Ren

108

0.7

0.8

0.9

1 1.1 1.2 Mean of Exponential Distribution

1.3

1.4

Figure 3. Power Curves of GOF Test Using Density with Complete Sample of Size 400.

1.002

1.004 1.006 b of Uniform Distribution (0,b)

1.008

Figure 4. Power Curves of Support Test with Complete Sample of Size 400.

1.01

109

The Bootstrap in Hypothesis Testing

tiable with respect to || H^ (in loo(Hx)) with derivative θ : T -> Rd. Then F —>- ify F ) is Hadamard diίferentiable since θ —>• i ^ is Hadamard differentiable from i? d to ^Ό by the regularity of the model and thus the composition F -> Θ(F) -» ify F ) is also. Example 3 The satisfaction of the conditions here on the sets T = {F : EF\X\2+δ < oo, ί > 0} and 7* = {F : / ' > 0} is well known. Example 4 The appropriateness of the conditions for right censored data may be obtained from Anderson, Borgen, Gill and Keiding (ABGK) (1993) and for the doubly censored case in Bickel and Ren (1996). Example 5 Appropriate references are cited in the example. Example 6 The arguments for the uncensored case is in Matthews et al (1985). The censored case modifications axe clear from the theory of the Kaplan-Meier for right censored data or the NPMLE for doubly censored data (see Bickel and Ren, 1996). E x a m p l e 7 The arguments based on Bickel and Rosenblatt (1973) are sketched in the example. Example 8 The arguments are given in BGvZ.

REFERENCES

Anderson, P. K., Borgan O., φ, Gill, R. D. and Keiding, N. (1993). Statistical Models Based on Counting Processes. Springer-Verlag. Arcones, M. and Gine, E. (1993). Limit theorems for U processes. of Probability 4, 1449-1452. Beran, R. (1986). Simulated power functions. Ann. Statist

Annals

14, 151-173.

Beran, R. and Millar, P. W. (1987). Stochastic estimation and testing. Ann. Statist 15, 1131-1154. Bickel, P. J. and Preedman, D. A. (1981). Some asymptotic theory for the bootstrap. Ann. Statist Vol. 9, 1196-1217. Bickel, P. J., Gδtze, F. and Van Zwet, W. R. (1997). Resampling fewer than n observations: gains, losses and remedies for losses. Statistica Sinica. (To appear). Bickel, P. J., Klaassen, C., Ritov, Y. and Wellner, J. (1993,1998). Efficient and Adaptive Estimation in Semiparametric Models. Johns Hopkins Press, Baltimore, Springer, New York.

110

Peter J. Bickel and Jiaπ-Jian Ren

Bickel, P. J. and Ren, J. (1996). The m out of n bootstrap and goodness of fit tests with doubly censored data. Robust Statistics, Data Analysis, and Computer Intensive Methods. Lecture Notes in Statistics. Springer-Verlag, 35-47. Bickel, P. J. and Rosenblatt, M. (1973). On some global measures of the deviations of density function estimates. An. Statist. 1,1071-1095. Bickel, P. J. and Sakov, A. (1998). On the choice of m in the m out of n bootstrap. (Preprint) Bickel, P. J. and Yahav, J. A. (1988). Richardson extrapolation and the bootstrap. J. Amer. Statist. Assoc. 83, 387-393. Bretagnolle, J. (1981). Lois limites du bootstrap de certaines fonctionelles. Ann. Inst. H. Poincare, Ser. B 19, 281-296. Csόrgδ, M. and Revesz, P. (1981). Strong Approximations in Probability and Statistics. Academic Press. New York. Freedman, D. A. (1981). Bootstrapping regression models. Ann. 12, 1218-1228.

Statist.

Galambos, J. (1978). The Asymptotic Theory of Extreme Order Statistics. John Wiley &; Sons, New York. Gehan, E. A. (1965). A generalized two-sample Wilcoxon test for doubly censored data. Biometrika 52, 650-653. Gill, R. D. (1983). Large sample behavior of the product-limit estimator on the whole line. Ann. Statist. 11, 49-58. Gine, E. and Zinn, J. (1990). Bootstrapping general empirical measures. Ann. Prob. 18, 852-869. Gu, M. G. and Zhang, C. H. (1993). Asymptotic properties of self-consistent estimators based on doubly censored data. Ann. Statist. Vol. 21, No. 2, 611-624. Hardle, W. and Mammen, E. (1993). Comparing nonparametric versus parametric regression fits. Ann. Statist. 21, 1926-1947. Hawkins, D. L., Kochar, S. and Loader, C. (1992). Testing exponentially against IDMRL distributions with unknown change point. Ann. Statist. Vol. 20, 280-290. Hinkley, D. V. (1987). Bootstrap significance tests. Proceedings of the J^Ίth Session of International Statistical Institute, Paris, 65-74. Hinkley, D. V. (1988). 321-337.

Bootstrap methods. J. R. Statist.

Soc.

B, 50,

Kaplan, E. L. and Meier, P. (1958). Nonparametric estimation from incomplete observations. J. Amer. Statist Assoc. 53, 457-481.

The Bootstrap in Hypothesis Testing

111

Komlόs, J., Maior, P. and Tusnady, K. (1976). An approximation of partial sums of independent r.v.s. and the sample df. Z. Wαhrsch. Verw. Gebiete 32, 33-58. Mammen, E. (1992). When Does Bootstrap Work? Springer-Verlag, New York. Massart, P. (1989). Strong approximations for the multivariate empirical and related processes by KMT construction. Ann. Probab. 17, 266291. Matthews, D. E., Farewell, V. T. and Pyke, R. (1985). Asymptotic scorestatistic processes and tests for constant hazard against a changepoint alternative. Ann. Statist Vol. 13, 583-591. Mykland, P. and Ren, J. (1996). Algorithms for computing the selfconsistent and maximum likelihood estimators with doubly censored data. Ann. Statist. 24, 1740-1764. Politis, D. N. and Romano, J. P. (1996). A general theory for large sample confidence regions based on subsamples under minimal assumptions. Ann. Statist 22, 2031-2050. Putter, H. and van Zwet, W.R. (1996). Resampling: consistency of substitution estimators. Ann. Statist. 24, 2297-2318. Rao, C. R. (1973). Linear Statistical Inference and its Applications. Wiley, New York.

Reeds, J. A. (1976). On the Definition of von Mises Functionals. Ph.D. Dissertation, Harvard University, Cambridge, Massachusetts. Ren, J. (1995). Generalized Cramer-von Mises tests of goodness of fit for doubly censored data. Ann. Inst Statist Math. 47, 525-549. Romano, J. P. (1988). A bootstrap revival of some nonparametric distance tests. J. Amer. Statist. Assoc. 83, 698-708. Romano, J. P. (1989). Bootstrap and randomization tests of some nonparametric hypotheses. Ann. Statist 17, 141-159. Sakov, A. (1998). Ph.D. Thesis, University of California - Berkeley. Shao, J. and Tu, D. (1995). The Jackknife and Bootstrap. Springer, New York. Shorack, G. R.and Wellner, J. A. (1986). Empirical Processes with Applications to Statistics. John Wiley & Sons, Inc. Turnbull, B. W. (1974). Nonparametric estimation of a survivorship function with doubly censored data. J. Amer. Statist Assoc. 69, 169-173. Vaisman, I. (1984).

A First Course in Differential Geometry. Marcel

Dekker, Inc., New York.

112

Peter J. Bickel and Jian-Jian Ren

van der Vaart, A. and Wellner, J. (1996). Weak Convergence and Empirical Processes. Springer, New York. PETER J. BICKEL DEPARTMENT OF STATISTICS UNIVERSITY OF CALIFORNIA BERKELEY, CA 94720

USA bickeWstat. Berkeley, edu JIAN-JIAN REN DEPARTMENT OF MATHEMATICS TULANE UNIVERSITY NEW ORLEANS, LA 70118

USA [email protected]

AN ALTERNATIVE POINT OF VIEW ON LEPSKΓS METHOD

LUCIEN BlRGE Universite Paris VI Lepski's method is a method for choosing a "best" estimator (in an appropriate sense) among a family of those, under suitable restrictions on this family. The subject of this paper is to give a nonasymptotic presentation of Lepski's method in the context of Gaussian regression models for a collection of projection estimators on some nested family of finitedimensional linear subspaces. It is also shown that a suitable tuning of the method allows to asymptotically recover the best possible risk in the family. AMS subject classifications: Primary: 62G07, Secondary 62G20. Keywords and phrases: Adaptation, Lepski's method, Mallows' C p , optimal selection of estimators. 1

Introduction

The aim of this paper is threefold. First we want to emphasize the importance of what is now called "Lepski's method", which appeared in a series of papers by Lepski (see Lepskii, 1990, 1991, 1992a and b). Then we shall present this method from an alternative point of view, different from the one initially developed by Lepski. Finally we shall introduce some generalization of the method and use it to prove some nice properties of it which, as far as we know, have not yet been considered, even by its initiator. Let us first give a brief and simplified account of the classical method of Lepski. This method has been described in its general form and in great details in Lepskii (1991) and the interested reader should of course have a look at this milestone paper. Here we shall content ourselves to consider the problem within the very classical "Gaussian white noise model". According to Ibragimov and Has'minskii (1981, p.5), it has been initially introduced as a statistical model by KoteΓnikov (see KoteFnikov, 1959). Since then, it has been extensively studied by many authors from the former Soviet Union (see for instance Ibragimov and Has'minskii, 1981, Pinsker, 1980, Efroimovich and Pinsker, 1984) and more recently by Donoho and Johnstone (1994a and b, 1995, 1996) and Birge and Massart (1999), among many other references. Although not at all confined to this framework, the method has been often considered in the context of the Gaussian white noise model for the sake of simplicity. This model can be described by a stochastic differential equation of the form (1.1)

dYε{t) = s(t) dt + ε dW(t),

ε > 0,

0 < t < 1,

114

Lucien Birge

where s E Lβ([0,1]) and W is a standard Brownian motion originating from 0. One wants to estimate the unknown function s using estimators s(ε), i.e. measurable functions of Yε and ε. By "estimator", Lepski actually means a family {s(ε)} of estimators depending on the parameter ε which is assumed to be small enough. In order to measure the performances of such estimators, a classical way is to fix some distance d on L2 ([0,1]) (or some pseudo-distance if d(s,t) = 0 does not necessarily imply that s = t in 1L^([0,1])), some numq ber q > 1 and define the risk of the estimator at 5 as Έs[d (s,s(ε))]. The point of view chosen by Lepski is then definitely minimax and asymptotic. an He considers a family of parameter sets {Sθ}βeΘ d uniform rates of convergences of estimators over those parameter sets. For a given estimator 5, he defines its rate r[s, 0] on Sβ and the minimax rate TM[Θ] on SQ given respectively by r[S, θ](ε) = sup E s [dq(s, s(ε))] sesθ

and

rM[θ] = inf r[5,0], «

where the infimum is taken over all possible estimators. Comparing estimators then amounts to comparing their rates, the rate r being better than the rate r' (r ^ r') if and only if limsup ε _> o r ( ε )/ r / ( ε ) < +00 and two rates being equivalent (r x r') if r ^ r' and r1 •< r. An estimator s is "rate asymptotically minimax" on S0, and therefore optimal from this point of view, if r[S, θ] x ΓM[0] The problem that Lepski considers in his papers is the following: starting from a family of rate asymptotically minimax estimators {sg}eeGi how can one get adaptation over the family {Se}θeθ, i e. build a new estimator s which is simultaneously rate asymptotically minimax over all the sets 5^, i.e. satisfies r[5, θ] x TM[Θ\ for all θ G Θ. Let us give a brief and rough account of his solution, rephrasing and simplifying his assumptions in the following way (see Lepskii, 1991 for the precise ones). Lepski's assumptions are essentially equivalent to 1. Θ is a bounded subset of M+; 2. the family {Sβ}θeθ

ιs

nondecreasing with respect to 0;

3. the minimax rates TM[Θ] are, in a suitable sense, continuous with respect to 0; 4. for each 0 E Θ, one has available a rate asymptotically minimax estimator §Θ on SΘ] 5. for ε small enough and each 0 E Θ, d ς (s, Sfl(ε)) is suitably concentrated around its expectation.

115

Lepski's Method

Lepski then chooses, for each ε, a suitable finite discretization θ\ < ... < θn(ε) of Θ and, given some large enough constant K, defines θ(ε) = θ-(ε) where q

j = mϊ{j < n{ε) \ d (sθj(ε),sθk(ε))

< Kr[sθk,θk](ε)

for all * G (j,n(ε)]}.

He shows that s = 3$ is simultaneously rate asymptotically minimax over all the sets SQ. This problem of asymptotic adaptation can also be considered from a quite different point of view: if s G S = UΘ(ΞΘSΘ, there exists a smallest value θ(s) of θ such that s G SΘ and, since we have therefore no idea of the q behaviour of the risk E s [d (s, sθ(ε))] for θ < 0(s), among the estimators at hand, SQ^ can be considered as the best estimator for estimating s, among the family of estimators {sβ}θeΘ Prom this point of view, the problem to be solved is to find a best estimator in a family of such estimators and it still makes sense without any reference to the minimax and even to the family {$θ}θeΘ It can also be considered from a purely nonasymptotic point of view and set up as follows. Given Model (1.1) with a known value of ε and an unknown value of s, a family of estimators {se(ε)}βeQ and some loss function t, is it possible to design a method for selecting an "almost best" estimator in the family? More precisely, assuming that s G S C I^QO, 1]), does there exist a constant C, independent of ε and s G S and a random selection procedure θ based on Yε such that the estimator s = s$ satisfies (1.2)

E s [φ,s(ε))] < C inf Es[£(s,sθ(ε))]

for all 5 G S and ε > 0.

This is precisely the problem we shall deal with in this paper by a suitable modification of Lepski's initial recipe. In order to allow an easier understanding of our method and avoid technicalities, we shall stick to the Gaussian white noise model and restrict our study to the case of a family {sθ(ε)}θe& of projection estimators over a nested family of finite-dimensional linear subspaces Sβ of lfl([0,1]) indexed by some subset Θ of N. We shall show that (1.2) actually holds with S = l^([0,1]) and that one can even take C arbitrarily close to 1 when ε goes to zero under some suitable restrictions on s. The framework we use here is just the one we considered in Birge and Massart (1999) for studying penalized least squares estimators. Since penalization can also be viewed as a method for selecting estimators, this allows us to make a parallel between these two methods. Indeed, under the assumptions we use here, they are essentially equivalent. A discussion of the relative merits of the two methods within a more general framework is beyond the scope of this paper. Let us merely mention that Lepski's method allows to handle more general loss functions, while penalization allows to deal with more general families of estimators.

116

Lucien Birge

Lepski's method has been put to use in various contexts and by several authors. Let us mention here the papers by Efroimovich and Low (1994), Lepski and Spokoiny (1995), Juditsky (1997), Lepski, Mammen and Spokoiny (1997), Lepski and Levit (1998), Tsybakov (1998) and Butucea (1999). Recently, Lepski has substantially improved his method by relaxing the monotonicity assumptions he previously imposed and which were in particular inadequate to deal with estimation of multidimensional functions with anisotropic smoothness. His new method, which he explained in a series of lectures (Lepski, 1998) could analogously be carried out in the context we use below. In order to keep our presentation simple and short, we shall dispense with this extension and content ourselves to present our point of view derived from the initial method from Lepski (1991). The procedure for selecting an estimator among some family that we develop below is actually not exactly the original procedure proposed by Lepski, but rather some modification of it which is better suited to our nonasymptotic approach and avoids any reference to minimaxity. Nevertheless, the ideas underlying our construction definitely belong to Lepski.

2 2.1

Preliminary considerations The problem at hand

The problem we want to deal with is the estimation of some unknown function s G LQ ([0,1]) in the Gaussian white noise model (1.1). In order to accomplish this task, we have at our disposal a family of projection estimators {sm}m£M corresponding to some nested family {Sm}m^M of finitedimensional linear subspaces of L2 ([0,1]) with respective positive dimensions Dm. Here M C N is either N* = N\ {0} or finite and equal to [1; M] ΠN and the sequence (Dm)m£M is strictly increasing. We recall that the projection estimator sm onto Sm is derived from Yε by the formula Dr,

sm = ] Γ ιs a n

where (ψι,... ,ipDm) Our purpose is then {s<m}meMi build a new sequence (Dm)meM and

/ ψj(t)dYε(t) ψj,

arbitrary orthonormal basis of 5 m . as follows: starting from the family of estimators one, denoted by s, function of those, of ε and of the such that

with a constant C independent of s and ε. Here and in the sequel, E denotes the expectation with respect to the distribution of the process Yε, as defined by (1.1).

Lepski 's Method

117

Since the family {Sm}meM is nested, one can always find an orthonormal basis φ\,... ,ψj,... of L2QO, 1]) such that Sm is the linear span of (y>i,...,¥>Dm) for each m e M. Then, if s = Σjyiβjψj, it follows from (1.1) that, for all m E M, Dm

(2.3)

w i t h

βj = / Ψj(t)dYe(t) = βj + eZj,

where the random variables Zj, j E ΛΊ, are i.i.d. with distribution Λ/*(0,1).

2.2

Some properties of projection estimators

In order to describe some elementary properties of the projection estimators, it will be useful to introduce some notations. Setting Do = 0 and D^ = +00, we consider for all pairs (m, q) with 0 < m < q < +00 the quantities 5 ^ , Vm and Um given by = ε"

and

2

with the convention that Σj=k = ^ when k > I. Since the variables Zj are i.i.d. Λ/"(0,1), it then follows that V&, has the distribution χ2(Dq — Dm) of a chi square with Dq — Dm degrees of freedom and Um the distribution χ/2(Dq — Dm, λ/Bm) of a non-central chi square with Dq — Dm degrees of freedom and noncentrality parameter y/Bm> Therefore and

= Dq - D

One then derives from (2.3) that (2-4)

Φ — c2

||Sm - Sll 2 =

and for any pair (j, m) G Λ1 2 with j < m, (2.5)

2.3

\\sm~sj\\2 = ε2up

E [||S ro - ^ | | 2 ] = ε 2

and

f + Dm -

Optimal projection estimators

Given any sequence (xm)meM s u c h that limyn-^+oo xm = +00 when Λl is infinite, one defines in a unique way (2.6)

argmin{x m , m G ΛΊ}

xj = inf x m >

=

in

=

inf {j I Xj < xm

raEΛΊ J

for all m > j} .

118

Lucien Birge

Then, given s, a best estimator in the family {sm, m E M}, i.e. one minimizing the quadratic risk E [||Sm — s||2] = ε2 (Dm + B™) at s is % with m = argmin{Z)m + i?£^, 771 E Λ4}. More generally, given some number 7 > 0 one can define J (2.7)

= argmin{7Z)m + β~, m e = inf {j I β?° - £?~ < 7 ( £ m - Dj) for all m > 3} .

Since 5?° - B% = Bψ, it follows from (2.5) that (2.8)

J - inf {j I E [ p m - sj\\2] < (1 + 7 ) ε 2 ( A » - Dj) for all m > j } .

On the other hand, by (2.7) (2.9)

Dj + B?<

Ί

Dm

Ί

+ B£ < (1 V 7) (Dm + B£),

and therefore the risk of s j satisfies (2.10)

e-*R[\\ij - β\\2] = Dj + Bf

< <

(l V7-1) ir/Dj + Bf) {ΊVΊ-')(D^

+ B%),

which is equivalent to E[||5J-ί||2]<(7V7-1)fmf<E[||ira-β||2]. One can therefore conclude that the risk of sj remains within a factor ηVη~ι of the optimal risk. Of course, from this point of view, the best value of 7 is clearly one. Nevertheless, in order to deal with more general versions of Lepski's estimators, it is interesting to consider general values of 7.

2.4

Some heuristics

Since the definition of J involves the sequence (B^)meM which depends on the true unknown parameter s, it cannot be computed but only estimated from the data. Our purpose here is to find an estimator J of J with the property that the quadratic risk of the estimator 5 = Sj is close to the quadratic risk of sj. In order to define J, one first observes that (2.8) gives a characterization of J in terms of the sequence of estimators (sm)meΛΊ rather that in terms of the true unknown function 5 to be estimated. Lepski's method is actually based on this argument. Since J depends on the 2 quantities \\SJ — sm\\ through their expectations and we only have at hand a single realization of these variables, we have to replace these expectations by the random variables themselves and since these variables clearly fluctuate around their expectations and tend to be larger than them with a nonnegli2 gible probability we have to suitably enlarge the bound (1 +^)ε (Dm — Dj) in (2.8) in order to derive a sensible estimator J of J.

119

Lepski 's Method

To see what should be added to (1 + η)ε1(Drn — Dj) in (2.8) we observe that J can equivalently be defined as the smallest index j such that 2

\\Sj-SmW

<

2

(l+Ί)ε (Dm-Dj)

+ ( P j " Sm\\2 - E [WSj - sm\\2]),

for all m > j .

Therefore a sensible definition of an estimator J of J is obtained by replacing in this formula \\Sj - sm\\2 - E [\\SJ - sm\\2] by some quantity which bounds it with a large enough probability. In order to derive such a quantity, we recall from (2.5) that ε~ 2 ||sj — s m | | 2 has a noncentral chi square distribution and appeal to Lemma 8.1 in the Appendix which controls its deviations from the mean. More precisely, it follows from this lemma with D = Dm — Dj and B = B™ < η{Dm — Dj) that whatever m> J and x m ? j > 0 2

p j - S m || > (1 + Ί){Dm

- Dj)

+ 2η)(Dm - Dj)xmj

+ 2x m ? J J < e x p ( - x m , j ) .

In order to control those deviations for all values of m simultaneously, one should require that the series Σm>J exp(—x m ,j) be summable and suitably small. This suggests the following version of Lepski's estimator.

3

Construction and existence of our estimator

Let us now define our estimator recalling that the framework we use has been described in Section 2.1 Definition 3.1 Given the increasing sequence (Dm)meM of positive integers, the projection estimators {sm} described by (2.3), a family of nonnegsuc ative numbers (λm)meM h that

(3.H) meM

and a family of numbers Kmj

(3.12)

defined form>2,

K m , i > [ ( l + 2 7 ) λ m ( D m - ^ )] 1 / 2 + λ m ,

1 < j < m and satisfying

for some 7 > 0,

we consider the random integer J = mϊ{jeM (3.13)

|||Sj-Sm||2

< ε 2 [(l + 7 ) ( D r o - Dj) + 2KmJ]

Our estimator is then given by s = sj.

for all

m>j}.

120

Lucien Birge

Remark: The convergence assumption (3.11) is quite analogous to the asex sumption Σ m G .M P(~~^m An) < +00 which appears in Assumption B p.70 of Birge and Massart (1997) and in various places in Barron, Birge and Massart (1999). Its aim is the same, as shown by the proof of the next proposition, namely to ensure that a large number of deviation inequalities be satisfied simultaneously. One should observe that J is well-defined when M = [1; M] ΠN is finite since then the set {m > M} is empty and therefore J < M (an empty restriction being always true). If M is infinite, one has to check that J < +00 a.s. in order that s be well-defined, which follows from the next proposition. Proposition 3.1 Under the conditions of Definition 3.1, J < +00 a.s. Proof We only have to study the case M. = N*. Let us consider the subset J = {Jo; Λ; •} of M of those indices j which satisfy Bj° + jDj < B™ + ηDm for all m> j . By definition, Jo = argmin {Bψ + Ί{Dm

- Dά\ m <Ξ M] = J,

as defined by (2.7), and for k > 0, J f c + 1 = argmin {j e M, j > Jk \ B]°+ΊDj

for all m > j} .

< B™+ΊDm

Moreover, since Dm -> +00 when m —> +00, J is infinite. Let now j G J and m> j . Then BJ1 < η{Dm - Dj) and it follows from (3.12) that

Kmj > J (θm - Dj + 2Bf} λ m + λm. Consequently ^

m

- Dj +

2Bf)λm+2\m.

Let us now set, for j (Ξ 3', Fj,m = { P i - Bmf > e 2 [(l + Ί)(Dm - Dj) + 2Km,j)} , and

(3-14)

Aj = Π Flm.

Since Uψ has the distribution χ' 2 \Dm - Dj, JBp\

it follows from Lem-

ma 8.1 that (3.15)

V[Fjtm] = P [Up > (1 + 7)(A* - Dj) + 2KmJ]

< e"λ-.

121

Lepski's Method

Then, by (3.13), J < j on A, and therefore {J > j} C A) = U m > i ίj,m- We conclude from (3.15) that, for any j E JΓ,

(3.16)

P[

]

[ $

Σ nj,]

j τn>j

which implies by (3.11) that P J > j 1 converges to zero when j tends to infinity in J. • 4

The performance of our estimator

Let us first set the assumptions we shall need to prove our results, recalling that the numbers λ m and Kmj have been given in Definition 3.1. Assumption 4.1 1. (Drn)rneM ^ a strictly increasing sequence of positive integers such that, if M is infinite, sup m >! Dm+ι/Dm < +oo. 2. There exists some integerp>2

such that

(

(4.17)

Σ^

Σe m>j

3. The numbers Kmj (4.18)

satisfy (3.12) and sup DmL

ra>2

sup KmJ Vl<

< +oo J

Let us first observe that, apart from the fact that it should not grow faster than exponentially, the sequence (Dm)meM can be fairly arbitrary. In practice, one typically encounters two situations. Either Dm = u + v(m — 1) (trigonometric type expansions) or Dm = u + vrn~ι (wavelet type expansions) for some suitable nonnegative constants u and v. The numbers Kmj have to satisfy simultaneously (3.12) and (4.18) and it is not at first sight obvious to choose the λ m 's in such a way that this is possible when M is infinite. The following proposition gives some hints for a proper choice of the parameters involved in our construction. Proposition 4.1 Assume that a > 3, TUQ > 1 and (4.19) then (4.17) holds.

λ m > a log Dm for m > m o ,

122

Lucien Birge

Proof Recalling from (3.16) that Σj = Σ m > j e x p ( - λ m ) , we consider some integer p > 2 such that a > 3 + 2/(p - 1). We want to prove that Σj:>1 DjΣ^~lfp < +oo. By (4.19) and the convexity of the function x \-> x'a one gets for j > UIQ -j \

-α+1

i and it follows that DjΈ-

p

p

< Dj

3 + 2/{p — 1), the series Σj>i ^3^j~

for j > TUQ. Since α >

converges and (4.17) is satisfied. •

Let us observe that (4.19) is in particular compatible with a choice of numbers Kmj satisfying for some positive constants A > a > 0, (4.20)

aDm < Kmj < ADm

for all m > 2, 0 < j < m,

which ensures that (4.18) holds. In particular, the original method of Lepski is based on the choice J = inf {j e M I \\sj - sm\\2 < Kε2Dm

for all

m>j}

with a suitably large constant K, Choosing K > 1 and 0 < 7 < K — 1 leads then to (If - 1 - η)Dm < 2KmJ = (K-1-

)Dm + (1 + Ί)Dj < KDm,

Ί

which is (4.20). Such a choice is therefore compatible with (3.12) and (4.19) for suitable values of the parameters λ m . In particular, the classical Lepski's method with a choice of K > 1 satisfies our assumptions. This is not true anymore if K < 1 and one could prove, in the same way that we proved lower bounds for the penalty term in Birge and Massart (1999), that K < 1 could lead to inconsistent estimators when ε converges to zero. One shall not insist on this here. On the other hand, if K > 1, the following theorem applies. Theorem 4.1 Under the above assumptions, there exists some constant C depending only on the various parameters involved in the construction of the estimator, but neither on ε, nor on s and such that

(4.21)

E [ p - S | | 2 ] < C inf E [ p m - S | | 2 ] . TTlt/Vl

If we fix the values of the various parameters involved in the construction of our estimator, C can then be taken as a universal constant. For instance, the particular choice of λm = 4log(Dm + l),p = 4, 7 = 1 and

123

Lepski 's Method

Kmj = \fi\mDm]ιl2 + λ m together with the assumption that An+i < for all m satisfies our requirements and, although this particular choice of the parameters has nothing special, it can cope with almost all practical situations. 5

Proof of Theorem 4.1

For the sake of simplicity we shall prove it below only under the assumption that M = N*. Only minor modifications in Section 5.5 below are needed to handle the finite case. 5.1

Basic inequality

It follows from (2.4) and the monotonicity of the sequence B™ that ε-2\\s-s\\2 =

= (VOJ + Bf) l^jy + (vj + Bf) 1{J>J} j + BJ + Bf) 1{J<J} + (VJ - Dj) 1

< Bf J O

VQJt{J>J} +

- Dj)

Vjh{J>J},

and therefore after integration

(5.22) ε- 2 E[p- S || 2 ]

<

Bf+Έ[v0Jt{j>J}] [\VOJ - Dj\ H { i < J } ] + E

[Vj%>J}\

We shall now bound successively each of the four expectations in the righthand side of (5.23). 5.2

Control of the first expectation

Recalling that the set Aj is defined by (3.14), we see that it only depends on the random variables U™ for m > j and therefore on the variables βm for m> j which implies that Aj is independent of VQ and therefore by (3.16) E

[VoJtίJ>J}]

< E[V0JlA<]

= E[V0J] P[Acj] <

DJΣJ.

124

Lucien Birge

5.3

Control of t h e second expectation

We want to bound E ί (βj + ηϋλ

l { j<j>]

W e

firs

t notice that if J = 1

and J < J, then J = J and a trivial bound is ΎDJ. Assuming now that J > 2, we define for ί > 0

Recalling that U^ has a distribution χ

/2

ίi)j- Dm, V^^)'

w e

derive from

(8.35) of Lemma 8.1 that

0<m<J

(5.23)

< (J-l)

Since E7jJ < (1 + 7)(JDJ - Dj) + 2Kj3

for J < J by the definition of J we

derive that, on the set Et Π {J < J } ,

t/

> Bj -

Using the fact that —2^/xy = [y/x — y/y) — x — y we then derive that, on the set EtΠ{J < J } , one has

[(Dj - Dj)/2 + Bj] -y/2tj

< ( 7 + l/2)(Dj - Dj) + 2Kj3 + 2ί,

and therefore

- Dj)/2 + Bj]
V2

1

aj = DJ sup KJiTn. < ajDj. Therefore, if ί > 1,

(5.25) BJ+ηDj < (y+2aj)Dj+Δt

with Δ = (β + \/(2j + 1 + ±aj)Dj}

125

Lepski 's Method

on the set Et Π {J < J} since this inequality clearly also holds if J = J. One then derives from (5.23) that P [ ( # j +7 £>j) 1 { J < J } > (7 + 2aj)Dj + Δt] < (J - l ) e - *

for t > 1.

Integration with respect to t finally leads, for J > 2, to < (7 + 2aj)Dj + [1 + (1 V log( J which clearly remains true when J = 1. Therefore whatever J > 1,

+ (δ + 7(27 + 1 + 4αj)Aj) log(3J + 5). 5.4

Control of the third expectation

Since VQ71 has a χ 2 distribution with £>m degrees of freedom, one can use Lemmas 8.3 and 8.4 to bound E \\VQ — Dj l/j< n in the following way:

- Dj m=l J

m=l

5.5

Control of the fourth expectation

In order to control E V/lr j> j , we introduce an increasing sequence (/fc)fc>o of elements of J", starting with /o = J, to be defined below. One can therem fore write, using the monotonicity of the sequence (Vo )m>o that

k>0

fc>0

126

Lucien Birge 2

Since V* has a χ (Dk—Dj) distribution and p > 2, it follows from Lemma 8.4 that E Γ (vfY] < (Dk -Dj+p-l)Pfork> Holder Inequality and (3.16) that

j and we then derive from the

fc>0

k>0

We now have to specify how we choose Ik for k > 0. Let us introduce K = inf Ij Bj° < Ύ{DJ — p + 2) \ which exists since Z)m tends to infinity ih m and d define dfi with h = argmin {Bf + ΊΌj \ j > (J + 1) V K} /fc+i = argmin { £ f + 7 ^ | j > Ik + 1} forfc> 1. Now, if Ik > K — 1, according to the definitions of Jfc+i and If, £/~+1 + ΊDh+1

< Bfk+ι + 7 ^ + 1 < 7(2£>/fc+i - P + 2),

and therefore (5.27) DIk+1 + p - 1 < 2.0/,+! < 2<5j£>/jb with ί j = sup

Dm+i/Dm.

m>J

This inequality holds for all k > 0 if J o = J > if - 1. Otherwise ΛΓ - 1 > J > 0 and (5.27) only holds for k > 1. Nevertheless, the same arguments then show that Bf^+ηD^ < η(2Dκ —p + 2). Therefore, using the definition of K one gets Dh+p-l

< 2DK < 2δjDK-ι < < 2δj{Ί-ιB?+p-2).

2δJ(Ί-1B%_1+p-2)

Therefore in both cases Dh +p- 1 < 2δj [(-f~ιBf + p - 2 ) v D j ] . It then follows from (5.26) that <

<

2δj[{Ί-1Bf+p-2)yDJ-(Dj/2)}ΣιJ-l/p

2δj [(Ί-ιB?+p+ (2δj - 1 m>J

2)vDj-

127

Lepski's Method 5.6

Completing the proof

Putting all four bounds together leads to 2

2

ε" E[p- S || ] < Bf V 7- 1 ) (s + V(27 + l + 4αj)£>j) log(3J + 5)

l

+ 2δj [(Ί- BT + P ~ 2) V Dj - ^ j Σi~ + (2δj - 1 m>J

which can also be written, since Dj > 1 as (5.28)

E [||β - *||2] < Cjε2 (Bf + ΊDj),

with Cj

= (lVΊ-ι)(l

+ 2Ί-1aj)+Ί-1Σj

+ 7- 1 (1 V 7-1) (8 + s/2Ί + 1 + 4aή Dγ'2 log(3J + 5) + 7- 1 3i?J 1/4 + (2δj L L (5.29)

1

m>J

1 1/p

+2δjΊ- {p-l)Σ f

.

On the other hand, it follows from (2.9) that

and finally from the definition of m that (5.30)

Έ[\\s-s\\2]
inf E [ | | β m -

The constant Cj depends on the various parameters involved in the construction of the estimator and on ε and 5 only through the parameter J . Moreover, it is bounded independently of J since by assumption, the sequences (aj)j>u (Σj)j>i and (ίj)j>i are bounded and Σm>o DmΣ^lfp < +00. This completes the proof of Theorem 4.1. 6

Asymptotic optimality of a modified Lepski's method

Our purpose is now to understand what is going on when J goes to infinity. Since Dj > J and Σ m > 0 DmΣl^l'p < +00, all the terms in Cj then converge

128

Lucien Birge

to zero, except for the first one which involves α j , defined by (5.24). The assumption (4.18) only implies that aj is bounded but one could enforce it to 1

(6.31)

limsupZ)- ( sup Kmλ m

\l<j<τn

= 0. )

Let us observe that such a requirement is perfectly feasible. The choice λ m = O(Dm/logm) when m —> +oo is clearly compatible with (4.19) and one can choose the numbers Kmj in such a way that they satisfy (3.12) together with sup 1 < : j < m K m j = O(Dm/\ogm) which implies (6.31). It is now easy to prove the following corollary. Corollary 6.1 Let us assume that M is infinite as well as the set {m (Ξ •M I / sΨm Φ 0}. Choose the parameters λ m and Kmj in order to satisfy the assumptions of Section 4 with U 18) replaced by (6.31) and define the estimator s as before. Then (6.32) V

limsup

J

i

Proof We already noticed that all the terms in Cj, as given by (5.29) converge to zero when J goes to infinity, except for the first one, which tends to 1 V 7 " 1 since aj -> 0 by (6.31). We then remark that our assumption on s implies that ε2B™ is bounded away from 0 independently of ε whatever me M. Then E [ p m - s|| 2 ] = ε2{B% + Dm) remains bounded away from 0 for fixed m when ε —> 0 while it can be made arbitrarily small provided that both ε and m are suitably chosen. This implies that J ->• +oo when ε —> 0 and therefore Cj —> 1 V 7 " 1 when ε -> 0. The conclusion then follows from (5.30). • One should observe here that (6.31) rules out the initial choice of Lepski for the parameters Kmj which implies that (4.20) holds.

7

Conclusion

In the framework we have chosen here, an older and very popular method for choosing an optimal estimator in our family is Mallows' Cp which actually amounts to choose S = Sj with J (7.33)

=

a r g m i n { - p j | | 2 + 2ε 2 ίλ / , j (Ξ M}

=

inf {j e M I p

m

- Sjf

< 2ε2(Dm

- Dj) for all m > j } ,

since p m — Sj\\2 = \\sm\\2 — \\SJ\\ for m > j . One should observe that it is also the estimator derived from our extension of Lepski's method with 2

LepskVs Method

129

0 < 7 < 1 and 2Kmj = (1 — 7)(An — Dj). Unfortunately, such a choice of Kmj does not always satisfy (3.12) when j = m — 1 and m is large since λ m goes to infinity with m while Dm — Dm-\ may remain bounded. Nevertheless (3.12) will be satisfied with λ m = αlogJ9 m , as in Proposition 4.1 provided that Dm > Dm-ι + clogDm for some large enough c. In any case, it has been proved in Shibata (1981), that the estimator s = sj with J given by (7.33) satisfies (6.32) with 7 = 1 and by Birge and Massaxt (1999) that it also satisfies (4.21). As to the consequences of Theorem 4.1, they have been developed at length in Birge and Massart (1999) where an analogue of this result has been proved for penalized estimators. We therefore refer the interested reader to this paper for applications of this result, just mentioning here the following one. Assume that M = N* and that Dm = m, which implies that Sm is the linear span of {φi,... ,<£>m} Given a nonincreasing sequence a = (α m ) m >χ of numbers in [0, +00] such that a\ > 0 and α m -> 0 when m -> +00, we denote by 6 (a) the ellipsoid defined by +00

with the convention that 0/0 = 0, x/0 = +00 and x/(+oo) = 0 for x > 0. Let s be any estimator satisfying (4.21), then it follows from Section 7.2 of Birge and Massart (1999) that s is minimax, up to constants, over all such ellipsoids. More precisely, there exists some constant K, such that, whatever the sequence a satisfying the above requirements, sup E [ | | 5 - s | | 2 ] < K [l V (ε/αi) 2 ] inf sup E [ p - s | | 2 ] , where the infimum is taken over all possible estimators.

8

Appendix

The following lemma is a generalization of Lemma 1 of Laurent and Massart (1998). 2

Lemma 8.1 Let X be a noncentral χ variable with D degrees of freedom and noncentrality parameter Bιl2 > 0, then for all x > 0, (8.34)

Ψ \x > {D + B) + 2yJ(D + 2B)x + 2rr] < exp(-z),

and (8.35)

P \x < {D + B) - 2y/(D + 2B)xj < exp(-x).

130

Lucien Birge

Proof Since we can write X as (JB1/2 + 17)) + V where U and V are independent with respective distributions Λf(0,1) and χ2(D — 1), the Laplace transform of X can be written as

E [ e «] = (1 - 2ίΓ D/2 exp [ ^ y for t < 1/2, which implies that

< (8.36)

(D + 2B)

= (D + 2B)

tH

i^>t>o}

_t{t
+

2t]

Then (8.34) follows from Lemma 8.2 below. On the other hand, for z > 0, PLY

and it follows from (8.36) that, for t < 0 log ( E I e * l * - " - * + * ; I) < _ L - • — JL-^V-

-J

• -M +

ί z >

Setting 2; = y(25 + D) with 0 < y < 1, one observes that the minimum of the right-hand side is obtained for 2t = — y/(l — y) and therefore log(P[X < D + B - z]) < \[D + 2S]pog(l - y) + y] < - \ [ D + 2B]y2. The result remains clearly true for y > 1 since X > 0 and (8.35) follows by setting y = 2xλ/2(2B + D ) " 1 / 2 . .

Lemma 8.2 Let X be a random variable such that log(E[exp(ί-Y)]) < ^ y

forO
b~\

where a and b are positive constants. Then Ψ[X > 2ay/x + bx] < exp(-x)

for all x > 0.

131

Lepskϊs Method Proof For z > 0, Ψ[X >z}<

exp[-Λ(*)] with

sup

h(z)=

0
and the supremum is achieved for ί' = 6~1[1 — a(bz + α 2 ) " 1 / 2 ] . z = 6x + 2aφc gives ί' = χ/x/(α + 61/z) ^ ίu

.0

b χ /

AΛ v

a x

2 a x

r—= a + by/x

h (ox + 2a\/x) = v

+

;

Taking

—= — x, a + by/x

which allows to conclude. • Lemma 8.3 Let M be a finite or countable set of indices, {Xm}meM a set of nonnegative random variables indexed by M., rh a random variable with values in M and M some subset of M, then, whatever q > 1,

)

\meM

Proof It follows from Holder's Inequality that E[Xml{ήι=m}] /

\ !/?

/

\

\m£M

/

\τneM

/

1-1/9

which is the desired inequality. • Lemma 8.4 IfY has a χ2(n) distribution, then E [(Y - n)4] = 12n(2π + 3) and k k E \γ ] < {n + k - l) - 1 for k G N, k > 2. Proof It is well-known that E [Yk] = Π?=o( n + 2i) T h e first r e s u l t t h e n follows from elementary computations. As to the second, one derives from the strict concavity of the logarithm that

(

k-l

\

k~ι J^(n + 2Ϊ) I = log(n + k - 1) i=o

/

and the conclusion follows since E [Yk] is an integer. •

132

Lucien Birge

REFERENCES Barron, A.R., Birge, L. and Massart, P. (1999). Risk bounds for model selection via penalization. Probab. Theory Relat. Fields 113, 301415. Birge, L. and Massart, P. (1997). Prom model selection to adaptive estimation. In Festschrift for Lucien he Cam: Research Papers in Probability and Statistics (D. Pollard, E. Torgersen and G. Yang, eds.), 55-87. Springer-Verlag, New York. Birge, L. and Massart, P. (1999). Report.

Gaussian model selection.

Technical

Butucea, C. (1999). Doctoral Thesis. University Paris VI. Donoho, D.L. and Johnstone, I.M. (1994a). Ideal spatial adaptation by wavelet shrinkage. Biometrika 81, 425-455. Donoho, D.L. and Johnstone, I.M. (1994b). Minimax risk over Zp-balls for i ς -error. Probab. Theory Relat. Fields 99, 277-303. Donoho, D.L. and Johnstone, I.M. (1995). Adapting to unknown smoothness via wavelet shrinkage. JASA 90, 1200-1224. Donoho, D.L. and Johnstone, I.M. (1996). Neo-classical minimax problems, thresholding and adaptive function estimation. Bernoulli 2, 39-62. Efromovich, S.Yu. (1998). On global and pointwise adaptive estimation. Bernoulli 4, 273-282. Efroimovich, S.Yu. and Low, M.G. (1994). Adaptive estimates of linear functional. Probab. Theory Relat. Fields 98, 261-275. Efroimovich, S.Yu. and Pinsker, M.S. (1984). Learning algorithm for nonparametric filtering. Automat. Remote Control 11, 1434-1440, translated from Avtomatika i Telemekhanika 11, 58-65. Ibragimov, LA. and Has'minskii, R.Z. (1981). Statistical Estimation Asymptotic Theory. Springer-Verlag, New York. Juditsky, A. (1997) Wavelet estimators: adapting to unknown smoothness. Math. Methods of Statist. 6, 1-25. KotePnikov, V. (1959). The Theory of Optimum Noise Immunity. McGraw Hill, New York. Laurent, B. and Massart, P. (1998) Adaptive estimation of a quadratic functional by model selection. Technical report 98.81. Universite ParisSud, Orsay. Lepskii, O.V. (1990). On a problem of adaptive estimation in Gaussian white noise. Theory Probab. Appl. 35, 454-466.

133

LepskVs Method

Lepskii, O.V. (1991). Asymptotically minimax adaptive estimation I: Upper bounds. Optimally adaptive estimates. Theory Probab. Appl. 36, 682-697. Lepskii, O.V. (1992a). Asymptotically minimax adaptive estimation II: Statistical model without optimal adaptation. Adaptive estimators. Theory Probab. Appl. 37, 433-468. Lepskii, O.V. (1992b). On problems of adaptive estimation in white Gaussian noise. Adv. in Soviet Math. 12 , 87-106. Lepski, O.V. (1998). Lectures given for the Paris-Berlin Seminar at Garchy. Lepski, O.V. and Levit, B.Y. (1998). Adaptative minimax estimation of infinitely differentiate functions. Math. Methods Statist 7, 123-156. Lepski, O.V., Mammen, E. and Spokoiny, V.G. (1997). Optimal spatial adaptation to inhomogeneous smoothness: An approach based on kernel estimates with variable bandwidth selectors. Ann. Statist. 25, 929-947. Lepski, O.V. and Spokoiny, V.G. (1995). Local adaptation to inhomogeneous smoothness: resolution level. Math. Methods Statist. 4, 239-258. Pinsker, M.S. (1980). Optimal filtration of square-integrable signals in Gaussian noise. Problems of Information Transmission 16, 120-133. Shibata, R. (1981). An optimal selection of regression variables. 68, 45-54.

Biometrika

Tsybakov, A.B. (1998). Pointwise and sup-norm sharp adaptive estimation of functions on Sobolev classes. Ann. Statist. 26, 2420-2469.

LUCIEN BlRGE LABORATOIRE DE PROBABILITIES ET MODELES STOCHASTIQUES BOΪTE 188 UNIVERSITE PARIS VI, 4 PLACE JUSSIEU F-75252 PARIS CEDEX 05

FRANCE [email protected]

LOCALIZATION AND DECAY OF CORRELATIONS FOR A PINNED LATTICE FREE FIELD IN DIMENSION TWO

1

ERWIN BOLTHAUSEN AND DAVID BRYDGES

2

Universitat Zurich and University of Virginia We prove that the two-dimensional harmonic crystal with a weak local pinning to a wall has finite variance and exponentially decaying correlations, regardless how weak the pinning is. The proof is based on an improved pressure estimate and an application of reflection posit ivity. AMS subject classifications: 60K35, 82B41. Keywords and phrases: Reflection positivity, harmonic crystal. 1

A survey on models and questions

There is a natural class of generalizations of the standard random walks to higher dimensional "time", which are mainly considered in mathematical physics. To motivate them, we consider first a standard real valued random walk XQ = 0,Xi,... ,X n . For simplicity, we assume that the distribution of the increments has a symmetric density / which is bounded and positive everywhere. Therefore, we can write f(x) = exp(—φ(x)), where φ is bounded from below, and symmetric. The joint density of (Xi,... ,Xn) is then (#i,... , xn) —• exp(— Σ£=i Φ(χi ~~ χi-ι)) where we put XQ = 0. For the higher dimensional versions introduced below, it is usually more natural to look at random walks which are tied down at the endpoint, i.e. conditioned on Xn+1 = 0. As we have assumed the increments to have a density, there is no problem to define that properly: The conditioned random walk has just the n-dimensional density n+1

,

(1) n

where we now set XQ = x n +i = 0, and where Zn is the appropriate norming Zn

ί =

n

1 χ

x

exp - ^2 Φ( i ~ *-i)

" J

\ nJrl

J

Research partly supported by Swiss L National = 1 ScienceJFoundation grant 20-55648.98. Research partly supported by National Science Foundation grant DMS 9706166.

1 2

ί

Pinned Lattice Free Field

135

in order that (1) becomes a probability density. As we have to do this norming anyway, we can as well shift φ around by some constant, and therefore assume that 0(0) = 0. Using this formulation, one easily sees how a generalization to higher dimensional "time" should look like. We replace the discrete time interval {1,... , n} by a discrete box Vn = {-n, -n + 1,... , n - 1, n } d , define the outer boundary by dVn = {x e I*d\Vn : 3y G Vn with \x - y\ = 1}, and then our measure Pn on R^71 by its density with respect to Lebesgue measure:

Pn(dx) = ^ e x p \-H^\x)] ΓΓ where the so called "Hamiltonian" is Hφ(x) = Σi jevnudvn H-j\=i Φ(Xi ~" Xj), and where again, we put zero boundary conditions X Ξ O O Π dVn. Zn is the appropriate norming constant, in order that Pn is a probability measure. They are usually called gradient models for obvious reasons. The corresponding family of one dimensional projections X{ : R F π -* R, i € Vn, is called a "random field" or a "random surface". We could of course as well consider Z-valued random fields. In the mathematical physics literature, such models are often called SOS-interfaces, and there is a huge literature about them. (SOS means "solid on solid", but the background of this name is somewhat obscure.) Evidently these random fields are natural generalizations of the standard simple random walks, but it should be emphasized, that they may have quite different properties in higher dimensions than they do in dimension 1. The special case which is easiest is the so called harmonic case, where φ(x) is proportional to x 2 . By scaling 0, the proportionality factor is at our disposal. For later convenience, we take φ(x) = ^ ^ 2 In that case, of course, the measure is just Gaussian. It is sometimes called the "harmonic crystal" or the lattice massless free field. The covariances are easy to calculate. One of the nice features of the model is that one has a random walk representation for them: (2) where 770,771,... is a standard symmetric random walk on Z d , starting at i under Pf w , and τyn is the first exit time from Vn. This representation follows by observing that the covariance matrix of the field is the inverse

136

Erwin Bolthausen and David Brydges

of the discrete Laplacian on Vn with Dirichlet boundary condition, and this inverse is just the right hand side of (2). The random walk representation is very useful for the investigation of properties of this random field. For instance, by standard properties of random walks, one derives that n Var n (Xo) ~ { logn

for d = 1 for d = 2 . for d > 3

The one dimensional case is of course just the standard fluctuation of a random walk. One sees that the behavior of the random field in higher dimensions is quite different from the standard one dimensional random walk. Properties of the above type have been proved for more general cases, and not just for the harmonic one. See for instance [3]. It should be emphasized that although the random surface is localized for d > 3 in the sense that the fluctuations are of order 1 (which is in striking contrast to the one dimensional situation), the surface continues to have long range correlations: Prom the random walk representation it is evident that for points i,j which are not close to the boundary dVn (i, j G \Vn, say), the covariances En(XiXj) are of order \i — j \ ~ d + 2 . For dimensions d > 3 it is easy to see and well known, that there exists a limit P ^ , a measure on R z , which is just the centered Gaussian measure whose covariances are given by the Green's function of the discrete Laplacian G(i, j) = E ^ w (Σ%L0 lτ/fc=i) An important development in recent years led to the discovery that more general gradient fields with φ convex possess also random walk representations, see [18], [23], [9]. However, the random walks which have to be used in these cases are random walks in random environments, where the random environment is generated by an auxiliary diffusion process on M y n . Although many important properties have been extended in this way from the Gaussian case to more general ones, the fine properties of this more complicated random walk are difficult to discuss and many questions remain open. We will not pursue that line here. In mathematical physics, one has investigated questions about such surfaces which are quite natural in this context, but which have not attracted much attention in the standard literature of random walks. Some of these problems center around the interaction of the surface with {x G IR^" : x% = 0 Vi G V^}, which is sometimes called a "wall". For instance, one asks how the properties of the surface change if there is a small attraction to this wall, or if one considers only random surfaces being on one side of the wall. Often, there appear qualitative transitions of the "macroscopic" behavior if some of the parameters are changed smoothly. Examples are so called "wetting transitions", where the surface at specific values of the parameters ceases to cling to the wall. We briefly discuss the wetting transition in the last section.

137

Pinned Lattice Free Field

There is a survey paper by Michael Fisher [14], his lectures on the occasion of the Boltzmann prize, where he introduced many of these problems and discussed them for the random walk case. During the rest of this paper, we entirely stick to the harmonic case

Φ(χ) = uχ2A relatively simple question is how the random surface is influenced by the presence of a local attraction to the above wall. There are several ways to build in such an attraction. The standard one is to modify the Hamiltonian Hψ by adding a potential YJieVn V(xi) where V is a function which is symmetric and has its minimum at 0. If the potential V itself is quadratic, V(x) = %x2, μ > 0, say, we arrive at what in physical jargon is called the massive free field. We define the so called "massive Hamiltonian":

(3)

*$*(*) = &

Σ

(*i-*i)2+

Vn, \i-j\=i

and then the probability measure

(4)

Pn,μ{dx) = J - exp f-ffW (x)l i€Vn

where

(5) Pn,μ is still Gaussian. The random walk representation needs only a simple modification: We have to replace the standard random walk (ηk)k>o with one having a constant death rate. More precisely, the random walk has probability -ίpy of disappearing into a "graveyard" at every step it makes. We can also formalize that by introducing a geometrically distributed random variable ζ, and replace τyn in the formula (2) by τyn Λ ζ. ^From this, one easily checks that this massive field has exponentially decaying correlations, uniformly in n. Considerably more delicate is the case where V is flat at infinity, and does not grow. For instance, one can consider

in which case the "pinned" probability measure is no longer Gaussian: (7) Pn{dx) = ^ r n

8d

i,jevnυdvn, \i-j\=ι

Udx

ievn

138

Erwin Bolthausen and David Brydges

(We will always write P for such a locally pinned measure.) It is much less clear, but true, that also in the case of a local pinning, the field is localized in a strong sense, meaning that

(8) n

and there exist constants c, C > 0, depending on ε, α, such that (9)

supEniXiXj)
We will discuss such a property here for the most delicate two-dimensional case. However, we will consider a slightly different form of local pinning, which is technically somewhat easier to handle. We describe that in the next section. 2

Statement of t h e result

The question of localization for the two dimensional harmonic crystal as introduced above was first studied in the papers [10, 11, 12, 13] and Lemberger [22]. They proved that £?n|Xo| ι s bounded no matter how small the potential is. They also proved in a mean field regime that the correlation En(XkXι) decays exponentially in |A — Z|, uniformly in n. The exponential decay for the pinned measure given by (7) had however remained open for d = 2. This problem is addressed here (for a slightly modified model). It is remarkable that also in the case of a non quadratic potential V, there is a random walk representation of covariances, at least if V is symmetric. This has been described in [5] (and in a less general way in [4]) where it is shown that the covariances E(XkXι) can be written as a sum over paths connecting k and I. By using Osterwalder-Schrader positivity this idea was embodied in a bound (see [5, lemma 4.1 and below]) which uses that the typical walk tries to minimize the number of points it occupies and thus resembles a geodesic. We will explain this below precisely for a slightly modified case, where the random walk representation is technically a bit simpler. The outcome is that E(XkXι) exhibits exponential decay and is bounded for I = k. Thus results similar to Dunlop et al. can be and were obtained in [5] but only for the easier case where V(x) is even, monotone and increasing to infinity. The last condition was imposed because otherwise the required pressure estimates were lacking. In this note we use ideas developed in [1] to provide these pressure estimates which in turn then provide these decay properties. In comparison with the work of Dunlop et al. and Lemberger our arguments rest on special properties (even potential, Osterwalder-Schrader positivity and monotonicity), but it applies in a wide regime of coupling constants to potentials which are difficult to analyze by cluster expansions used by these authors. It also appears that the method

Pinned Lattice Free Field

139

presented here provides correct dependencies of the decay on the relevant parameters. The method of [5] works directly only for periodic boundary conditions. This means that we identify n and — n in Vn = {—n, — n + 1,... , n } d , getting a (discrete) torus T n , but we loose any outer boundary, and therefore loose also any boundary condition. It is evident that in that case, a Gaussian measure with Hamiltonian ^ Σk,ieτn,\k-i\=i(χk ~~ χι)2 n o longer exists (\k —1\ = 1 has to be interpreted on the torus). This is easiest remedied by introducing a (small) mass. Thus we start with the measure PUjμ defined in (4), but with periodic boundary conditions (i.e. where Vn is replaced by T n ). As remarked in the previous section, this automatically has exponentially decaying correlations. The decay however disappears in the μ -> 0 limit. For us, the "mass" parameter μ serves only to replace the 0-boundary condition which no longer can be implemented in the periodic case, and we want to have results which are uniform in μ, after taking the thermodynamic limit n —> oo. The random walk for which (2) is correct is then simply a random walk on the torus with killing rate ^ - . We can then perform the thermodynamic limit n —> oo, and obtain a measure P<χ>,μ on Mz . This measure evidently is a translation invariant centered Gaussian measure with exponentially decaying correlations, where the decay depends on the parameter μ, and disappears when μ —» 0 and where lim^-^o Var 0 0 j μ (Xo) = oo. What we will prove here is that if we introduce an additional pinning which acts only locally, as described in section 1, we get localization, i.e. estimates (8) and (9), which are uniform in μ. We stick to the two-dimensional case which is the most delicate. (In three and higher dimensions, there is actually a simple domination argument, as has been remarked by Dima Ioίfe, which does not work in the two dimensional case). We however change the pinning slightly, to make it purely local. Our measures will be

(10)

Pnfμ,j(ώ;) - jl— ^n^J

exp [-Hμ(x)] J ] (dXi ieτn

where

(11)

Zn^j = ίe~H^ J

Π ieτn

The parameter J G M regulates the strength of the pinning. The interesting case is when exp( J) is small, which means that the pinning is weak. Theorem 2.1 Assume d = 2. For αllJ G M the field is localized in the sense that (12)

sup lim sup 25n,Ai,j l^o| 2 < oo, μ>0

n—>oo

140

Erwin Bolthausen and David Brydges

and (13)

supliτnsupEn^j(XiXj) μ>0

for some positive constants tive J (14)

< Cjexp[-cj \i - j\]

n—> oo

CJ,CJ.

Furthermore, for sufficiently large nega-

suplimsup£ n > μ ,j|Xo| 2 < K\J\ μ>0

n—> oo

for some positive constant K.

3

Random Walk Expansion

The main advantage of using the "delta-pinned" measure (10) is that it has a more elementary random walk representation of the correlations than in the case of a general (symmetric) V in (7). It would however be only technically a bit more complicated to consider for instance the case (6) in combination with the random walk representation of [5]. On the other hand, it should be emphasized that the periodic boundary conditions are crucial for applying the chessboard estimates we are using here. For A C Tn and Ac — T n \A, let

keAc

This is a probability measure on K7™, but restricted to MA, it is just the free field on A. In particular, we will have the random walk expansions for the covariances under PA,μ exactly as (2), where only we have to replace τyn by TA Λ ζ — min(τ,4,C)> C being the geometrically distributed killing from the positivity of μ. The measure, we are interested in, namely Pn,μ,j can now easily be expanded in terms of these Gaussian measures: Expanding the product we get

The covariance is therefore

We insert the random walk expansion for the Gaussian expectation

fc=0

141

Pinned Lattice Free Field

Resumming over A under the random walk expectation noting that the constraint lTAΛζ>k is the same as requiring Ac to be disjoint from the range of the walk η^Q^ and the walk not being killed by the clock, we get

k=0

where, for any set B c T n , J

dxk f j {dxk + e δ0(dxk)) This is the random walk representation. It is essentially a special case of [5, Theorem 2.2] which applies to any even potential V. From this expression one also sees that the variables are always positively correlated.

4

Reduction to a pressure estimate

We define (17)

^=inΠiminf —

log

where Z n ? μ has been defined in (5). We prove in the next section the following Proposition 4.1 For all J E R, we have δj > 0 and

lim inf — log δj > 1 j-ϊ-00

J

As a special case of the estimates in section 4 of [5], it follows that δj > 0 implies that the variance of Xo is bounded in the thermodynamic limit. Exponential decay of the covariance is also an easy consequence. For the convenience of the reader, we prove here how the theorem now follows from this proposition, in particular as the argument is simpler in our "delta pinning"-case than for a potential V of the type (6). Following [5], we use Osterwalder-Schrader positivity in the form of the chessboard estimate [17], [15], [24], [16]

142

Erwin Bolthausen and David Brydges

so that from (16), we get rRW

l*?[offc]L '|τ»|

z •7ft 3 C>fc

k=0 oo
Γ Zn,μ 1 hθ,k]\/\Tn\~ _Zn,μ,Jm

We therefore have

(18)

α(ij) ά=

where on the right hand side the random walk is on Z 2 . Remark that we are done now with the positive μ and therefore with the geometric clock ζ, and also with any finite size of the torus, and so we can now use just estimates for the standard random walk. Using this expression, we can easily estimate the covariances and the variance. Let τyN be the first exit time from the box VN. Then (19)

α(0,0) < N=0 k=0

N=Q k=0

For i,j G VM define

k=0

This is a Green's function, the kernel of (—Δy^)"1, where ΔyN is the lattice Laplacian with Dirichlet boundary conditions at the boundary of V}v Prom (19) we get:

α(0,0) < ] Γ

(GVN+1(0,0)-GVN(0,0))

N=Q

= f;

GVN(0,0)

N=0

N=0

By [20, Theorem 1.6.6] we have the estimate GyN (0,0) < clogΛΓ = c{logNδj - logίj),

143

Pinned Lattice Free Field and therefore OO

α(0,0)
log{δjN)e-δ^N-ιϊ - δjlogδj J ] e"^^" 1 ) I < oo

N=0

N=0

'

which proves (12) of the theorem. Evidently, we have δj —> 0 for J -» — oo. As δj -> 0, the two sums on the right hand side of the estimate are Riemann sum approximations to /•oo

zoo

logί g e~*dt-logίj g j // e~fdt

/

Jo

Jo

and therefore

α(0,0)<\ogδj\l + o(-J)) as J -> — oo. This together with (18) proves (14) of the theorem. The decay of the correlations (13) follows by a simple modification. By translation invariance, we only have to consider α(j,0). Then in the above estimates (19), we can restrict the summation over N to N > \j\\ + \J2\ where j = (ji, J2) Therefore we get

α(i,0) < ciδj

Σ

log(δjN)e-WN-V - δjlogδj

N=\h\+\j2\

from which (13) is immediate.

5

Estimates on 5j, proof of proposition

We finish the proof of our main theorem by proving the proposition of the last section. In the sequel, c > 0 is a generic constant, not necessarily the same at different occurrences. We subdivide Tn into boxes B of side length 2L, where for convenience we assume that L divides n. The partition function Zn^j is expanded as in (15). A lower bound for Zn^j is then obtained by restricting the sum over A in the expansion to a special class 2le of sets defined by: A G 2le if, for every box £?, B Π Ac contains exactly one lattice point and this point lies within L/2 of the center of B. Thus

L

2(n/L) 2 e J(n/L)

The proof is now easily finished using the following result:

Z

A9μ

144

Erwin Bolthausen and David Brydges

Lemma 5.1 There exists Lo E N and depending on μ > 0, there exists no(μ) £ N, such that for n > no(μ) and L > Lo inf We postpone the proof of this lemma for a moment, and proceed with the proof of the proposition. Evidently, the right hand side in the above lemma is //-independent, but we have to remember that we have the restriction n > no(μ) which does not bother us, as the claimed μ-uniformity is after taking the thermodynamical limit only. Prom the lemma we get δj > L~2 (J - loglogL - c + logL 2 ) The key point is that the entropy in the sets 2lc has given rise to the last term which dominates at large L, so by optimizing over L, L2 = e- J + °( J ) as J -> -oo we achieve a strictly positive J+

J

δj > e °(- )

and this implies the statement in the proposition. Proof of Lemma 5.1 By definition of 2l£, every box B contains exactly one point, call it A;, which is not in B Π A. In the ratio ^βnΛ,μ/^β,μ of partition functions below we integrate first out all the variables except x — Xk which leads to a Gaussian law with variance a~ι = Gβ(fc, A;), and therefore

ZBφ

fe~2aχ2dx

where in the last inequality, we estimate the Green's function GB(A;, fc) using [20, Theorem 1.6.6]. This we do with every box £?, and therefore, we get TT ZBnA,μ

>

e

-(n/L) 2 (loglogL+c)

Noting that this is the right hand side of the expression in Lemma 5.1, we see that it is sufficient to prove that Q defined by (20)

145

Pinned Lattice Free Field

satisfies

Q < c{n/Lf. These Gaussian partition functions can be integrated in terms of determinants of lattice Laplacians, which then can be expressed with a random walk representation. This is reviewed in details in [1], Section 4.1. The outcome is that

keAm=l

Formally, this is just coming from expanding the logarithm in the equality det [1 + A] = exp(Tr log [/ + A]) and using a random walk representation for the resulting terms. Implementing the above expression into (20), we arrive at oo

1

fcW(%m = k,τB<

2m,τA < 2m,C > 2m)

because the partition function Z^μ/Z^μ involves the sum over all paths in Tn that leave A. Amongst these are paths that stay inside some box B but leave A at the single point in B Γ\ Ac. These are divided out by the denominator in Q so we are left with paths that exit A and whichever box B they started in. We can replace the random walk on the torus by the free random walk, making an error in the above expression of order n2 exp [—cμn] Σm=i ^ (C > 2m) /2m, which for any μ > 0 is at most 1, if n is large enough, n > n o (μ), say, and can therefore be neglected. After having made this replacement, we write Q = Q- + Q- corresponding to ^

= ^

+ 2

m

m
^ m>L2

Thus 2

keTn πι>L 2


92

v^

}

1 11 2mm

rrn rr

2


m>L2

where the second inequality rests on the local central limit theorem. Also

Q-<ΣΣ

B

Σ 2^ keT m
n

m
W(τ?2m =

TB

2m TA

*' - ' -

2m)

146

Erwin Bolthausen and David Brydges

Stop the walk on first exit from B and write it in terms of the hitting distribution on the boundary of the box. The sum over walks that start with this hitting distribution and return to 0 is bounded by (1/m) exp(—cL2/m) by the local central limit theorem. Therefore

because as L —> oo

We have now proved Q < c(n/L)2 as required. • 6

Concluding remarks

There have been some very recent developments concerning the topics of this paper. In particular, in Deuschel and Velenik [8] and Ioίfe and Velenik [19], these authors have been able to prove (among other things) the exponential decay of correlations in two dimensional gradient models with convex interactions in quite some generality. They are using a refined version of the pressure estimate as developed here, and a renormalization procedure. A particularly interesting topic with similar questions is the so called "wetting transition". Here one considers the random field which stays on one side of the "wall" {x : xι = 0}, say on the positive side. One therefore considers the probability measures P+( ) d = P n ( |Ω+), where Ω+ d= {X; > 0, Vi}, and similarly for the pinned situation P^ji') = Aι,j( |Ω + ). (We consider again measures here with zero boundary conditions outside a box V^). In the one dimensional case, the measure P£ just describes a random walk tied down at the endpoints of the interval, and conditioned to stay positive. It is well known that after Brownian rescaling, this measure converges weakly to the law of the Brownian excursion with base line [—1,1]. If we now consider the pinned measure P^3 (still in one dimension), it is not clear from the outset if the paths under this (for n large) should still look like a Brownian excursion or if the walk sticks close to 0 due to the pinning. In his survey paper [14], Michael Fisher proved the somewhat surprising fact that there is a transition in the behaviour depending on the pinning parameter. (Fisher actually looked at a discrete random walk, but similar results can easily be proved for the situation described here, see [2]). It turns out that for small J, the pinning is not sufficient, and the path indeed just looks like if the pinning would not be present, i.e. like a Brownian excursion (after rescaling). On the other hand, if J is large, then the pinning takes over and the

147

Pinned Lattice Free Field

path becomes localized in the strong sense described by (8) and (9). This transition is what is called a "wetting transition". The phenomenon had actually been known before, see [6], [21]. The question arises if there is a similar transition occurring in higher dimensions. Quite recently, we were able to show that there is no such transition for d > 3 in the harmonic case [2]. On the other hand, Caputo and Velenik [7] showed very recently the existence of a wetting transition for d = 2, and actually in all dimensions for the continuous SOS case, where the interaction function φ of Section 1 is not quadratic but the absolute value. The results proved in these papers are however somewhat weak in the sense that only the pressure estimates have been provided. To be precise, it is proved in [2] that

>0 for all J € K, and d > 3, where

Zt = ί e-»V Π ** and

Kτn = ί e~H{x) Π (<** + eJδo(dxi)) , Jn+

ievn

and in [7] it is proved that 5j~ = 0 for small enough J. The proof that δ~j > 0 for large enough J is actually very easy in all dimensions (see [1]). Prom Jj > 0, it is easy to see that under P*j, there is a positive density of zeros in the sense that there exists an ε( J) > 0 with lim P+j(%{i eVn:Xi

= 0}> ε(J)nd})

=

It seems however to be quite delicate to conclude from that that one has a behavior of the type (8) and (9). In [22] such a result was proved by cluster expansion techniques for a slightly different model in a situation which would correspond to J being large. These methods however appear to be powerless to prove the result for arbitrary J for which δ~j > 0, which certainly should be true, but cannot be proved by the methods discussed in this paper (or any other methods presently available).

148

Erwin Bolthausen and David Brydges

REFERENCES

[I] Bolthausen E. and Ioffe, D.: Harmonic crystal on the wall: a microscopic approach. Comm. Math. Phys. 187: 523-566, (1997). [2] Bolthausen, E., Deuschel, J.D. and Zeitouni, O. Absence of a wetting transition for lattice free fields in dimensions three and larger, to appear in J. Math. Phys. [3] Brascamp, J., Lieb, E. and Lebowitz, J.L. The statistical mechanics of anharmonic lattices. Proceedings of the 40th Session of the ISI (Warsaw 1975). Vol 1. Bull. Inst. International Stat. 46, 393-404 (1976) [4] Brydges, D.C. and Federbush, P.: A lower bound for the mass of a random Gaussian lattice. Comm. Math. Phys. 62, 79-82 (1978) [5] Brydges, D . C , Frόhlich, J. and Spencer, T.: The random walk representation of classical spin systems and correlation inequalities. Commun. Math. Phys., 83:123-150, (1982) [6] Burkhardt T.W.: Localization-delocalization transition in a solid-onsolid model with a pinning potential. J.Phys.A: Math.Gen. 14 (1981), L63-L68. [7] Caputo, P. and Velenik, I.: A note on wetting transition for gradient fields, Preprint [8] Deuschel, J.D and Velenik, I.: Non-Gaussian surface pinned by a weak potential, to appear in Prob. Theory and Rel. Fields [9] Deuschel, J.D., Giacomin, G. and Ioffe, D. Large deviations for Vφ interface models, Preprint [10] Dunlop, F., Magnen, J., Rivasseau, V. and Roche, Ph. Pinning of an interface by a weak potential J. Statist. Phys. 66, 71-98, (1992) [II] Dunlop, F., Magnen, J., Rivasseau, V. Mass generation for an interface in the mean field regime. Ann. Inst. H. Poincare Phys. Theor., 57, 333360, (1992). [12] Dunlop, F., Magnen, J., Rivasseau, V. Addendum: Mass generation for an interface in the mean field regime. Ann. Inst. H. Poincare Phys. Theor., 61, 245-253, (1994).

Pinned Lattice Free Field

149

[13] Dunlop, F. Random and interacting surfaces. In Constructive physics (Palaiseau, 1994) Volume 446 of Lecture Notes in Phys., pages 249-265. Springer, Berlin, 1995. [14] Fisher, M. Walks, walls, wetting and melting. J. Stat. Phys. 34, 667-729 (1984) [15] Frohlich, J. and Lieb, E. Phase transitions in anisotropic lattice spin systems. Comm. Math. Phys. 60, 233-267, (1978) [16] Frohlich, J., Simon, B. and Spencer, Th. Infrared bounds, phase transitions and continuous symmetry breaking. Comm. Math. Phys. 50, 7995, (1976). [17] Glimm, J., Jaffe, A. and Spencer T. Phase transitions for fields. Comm. Math. Phys.45, 203-216, (1975).

φ\-quantum

[18] Helffer, B. and Sjόstrand, J. On the correlation for Kac-like models in the convex case. J. Statist. Phys. 74, 349-409, (1994) [19] Ioffe, D. and Velenik, I. A note on the decay of correlations under δpinning, to appear in Prob. Theory and Rel. Fields [20] Lawler, G.F. Intersections of random walks. Probability and its Applications. Birkhauser Boston Inc., Boston, MA, 1991. [21] van Leeuwen J.M.J. and Hilhorst H.J.: Pinning of a rough interface by an external potential. Physica 107A (1981), 319-329. [22] Lemberger, P. Large field versus small-field expansions and Sobolev inequalities. J. Statist. Phys. 79, 525-568, (1995). [23] Naddaf, A. and Spencer, Th. On homogenization and scaling limit of some gradient perturbations of a massless free field. Comm. Math. Phys. 183, 55-84 (1997). [24] Seiler, E. and Simon, B. Nelson's symmetry and all that in the Yukawa2 and φ\ -field theories. Ann. Physics 97, 470-518, (1976).

D E P T . MATHEMATICS UNIVERSITAT ZURICH WLNTERTHURERSTRASSE 1 9 0

CH-8057 ZURICH SWITSERLAND

eb@amath. unizh. ch

QUADRATIC STATISTICS IN TESTING PROBLEMS OF LARGE DIMENSION

D.M. CHIBISOV1

Steklov Mathematical Institute We consider testing a simple hypothesis about the mean vector of an iV-variate normal distribution against shift alternatives in a Bayesian setting specifying a prior distribution of the mean vector under the alternative. We treat the problem asymptotically, as N —• oo, and state fairly general conditions on the sequence of prior distributions under which the Bayes tests have asymptotically ellipsoidal acceptance regions. AMS subject classiBcations: 62F03, 62F05, 62F15.

Keywords and phrases: Hypothesis testing, Bayes tests, multivariate normal distribution, large dimension.

1

Introduction

We consider testing a simple hypothesis about the mean vector of an Nvariate normal distribution against shift alternatives in a Bayesian setup specified by a prior distribution of the mean vector under the alternative. Specifically, based on a single observation of an iV-variate normal vector with identity covaxiance matrix we test the hypothesis that it has zero mean vector. We assume that for each N the prior distribution is the product of symmetric univariate distributions, or, in other words, under this prior the mean vector has independent symmetrically distributed components. Furthermore, we require these components to be in a certain sense asymptotically uniformly negligible. The result obtained can be viewed as an asymptotically complete class theorem saying that for this kind of alternatives in large dimension one can restrict oneself to tests with ellipsoidal acceptance regions. At the end of this section we give an example of a prior distribution for which our conditions fail. The normal shift model of fixed dimension arises in asymptotic hypothesis testing problems about a multivariate parameter, the normal vector under consideration being the limit in distribution of a sequence of (vector-valued) asymptotically sufficient statistics, see, e.g., Roussas (1972), Chapter 6. (The general case of a known positive definite covariance matrix treated therein 1

Research supported by the INTAS-RFBR grant 95-0099, by DFG-RFBR grant 9801-04108, and by the RFBR Grant 96-15-96033 for Leading Scientific Scools..

Quadratic Statistics

151

reduces to the case of the identity matrix by a linear transformation.) It is customary for multivariate analysis to use the chi-squared test for this testing problem. However, the underlying property of rotational invariance may often be inadequate, and discarding it we are left with a variety of tests neither of which is intrinsically dominant. Hence we look for a reduction which could be achieved under some natural additional requirements. It is well known that the shifts of a normal distribution form an exponential family in which case tests with convex acceptance regions constitute an essentially complete class of tests; these are Bayes tests and their weak limits, see, e.g., Roussas (1972), Appendix 4, and references therein. Our study is motivated by nonparametric goodness of fit and signal detection problems, see Ingster (1993), Spokoiny (1998), and references therein. In the former problem the normal shift model again is obtained asymptotically, for large sample size, while in the latter case, when the signal is observed in a white Gaussian noise, it is obtained directly by taking the Fourier coefficients of the observed process with respect to some orthonormal basis on the observation interval. In both these cases it appears natural to consider prior distributions rendering the components of the mean vector independent and symmetrically distributed. (Thus we think, say, of a possible signal as being composed of random and independent harmonics.) For fixed N these assumptions provide a certain reduction (in particular, the acceptance regions become symmetric in each coordinate). However, the treatment of this problem for large dimension (as N -> oo) allows for a substantial reduction to the class of tests of a specific structure, viz., tests with ellipsoidal acceptance regions, provided the priors satisfy a certain uniform negligibility condition. To explain the nature of the result we state in this section a corollary to the main theorem having a more transparent form. Thus we observe the random vector (1.1)

X =

(XU...,XN)

having normal distribution N(μw, IN) with μπ = ( μ M , . . . , μNN) £ R N and Ijy the NxN identity matrix. We test the hypothesis HNO ' βN = 0 against HN\ VN Φ 0. In the Bayesian setup we assume that μjy under HNI has a prior distribution TΓΛΓ, which is the product of N coordinate distributions, N

(1.2)

πN(dμN)

= x 2=1

πNi(dμNi),

so that //ΛΓ is a random vector with independent components having distributions TΓM, . . . , KNN. We assume throughout that TΓJV», i — 1? > -ΛΓ? are symmetric about the origin. By (1.1) for a given μjv the distribution of X has Lebesgue density N

(1.3)

152

DM. Chibisov

where φ(-) denotes the density of the standard normal distribution. We denote this distribution by P;v,μ and the corresponding expectation by E^μ. In particular, the distribution of X under HNO has density
The power of a test with test function ΨN(X) against a particular alternative μw equals (1.4)

/MAW ΦN) =

ENtμψN(X).

In the Bayesian setup, (1.3) is a conditional density of X given μ^, and the marginal distribution of X has density (1.5)

PJV(X) =

/

Then the power of the test ΦN is (1.6)

/?ΛΓ(TΓΛΓ; ΦN)

= I ^Λτ(x)PΛr(x)rfx = /

/MAWφN)πN{dμN)>

We will refer to /?jv(μjv; ΦN) given by (1.4) as the power function of the test ΦN and to PN(^N\ΦN) given by (1.6) as the average power. For a preassigned size α, the Bayes test maximizing PN{^N\ΦN) over size a tests ΦN rejects HNO for large values of the likelihood ratio (LR) (1.7) More precisely, the level a Bayes test has critical function (1.8) with

) < cN, CN

and

^ΛΓ(X)

on {x : /iiv(x) =

CN}

defined so that

= / ^iv(x)^Λr( The level a > 0 will be kept fixed as N —> oo. In Theorem 2.4 we state conditions on the priors TΪN under which the LR IIN is asymptotically approximated by

(1.9)

gN(x) = exp[i £ bm (x] - l) - \BN] ,

where b^ > 0 are certain characteristics of TΓJVI and BN = Σ ^2Ni (which is assumed to be bounded as N —> oo). Namely, QN approximates HN in Li-norm w.r.t. PΛΓ,O, ί e., (1.10)

ENio\hN

- 9N\ -> 0

as

N -^ oo.

153

Quadratic Statistics

It follows from (1.10) that the test ψ9N(x) defined for gx similarly to (1.8) has asymptotically the same average power as the Bayes test V>JV(X)> i e > 9

βN(τrN;φ N)-βN(πN;φ^)^0

as N -> oo.

To illustrate Theorem 2.4 we state here a special case. Suppose that the distributions n^i in (1.2) are scale transforms of one and the same distribution π on R with scale factors UNi(μ) = U(μ/bNi),

i = 1,..., N,

where Πjvj(μ) and Π(μ), μ E R, denote the distribution functions corresponding to ΈNi and π. Let π and {&#;} satisfy the following conditions: (Πl)

π is symmetric, i.e., Π(μ) = 1 — Π(—μ), μ E R;

(Π2)

/ μ 2 π ( d μ ) = 1, / μ 4 π ( d μ ) < oo;

(Bl)

bNi > 0, fciv,max : = maxi 0 as N -* oo;

(B2)

Σ i I i ^ - ^ J B > 0 asiV-^oc.

Note that the first condition in (Π2) is merely a normalization of π, under which b2Ni is the variance of μNiCorollary 1.1 Let Conditions (Πl), (Π2), (Bl), (B2) be fulfilled and let gN be defined by (1.9) with BN = B. Then (1.10) holds. Consider the particular case where 6 M = . . . = b^N- Obviously, (Bl), (B2) are satisfied for 6 ^ = ( J 3 / J V ) 1 / 4 , i = 1,...,JV. Then Corollary 1.1 says that, under the independence assumption on the components of μ # , Conditions (Πl) and (Π2) are sufficient for the Bayes test to be asymptotically chi-squared. It is well known that for any spherically symmetric prior distribution the Bayes test is exactly chi-squared. Under the independence assumption spherical symmetry holds only for π normal. Corollary 1.1 says, however, that Bayes tests become approximately chi-squared for large dimension under arbitrary symmetric π unless π is heavy-tailed (the second condition in (Π2)). Note that in the setup of Corollary 1.1, when the prior distribution has independent symmetric components differing only by scale factors, (Π2), (Bl), and (B2) are exactly conditions for asymptotic normality of

The same is true in the general case (see Remark 2.2). In the literature Bayes tests in the normal shift model of increasing dimension are used in asymptotically minimax nonparametric hypothesis testing, see Ingster (1993), (1997), and Spokoiny (1998), where further references

154

DM. Chibisov

can be found. In these studies the original problem of signal detection or goodness of fit reduces by a suitable orthogonal decomposition to a testing problem in the normal shift model (possibly, infinite-dimensional). Typically in the minimax setting this is the problem of testing for zero mean against the set of alternatives specified by a "big" ball or ellipsoid in a certain norm, say, Zς-norm, with a "small" ball or ellipsoid in, say, ίp-norm around the origin removed. The problem is treated asymptotically as the size of these domains varies and/or the common variance of the X^s tends to zero. For some particular prior distributions used in those papers the asymptotically ellipsoidal form of the Bayes tests was established directly. For example, Ingster (1993) uses "Bernoulli priors" specified by symmetric two-point prior distibutions of components. These distibutions obviously satisfy conditions (Πl) and (Π2). The choice of the prior distribution depends on the shape of the parameter set, specifically, on the degrees p and q of the norms. If the normal shift model originates from, say, a signal detection problem, these degrees are related, qualitatively, to smoothness properties of the least favorable signals and restrictions on their "energy". In this respect Spokoiny (1998) distinguishes four types of alternative sets. Apparently the type of alternatives treated here fits in one of those classes, viz., that of "smooth" signals. Another type of prior distributions used by Ingster (1993) and Spokoiny (1998) for other types of alternatives has three-point component distributions Έ^I with masses pw at points ±1 (up to scale factors) and mass 1 — 2pχ at 0 with PN -> 0 as N -» oo. Note that the ratio of the fourth moment to the squared variance equals here 1/PN —^ oo For this prior distribution the conditions and the conclusion of Theorem 2.4 fail. We state the main Theorem 2.4 in Section 2 and give its proof in Section 3. Section 4 contains the proofs of auxiliary results and Corollary 1.1. 2

Main Theorem

Recall that we consider testing the hypothesis Ho : μjsf = 0 based on the observed iV-variate random vector X = (Xi,..., XJV) with normal distribution ΛΓ(μw,/#). Under the alternative μN has prior distribution (cf. (1.2)) πN(dμN)

N

= x

i=l

πNi(dμNi).

Thus under this prior {μNί} form a triangular array of r.v.'s independent within each row (for each N) with corresponding distributions π/vι, i — 1,...,ΛΓ.

Assumption ( A l ) . The distributions TΓ^, i = 1,...,JV, JV E N = {1,2,...}, are symmetric, i.e., πjVi(A) = fκ^i{—A) for any Borel set A.

Quadratic Statistics

155

In terms of the corresponding distribution functions this assumption means that UNi(μ) = 1 - ΠΛΓΪ(—μ), μ G R (cf. (Πl) in Section 1). For α > 0, denote (2.1)

ΊNi(α) = 1- 7Γjvi([-α, α]) = 2πNi((α, oo)).

Assumption (A2). For any α > 0,

For a measure ζ) and a measurable function / (on the same space) we will write

(2.2)

Q(f) = Jf(x)Q(dx).

For α > 0, denote by τr^\ the measure πjsn restricted to the interval [—α, α], π(£)i(A)=πNi(An[-α,α}).

(2.3)

Define the corresponding truncated moments as (2.4)

^ v > ) = 41(μjvi),

* = 0,1,2,...

Note that due to symmetry of π ^ (see (Al) and (2.3)), ^fc,ΛΓ,i(α) = 0 for odd k. Obviously, Vk,N,i(β) for &ny even k is a nondecreasing function of α. Lemma 2.1

Under Assumptions (Al), (A2), N

N

(2.5) for any fixed αi, α<ι > 0 and any even fc > 0. Proof For 0 < αi < α2 the left-hand side of (2.5) is nonnegative and bounded by αί> Σ7ΛΓi(αi), which tends to zero by (A2). •

Assumption (A3). For any α > 0, ΛΓ t=i

By Lemma 2.1 the requirement "for any α > 0" can be equivalently reduced to the requirement "for some α > 0". Since ^N,i(α) ^ "4,ΛΓ,i(α)> Assumption (A.3) implies

156

DM. Chibisov

Corollary 2.2

Under Assumption (A3),

(2.6)

limsupί?Λr(α) < oo,

for any α > 0, where (2.7) L e m m a 2.3

Under Assumptions (A1)-(A3), for any fixed oi, α^ > 0 AN := BN{α2)

- BN{α{)

-» 0

as

iV ->• oo.

The proof of this lemma will be given in Section 4. Theorem 2.4

Under Assumptions

(2.8)

E | M X ) ~ ffiv(X; α)| -> 0

(A1)-(A3) as

N -> oo

for any α > 0, where (see (2.4), (2.7))

(2.9)

1 5n(x,α) = e x p ( -

N

i

Remark 2.1 The relation (2.8) implies, in particular, that the functions ^jv( ,α) for different choices of α approach each other in L\ norm. This can also be verified directly by using Lemmas 2.1 and 2.3. Remark 2.2 Assumptions (A1)-(A3) imply asymptotic normality of the sequence Σιμ2Ni with mean BN(O) and variance X) ί^4,ΛΓ,i(α) ~^|,iv,i( α )) for any α > 0, see Loeve (1960), Section 22.5. In this respect Corollary 1.1 relates to Theorem 2.4 in the same way as Theorem V.1.2 in Hajek and Sidak (1967) to the general normal convergence theorem in Loeve (1960) mentioned above.

3

Proof of Theorem 2.4

Take an α > 0. Without loss of generality we will assume that there exists the limit B(α) := limjv->oo-Biv(α). (Otherwise assume that (2.8) fails, select a subsequence where the left-hand side of (2.8) stays bounded away from zero and find by (2.6) a further subsequence where Bjsf(α) converges.) The proof relies on the following one-sided version of Scheίfe's Lemma (see Chibisov (1992), Lemma 3.1).

157

Quadratic Statistics

Lemma 3.1 Let for each J V G N the random variables UN > 0 and VN > 0 be denned on a probability space (AJV,*AΛΓ,PΛΓ) Assume: (i) ENUN -> 1, -> 1; (ii) V# are uniformly integrable w.r.t. PN, or, equivalently, EΛΓ[VΛΓ; AN]

:= /

Vfr dPjv -> 0

whenever

-» 0;

PN(AN)

(iii) PN(UN < VN - ε) -» 0 for any ε > 0. Then EN\UN-VN\->0. We will apply this lemma with PN : = PJV,O = iV(0,/JV)? ^ : = ^iv? and VJV : = 9N(' >°>)' Condition (i) for h^ holds by definition (see (1.5), (1.7)), since EJV,O^JV = 1. The following lemma will be used to verify Condition (i) for gN. Lemma 3.2 For any even k > 0 and any α > 0 max Vk Ni(α) ~+ 0. l
Proof For an arbitrary ε > 0, limsup max i/fe^ifα) < εfc + li 1<<JV

ΛΓ-)-oo

By Lemma 2.1, the latter term equals 0. Hence the lemma follows. • To check Condition (i) for QN = 5iv( , α), we use the formula: for a r.v. X with standard normal distribution and any b < 1,

When applied to (2.9), this yields (with dependence on α suppressed) N

χ

1

1

zy

1/2

EN,o9N = Π t " 2,iv,i)~ exp(--i/2,ivJi - 4^2,;v,i) By Taylor we have with some 0 < ΘNΪ < 1

which tends to zero by (2.6) and Lemma 3.2. To verify condition (ii), assume the contrary. Then there exist ε > 0, a subsequence {N'} C {iV}, and sets ANι C R ^ ' with PN'${A I) -> 0 such that N

(3.1)

EN'fl\gN'l AN>] > ε

for

all N'.

158

DM. Chibisov

Recall that we assume the limit B(a) = limw^ooiJjv'^) to exist. Let B(a) > 0. Then (3.2)

(2S(α))~ 1 /2 £ u2^(a)(Xf

- 1) ^

d

ΛΓ(0,1)

because Lemma 3.2 implies the Lindeberg condition (see Theorem V.I.2 in Hajek and Sidak (1967)). Hence gw( -,a) converges in distribution to g = e x p [ i y - i β ( α ) ] , where Y ~ N(Q,2B(a)). If B{a) = 0, one checks directly that Sjv( α) converges in distribution to g = 1. In both cases Eg = 1. Therefore gw are uniformly integrable (see Loeve (1960), 9.4.e and 11.4.A), which contradicts (3.1). Thus it remains to prove that (3.3)

PNfl(hN
for any

ε > 0.

If B(ά) = 0, one can check directly that both hjsr and pjv(a) converge to 1 in probability, which implies (3.3). So, assuming B(a) > 0 we will prove that (3.4)

P#,o(log h,N < log <7;v(α) — ε) —> 0 for any

ε > 0.

It is readily shown that (3.4) implies (3.3). Indeed, the inequality in (3.4) entails an inequality as in (3.3) unless gx takes large values, which occurs with a small probability by an argument similar to the one used when checking condition (ii). Thus having shown (3.4) we will have established the conditions of Lemma 3.1, which then implies the theorem. Now we proceed to the proof of (3.4). By (1.2), (1.3), (1.5), and (1.7),

l

1

where πjvi[- •] means the integral w.r.t. μjsn as in (2.2). Obviously, N

(3.5) where (3.6)

h>Ni(x, a) = πNi [exp(xμNi — ^Ni)]

(see (2.3)). Now we use the fact that for odd m G N m

kk

x x e > 1 + Σ 7τ for any x 6 R. k x

k=i

159

Quadratic Statistics Applying this inequality with m = 5 to (3.5) we obtain (3.7)

hNi(x,α) >

l+ξNi(x,α),

where, using the notation (2.1) and the abbreviation v^ = Vk,N,i{Q)->

(3.8) ξmM

= -7ΛΓ»(α) - -u2 + ^ ( Λ + ^ Ά ) - ^ A l/

, 3 o

4

1

^

1/^4

5

+ 1

2

To complete the proof, we need the following two lemmas. Their proofs will be given in Section 4. L e m m a 3.3 (3.9) Lemma 3.4

(3.10)

(A1)-(A3), for any δ > 0,

Under Assumptions

< -δ) -> 0.

PNfi(minξNi(Xuα)

(A1)-(A3)

Under Assumptions

~ 1) ->P^,O 0.

J2^i(Xuα) - iΣ^Mti&i l

t=l

i=l

By Taylor, for any ε > 0 there exists δ = δ(ε) > 0 such that log(l + x) > x-

-{l + ε)x2

for

x > -δ.

Hence by (3.5), (3.6), and (3.7), Lemma 3.3 implies that (3.11)

P * f o ( l o g M X ) > /ΛΓ(X)) ~> 1,

where

(3.12)

1 -I

Lemma 3.4 and (3.2) imply that (2/J3w(a)) cally normal iV(0,1). Therefore

ley

s

Σ£/Vi(^ΐ) ^ asymptoti-

(see Gnedenko and Kolmogorov (1949), Section 28, Theorem 4). comparing (2.9) and (3.12) we obtain by Lemma 3.4 (3.13) Since {Bχ(a)} theorem. 4

fN(X)

Hence

- loggN{X,a) + 6-BN(a) - > P i V 0 0. 4

'

is bounded, (3.11) and (3.13) imply (3.4) and hence the

Proofs of auxiliary results

160 4.1

DM. Chibisov Proof of Lemma 2.3

Assume 0 < aγ < a2. Then, obviously, AN

> 0. By (2.7),

f

(4.1)

N

For each i = 1,..., TV, when the inequality

holds, we have (4 2) Otherwise,

Hence in this latter case (4.3)

~ 4iv,i( α i) < 2ε"1(ι/2

"IN^M

Therefore (4.1), (4.2), and (4.3) show that N

(4.4)

Δiv < 2ε ^

N

^lN^{a2)

+ 2ε~ι

The second term in (4.4) tends to 0 as N -> oo by Lemma 2.1, while the sum in the first term is bounded by Corollary 2.2. Hence limsup^ AN can be made arbitrarily small by the choice of ε. 4.2

Proof of Lemma 3.3

Rewrite (3.8) as (4.5)

6\ri(α) = - 7ΛΓi(α) + VNΪ ( α ) + ^Ni ( α ) + ^Ni (α)>

where τ$](α) = r$](-Y<,α) with /J

Λ\

(A

fγ\

(1) /

\

(2) /

\

i

-*• /

14 O

VNi\X^a)

(4.8)

ηNi(x,a) =

-I \

2

4

fi^y.2

~ WΛ\X ~~ΌX

/

\

161

Quadratic Statistics

The sum in (4.8) contains a finite number of terms (actually, six) with even j > 0 and k > 6. For the proof of Lemma 3.3 it suffices to establish the corresponding assertions for each term in the RHS of (4.5). The ones for 7 ^ , ηNi, and ψN\ follow from Assumption (A2) and Lemma 3.2 (notice that the polynomials in (4.6) and (4.7) are bounded from below). The counterpart of (3.9) for ψNl is obtained from the following two lemmas. Lemma 4.1

For any α > 0 and any even k > 4 N

(4.9) Proof By Lemma 2.1, for an arbitrary ε > 0, N

N

limsupΣ^,ΛΓ,i(α) = limsup^ I / Mr,*( e ) N-ϊoo

i =

ι

N-ϊoo

i = 1

N 4jiV

,i(ε) = Cek~4

with C < oo by Assumption (A3). Hence (4.9) follows. • Lemma 4.2 Let YΊ, Y2, be i.i.d. r.v.'s with E|YΊ| < 00, and let {CNΪ,i = 1,..., N}, N £ N, be a triangular array of nonnegative numbers such that N

(4.10)

ΣcM->° z'=l

Then maxi<ί<jv Cjvi|Yί| —tp 0. Proof For an arbitrary ε > 0 we have, using the Markov inequality,

which proves the lemma. • Now the counterpart of (3.9) for each term of ψNl(Xi,α) (see (4.8)) follows by Lemma 4.2, with condition (4.10) for CJV; := i>fc,jv,i(a) fulfilled by Lemma 4.1.

162

DM. Chibisov

4.3

Proof of L e m m a 3.4

Comparing (3.10) with (4.5) and taking into account Assumption (A2), we see that it remains to show

2=1

and N

(4.12J

2^3 : = /

J

ηNi

(Q

2=1

It is directly verified that E(Xf-6X^+3)

= 0, so that E Σ 2 = 0; further,

var Σ2 = const ] P v%,N,i(a) ^ const maxu^N^a)

^

^ j v ^ α ) -> 0

by Assumption (A3) and Lemma 3.2. This implies equation (4.11). Next, by Lemma 4.1, E | Σ 3 | -> 0, which proves (4.12). 4.4

Proof of Corollary 1.1

We have (a) to check that (Πl), (Π2), (Bl), and (B2) imply Assumptions (A1)-(A3) and (b) to show that the truncated moments V2,N,i{a) &nd the quantity J5jv(α) can be asymptotically replaced by b2Ni and B respectively. Assumption (Al) obviously follows from (Πl). The 4th moment assumption in (Π2) implies 14

(4.13)

ηNi{a) = π{bNi\μNi\

p

> a) < -& /

\μ\4π{dμ)

The last integral tends to zero uniformly in 1 < i < N by (Bl) and (Π2), so (A2) follows from (B2). To check Assumption (A3), note that

J\μ\
Hence (A3) follows from (B2) and (Π2). For (b) we have to show that for any a > 0 (4.14)

Σ > 2 , J V » - b2Ni) (Xf - 1) -» P j V j 0 0 2=1

and (4.15)

BN(a) -> B.

163

Quadratic Statistics

We establish (4.14) by showing that the 2nd moment of the LHS tends to 0, which amounts to (4.16) Observe that

/ \

12 f Nι JM
'

hence (δ/Vi "" ^2,ΛΓ,ί(α)) = bNi[

/ K

J\μ\>a/b \μ\>α/bNiNi

μ

μ dπ) J

'

% f

< bNi I{

J\L J\μ\>α/bNi

|/|

Thus (4.16) is obtained by (B2) and the argument following (4.13). Now (4.15) follows from (4.16) by the triangle inequality. It remains to show that under the assumptions of Corollary 1.1
Acknowledgements The author thanks the referees and editors for suggestions that helped to improve the presentation.

REFERENCES Chibisov, D.M. (1992). Asymptotic optimality of the chi-square test with large number of degrees of freedom within the class of symmetric tests. Mathematical Methods of Statistics 1, 55-82. Gnedenko, B.V. and Kolmogorov, A.N. (1949). Limit Distributions for Sums of Independent Random Variables. Gostekhizdat, MoscowLeningrad (Russian). English translation: Addison-Wesley, Reading, MA, 1954.

164

DM. Chibisov

Hajek, J. and Sidak, Z. (1967). Theory of Rank Tests. Academia, Prague. Ingster, Yu.I. (1993). Asymptotically minimax hypothesis testing for nonparametric alternatives. I, II, III. Mathematical Methods of Statistics 2, 85-114, 171-189, 249-268. Ingster, Yu.I. (1997). Some problems of hypothesis testing leading to infinitely divisible distributions. Mathematical Methods of Statistics 6, 47-69. Loeve, M. (1960). Probability Theory, 2nd edition. VanNostrand, Princeton, NJ. Spokoiny, V.G. (1998). Adaptive and spatially adaptive testing of a nonparametric hypothesis. Mathematical Methods of Statistics 7, 245-273. STEKLOV MATHEMATICAL INSTITUTE GUBKINA STR. 8

GSP-1 117966 Moscow RUSSIA

chibisov@mi. ras. ru

SOME REMARKS ON LIKELIHOOD FACTORIZATION

D.R. Cox Department of Statistics and NufEeld College, Oxford Various types of likelihood factorization are reviewed and some general statistical consequences noted. In one broad class there is noniterative asymptotically efficient combination of information across factors via generalized least squares. This is used to discuss missing information in simple binary problems. It is shown that with observations on a 2 x 2 table supplemented by independent observations on each margin the maximum likelihood estimate of the odds ratio differs from that based on the complete table but is not asymptotically more efficient. AMS subject classifications: Primary 62F 15. Keywords and phrases: asymptotic theory; concentration graph; conditional likelihood; marginal likelihood; 2 x 2 table .

1

Introduction

The likelihood function plays a key role in all approaches to the formal theory of parametric statistical inference and has implications also for semiparametric problems. We deal here primarily with the former. We consider a number of types of factorization of the likelihood function. For the important distinction between parameter-based and concentration-graph based factorizations and some applications to mixed binary and continuous variables, see Cox and Wermuth (1998). We assume throughout what are known in some quarters as the British regularity conditions. Suppose that for an observable random vector Y with observed value y there is a parametric statistical model leading to a likelihood function L(θ; y) with the parameter vector θ taking values in Ω#. Suppose further that we factorize the likelihood in the form (1)

L(θ;y) =

L1(φ1-y)L2(φ2;y),

where {φ\,φ2) determines θ. There is no essential loss of generality in restricting the discussion mostly to two factors. For sampling theory discussions we suppose that (1) is available for all y whereas for Bayesian calculations it is enough that (1) holds for the particular observed y.

166

2

D.R. Cox

Some types of factorization

We call the factor L\ directly realizable if it is the full likelihood function for a random system directly associated with that defining Y. The direct association is typically by conditioning on and or marginalizing over observed features of the initial random system. Example 1. A cut in an exponential family (Barndorff-Nielsen, 1978) involves a factorization based on the marginal distribution of a statistic S and the conditional distribution given S = s. Both marginal and conditional factors are directly realizable. Example 2. There is a trivial factorization of the form (1) if the full data derive from observations on two or more independent random systems. Example 3. The normal-theory of the recovery of information in incomplete block designs (Yates, 1940) is in effect based on a likelihood factorization in which the first factor is the likelihood of the parameters based on data transformed to eliminate block effects and the second factor is the likelihood derived from the block totals. Both factors are directly realizable. We now give an example of a factorization that is not directly realizable. Example 4- In right censored survival data, taken without explanatory variables for simplicity, the observations can be written yι = (U,di), for i = 1,..., n, where U is a recorded time and d{ = 1 for a failure and d{ — 0 for censoring for the i th individual. Then if f(t) and Sf(t) are respectively the density and survivor function of failure-time and g(t) and Sg(t) are the corresponding functions for censoring time, the likelihood under random uninformative censoring can be factorized in the form (2) (3)

L2(φ2;y)

=

Here φ\ and φ<ι are parameters characterizing respectively the distributions of failure and censoring times. One or indeed both factors could be left nonparametric. The factorization is not directly realizable, any interpretation of this on its own being remote from the original generating process. We shall give further examples later. We call the factor L\ asymptotically second-order valid if, in some notional limiting operation in which the amount of information contained in Y tends to infinity, the asymptotic distribution of i, the formal maximum likelihood estimate of φ\ derived from Li, is asymptotically normal with mean φ\ and covariance matrix estimated via the inverse of the observed information matrix, i.e. by (4)

-1

167

Likelihood Factorization

where l\ = logLi and V is the gradient operator with respect to φ\. The factor L\ is asymptotically first order valid if φ\ is asymptotically normal with mean φι, the covariance matrix not necessarily being determined from (4). Well-behaved directly realizable factors will typically satisfy (4). Example 4, censored survival data, also satisfies (4). Example 5. A simple example of first-order asymptotic validity is provided by quasi-likeihood analysis of overdispersion in a Poisson distribution, treating the case without explanatory variables for simplicity. Let hp(y; μ), hχB(y\ μ?«) denote the probability of value y in respectively a Poisson distribution of mean μ and a negative binomial distribution of mean μ and index n. For independent observations, write

(5) Lι(μ y) = Uhp(yi\μ),L2{μ,κ;y)

= τihNB(yϊ, μ, κ)/hP(yn μ).

Then under the negative binomial model, the maximum likelihood estimate of μ, μi, say, calculated from L\ is asymptotically first-order valid but its variance is not given via the second derivative of L\ unless the special case of the Poisson distribution applies. The negative binomial distribution can be replaced by other distributions of mean μ. The second factor contains information about K,. Asymptotic expansion in powers of κ~ι shows that to first order the dependence of I/2(μ,«; y) on the data is essentially via a comparison of variance and mean. We return to this and to another example in Section 3. Next we call (1) the factorization (1) parameter-based if φ\ and 02 are variation independent and

There are various extensions, such as to situations in which there are a small number of common parameters to the two factors possibly orthogonal to one or both of the disjoint component parameters. Finally suppose for simplicity that we observe a p x l vector on independent individuals and the components are represented by the nodes of a concentration graph (Lauritzen, 1996; Cox and Wermuth, 1996). Then if the set C of nodes separates the sets A and B we have that YA is conditionally independent of Yβ given Yc, in a natural notation associating random variables with the sets of nodes. It follows that the likelihood of the full data can be factorized, for example in the forms (6)

LA\c(θ; y)LB\C(θ', y)Lc{θ; y) = LAC{β\ y)LBC(θ;

y)/Lc(θ;

y).

We call these factorizations concentration-graph based. For such factorizations to be directly useful in statistical analysis they need to enjoy one of the other properties listed above. Cox and Wermuth

168

D.R. Cox

(1998) discuss the connection with parameter-based factorizations and the implications for the analysis of mixtures of discrete and continuous variables.

3

Some properties and a further example

We first give another example of a factorization that is asymptotically only first-order valid. Example 6. Azzalini's (1983) first-order factorization of a recursive likelihood can be written in a condensed notation as 1,2/2)

(7)

• If r |r-l(0; 1/r, Vr-l)

£ n |n-l(0; I/nj 1/n-l)

xΠL^x,...^; yΓ1 y^/L^β;

yr, yr_χ),

where y^r~^ = (3/1,... ,y r _i). The final term can be written in the more enlightening form, in an even more condensed notation,

We may compare this with Besag's (1977) pseudo-likelihood in which the conditioning in the first factor is on all other values, this often being more appropriate for spatial as contrasted with temporal processes. We now factorize the likelihood corresponding to the two lines of (7). If and only if the series is a first-order Markov process the second factor is identically one. For some non-Markov processes the first factor contains enough dependence on the full vector θ to allow estimation from it alone. In general, however, the condition for second-order asymptotic validity is not satisfied. To verify first-order asymptotic validity we have to check that (9)

E(Vh)=0

and for second-order validity that (10)

Γ

τ

JS(V/i V /i) + E{VV h)

= 0.

Here l\ = /i(0i; Y), the gradient operator, V, is with respect to φ\ and expectations are evaluated under the model as originally specified. A parameter-based factorization satisfies both conditions. It is easily checked that Examples 5 and 6 satisfy the first condition but the second only in degenerate cases. In both these examples the evaluation of the asymptotic variance of φ\ is equivalent to finding the variance under one model of a maximum likelihood estimate calculated under a different model (Cox, 1961) leading to the socalled sandwich estimate (Royall, 1986). Study of higher-order asymptotics associated with these factorizations will not be attempted here. It would, for example, be possible to define asymptotic third-order validity as the satisfying of standard identities for log likelihood derivatives of up to order three.

Likelihood Factorization

4

169

Combination of information

Suppose that we have a factorization into q factors each second-order asymptotically valid. If the factorization is also parameter-based, separate inference about component parameters can proceed directly. In general, however, there will be common component parameters and combination of information from different factors is required. As always, where merging distinct sources of information mutual consistency should be checked, at least informally. Once separate maximum likelihood estimates and their associated observed information matrices have been found, noniterative combination by generalized least squares is possible without loss of asymptotic efficiency, i.e. with an error O p (l/n), where n is a notional sample size. To see this we reparameterize in terms of components of the original p x 1 parameter vector θ. Let As(s = 1,..., q) be a ps x p matrix specifying which components of θ are estimated by the components of 0 S , i.e. asymptotically

E(φs)=Asθι

(11)

each row of As will contain a single one and p — 1 zeros. Then if j 3 is the observed information matrix associated with L s , the log likelihood functions are equivalent to quadratic forms centred on the maximum likelihood points and the system is equivalent asymptotically to a least squares estimation problem with data (0i,..., φq) and with

M \

\ (12)

θ

E \ Φq /

\

and covariance matrix the block diagonal matrix

so that the estimate

with cov(6ί) = ι

differs by Op{n~ ) from the maximum likelihood estimate θ.

170

5

D.R. Cox

A missing value problem

Consider two binary variables (Yi,!^) each taking values 0,1 and with (13)

=P(Y1=i,Y2=j).

πij

It is convenient to think of the variables as defining the rows and columns of a 2 x 2 table and to write (14)

λi = P(y< = i)

for the marginal probabilities. Suppose that the parameter of interest is the log odds ratio (15)

ψ = log{(πiiπoo)/(7Γioπoi)};

we suppose that the parameters π ^ are expressed implicitly in terms of Now suppose that there are three types of observation, complete observations on the pair (Yi, Y2), observations in which only Y\ is observed and observations in which only Y2 is observed. We suppose that component observations are missing completely at random so that the same set of TΓJJ prevail throughout. The issue for discussion is: do the observations with missing components contribute information about ψ? This is not the same as, although indirectly related to, the much-discussed question of conditioning on the margins of a 2 x 2 table. The full log likelihood can be written (16)

lτ =

corresponding to the three types of data. Now direct calculation shows that $T Φ Φ(12)

the latter being the log cross-product ratio from the full data, i.e. on the complete contingency table. We show, however, that

(17)

1

Ψτ-Ψ(n)=θp(n- ),

so that any information from the incomplete data is asymptotically negligible. We prove this, essentially without detailed calculation, by appeal to the asymptotic quadratic form of the three log likelihoods, essentially of the form λi ? λ 2 j (i2) - λ 2 ; Γ ( 1 2 ) ) + (18)

™Q(i)(λi(i) - λ ( 1 );Γ(!)) +n<3( 2 ) (λ 2 ( 2 ) - λ 2 ;Γ( 2 ) ),

Likelihood Factorization

171

where n is a notional total sample size, and the standardized covariance matrices Γ can be regarded as known for asymptotic inference. They can be obtained theoretically or derived via the observed information matrices. The estimation problem is thus formally equivalent to a generalized least squares problem with observational vector (V>(12) 5 λ 1 ( 1 2 ), λ2(i2), λ 1 ( 1 ), λ2(2)) coming from the three likelihood factors. The asymptotic covariance matrix is in fact diagonal except for the element cov(λi(i2), λ2(i2)) = {eφ - I)λiλ 2 /n ( i 2 ),

(19)

where n^) ι s the number of complete observations. The position of the zeros in the asymptotic covariance matrix is such that on applying the method of generalized least squares that to this order the maximum likelihood estimate of φ is φ(\2) but that for the other parameters, and therefore for any parametric function other than φ itself, combination of information from the factors is needed. Note in particular that except when φ — 0 efficient estimation of say λi is not obtained merely by pooling all information on Y\. The asymptotics involved in this are essentially that the three sample sizes all increase at the same rate. It can be shown, however, that even if the numbers of incomplete observations increase much more rapidly than the number of complete observations the asymptotic efficiency of φ{\2) is retained. We shall not consider possible generalizations, for example to higher dimensional contingency tables.

6

Discussion

Likelihood plays a central role in all formal approaches to parametric analysis. Correspondingly maximum likelihood occupies a central role in asymptotic theory. The notion that it may be beneficial, or indeed in some cases essential, to use some form of likelihood different from that based directly on the distribution of the full data stems from Bartlett (1937). Conditional and marginal likelihood were studied explicitly by Kalbfleisch and Sprott (1970) and a generalization, partial likelihood by Cox (1975). Quasi-likelihood was introduced by Wedderbrun (1974) and some optimality properties were shown by McCullagh (1983); see Example 5. Some other important papers are referred to in the text. Numerous variants have been proposed many based on marginalizing or conditioning with respect to well-chosen statistics or using inefficient estimates of some components while retaining asymptotic efficiency for the components of interest. The incorporation of all these into a systematic theory would be welcome.

172

D.R. Cox

Acknowledgements. It is a pleasure to thank Els Goetghebeur for asking the question leading to Section 5 and hence to much of the rest of the paper and the referees for some helpful suggestions.

REFERENCES Azzalini, A. (1983). Maximum likelihood estimation of order m for stationary stochastic processes. Biornetrika 70, 381-387. Barndorff-Nielsen, O.E. (1978). Chichester: Wiley.

Information

and exponential

families.

Bartlett, M.S. (1937). Properties of sufficiency and statistical tests. Proc. Roy. Soc. (London) A, 160, 268-282. Besag, J.E. (1977). Efficiency of pseudo-likelihood estimation for simple Gaussian fields. Biometrika 64, 616 - 618. Cox, D.R. (1961). Tests of separate families of hypotheses. Proc. 4th Berkeley Symp. 1, 105-123. Cox, D.R. (1975). Partial likelihood. Biometrika 62, 269-276. Cox, D.R. and Wermuth, N. (1996). Multiυariate dependencies. London: Chapman and Hall. Cox, D.R. and Wermuth, N. (1998). Likelihood factorizations for mixed discrete and continuous variables. Scand. J. Statist. 26, 209-220. Kalbfleisch, J.D. and Sprott, D.A. (1970). Applications of likelihood methods to problems involving large numbers of nuisance parameters (with discussion). J.R. Statist. Soc. B 32, 175-208. Lauritzen, S.L. (1996). Graphical models. Oxford University Press. McCullagh, P. (1983). Quasi-likelihood functions. Ann. Statist.

11,59-67.

Royall, R.M. (1986). Model robust confidence intervals using maximum likelihood estimators. Int. Statist. Rev. 54, 221-226. Wedderburn, R.W.M. (1974). Quasi-likelihood functions, generalized linear models and the Gauss-Newton method. Biometrika 61, 439-447. Yates, F. (1940). The recovery of interblock information in balanced incomplete block designs. Ann. Eugenics 10, 317-325. D E P T . STATISTICS AND NUFFIELD COLLEGE OXFORD DX1

INF

UK David. Cox@nuf. ox. ac. uk

TRIMMED SUMS FROM THE DOMAIN OF GEOMETRIC PARTIAL ATTRACTION OF SEMISTABLE LAWS1

SANDOR CSORGO AND ZOLTAN MEGYESI

Universities of Michigan and Szeged and University of Szeged

We show that all possible moderately trimmed sums from a domain of geometric partial attraction of a semistable law G are asymptotically normal, along the whole sequence of natural numbers, under the necessary condition that the Levy functions of G do not have flat stretches. We also show that asymptotic normality prevails still along the whole sequence at least for suitably chosen moderately trimmed sums from such a domain for every G. It then follows that after removing any moderately trimmed sum from the middle the remaining sums of extreme values still produce every semistable limiting distribution G that the original full sums have, along exactly the same geometrically growing subsequences of the natural numbers. AMS subject classiήcations: Primary 60F05; secondary 60E07. Keywords and phrases: Trimmed sums, domains of partial attraction, semistable laws, asymptotic normality.

1

Introduction

Let XL, X2,. be a sequence of independent and identically distributed random variables, and for each natural number n G N consider the order statistics Xι,n < < Xn,n pertaining to the sample X\,..., Xn. Trimmed sums ΈrZi+i Xj,n for Z, ra E N, l + m < n, are the initial basic objects in statistical theories of robust estimation, so it is not surprising that there has been considerable interest in the investigation of their asymptotic distribution. The large literature on a number of versions of the problem may be traced back from our references; see in particular the collection edited by Hahn, Mason and Weiner (1991). Here we deal only with trimming according to natural order, as in the sums Σ?=/+i ^j>> a n d n o t w ^ h the case when trimming is done with respect to ordering the moduli \Xi\,..., \Xn\ of the observations. Work partially supported by the NSF Grant DMS-9625732..

174

Sandor Csδrgδ and Zoltan Megyesi

Subsequent to specific results on asymptotic normality, Stigler's (1973) theorem completely solves the problem of the asymptotic distribution for the classical trimmed mean n-{n-\nβ\)

Σ j=[nα\+l

Xj,n/(lnβ\-lnα\)=

\nβ\

£

Xj,n/([nβ\ - [nαj),

j=[nα\+l

where 0 < α < β < 1 and, with Z standing for the set of integers, [x\ = max{r E Z: r < x} is the integer part of a real number x E M: In this case of heavy trimming enough extreme values are discarded so that, with suitable centering and norming, the remaining mean has an asymptotic distribution as n -> oo for every underlying distribution function F(x) = Ψ{X\ < x}, xGR, where the basic probability space is denoted by (Ω, A, P). Introducing the associated quantile function Q(s) = inf{x E R: F(x) > s}, 0 < s < 1, this asymptotic distribution is normal if and only if Q is continuous at both a and β. A proof of the general result, different from Stigler's, and one that shows his theorem to be a boundary case of asymptotic distributions for moderately trimmed sums discussed below, is given by Csόrgό, Haeusler and Mason (1988b); see also Cheng (1992) for further elaborations. At the other trimming extreme, it is conceivable that for fixed pairs of positive integers I and m the existence and nature of asymptotic distributions of the lightly trimmed sums Snk(l,m) = Σ"=/+i ^ > f c , generally along subsequences { n ^ } ^ C N, are closely connected with those of the limiting distributions of the whole untrimmed sums Snk = Snfc(0,0) = Σj=i Xj (Asymptotic distributions for any of the sums here and in the sequel are always meant with suitable centering and norming and all infinite subsequences of N are assumed unbounded throughout) Indeed, it was shown in their Corollary 6 by Csόrgδ, Haeusler and Mason (1988a) that Snk converges in distribution along some {rik} to a nondegenerate random variable, in other words, F is in the domain of partial attraction of some infinitely divisible distribution, if and only if Snfc (Z, m) converges in distribution to nondegenerate random variables for every pair (/, m), along the same {n^}. The limiting distributions of the latter are some "trimmed" forms of a special representation of an infinitely divisible random variable, the distribution of which is the limiting distribution of the former; the representation is given in the next section. One may conjecture that it is sufficient to require the distributional convergence of 5njb(/,m) for a single pair (/,m) G N2 to achieve the same conclusion for 5 nfc , and hence also for all (Z,m) E N 2 , along the same {n^}. For the whole sequence {n} = N this was proved by Kesten (1993), in which case the conclusion is that F is in the domain of attraction of a stable law. The general subsequential version is still open.

Semistable Trimmed Sums

175

Perhaps the most interesting case, the topic of the present note, is that m of moderately trimmed sums Sn{ln,πιn) = Σ?=ί +i Xj,m where (1.1)

/

ln -> oo, — -» 0 n

and

7T7

mn -> oo, — - —> 0 n

a s n —> o o .

The first deeper result is due to Csόrgδ, Horvath and Mason (1986), who proved that if the full sums Sn have a nondegenerate asymptotic distribution along the whole {n} = N, i.e. if F is in the domain of attraction of a (normal or nonnormal) stable law, then with ln = mn and suitable centering and norming sequences Sn(mn,mn) is asymptotically normal as n —> oo. Csόrgδ, Haeusler and Mason (1988b) then determined the class of all possible asymptotic distributions for Sn(ln,mn) along all possible subsequences {rik}, together with necessary and sufficient conditions for the convergence in distribution of Snk(lnk,mnk) as k —> oo. To formulate at least the condition for asymptotic normality, define f o r O < θ < l — £ < 1 , l-ί

/

(1.2)

/ Js

rl-t

[min(u, v) - txv] dQ(tί) dQ(v)

= β Q ( s ) + tQ2(1 -t)+

Q (u) du Js

- \sQ(s) + tQ(l -t) + f'*

Q(u) du} ,

a basic function in Csδrgo, Haeusler and Mason (1988a,b). For given sequences {ln} and {mn} set (1.3)

an(ln,mn)

= y/nσ[ — ,l \n

), n J

and introduce the two sequences of functions 2 '

ψl,n{X) = < X < OO,

and -oo
< X < OO.

176

Sandor Csδrgδ and Zoltέn Megyesi

Also, let —> denote convergence in distribution and let Z be a standard normal random variable. According to Theorem 4 of Csόrgδ, Haeusler and Mason (1988b), for sequences {ln} and {mn} satisfying (1.1), there exist centering and normalizing constants Cn G R and An > 0 such that l A-n [Sn{ln-> ran) — Cn] —> Z as n -» oo if and only if (1.4)

lim ψjnix) = 0 for every x E l , j = l,2, n—>oo n

in which case Cn = c n (/ n ,m n ) := n / i n ± i Q{u)du and Λn = αn(ln,mn) work. The subsequential version of this result is also true. If at least one of the m functions φj,n{ ), or one of the renormalized functions αn(ln,mn)φj,n( )/An for some An > 0 for which αn{ln,mn)/An —> 0, j = 1,2, converges to a nonzero function either along the whole {n} or along a subsequence, then extra terms appear in the limiting random variable so that the asymptotic distribution, typically obtained along a further subsequence, is no longer normal; it does not even have a normal component in the renormalized case. The conditions appearing are optimal; for the precise statements the reader is referred to Csόrgδ, Haeusler and Mason (1988b, 1991b). Griffin and Pruitt (1989) rederived this theory by a different method, obtaining the conditions and the description of limiting random variables in alternative forms, with numerous additional observations. While the "asymptotic continuity" condition (1.4) solves the problem of asymptotic normality of moderately trimmed sums completely from a general mathematical point of view, its probabilistic meaning is not so clear until it is tied to better understood conditions that govern the asymptotic distribution of the entire untrimmed sums. Indeed, it was pointed out by Csόrgδ, Haeusler and Mason (1988b) and then by Griffin and Pruitt (1989) that if F is stochastically compact, meaning that the full sums are stochastically compact in the sense that there exist sequences of constants bn G R and dn > 0 such that every subsequence of N contains a further subsequence along which [Sn — bn]/dn converges in distribution to a nondegenerate random variable, then the sequences of functions {ψj,n(')}^Lι a r e uniformly bounded, j = 1,2, and hence the sequence S^(ln,mn) := [Sn(ln,mn) cn{ln,mn)]/αn(ln,mn) of centered and normed trimmed sums is also stochastically compact for any pair (/n?^n) of sequences satisfying (1.1). However, nonnormal subsequential limiting distributions do arise in this case. Thus, to date, the only explicitly determined family of underlying distributions for which 5*(m n ,m n ) is known to be asymptotically normal along the whole N for every sequence {mn} satisfying (1.1) is the family of those F that are in the domain of attraction of a stable law [Csόrgδ, Horvath and Mason (1986)], and the only explicit family for which 5*(ί n ,m n ) is known

177

Semistable Trimmed Sums

to be asymptotically normal for every sequence {(/n, mn)} of pairs satisfying (1.1) is the subfamily attracted by not completely asymmetric stable laws [Griffin and Pruitt (1989)]. The question arises whether there is a probabilistically meaningful larger class of distributions, necessarily within the class of stochastically compact distributions, which would respectively contain the families above and for which the same conclusions for the asymptotic normality of trimmed sums would still hold true. A feature of the phenomenon would of course be that the full sums, [Sn — bn]/dn, would no longer converge in distribution themselves along the whole {n} = N. The aim of this paper is to show that a larger class of distributions within the class of stochastically compact distributions does indeed exist with these properties: it is a proper subfamily of the family of distributions in the domain of geometric partial attraction of semistable laws. In the next section we describe this family of distributions, while Section 3 contains the new results and their proofs. 2 Semistable distributions and their domains of geometric partial attraction Let Φ be the class of all non-positive, non-decreasing, right-continuous functions φ( ) defined on the positive half-line (0, oo) such that /ε°° ψ2(s) ds < oo for all ε > 0. Let 2£f', E2 , -.., j = 1,2, be two independent sequences of independent exponentially distributed random variables with mean 1. With their partial sums F n = E± + + En as jump points, n E N, consider the standard left-continuous independent Poisson processes Nj(u) := ΣΪZLi I{YnJ>} < u), 0 < u < 00, j = 1,2, where /(•) is the indicator function. For a function φ G Φ, consider the random variables

~ JiΓ

- s]dφ(s)

where the first integrals are almost surely well defined, by the condition that φ G Φ, as improper Riemann integrals. For ψ i G Φ and φ<ι G Φ, consider the constant

J

1+ ^ W '

let Z be a standard normal random variable such that iVi( ), Z, and N2( ) are independent, and for a finite constant σ > 0 finally introduce the random variables (2.1)

178

Sandor Csδrgδ and Zoltan Megyesi

'ψ2), the latter of which by Theorem 3 in Csδrgό, Haeusler and Mason (1988a) has characteristic function

, x (2.2)

1

2

7-cx5\

1 + x2/

l

+ for all ί E M, where L(x) = inf{s > 0: ψι{s) > χ } for α: < 0 and R(x) — — inf{s > 0: ^2(5) > —x} for x > 0. Here L( ) is left-continuous and nondecreasing on (—oo,0) with L(—oc) = 0 and R( ) is right-continuous and non-decreasing on (0,00) with R(oo) = 0, and f^ε x2dL{x)+^ x2dR(x) < 00 for every ε > 0 since ψutfa G Φ. Thus V(^i,^2,σ) is infinitely divisible by Levy's formula [see e.g. in Gnedenko and Kolmogorov (1954)]. Conversely, given the right side of (2.2) with L( ) and R( ) having the properties just listed, the variable W ^ i , ^ ? ^ ) has this characteristic function again if we choose ^i(s) = inf{x < 0 : L(x) > s} and ^2(5) = inf{x < 0: -R(-x) > 5}, 5 > 0, for then ^ 1 , ^2 G Φ. Thus the class I of all nondegenerate infinitely divisible distributions can be identified with the class {(Ί/>I,^2,CΓ) φ (0,0,0) : ^1,^2 G Φ,σ > 0} of triplets. Then F being in the domain of partial attraction of a G = Gψιψ2Ί(T G J , written F G Bp(G), means that there exists a subsequence {kn}™=1 C N and centering and norming constants Ckn G R and Ajςn > 0 such that ί k~

λ

v where a convergence relation is meant to hold as n -> 00 unless otherwise specified and Gφli7/,2iσ is the distribution function of the random variable V(^i,^2,σ) from (2.1); the characteristic function of V(VΊ>^2,σ) — θ(Ψι 1Ψ2) is in (2.2). By classical theory [Gnedenko and Kolmogorov (1954) or Corollary 5* in Csόrgό (1990)] this happens for {kn} = {n} = N if and only if either ( ^ I J ^ J G Γ ) = (0,0, σ) for some σ > 0, in which case F is in the domain of attraction of the normal distribution, written F G D(2), or W>i?^2j0") = (mi^ α ,m2'0 Q ,O) for some constants α G (0,2), mi,m2 > 0, m\ + 777,2 > 0, where ψα(s) = — s" 1 /", s > 0, in which case F is in the domain of attraction of a stable distribution of exponent α, written F G B(α). α (The superscript α in i/> , and in t/>f and ^2 beginning with (2.4) below, is meant as a label, not as a power exponent.) The normal being the stable law of exponent 2, let S denote the class of all stable laws. Levy (1937) introduced the class 5* C I of semistable laws by extending a defining property of stable characteristic functions and, as translated into

179

Semistable Trimmed Sums

the framework of the present description of infinitely divisible laws, showed that Gψliψ2i(T G S* if and only if either (^1, ^2 5 cr) = (0,0, σ) for some σ > 0, giving the normal distribution as a semistable distribution of exponent 2, or (tl>uih,σ) = (ψf,ψξ,0), where (2.4)

α

1α

ψf(s) = Mj(8)tl> (8) = -Mj(8)8- ' ,

s>0,

j = l,2,

for some α G (0,2), defining a semistable distribution of exponent α G (0,2), where M\ and M<ι are nonnegative, right-continuous functions on (0, oo), either identically zero or bounded away from both zero and infinity, such that Q M1+M2 is not identically zero, the functions Mj( )^ ( ) are nondecreasing, j = 1,2, and Mj(cs) — Mj(s) for all 5 > 0, j = 1,2, for some constant c > 1; the latter property will be referred to as multiplicative periodicity with period c. For α G (0,2), Levy's original description of the property in (2.4) in terms of L and R in (2.2) is that there exist nonnegative bounded functions ML( ) on (—00,0) and MR( ) on (0,00), one of which has a strictly positive infimum and the other one either has a strictly positive infimum or is identically zero, such that L(x) = ML(x)/\x\α, x < 0, is left-continuous and nondecreasing on (—00,0) and R(x) = —Mjι(x)/xα, x > 0, is rightcontinuous and nondecreasing on (0,00), while Mχ/(c1/αa:) = ML(X) for all x < 0 and Mβ(c 1 / α x) = MR(X) for all x > 0, for the same period c > 1. Because of the inversions given above, the two descriptions are equivalent. The realization of a tangible significance of 5* D S starts with a remark of Doeblin (1940), without any elaboration or, for that matter, even a precise statement, to the effect that semistable laws arise in the limit in (2.3) if the normalizing constants A*n satisfy a geometric growth condition. Thirty years later, Shimizu (1970) and Pillai (1971) came close while Kruglov (1972) and Mejzler (1973) fully achieved that realization, all four of them acting independently of one another. It turned out that the following Characterization Theorem is true: If (2.3) holds along a subsequence {kn} C N for which (2.5a)

liminf - ? — = c for some c G (1,00),

then the distribution Gψuψ2tσ of V(ψι,ψ2,σ) is in 5* such that, in the case when the exponent of Gψλψ2^σ = Gφa^a^ is α < 2, the multiplicative period of the functions M\ and M2 in (2.4) is the c from (2.5a). Conversely, for every Gψltψ2tσ G 5* there exists an F such that if ΛΊ, X2,. are independent random variables with the common distribution function F, then there exists a subsequence {kn} C N such that (2.5b)

lim -£±^ n->oc

k

=

c

for some c G [1,00)

180

Sandor Csδrgδ and Zoltan Megyesi

and (2.3) holds along {kn}. An equivalent version of this theorem, in terms of the Levy type description of 5* was proved by Kruglov (1972) and Mejzler (1973), while the present version was obtained by Megyesi (2000) with an independent proof within the framework of the 'probabilistic' or 'quantiletransform' approach of Csδrgό, Haeusler and Mason (1988a,b; 1991a,b) and Csδrgό (1990) to domains of attraction and partial attraction. For G = Gψuψ2i(T G 5*, we say that F is in the domain of geometric partial attraction of G with rank c > 1, in short F G B^(G), if (2.3) holds along a subsequence {kn} C N satisfying (2.5b). Of course, the geometric n n subsequence kn = [c j, the integer part of c , is unbounded and satisfies the (quasi)geometric growth condition (2.5b) if c > 1. Recalling that (Ψuψ2,σ) Φ (0,0,0) for G = Gψuψ2tσ G S*, define c = c(Go,o,σ) = 1 for any σ > 0 and c = c(G^«^«j0) = inf{c > 1: Mj(cs) = Mj(s), s > 0, j = 1,2}, the minimal common period c of the factor functions M\ and M2 in ψf and ψξ in (2.4) for a G (0,2). Thus c = c(G) is defined for all G G S*. It turns out for the whole domain B gp (G) := U c > 1 1 % ; (G) of geometric partial attraction of G G S* that B g p (G) = ΠmeN^gP™^) = ^gϊ>(G). Also, if c(G) = 1 for G G <S*, then G G S and B g p (G) = B(G), the domain of attraction of the stable G. In other words, if B(5) := \JσesΌ(G) = Uo<α<2ID>(α) is the classical domain of attraction and IDgp(G) := UGe^^gp(^) ι s ^ e domain of geometric partial attraction of a class 5 c 5 # , then 0 g p (5) = B(5). Some of these results were first proved by Mejzler (1973), all of them and related other observations are obtained by Megyesi (2000). The first characterization of an F G Dgp(«ί>*) was obtained by Grinevich and Khokhlov (1995). However, besides the fact that it contained an error, this characterization is in terms of the norming factors Akn in (2.3) and the tails of F, and so it is not useful when trying to apply the criterion (1.4) to trimmed sums. The following alternative characterization is due to Megyesi (2000). Consider a subsequence {kn}<^>=1 C N satisfying (2.5b). If c = 1 in (2.5b), then put 7(s) = 1 for every 5 G (0,1). If c > 1, then the sequence {kn} is eventually strictly increasing to 00. Hence, for all s G (0,1) small enough there exists a uniquely determined fcn*(s) such that k~h^ < s < k~},\v For any such s we define 7(s) = sfcn*(s), so that for any fixed ε > 0 and all 5 G (0,1) small enough we have 1 < 7(5) < c + ε for the limiting c > 1 from (2.5b). In particular, for any sequence sm > 0 for which \imm-+oosm = 0, the limit points of the sequence {7(s m )}^ = 1 are in the interval [l,c]. Let Q+(') denote the right-continuous version of the quantile function Q( ) of the underlying distribution function F( ). Since D gp (G) = D(G) for a normal G G 5*, we only have to describe the domain of geometric partial attraction of nonnormal semistable laws, for which the Domain Theorem is this: If GφaφaQ G 5* is semistable with exponent α G (0,2), so that ψf and ψξ

181

Semistable Trimmed Sums

satisfy (2.4), and F e Bgp(Gψ<*ιφ«β) such that (2.3) holds for V(ψ?,ψ§,0) and a subsequence {kn}^=ι C N satisfying (2.5b), then for all s G (0,1), (2 6)

Q+{S)

=

Q(l-8)=

s

£

W [ M i ( 7 ( ^ ) ) + Λi(*)]

and

8-VH{8) [M2{Ί{S)) + h2(S)]

for some a G (0,2), where έ( ) is a right-continuous function, slowly varying at zero, and the errors hi and Λ2 are right-continuous functions such that if Mj is continuous, then lims^o hj(s) = 0 for the corresponding hj, while if Mj has discontinuities, then the corresponding hj(s) may not go to zero as s I 0 but limn^oo hj(t/kn) = 0 for every continuity point t > 0 of Mj, j = 1,2. Conversely, if for the quantile function pertaining to F the equations in (2.6) hold with the properties of I and of h\ and Λ2 just described, for some a G (0,2) and functions M\ and M<ι satisfying the properties described at (2.4), and for 7( ) determined by a given subsequence {kn}™=1 C N satisfying (2.5b), then F G ^gp(G^^fi) for the φf and ψ% given by (2.4), and, in particular, the relation (2.3) can be specified as

£(l/kn)

[j=

Finally we note that if F G ^gpiG^^β) for some a G (0,2), so that (2.6) holds with all the properties of the ingredients described above, then it is easy to see that da'^iis) ι

C- lH{)

< \Q+(s)\ < DιS-ιl"ί{s) < |Q(1

and

-s)\<

for all 5 > 0 sufficiently small, where 0 < C\ < D\ < 00 and 0 < C2 < D2 < 00 are constants such that C\ + C2 > 0 and Cj = 0 if and only if Dj > 0 can be chosen as small as we wish, which happens if and only if Mj(-) = 0, J = 1,2. 3 Asymptotic normality of moderately trimmed sums from Bgp(S*) Our main result is Theorem 3.1 Suppose that F G D gp (G) for some nondegenerate semistable law G = Gψuψ2tσ such that both φ\ and Ψ2 are continuous on (0,00). (i) If neither ofψi and Ψ2 is identically zero, then for any two sequences {ln}™=ι and {mn}^=ι of positive integers satisfying (1.1),

( 3 1 )

182

Sandor Csόrgδ and Zoltan Megyesi

where α n (/ n , mn) is as in (1-3) and Z is a standard normal random variable. (ii) If at least one ofψi and Φ2 is identically zero, then (3.1) holds true for any two sequences {/n}^Li and {mn}^L1 of positive integers satisfying (1.1) such that

(3.2)

0 < liminf — < Km sup — < 00. n^HX)

rnn

Π-KX5

τnn

In this theorem, the distribution G is either normal, i.e. G = (?o,o,σ for some σ > 0, or G — ί?ψ«,ψ«,θj a semistable law of exponent α E (0,2) with continuous ψf and ψ% satisfying (2.4). In the first case, the continuity condition is trivially satisfied and part (ii) for this case is just a restatement of part of Theorem 1 in Csόrgδ, Horvath and Mason (1986) when ln = mn. In the second case, the two parts (i) and (ii) here extend results of Csόrgδ, Haeusler and Mason (1988) and Griffin and Pruitt (1989) mentioned in the introduction. By (2.4), the theorem's continuity condition is nothing but the requirement of continuity of the corresponding functions Mi and M2. This condition cannot be dropped in general as the example of the St. Petersburg game shows, where the underlying distribution is in the domain of geometric partial attraction of a semistable law with exponent 1 and Theorem 3.2 of Csόrgδ and Dodunekova (1991) shows that nonnormal limits do arise for moderately trimmed sums along subsequences of N. The generalized St. Petersburg games considered by Csόrgδ and Simons (1996) in a different context and their symmetrized versions may serve to show the same for all exponents α E (0,2). In terms of the Levy functions L and R in (2.2), we see that a nonzero ψf (or φξ) 1S continuous, or equivalently the corresponding Mi (or M2) is continuous if and only if L (or R) does not have flat stretches in the sense that it is not constant on intervals with positive length. We emphasize that even though (2.7) holds for the full sums only along a subsequence satisfying (2.5b), the convergence in (3.1) takes place along the whole N. If the continuity condition is violated, we still have an existence result along the whole N. Theorem 3.2 If F G Όgp(G) for a nondegenerate semistable law G, then there exist two sequences {/n}ί£Li and {mn}^Lι of integers satisfying (1.1) such that (3.1) holds. With some extra work the proof can be modified to allow the choice ln = mn. Also, if G = Gψ^ψβ^ for some exponent α G (0,2), neither of ψf and Ψ2 ι s identically zero and ψf is continuous, then there is an {m n }^_i satisfying (1.1) such that (3.1) holds for every {ln}^Lι satisfying (1.1); an analogous statement is true when ψ% is continuous. Our last result extends Theorem 3 of Csόrgδ, Horvath and Mason (1986)

Semistable Trimmed Sums

183

and demonstrates that asymptotic semistability in (2.7) is determined only by arbitrarily small moderate portions of upper and lower order statistics in the sample. T h e o r e m 3.3 IfF € BgP (G) for a nonnormal semistable law G = fy^> of exponent a G (0,2), so that (2.7) holds along a subsequence {kn}%L1 C N satisfying (2.5b), then, for the slowly varying function £(-) from (2.6) and (2-7), (3-3)

(34) where —> denotes convergence in probability, and

where the independent random variables W\(ψf) and W^iΦx) are given at f2.1^, and so

for any two sequences {/n}£Li and { m n } ^ = 1 of positive integers satisfying (1.1). The general theory in Csόrgό, Haeusler and Mason (1988a, 1991b) and Csόrgδ (1990) ensures the existence of sequences {/n} and {mn} satisfying (1.1) for which these statements hold, the point of Theorem 3.3 is that they hold for all such sequences. If Mj = ψ* = 0, which is allowed in (2.7) and in Theorem 3.3 above for one of the j , then of course Wj(0) = 0. A more general version of Theorem 3.3, in which a fixed number of the smallest and the largest extremes may be discarded from the sums in (3.3) and (3.5) is also true; the way in which the centering sequences and the limiting random variables should be changed in (3.3) and (3.5) for this version is clear from the general scheme in Csόrgδ, Haeusler and Mason (1988a), Csδrgδ (1990),

184

Sέndor Csδrgδ and Zoltan Megyesi

or Megyesi (2000). The formulation of Theorem 3.3 above suits well the genuinely two-sided case. In the completely asymmetric case when one of Φι and ψξ is identically zero, a somewhat stronger statement can be made, even in the more general version with possible light trimming: see the end of the proof of Theorem 3.3 for this in the present case of full extreme sums. Turning now to the proofs and recalling the notation in (1.2) and the statement in (2.8), Theorem 3.1 requires the following Lemma 3.4 If F G Bgp(G-0<*,ψαfi) for a semistable Gψ^ψ^o of exponent α e (0,2), then (i) there exist some constants K\,Ki G (0, oo) such that

Kλ < limiπf " ( ' -

40

1

S

1

- ' ) < limsup

-I^2(s)

-

40

σ ι

l^A 2

s «l {s)

< K, . ~

(ii) ifCi > 0 in (2.8), then there exist some constants κ[l\κ^ such that

P

liminf « y / 2 > < limsupσ \

€ (0, oo)

$\

and if C2 > 0 in (2.8), then there exist some constants K\ ,K^ G (0, ex)) such that

K?< liminf ~

40

Proof We follow the proof of Lemma 1 in Csόrgό, Horvath and Mason (1986). Obviously the inequalities in (2.8) directly imply that

C?+ Cl < liminf -VM + W-') - 40 s^ϊPis)


Also from (2.8), similarly as at (3.11) and (3.12) in Csδrgδ, Horvath and Mason (1986),

(3.6)

Kj liminfSS " ?? ((°°' '** << limsup limsup '' • 'f" Wf Wd d"" < K>2 j <
i'-i£2(s)

~

iθ

s'-ip(s)

~

185

Semistable Trimmed Sums

for some constants K^Kζ

G (0, oo) and

3.7)

(

; S2-U{s)

40

0

Using these four relations in the second formula in (1.2), the inequalities in (i) follow. The symmetric two statements in (ii) are obtained in a similar fashion. Considering the first, for example, the first pair of inequalities in (2.8) imply that C\ < liminf - ^ L < l i m s u p sQ*ω < D2 40 sι-«i2(s) 40 sι-«i2{s)

s\Q{s)\ = 40 S2-U(s)

a n d l i m

and (3.6) and (3.7) remain true by the same argument if 1 — s in the upper limits of the integrals is replaced by 1/2. • Proof of Theorem 3.1 To prove part (i), consider any two sequences {ln}^=ι and {mn}'^)=1 satisfying (1.1) and introduce the "renormalized" halfsided functions

(x e R), the original functions φι,n( ) and ψ2,n(') being given between (1.3) and (1.4), where a>i,n(Q = Vnσ( — , - ) and a2,n(mn) = y/nσ[ - , 1 - — ). \n 2) \2 nJ Since none of φι = ψf and ψ\ = ψξ is zero anywhere, 0 < a < 2, it follows from (2.6) and (1.2) that aι,n{ln),a2,n{mn) > 0, and so the definitions of the renormalized functions are meaningful for all n large enough and, of course, αi,n(O,α2,n(raτι) < a>n(l"n,™>n)' Hence, to prove (1.4), it suffices to show that

lim φ[n]Sx) = °

(3.8)

a n d

Jl™, φ$™n(x) = °

for

To deal with φ^]n(x) at any fixed x e M, note that by the domain theorem at (2.6),

n

+ x

)

n J

+ x

\n

γ

e

n J

(

+ x \n

)

nJ

l

\Λ/Γ ( (In ^ >fa\\>h ( n , Vk x Mi 7 — + x +hι[ — +x

[

\\n

n JJ

\n

n

Sandor Csόrgδ and Zoltan Megyesi

186 and

for all n large enough. We substitute these into the formula for ψn \ (x) through the formula given for φ\tn(x). Using then the fact that K :=

< lim inf ... < limsup

σ{ln/n,1/2) σ{ln/n,1/2) ^ —{—j——-— <

by the first statement of Lemma 3.4(ii), for all n large enough we obtain

+ Λi( —

where

s/ΓJ by the slow variation of ^( ) at zero. Since Mi( ) is bounded, we see, therefore, that the first convergence in (3.8) will follow if we show that (3.9)

+ Mtn(*))] " [Mi{-y(an)) +

0,

where s n = U " 1 -> 0 and tn(a;) = lnn~ι + xy/ΐnΠ'1 = sn [1 + xίή1^2] -> 0. Since also, as a result of our continuity assumption, lims^o h\(s) = 0 by the domain theorem at (2.6), and tn(0) = sn of course, for (3.9) it suffices to show that (3.10)

υn(x) := |Mi(7(tn(a?))) -

0 for each x φ 0.

Let c > 1 be the limit in (2.5b) for the sequence {kn}^=ι which defines 7( ) preceding (2.6). We may and do assume that c > 1 since in the case of c = 1, when F E D(α) for the given α G (0,2) at hand and Mχ( ) is a constant

187

Semistable Trimmed Sums function, (3.10) is trivial. Then for all n large enough, η{sn),η{tn{x)) [l,c 2 ], say, the definitions if7 ( ( ) )

-y(tn(x))> Ίn(x) := < I(ίn(x)),

7 ( ) ,

G

for x > 0,

if 7(*»(z))<

and

7*0*0 : = <

if 7(*n(a;)) < 7(«n),

(tn{x)),

Ί

for a: < 0,

^ ( W ) ( )

U(())

are meaningful and c~ι < ηn{x) < c2 for x < 0 and 1 < ηn(x) < c 3 for a; > 0. Since Mι(^y(tn(x))) = Mι{ηn{x)) by the multiplicative periodicity of Mi( ), we have vn(x) = \Mι{ηn(x)) - Λfi(7(s n ))| and, using the continuity condition for the second time, the function Mχ( ) is uniformly continuous on the closed interval [ c " 1 , ^ ] . Now, based on the definition of 7( ) above (2.6), the asymptotic equality 7nQg) —.—r- ~ Ί{S)

tn{x) S

,

where

tn(x)

> 1,

S

can be shown by elementary arguments, which since the sequence {7(s n )} is bounded, implies that |7n(ar) —j(sn)\ -> 0. The uniform continuity of Mi( ) then implies (3.10), proving the first statement in (3.8). Using the second statement of Lemma 3.4(ii), the proof of the second statement in (3.8) is completely analogous, and hence we have part (i) of the theorem. Condition (3.2) for part (ii) of the theorem implies the existence of some finite positive constants A\ < 1 < A<ι such that A\mn < ln < A2mn and A^ln ^ η^n ^ A^[ιln for all n large enough. When proving (1.4), m we renormalize φ\,n{') and ψ2,n{ ) replacing αn(ln,mn) in the denominator by the sequences α n (Λ^ 1 / n , A^lln) < αn(ln,mn) and α n (A 2 ra n , A 2 ra n ) < α>n{ln, ™>n) to obtain the present versions of φ^ J (•) and φh;mn(') of the proof above, respectively. For unified notation, we write r^n = ln and Γ2,n = mn. Part (ii) itself has two cases. When G = C?o,o,σ ι s normal for some σ > 0, we see by the criterion (1.26a) in Corollary 1 of Csόrgδ, Haeusler and Mason (1988a) for the domain of attraction of a normal distribution that both terms in φn,rjjn(x) go to zero separately at every x G K, j = 1,2, and hence (3.8) holds and implies (1.4) again. Finally, the other case of part (ii) is when one of Mi( ) and Λf2( ) in (2.4) and (2.6) is identically zero while the other is nowhere zero. Replacing K by \fK{ of part (i) of Lemma 3.4, the proof of (3.8) for that one of the present two sequences {φn}rjtn{')} for which Mj(>) > 0, j e {1,2}, is practically the same as the one above for case (i), while it is simpler for the other j G {1,2}

188

Sandor Csδrgo and Zoltan Megyesi

for which Mj( ) = 0 because (3.9) for that Mj(-) is trivial. Thus condition (1.4) for asymptotic normality holds true once more. • We also separate two lemmas for the proofs of Theorems 3.2 and 3.3, respectively. Lemma 3.5 Suppose that F G ^gpiGφ^^β) with & quantile function given by (2.6). Then for any α, b G [ 1, c), α < b, that are continuity points of both Mj, j = 1,2, and for any δ > 0 and ε G (0,1) there exists a threshold number N(α, 6,5, ε) such that the inequality

(3.11) < 2 [\Mj (α) - Mj (6) I + C(α, b, α, ε) + δ] holds true for all n > N{α, 6, δ, ε) and yi, j/2 £ [α> &L i

=

1,2, where

with the constants D\ and Z?2 from ^2.8^ and M*j(-) = Mj{η(Ίkn)) hj( /kn),j = 1,2.

+

Proof Notice first that (3.11) is trivial if Mj = 0. Thus, since the half-sided version of the proof below will be an obvious special case when exactly one Mj = 0, it suffices to deal with the situation when Mj φ 0, j = 1,2. In this situation M\ and M<ι both have positive infima on (0, oo) and we see by applying (2.6) for s = t/kn, where t > 0 is a continuity point of M\ and M2, and by the monotone nondecreasing nature of Q that lims^o Q(s) — ~°° a n d lim s |i Q(s) = 00. We choose iV* = Λf(α, 6, <5, ε) so that, for j = 1,2, Q J — J < 0

and Qyl - — J > 0,

and so M*j(y) > 0,

α
η/(α/kn) = α and "y{b/kn) = 6, ' e, 2/1,1/2 ^ fα,&L

and

hold simultaneously whenever n > N* and show (3.11) with this choice of the threshold. Assuming without loss of generality that α < y\ < 3/2 < 6, notice that

and -^

* 4 > 1 , that is,

Vfc

"

;

K nJ

*

3

'

( ) " M ) " t

> 1,

189

Semistable Trimmed Sums (j = 1,2), and so

if n > JV*. Recalling (2.8) we see that for n> N* the inequality (3.12)

\Kj(Vi) - Kj(V2)\ < C(a,6,a,e)

holds true for any choice of yuy2 6 [α,6], yi < y 2 , provided M*j(yι) < M nj(V2), 3 = 132. If this is the case indeed, then (3.12) in itself proves (3.11), but the following considerations apply in general. Indeed, observe that the choice of N* ensures that

(3.13)

\KAa)-KM

< I M J M - M J WI + J,

j = 1,2,

for all n> JV*, and note also that

Here \M*j(α) - M*j(yι)\ < C(α,b,α,ε) by (3.12) if M*d(α) < M l,j G {1,2}, but if this fails for some /, j G {1,2} then we still have

where the first term can be estimated using (3.13) and C(α, 6, α, ε) is an upper bound on the second one, provided M*j(yι) < M*j(b). However, if the latter inequality is not the case either, then \M*j(α) — M*j(yι)\ < \M*j{α) - M*j(b)\9 since M * » > M*j(yι) > M*d(b). All this together imply (3.11). • Proof of Theorem 3.2 We only have to deal with the case when G = G ψ β ^ o for some α G (0,2), where at least one of φf and ψξ is not identically 0. The other case being analogous, suppose that ψf φ 0. Retaining the notation in the proof of Theorem 3.1, we show that a sequence {ln} of positive integers can be chosen to satisfy both (1.1) and the first convergence relation in (3.8). The latter follows, through the same considerations as there, if {ln} is chosen to make sure that (3.9) holds. By the monotonicity of ψf we can pick a sequence of pairs (αj,6j), 1 < αj < bj < c, and constants Sj £ (0,1) such that both αj and bj are continuity points of M\ and the inequalities \Mχ(αj) — M\{bj)\ < gk and C(α,j, 6j, α, Sj) < jp hold for all j E N. Next, put NQ := 0 and, by means of the threshold numbers of Lemma 3.5, define an increasing sequence {Nj} of positive integers by setting Nj := max{N(αj,bj, gj,ej),-/Vjli + 1}, j € N. Elementary consideration shows now that for each j G N there exists a

190

Sandor Csδrgδ and Zoltan Megyesi

threshold number Nj G N such that for every n > NJ one can choose an ζ ; GN with the properties that (3.14)

in:

^~ n

' n

π

n

and

C

and it can clearly be stipulated that 1 < N* < Nζ < see that /I*

. By Lemma 3.5 we

/I*

i for all x 6 [—j, j] and n> Nf. Now we axe ready to choose the desired sequence {ln}. We set ln : = 1 for n < N* and define {ίnj^L v* by the following algorithm, in which T G N is a new auxiliary variable: Step 1. Let the initial values of j and n be j := 1 and n := ΛΓ*, and put T:=JVf. Sfep «. If NJ iVJ+1 then put ln := /* j or ln := l^j+i according as lnj+i ^ fa or Inj+i > fa->a n d if 'n,j+i > ' r then set also j := j + 1 and T := n. 5ίep ^. Set n := n + 1 and go to Step 2. Then ln -> ex) by the choices of T and, since iV? —>• ex) as j -> oo, we also have Zn/n -> 0 by (3.14). Thus (1.1) holds for the chosen sequence {ln} and the displayed inequality following (3.14) above shows that (3.9) is also satisfied for any fixed i G l If ψ% φ 0, then the sequence {mn} can be chosen in a similar fashion. If Φ2 = 0? then simply put mn := ln for every n G N, and the desired asymptotic normality follows as in the proof of part (ii) of Theorem 3.1. • Lemma 3.6 If a function ί( ) on (0,1) is slowly varying at zero and {rn} is a sequence of positive numbers such that rn —> 00 and rn/n -> 0, then

If, in addition j F G B g p (G^« ,«0<*,o) f°r αe (0,2), then r r ( )

a

semistable

G-0«,^α,o of exponent

0.

Proof The first statement is just a special case of Lemma 2 in Csόrgό, Horvath and Mason (1986), while the second follows from the first and part (i) of Lemma 3.4. •

191

Semistable Trimmed Sums

Proof of Theorem 3.3 We see by (2.8) and the first statement of Lemma 3.6 that for all x G M,

rθ

Ώ

and

\.

In fact, these convergences take place along the whole sequence {n} = N. Next, according to Megyesi (2000), since the main source of the domain theorem at (2.6) is that Mj(η(y/kn)) —> Mj(y) for every continuity point y > 0 of Mj, we have

at all the respective continuity points y > 0 of the limiting functions. Furthermore, Lemma 3.4(i) implies that l i m s u p —-rη-^

«^oo

/α

ki e(i/kn)

—<

~

lim sup

kTl{llkn) Finally, Lemma 3.6 implies

for any sequence {r n } of positive numbers such that r n —)• oo, r n // n —> 0 and rn/™>n -> 0; in fact, these are true along the whole N again. These four pairs of facts allow a subsequential application of that variant of a two-sided version of Theorem 1 in Csδrgδ, Haeusler and Mason (1991a), the version alluded to on p. 789 there, in which the basic functions Q+{s) and Q(l — 5), 0 < s < 1, are taken right-continuous and the Poisson processes A^i( ) and A^(-) are taken left-continuous as in the present paper. Using the eight facts above, this variant implies that every subsequence of N contains a further subsequence such that (3.3) and (3.5) hold jointly along that subsequence. This implies that (3.3) and (3.5) hold jointly as stated. By the convergence of types theorem, (3.3) and (3.5) already imply (3.4) for the subsequence {fcn}. However, if neither of Mi and M2, or equivalently, neither of ψf and ip% is identically zero, then the left side of (3.9) is bounded, by 2(Dι + D2) from (2.8), for both Mi and M 2 even if they are not continuous, implying that the two sequences of functions in (3.8) are pointwise

192

Sandor Csόrgό and Zoltan Megyesi

bounded. Hence the same is true for the sequences {φj,n{')}, j £ {1?2}. Also, setting rn = min(Zn, ran), we have αn(lni mn) < αn(rn, rn) for all n £ N 1 α and α n (r n ,r 7 l )/[n / ^(l/n)] -> 0 by Lemma 3.6. Therefore, the discussion at (1.13) in Csόrgδ, Haeusler and Mason (1988b) yields (3.4) as stated. If, on the other hand, M\ = 0 and M2 φ 0, then by the same argument ^>0

._

and, since in this case the first convergence in (3.15) takes place along the whole {n} = N with an identically zero limiting function, we also get

for both rn = ίn and r n = m n , which together prove (3.4). We see that if Mi( ) = 0 and M2( ) > 0, then in fact we have

along with (3.5). Similarly, if M2( ) = 0 and Mi( ) > 0, then again we have (3.4) and, in fact, U

along with (3.3).

REFERENCES Cheng, S.H. (1992). A complete solution for weak convergence of heavily trimmed sums. Science in China, Ser. A 35 641-656. Csδrgδ, S. (1990). A probabilistic approach to domains of partial attraction. Adv. in Appl. Math. 11 282-327. Csδrgδ, S. and Dodunekova, R. (1991). Limit theorems for the Petersburg game. In: Sums, Trimmed Sums and Extremes (M. G. Hahn, D. M. Mason, D. C. Weiner, eds.), pp. 285-315, Progress in Probability 23, Birkhauser, Boston.

Semistable Trimmed Sums

193

Csδrgδ, S., Haeusler, E. and Mason, D.M. (1988a). A probabilistic approach to the asymptotic distribution of sums of independent, identically distributed random variables. Adv. in Appl. Math. 9 259-333. Csόrgδ, S., Haeusler, E. and Mason, D.M. (1988b). The asymptotic distribution of trimmed sums. Ann. Probab. 16 672-699. Csδrgδ, S., Haeusler, E. and Mason, D.M. (1991a). The asymptotic distribution of extreme sums. Ann. Probab. 19 783-811. Csόrgδ, S., Haeusler, E. and Mason, D.M. (1991b). The quantile-transformempirical-process approach to limit theorems for sums of order statistics. In: Sums, Trimmed Sums and Extremes (M.G. Hahn, D.M. Mason and D.C. Weiner, eds.), pp. 215-267, Progress in Probability 23, Birkhauser, Boston. Csόrgδ, S., Horvath, L. and Mason, D.M. (1986). What portion of the sample makes a partial sum asymptotically stable or normal? Probab. Theory Rel. Fields 72 1-16. Csόrgδ, S. and Simons, G. (1996). A strong law of large numbers for trimmed sums, with applications to generalized St. Petersburg games. Statist. Probab. Letters 26 65-73. Doeblin, W. (1940). Sur l'ensemble de puissances d'une loi de probability. Studia Math. 9 71-96. Gnedenko, B.V. and Kolmogorov, A.N. (1954). Limit Distributions for Sums of Independent Random Variables. Addison-Wesley, Reading, Massachusetts. Grinevich, I.V. and Khokhlov, Y.S. (1995). The domains of attraction of semistable laws. Teor. veroyatn. primen. 40 417-422 (in Russian). [English version: Probab. Theory Appl. 40 361-366.] Griffin, P.S. and Pruitt, W.E. (1989). Asymptotic normality and subsequential limits of trimmed sums. Ann. Probab. 17 1186-1219. Hahn, M.G., Mason, D.M. and Weiner, D.C, editors (1991). Sums, Trimmed Sums and Extremes. Progress in Probability 23, Birkhauser, Boston. Kesten, H. (1993). Convergence in distribution of lightly trimmed and untrimmed sums are equivalent. Math. Proc. Cambridge Philos. Soc. 113 615-638. Kruglov, V.M. (1972). On the extension of the class of stable distributions. Teor. veroyatn. primen. 17 723-732 (in Russian). [English version: Probab. Theory Appl. 17 685-694.] Levy, P. (1937). Theorie de Vaddition des variables aleatoires. GauthierVillars, Paris. Megyesi, Z. (2000). A probabilistic approach to semistable laws and their domains of partial attraction. Ada Sci. Math. (Szeged) 66, to appear.

194

Sandor Csδrgδ and Zoltan Megyesi

Mejzler, D. (1973). On a certain class of infinitely divisible distributions. Israel J. Math. 16 1-19. Pillai, R.N. (1971). Semi stable laws as limit distributions. Ann. Math. Statist. 42 780-783. Shimizu, R. (1970). On the domain of partial attraction of semi-stable distributions. Ann. Inst. Statist Math. 22 245-255. Stigler, S.M. (1973). The asymptotic distribution of the trimmed mean. Ann. Statist. 1 472-477.

DEPARTMENT OF STATISTICS

BOLYAI INSTITUTE

UNIVERSITY OF MICHIGAN 4062 FRIEZE BUILDING ANN ARBOR, MICHIGAN 48109-1285 USA

UNIVERSITY OF SZEGED ARADI VERTANUK TERE 1 H-6720 SZEGED HUNGARY

scsorgo @umich. edu

csorgo ©math, u-szeged. hu [email protected]

STATISTICAL PROBLEMS INVOLVING PERMUTATIONS WITH RESTRICTED POSITIONS

PERSI DIACONIS, RONALD GRAHAM AND SUSAN P. HOLMES

Stanford University, University of California and ATT, Stanford University and INRA-Biornetrie The rich world of permutation tests can be supplemented by a variety of applications where only some permutations are permitted. We consider two examples: testing independence with truncated data and testing extra-sensory perception with feedback. We review relevant literature on permanents, rook polynomials and complexity. The statistical applications call for new limit theorems. We prove a few of these and offer an approach to the rest via Stein's method. Tools from the proof of van der Waerden's permanent conjecture are applied to prove a natural monotonicity conjecture. AMS subject classiήcations: 62G09, 62G10. Keywords and phrases: Permanents, rook polynomials, complexity, statistical test, Stein's method. 1

Introduction

Definitive work on permutation testing by Willem van Zwet, his students and collaborators, has given us a rich collection of tools for probability and statistics. We have come upon a series of variations where randomization naturally takes place over a subset of all permutations. The present paper gives two examples of sets of permutations defined by restricting positions. Throughout, a permutation π is represented in two-line notation 1 π(l)

2 3 π(2) π(3)

... n ••• τr(n)

with π(i) referred to as the label at position i. The restrictions are specified by a zero-one matrix Aij of dimension n with Aij equal to one if and only if label j is permitted in position i. Let SA be the set of all permitted permutations. Succinctly put: (1.1)

SA = {π : UUAiπ{i)

= 1}

Thus if A is a matrix of all ones, SA consists of all n! permutations. Setting the diagonal of this A equal to zero results in derangement, permutations with no fixed points, i.e., no points i such that π(i) = i.

196

Persi Diaconis, Ronald Graham and Susan P. Holmes

The literature on the enumerative aspects of such sets of permutations is reviewed in Section 2, which makes connections to permanents, rook polynomials and computational complexity. Section 3 describes statistical problems where such restricted sets arise naturally. Consider a test of independence based on paired data (Xi, Yi), (X2, Y2) {Xn, Yn)> Suppose the data is truncated in the following way: For each x there is a known set I(x) such that the pair (X, Y) can be observed if and only if Y E I(X). For example, a motion detector might only be able to detect a velocity Y which is neither too slow nor too fast. Once movement is detected the object can be measured yielding X. Of course, such truncation usually induces dependence. Independence may be tested in the following form: Does there exist a probability measure μ on the space where Y is observed such that {1.2) P{YieBi,l
2=1

when π is chosen uniformly in SA In Section 3 we discuss natural examples where the restriction matrix A has an interval structure (the ones in each row are contiguous). For one-sided intervals some of the standard limit theory can be pushed through, although much remains to be done. A classical contingency table with structural zeros of Karl Pearson is treated as an example. For two-sided intervals, we offer an exchangeable pair so that Stein's method can be used. The exchangeable pair gives a Monte Carlo Markov chain for calibrating the permutation distribution. A red-shift data set of Efron and Petrosian (1998) is treated as an example. Permutations with restricted position appear in a different guise in skill scoring, a technique for evaluation of taste testing and extra sensory perception experiments with feedback to subjects permitted. This application

Independence with Truncation

197

is reviewed in Section 4. Some of the tools developed to prove the van der Waerden permanent conjecture are applied here to give a simple proof of a natural monotonicity conjecture. We hope that these two examples of permutation testing with restricted positions will interest Bill. This part of the subject can certainly use his help.

2

Permanents

Let A be a zero-one square matrix of dimension n. The number of elements in SA of (1.1) is determined by the permanent of A: yZi.oj

\&A\ ~

The sum in (2.1) is over all permutations in the symmetric group. Thus the permanent is like the determinant without signs. There is a large mathematical literature on permanents. We review some of this pertaining to matching theory (Section 2.1), rook theory (Section 2.2) and complexity theory (Section 2.3). We believe that many of the nice developments in permutation enumeration and testing will work out nicely for the case of interval restricted permutations. An example, Fibonacci permutations, is developed in Section 2.4. It may be consulted now for motivation.

2.1

Bipartite Matchings

Let [n] = {1,2,..., n} and [n1] = {I 7 ,2',..., n'} be two disjoint n element sets(n — n'). A bipartite graph G is specified by giving a set of undirected edges ε = {(ii,ii), (^2^2) (^e^e)} For example, when n = 3 the graph might appear as:

A matching in G is a set of vertex disjoint edges. Thus (1, l')(2,3') is a matching in the figure above. A perfect matching is a matching containing n edges. There are three perfect matchings in the figure above. There is a one-to-one correspondence between perfect matchings and the set SA of (1.1) when A is taken as the adjacency matrix of the graph G.

198

Persi Όmconis, Ronald Graham and Susan P. Holmes

Thus for the figure above: 1' 2' 3' 1 1 1 0 A

=

1 1 1

2 3

0

1 1

and Per (A) = 3. This correspondence allows standard graph theory algorithms to be used for permutations with restricted positions. For example, given a restriction matrix A there is a polynomial time algorithm (order n2 5/y/logn) for finding if there exists a perfect matchings using the widely available algorithm for solving the assignment problem of combinatorial optimization (see Cook (1998)). Naively computing the permanent from its definition (2.3) takes n.nl steps. Ryser's algorithm (see van Lint and Wilson (1992); theorem 11.2) improves this to order n.271. As discussed in Section 2.3 below, no substantial improvement can be expected for general restriction matrices (the problem is #-P complete). For some special cases (Sections 2.2,3.3,4.1) enumeration is feasible. For matrices A with row sums ri, Γ2,..., r n , van Lint and Wilson (1992 theorem 11.3) gives the bound Per{A) < Π ? = 1 ( r i ) ! ^ . Bipartite matching is the easiest case of matching theory in general graphs. Many further results applications and references are collected in the splendid book by Lovasz and Plummer (1986).

2.2

Rook Theory

This is an algebraic technique for enumerating SA The classical setting is a set of squares B of an n x n chess board. Let r(B,k) be the number of ways of placing k non-attacking rooks on the squares of B (that is, choosing k squares in B no two in the same row or column). A board B is identified with a bipartite graph G with edge (ΐ,i') if and only if {i,i') is in B. Thus r(B, k) is the number of matchings with k edges. The rook polynomial r(G,x) is defined as (2.4)

r(G,x) =

Thus the number of perfect matchings is the value r(G,0). Rook polynomials have been extensively studied. Stanley (1986, Chapter 2) gives a useful treatment of the essentials including extensive references to the work of Goldman, Joichi, White.

Independence with Truncation

199

Riordan (1958) reviews the extensive classical literature. Godsil (1981) and Lovasz/Plummer (1986) treat the generalization to matching polynomials of a general graph due to Heilmann-Lieb. Rook and matching polynomials satisfy useful recurrences, have real zeroes and, for neat graphs, give rise to orthogonal polynomials.

2.3

Complexity Theory

Evaluation of the permanent of a square matrix is a celebrated problem in modern complexity theory. Indeed, Valiant (1979) used this as the first example of a #P-complete problem. Recall (Garey and Johnson, 1978) that NP-complete problems have "yes" or "no" answers, e.g.,"Is there some subset sum of this list of integers equal to 137?" The class of #P-complete problems are counting problems "How many subset sums are equal to 137?" For bipartite matching there is a fast way to check existence, but the counting problem is #P-complete; if a polynomial time algorithm exists then thousands of other intractable problems (e.g., computing the volume of a convex polyhedra) can be solved in polynomial time. A review of the work on the permanent from a complexity viewpoint can be found in Jerrum and Sinclair(1989) or Sinclair (1993). Modern computer science has produced efficient randomized algorithms for approximating the permanent of a dense bipartite graph (every vertex having degree at least | ) . These algorithms perform a random walk on the set of perfect matchings and almost matchings (at most one edge missing). It has been proved that these walks converge rapidly and allow efficient selection of an essentially random perfect matching. This makes Monte Carlo quantification of the tests outlined in the introduction feasible for dense graphs.(Jerrum and Sinclair (1989), Sinclair (1993)) Unfortunately, arguments used by Jerrum and Sinclair and later workers really seem to depend on the denseness assumption. Despite extensive work over the past ten years the rate of convergence of natural random walks on the set of permutations consistent with a general restriction matrix remains open. Among interesting recent developments we mention: Kendall, Randall and Sinclair (1996) show that the random walk algorithm works in polynomial time for the perfect matchings in bipartite graphs which are vertex transitive (e.g. d-dimensional rectangular grids with toroidal boundaries). Jerrum and Vazirani (1992) give an algorithm that approximates the permacnΊ ι n nent of any n x n zero-one matrix in time e ( °9 ) . This is superpolynon mial but better than the n2 of Ryser's algorithm. Rasmussen (1994) gives a simple greedy approximation algorithm for the permanent which is shown to run in polynomial time for almost every graph for the usual G(n,p) model of random graphs. Finally, Karmarkar et al.(1993) followed by Barvinok

200

Persi Diaconis, Ronald Graham and Susan P. Holmes

(1998) and Rasmussen (1998) show how to approximate the permanent by a stochastic algorithm which replaces the ones by cube roots of unity and takes the squared modulus of the determinant. This algorithm has good average case behavior but exponential worst case behavior. In Section 3 below we describe walks with interval restrictions where some things can be proved.

2.4

The Example of Fibonacci Permutations

This Section treats a simple example which shows that elegant theory can be developed for enumeration, random generation and the study of cycle structure. Let An be the n x n matrix with ones on, just above and just below the diagonal. Thus when n = 4 1 1 0

0

1 1 1 0 0 0 0

1 1 1 1 1

From the definition (2.1), Per (A) is multilinear. Expanding by the first row, we derive a result first noted by Lehmer (1970): PerAn

— PerAn-ι

+ PerAn-2-

From direct computation, Per(A\) = 1, Per(A2) = 2, so Per(An) = F n + i is the (n + l)st Fibonacci number. The Fibonacci numbers are defined as ΓQ

— U, r i = 1, Γ2 = 1, Γ3 = ^ , Γ4 = o, Γ5 = O,. . . , ^n+1

=

^n "T ^n-1?

We will call the elements of SAU Fibonacci permutations. We first develop several bijections with combinatorial objects well known to be counted by Fibonacci numbers. Permutations counted by Per(An) can be described in their cycle form. They are all permutations consisting of fixed points and pairwise adjacent transpositions. To see this, observe that permutations can be constructed as follows. Place symbols 1,2,3,..., n in a line, put a left parenthesis at the start and a right parenthesis at the end. Proceeding sequentially, decide to pass on or to place parentheses ) ( between i and i + 1. This results in the following five permutations consistent with A\\

From this description it is easy to see that the permutations enumerated by An are in one to one correspondence with the following well-known Fibonacci equivalents.

201

Independence with Truncation

Proposition 2.1 The set of Fibonacci permutations on n letters is in oneto-one correspondence with: • Subsets of [n — 1] with no consecutive elements: φ, {1}, {2}, {3}, {1,3} • binary (n-l)-tuples without two consecutive ones: 000; 100; 010; 001; 101 • Matchings in an n-path: o

o

o

o

J o—o

o

o

!

o

o—o

o

' o

o

o—e

' o——o

o—o

• Compositions ofn with all parts equal to one or two 1111,211,121,112, 22 The next proposition gives an easy, direct method for uniformly choosing a random Fibonacci permutation. It is based on a theorem of Zeckendorf (1972) and the Fibonacci numbering system. Proposition 2.2 Any positive integer n can be uniquely expressed as n = Fki + Fk2 H + Fkt with F^ distinct Fibonacci numbers starting with F2, no two adjacent. Thus 1 = F 2 , 2 = F 3 , 3 - F 4 , 4 = F 2 + F 4 , 5 = F 5 , 6 = JF 5 + F 2 , 7 F 5 + F 3 , 8 = F 6 , 9 = F 6 + F 2 ,10 6 - F 3 0 + F26 + F 2 4 + F12 + F 1 0 , these topics are covered in Graham, Knuth, Patashnik(1989, pp.281-283). They show that the representation is easy to find, each time substracting off the largest possible Fibonacci number. For our purposes, consider a path with n vertices and edges labeled with consecutive Fibonacci numbers starting with F 2 ;eg for n = 8 : F2

F3

F4

F5

FQ

1

2

3

5

8

FJ

13

F%

21

The weight of a matching in this graph is the sum of the weights of the edges. The largest possible sum comes from choosing the unique maximal matching. It is easy to prove that this has weight F n + i — 1. Thus in the example above, 21 + 8 + 3 + 1 = 33 = Fg — 1. With this preparation, the algorithm is simple. Proposition 2.3 The following algorithm produces a randomly chosen Fibonacci permutation on n letters. Choose an integer U, uniformly between zero and F n + i — 1. Express U in the Fibonacci numbering system and use the edges as a matching. Using the correspondence between matchings and Fibonacci permutations completes the construction.

202

Persi Diaconis, Ronald Graham and Susan P. Holmes

For a randomly chosen permutation π 6 Sn the number of fixed points F(π) has an approximate Poisson(l) distribution, the number of transpositions T(π) has an approximate Poisson(^) distribution, the number of cycles C(π) has an approximate Normal distribution with mean logn and variance log n, and finally the number of inversions has an approximate Normal distribution with mean \ and variance |g. The following proposition shows how these results carry over to Fibonacci permutations. In this case, the four results coalesce since T(π) = /(π), C(π) = T(τr) + 7(τr) and n = F(π) + 2T(τr). Proposition 2.4 Let π be α randomly chosen Fibonacci permutation on n letters. Let T(π) be the number of transpositions in π. Then /n-k\

(2.5)

DίΦ __ M __ V *• '

~ -.

. tn

(2.6) with φ = (2.7)

(2.8) Proof A bijection between Fibonacci permutations with T(π) = k and fe sets of an n — k set is easy to see; Put n — k dots down in a row. Circle each element in a subset of size fc. Now working from left to right, all encircled points are expanded to two points and correspond to transpositions. Thus

®

®

<

> (12)(34)5

The generating function for (n^fc) is a classical result:

m - Σ(n;k) '* (2.9) Setting z = 1 gives the classical expression (2.10)

Independence with Truncation

, l + \/5 Λ where φ = — -2— ,' ψ " = Differentiating (2.9) at z = 1 gives

203

1-Vb 2 2

Φn) - 5\/5

25 Dividing by F n +i and using (2.10), routine simplifications give (2.6,2.7) . An interesting proof of the liming normality uses the identification of Fibonacci permutations with the set of all matchings in an n-path. In this identification, the number of transpositions corresponds to the number of edges in the associated matching. Godsil(1981) has shown that the number of edges in a random matching of a general graph tends to a Normal distribution provided only that the variance tends to infinity. • Remark 2.1 The limiting normality can be proved directly from (2.5) using Stirling's formula to give a local central limit theorem. Alternatively, the generating function (2.9) can be used. Godsil's proof used above itself uses Harper's method and depends on the fact that the zeros of the matching polynomial are all real. Pitman(1997) develops Harper's method and gives easily computed error bounds to the central limit theorem. Remark 2.2 Similar results can be developed for the case where the restriction matrix has ones on the diagonal and k to the left and right of the diagonal in each row. The Fibonacci example has k = 1. 2.5

Other applications of permanents

The above surveys only mention features of the permanent literature directly related to the present project. There are many further applications. Mallows (1957) shows how permanents appear in computing normalizing constants in non-null ranking models. Bapat (1990) surveys a variety of appearances of permanents in statistics and further applications appear in Sections 3 and 4 below. Daley and Vere-Jones (1988) show how permanents occur for point-process theory and computing moments of complex normal variables. One of the most active recent developments is the immanants. These are expansions of the form: (2.11)

204

Persi Diaconis, Ronald Graham and Susan P. Holmes

with Ξ a character of the symmetric group. Taking Ξ = 1 gives the permanent, taking Ξ(τr) = sgn(π) gives the determinant. There is active work giving inequalities for other immanants (see Lieb (1966) and Stembridge (1991, 1992) for surveys). 3

Testing for Independence

3.1

Introduction

Consider the classical problem of testing for independence without truncation. One observes pairs (AΊ, YΊ), (X2? ^2), > {X n, Yn) drawn independently from a joint distribution V with Xi e X,Yi £ y, suppose that V has margins V1 and V2. A test of the null hypothesis of independence: V — φλ x φ2 may be based on the empirical measure Vn. Let δ be a metric for probabilities on X x y. One class of test statistics is (3.12)

Tn =

δ{Vn,VιnxV2n)

Extending classical work of Hoeffding (1948), Blum, Kiefer and Rosenblatt (1961), and Bickel (1969), Romano (1989) show that under very mild regularity assumptions, the permutation distribution of the test statistic Tn gives an asymptotically consistent locally most powerful test of independence. Consider next the truncated case explained in Section 1. The hypothesis (1.2) may be called quasi-independence in direct analogy with the similar problem of testing independence in a contingency table with structural zeros. Clogg (1986) and Stigler (1992) review the literature and history of tests for quasi-independence with references to the work of Caussinus and Goodman. While optimality results are not presently available in the truncated case, it is natural to consider the permutation distribution of statistics such as (3.12). This leads to a host of open problems in the world of permutations with restricted position. We were led to present considerations by a series of papers in astrophysics literature dealing with the expanding universe. The red shift data that is collected for these problems suffers from heavy truncation problems. For example, Figure 1 from Efron, Petrosian (1998) shows a scatterplot of 210 x — y pairs subject to interval truncation, the x coordinate corresponds to red-shift, the y coordinate corresponds to log-luminosity. A suggested theory of 'luminosity evolution' says that early quasars were brighter. This suggests that points on the right side of the picture are higher because the high redshift corresponds to high age. Astronomers beginning with Lynden-Bell (1971, 1992) have developed permutation type tests based on Kendall's tau for dealing with these problems. There is a growing statistical literature on regression in the presence of truncation; see Tsui et al.(1988) for a survey.

205

Independence with Truncation 210 observed data points and their boundaries

0

0.5

1

1.5

2

2.5

3

3.5

Figure 1. Quasar Data, 210 points, upper and lower limits.

Most previous work deals with one-sided truncation of real-valued observations. The theory and practice is easier here as explained in Section 3.2 . Efron and Petrosian (1998) have recently developed tests and estimates for the case of two-sided truncation. We develop some theory for their setup in Section 3.3. The following preliminary lemma shows that interval truncation of real valued observations leads to restriction matrices with intervals of ones in each row.

Lemma 3.1 Let a?i,... ,x n take values in an arbitrary set. Let I(xt ) be a real interval. Let yi,2/2, ,2/n be real numbers with yι G I(xχ). Suppose the ordering is chosen so that y\ < y<ι < 2/3 < < yn Finally, define a zero-one matrix A of dimension n by f Vj € I(xi) ^ ~ \ 0 eelse Then each row in A has its ones in consecutive positions with a one in position (i,z), 1 < i < n.

Proof

The intervals may be pictured as in Figure 2, which is translated

206

Persi Diaconis, Ronald Graham and Susan P. Holmes

3 -

(

>

< 1-

<>

0 -

<

-2 -

-3

0

1

2

3

4

5

6

7

10

Figure 2. Seven permitted intervals and interior points

to the matrix whose transpose is

(3.13)

1 0 0 0 0 0 0

1 1 0 0 0 0 0

0 1 1 1 1 0 0

0 0 1 1 1 0 0

0 0 0 1 1 0 0

0 0 0 0 0 0 0 0 0 0 1 1 1 1

Consider row i of A. By definition, An = 1. Suppose that Aij = 1, for some i < j . Thus yj G I(xi) and of course y\ G I(xi). By monotonicity, yi G I(xi) for i < ί < j . A similar argument for j
3.2

One-sided restriction

Paired data with {y^Xi) observable, if and only if, yi G (αi,oo) give rise to one sided restriction matrices (yι G (—oo, αi) is similarly treated). This Section shows that a neat theory emerges for one-sided truncation. Sets of permutations consistent with one-sided truncation can be described as follows. Let & = (δi, &2? i &n) with 1 < b{ < n be positive integers. Let (3.14)

Sb = {π : τr(t) > h, 1 < i < n}

207

Independence with Truncation

Equivalently, the matrix A has Aij = 1 iff j > bi. Without loss of generality, we take b\ < 62 < * h < < bn in the sequel. Some examples: • If all bi = 1, Sb contains all permutations. • If bi = 1,1 < i < α, b{ = 2, α + 1 < i < n, Sb contains all permutations with 1 in the first α places. • If bi = ΐ, 5b contains only the identity permutation. • If bi = b2 = I, bi = i — 1,1 bn}. list of choices.

Delete π(n) from the

• Choose π(n—1) from the elements j in the current list J with j > bn-\. Delete π(n — 1) from the list • : and so on... Proof The algorithm produces an element of Sb without getting stuck because of the restriction bi
(2) Σxesb (3) Σ es

208

Persi Όiaconis, Ronald Graham and Susan P. Holmes

Corollary 3.2 Suppose that X{ is uniform on {0,1,... ,i — b{}. Let Y{ be independent binary variables with P{Yi = 0} = ι-j1, P{Y% = 1} = -f. Let π be chosen uniformly from Sn. Then (1) (2) P(C(π)=j)=P(Y1+Y2

+

.Yn=j)

Remark 3.1 The statistics J(π) and n — C(π) are standard distances on the permutation group used as measures of disarray (see Diaconis and Graham(1977) or Diaconis(1988, chapter 8)). Indeed, I(τr) is the minimum number of pairwise adjacent transpositions required to bring π to the identity and n — C(π) is the minimum number of transpositions required to bring π to the identity. Further, J(π) is affinely related to Kendall's tau, a non parametric measure of association. The corollary represents these statistics as sums of simple independent random variables. This allows easy calculation of means and variances and a proof of the central limit theorem with Edgeworth corrections. For further details see Feller (1968). Remark 3.2 One natural notion of rank test involves statistics of form Σ?=i m (*> π W)? with π uniformly chosen in S&. Efron and Petrosian (1992) have introduced an apparently different notion of rank test with a simple distribution theory based on lemma 3.2. The relation between these rank tests and the distribution theory above are open problems. While Corollary 3.1 is well known in the combinatorial literature, it is often rediscovered by statisticians, see Tsai (1990).

3.3

An example with historical insights

Karl Pearson considered a natural source of censored observations in his work on what is now called quasi independence. He considered families with one or more imbecilic children, cross tabulating the family size versus the birth order of the first such child. Clearly a family of j children can only have its first born special child in a position between one and j and consequently, T{j = 0 for i > j. Pearson carried out a test of independence with this truncated dataset in 1913! A historical report on Pearson's work and its later impact is given by Stigler (1992). It is worth beginning with an exact quote of Pearson's procedure from the article by Elderton et al. (1913). "Lastly we considered the correlation between the imbecile's place in the family and the gross size of that family. Clearly the size of the family must always be as great or greater than the imbecile's place in it, and the correlation table is accordingly one cut off at the diagonal, and there would certainly be correlation, if we proceeded to find it by the usual product

Independence with Truncation

209

moment method, but such correlation is, or clearly may be, wholly spurious. Such tables often occur and are of considerable interest for a number of reasons. They have been treated in the Laboratory recently by the following method: one variate x is greater than or equal to the other y; let us construct a table with the same marginal tables, such that y is always equal to or less than x, but let its value be distributed according to an "urn-drawing" law, i.e. purely at random. This can be done. We now have two tables, one the actual table, the other one with the same marginal frequencies, would arrive if x and y were distributed by pure chance but subject to the condition that y is equal or less than x, this table we call the independent probability table. Now assume it to be the theoretical table, which is to be sampled to obtain 2 the observed table, and to measure by χ and P the probability that the observed result should arise as a sample from the independent probability table." We find this paragraph remarkable as an early clear example of the conditional permutation interpretation of the chi-square test. A careful reading reveals that Pearson is not explicit about the "urn drawing" commenting only that this can be done. In the rest of this Section we give an explicit algorithm by translating the problem into that of generating a random permutation with restrictions of the one-sided type and showing that lemma 3.2 achieves a particularly simple form. To begin with, it may be useful to give the classical justification for Fisher's exact test of independence in an uncensored table. Let T{j be a table with row sums T{ and column sums Cj . Under independence the conditional distribution of Tij given r^, Cj is the multiple hypergeometric. This may be obtained and motivated as a permutation test as follows. Suppose the n individuals counted by the table have row and column indicators (Xi,Yi)ι
210

Persi Όiaconis, Ronald Graham and Susan P. Holmes

We can now mimic these computations for the triangular table considered by Pearson. Call the table entries Πij with Πij = 0 for j > i. Suppose n = Σt>j n%j τ h e original data can be regarded as (X^ Yk)ι
j

ι

. The algorithm of Lemma 3.2 translates

to the following algorithm to generate the triangular table will row sums r i , . . . 77, column sums ci,... cj, and Tij =0 if j > i. • Place c\ balls labeled 1 in an urn and sample τ\ of these without replacement. Let T\\ be the number of balls in the sample labeled 1. • Add C2 balls labeled 2 to the urn. Sample r<ι from the urn without replacement. Let T21 be the number of balls labeled i in the sample ί = 1,2.

Remark 3.3 It is clear from Pearson's discussion following the quote above that he was aware of essentially this algorithm. He used it to give a closed form expression for the maximum likelihood estimates. Very similar upper triangular tables arise in genetics in testing goodness of fit of the Hardy-Weinberg equilibrium model. The analogous exact sampling scheme is well-known. Recently, Markov chain Monte Carlo techniques have been used to to do the sampling; Guo and Thompson (1992) derive such an algorithm which is a further studied in Diaconis, Graham and Holmes(1999). Lazzeroni and Lange(1997) give a stopping time approach which is an early example of exact sampling. All of these ideas can be extended to the one-sided censoring case. 3.4

Two-sided restrictions

All of the neat factorizations and sampling schemes in Section 3.3 disappear in the case of two-sided truncation. In this Section we introduce a graph structure on permutations in SA This gives a reversible Markov chain on SA which can be run to calibrate permutation tests. Further, the graph structure gives an exchangeable pair so that Stein's method may be used to approximate the distribution of rank statistics as in Stein (1986), Bolthausen (1984), and Bai, Chao, Liang and Zhao(1997). Lemma 3.3 Let A be the zero-one matrix of dimension n. Suppose that for all i, An = 1 and that the ones in each row of A lie in an interval Define a graph G with vertex set SA and edges between σ and r for σ and r that differ by a transposition of labels. This graph G is connected.

Independence with Truncation

211

Proof By construction, Id € SA Connectedness is proved by showing that any σ € 5^ is connected to Id. This in turn is proved by regarding σ as a product of disjoint cycles and showing that any cycle can be broken into two resulting in another permutation still in SA Towards this end, it is useful to picture the permutations superimposed on the matrix A coloring the places in A corresponding to the permutation representation of σ. For . / 1 2 3 4 5 \ example σ — I I appears as / 1 0

0

(3.15)

\o

1 /

The rows correspond to positions, the columns to labels. An allowable transition can be pictured on A as well. For example, transposing labels 2 and 5 gives I

yy 2 2

11 4 4

o o 55 J

I G SA The transition is pictured via the paren-

theses in display (3.16):

1

(3.16)

Γ] 0 0 0

E 1 1 1 1

1

1

1 1

0

Ξ

i)

B 1 1

1

(1) 0 0 1

E

In general, transposing two labels is admissible if and only if there are two ones in the two available places in A. In the example, σ is a product of two cycles (1,5,2) and (3,4). It will be useful to picture moving along the cycle on the picture of σ on A. Prom a boxed square at (i, j) move to diagonal (j,j) and then to the unique box in row j . Finally, observe that given a cycle (ή,i2, . ,H) w i t h h smallest, the submatrix A of A formed by rows {ϊi, %2, . , k} and columns {ή, Z2,.. , π} has the row interval property with ones on the diagonal. For example, the cycle (l,5,2)above the gives A as / 1 1 1 \ (3.17)

1

0

0

1

Beginning the cycle with its smallest elements results in A having the leftmost boxed element in the first column.

212

Persi Diaconis, Ronald Graham and Susan P. Holmes

We may thus study the submatrix corresponding to the cycle (ii,i2j > it) in a matrix A with ones on the diagonal and having the ones in each row in an interval use {ή,22? 5 ^} to label the rows and columns. Thus the matrix appears as 1

1

1

1

1

(3.18)

1

1

1 1 1

1

1 1 1

1 1 1

1

1

ϊ]

1

1

1

1

1

1

1

1

1

1

. .. . ..

-I

1 X

1

1 1 1

1 1 1

1

1

X

X

1 1

1

1

The proof proceeds in two cases: Case 1 i£ > i2 Then, by the row interval property there is a 1 in positions (ii,ii) and (^,22)- Thus labels 12 and it can be transposed. Case 2 it
1 1 ... 1 1 1 1

(3.19)

(*2,*2)

1 1 1

1

1 1

The reader may picture the horizontal £JJ and the vertical line ly through (Ϊ2,Ϊ2) The path along the cycle next goes down to (13,12) and continues. Eventually, the cycle path must hit the box in position (it,ii). To do this, it must cross the vertical line £y. Consider the first time this happens, the path must cross the line from right to left winding up in a box at position (iΓ,ic) with ir >'i2,ic < h By the row interval property there are ones between (ir,ic) and (ir,ir) Thus AIM = 1. Further, in the first row A^ = 1. It follows that labels %i and ic can be transposed.

Independence with Truncation

213

This covers all cases and completes the proof. • Remark 3.4 The graph of Lemma 3.3 can be used to run a Markov chain on SA Prom σ G SA choose one of (£) transpositions uniformly at random and transform σ by switching the two chosen labels. If this new permutation is in SA, the walk moves there. If not, the walk stays at σ. This results in a symmetric connected Markov chain which has uniform stationary distribution on SA- The chain is aperiodic whenever A is not the matrix entirely filled with ones. This chain allows calibration of any test statistic. Its proper use needs an estimate of the relaxation time. We have carried out some preliminary work which suggests that order π2logn steps are sufficient in the case of interval structure for A, this remains a conjecture. We remark that Hanlon(1996) has determined all the eigenvalues of this Markov chain for the case of one-sided truncation. Remark 3.5 For more general zero patterns, the graph based on transpositions need not be connected. For example, consider the derangements in 53. /0 1 1\ The matrix is 1 0 1 . The two derangements are (1,3,2) and (1,2,3) \l 1 0/ in cycle notation. These are even permutations and cannot be connected by transpositions. The 3 x 3 matrix for derangements (call it D) can be used to construct larger examples which show the difficulty of making a general theory. For example, construct a 3n x 3n matrix A by placing copies of D down the diagonal, zeros in the remaining upper triangular part and ones in the remaining lower triangular part. It is easy to see that there are 2n permutations in SA and that none of these can be connected to any others by transpositions. As a second example, construct an n x n matrix A with a single copy of D in the upper left hand corner. Place zeros in the remaining places of the first three rows and ones everywhere else. Here, there are two giant components in SA which cannot be connected by transpositions. We have shown that for n > 4 the set of all derangements is connected by transpositions. The argument proceeds by showing that any derangement can be brought to the n-cycle (1,2,3,4,..., n) in at most n—1 transpositions. While not exploited in the current paper, there is a large class of examples of sets of permutations which are connected by transpositions; these are the linear extensions of a given partial order. See Brightwell and Winkler (1991) for an overview and references. Remark 3.6 The Markov chain of Remark 1 above gives an exchangeable w pair of random permutations (X, X')> ^ ^ X chosen from the uniform distri-

214

Persi Diaconis, Ronald Graham and Susan P. Holmes 1

but ion on SA and X one step of the chain away from X. Such an exchangeable pair forms the basis of Stein's approach to the study of Hoeίfding's combinatorial limit theorem. Bolthausen (1984) and Schneller (1989) used extensions of Stein's method to get the right Berry-Esseen bound and Edgeworth corrections. Zhao, Bai, Chao and Liang (1997) give limit theorems α π π for double indexed statistics (a la Daniels) of form Σ (iiJi (i)> U)) using Stein's method. Finally, Mann(1995) and Reinert(1998) have used Stein's method of exchangeable pairs to show that the chi-square test for indepen2 dence in contingency tables has an approximate χ distribution, conditional on the margins. We have used the exchangeable pair described above to prove normal and Poisson limit theorems for the number of fixed points in a permutation chosen randomly from the set SA- There is a lot more work to be done. We note in particular that the limiting distribution of linear rank statistics is an open problem with even one-sided truncation. The distribution of Kendall's tau is an open problem in the case of two-sided truncation. We close this Section with a statistical comment and a useful lemma. The widely used non parametric measure of association Kendall's tau applied to paired data {{xi,Vi)} can be described combinatorially as follows: Sort the pairs by increasing values of X{. Then calculate the minimum number of pairwise adjacent transpositions required to bring {yι\ into the same order as {xi}. When working with restricted positions, it is natural to ask if any admissible permutation can be brought to any other by pairwise adjacent transpositions. The following example shows that this is not so. For n = 3 consider the / 1 1 1\ / 1 2 matrix I 0 1 0 I. There are two admissible permutations (

Vi

i i/

V

3\ I ;

/ 1 2 3 \ and I I. No pairwise adjacent transpositions of the labels is al\ 3 2 IJ lowable. The matrix has the row interval property and all transpositions connect. It is not hard to see that pairwise adjacent transpositions connect all admissible permutations in the one-sided case.The following lemma proves connectedness in the monotone case: The intervals I(xi) = (αi,&i) can be arranged so that α\ < α<ι < as < < an;bι < 62 < 63 < * * < K- For the monotone case, order i by a{ increasing. Then, it is easy to see that the restriction matrix Aij = < . j ^ *' has both row interval property and column interval property, and An = 1. Call such a restriction matrix monotone.

215

Independence with Truncation

Lemma 3.4 Let A be α n-dimensionαl monotone restriction matrix. Form a graph with vertex set SA and an edge between two permutations if and only if they differ by a pairwise adjacent transposition of labels. This graph is connected. Proof By construction, SA contains the identity. The argument proceeds by showing that any permutation π G SA can be brought to the identity by pairwise adjacent transpositions. It is useful to picture π on A by boxing entry (i,j) if π(i) = j.

o\

1 1

(3.20)

0

DD 1

1 1

/ 1 2 3 4 represents I 3 4 j 2

A pairwise adjacent transposition corresponds to a basic move in two adjacent rows

Recall that π has an inversion at (i,j) if i < j and π(z) > π(j). Only the identity has no inversions. Consider two consecutive rows z,z + 1 , if π(i) > π(i + 1) the picture appears as

Consider the first row for which π(i) > ΐ, if π Φ identity, such a row must exist. By the row and column interval property a basic move can be made. This reduces the number of inversions by one. Continue until no pair of adjacent rows has an inversion. This gives the identity. •

Remark 3.7 While lemma 3.3 shows the graph is connected we have found simple examples of monotone restriction matrices where the distance between pairs and agreeing permutations is not given by Kendall's tau. For example, let A be a 4 x 4 matrix with ones everywhere except in the upper right and lower left corners. The two permutations . 1

( 9 1 4 0 ) and

2

3 4 \ I have Kendall's distance 5, but their graph distance is 6.

J.

4c

Δ

J

216

Persi Diaconis, Ronald Graham and Susan P. Holmes

Remark 3.8 In the case of discrete data(contingency tables) arbitrary patterns of truncation can be handled using the moves in Diaconis and Sturmfels (1998). For this case, the moves are easy to specify. Suppose the x-variable takes on / levels and the y variable takes on J levels. Form the complete bi-partite graph on the set {1,2,..., 1} x {1,2,..., J } . If cell (i, j) is unobservable, delete this edge. The circuits in the remaining graph form a connecting set of moves successively adding and deleting one while traversing the circuit. This suggests two lines of generalization. First, continuous data can be treated by discretization. Second, truncated multivariate data can be approached using the multiway table moves from Diaconis and Sturmfels (1998). 4

An application to ESP guessing experiments

This final Section gives a different set of applications for permutations with restricted positions. A classical test of parapsychology involves a deck of 25 cards made up of the following five symbols of

#+ «• o Each repeated five times. Under ideal conditions the deck is well shuffled and a guessing subject attempts to guess at the cards in order. Under the natural chance model, each guess has chance | of being correct and so the expected number of correct guesses is five. Of course the distribution of the number of correct guesses depends on the guessing sequence. If the subject always guesses the same symbol, there is no variability. It is not hard to show that the variance of the number of correct guesses is largest if the subject guesses some permutation of the values. In Diaconis and Graham(1981), we studied variations where feedback was given to the subject after each guess. For example, suppose the subset is shown the card at position i after guess i (complete feedback). Then the optimal strategy is to guess a card with highest frequency among those remaining. Read (1962) shows in this case the expected number of guesses is 8.65. This is of some practical interest since many early experiments were done with feedback and 8.5+ is reported as the highest of average trials in actual trials. The most interesting type of feedback is yes/no feedback: if a subject guesses correctly, they are told so. If they are incorrect they are only given that information. Now, the subjects optimal strategy is not obvious. We have shown in Diaconis and Graham (1981), that the greedy strategy (guess the most likely value), is only close to optimal. We also determined that the

Independence with Truncation

217

expected number of correct guesses under the optimal strategy is 6.63. The reason for discussing these matters here is twofold. First; the evaluation of the probabilities involved uses permanents. Second; a natural monotonicity conjecture proved by a longish combinatorial argument in Chung, Diaconis, Graham and Mallows (1981) follows from one tool developed to prove the van der Waerden conjecture. To define things, let iV(αi, α2,..., α r ; 6χ, b2, , K) be the number of arrangements of a deck of α\ + α2 + αr = n cards with αι of type i, such that symbol one does not appear in the first b\ places, symbol 2 does not appear in places b\ + 1, b\ + 2 , . . . , &i + &2> and so on. This quantity allows evaluation of the probabilities of events like: The next card is type i given bj "no" responses on type j . From the definition, iV(a, b) = Per(M) where M is the n x n zero/one matrix of the form:

M =

[mi] 1 1

1

1 1

[m 2 ] 1

[m 3 ]

1

1

1

1

1

1

1

1 1

1 1

1 1

1 1

1

1

1

1 1 1

1

1

1

1

1

1 1

1 1

N

with mi an αι x bi block of zeros and the rest all ones. The inequality to be proved is: Proposition 4.1 For any a, b with JV(a, b) φ 0, and any i, 1 < i < r N{a,b) with e{ = ( 0 , 0 , . . . , 1,0,..., 0) the usual ith basis vector. Remark 4.1 In the card guessing context, the inequality has the following interpretation: in a yes-no feedback experiment; the chance that the guess at the next card is of type i cannot decrease if the next card is of type i and is incorrect. n

Proof The argument uses a quadratic form defined on R . Given positive vectors Vi, V2,..., VU-Ί in RΛ define < x\y >= Per[Vu V2,..., Vn-2, ^ v] n

This is a symmetric bilinear form on R . It is used in a crucial way in Egorychev's proof of the permanent conjecture. We follow the account in

218

Persi Diaconis, Ronald Graham and Susan P. Holmes

van Lint and Wilson (1992). They show (Theorem 12.6) that this form is Lorentzian having n — 1 negative eigenvalues and one positive eigenvalue. Such forms are easily seen to satisfy a "reverse Cauchy-Schwarz inequality" for x positive and y arbitrary. <x\y >2 > <x\x >

(4.21)

By continuity, (4.21) also holds if some of the entries in V* or x are allowed to be zero. Proposition 4.1 follows from (4.21). By symmetry of the permanent, it is enough to prove it for e r . Consider the three matrices corresponding to αr x 6r, αr x br + e r , αr x br + 2eΓ. Move these rth blocks to the right of the full matrix. The last two columns of the full matrices with these blocks appear as ' 1 1"

' 1 1"

"1 1 '

1

1

1

1

1

1

1

1

1

1

1

1

0

1

0

0

1

1

:

1

1

0

1

1

*

αr

0

1

0

All other columns axe the same, Call the two columns from the first matrix x and y and apply (4.21). • Remark 4.2 In Chung, Diaconis, Graham and Mallows (1981) it was in fact shown that Πk = AΓ(α, b + kei) is log-concave: n\ > n/-+infc_i. Their proof was combinatorial and only worked for zero-one matrices. It is not clear if there is an analog of log-concavity for more general Lorentzian forms.

Acknowledgements. We thank Brad Efron for providing the original motivation and examples of truncated data, as well as many ideas on the relation to quasi-independence, Steve Stigler for the explanation of Pearson's work, Alistair Sinclair for help with the permanent literature, Steve Fienberg for pointers to the literature on structural zeros and Marc Coram, Mark Huber and Jim Fill for reading the manuscript carefully.

Independence with Truncation

219

REFERENCES

Albers, W., Bickel, P. J,. van Zwet, W. R.(1976) Asymptotic expansions for the power of distribution-free tests in the one-sample problem (Corn V6 pll70), Annals of Statistics,^, 108-156. Bai,Z. Chao,C.C,Liang C , Zhao L. (1997) Error bounds in a central limit theorem of doubly indexed permutation matrices, Annals of Statistics, 25, 2210-2227. Bapat R. B. (1990) Permanents in probability and statistics, Linear Algebra and its Applications, 127, 3-25. Barvinok,A. (1998) A simple polynomial time algorithm to approximate the permanent within a simply exponential factor. Preprint, Mathematical Sciences Research Center, Berkeley. Bickel P. (1969) A distribution-free version of the Smirnov two-sample test in the p-variate case. Annals of Mathematical Statistics, 40, 1-23. Blum, J. R. Kiefer, J. and Rosenblatt, M., (1961) Distribution free tests of independence based on the sample distribution function, Annals of Mathematical Statistics, 485-49. Bolthausen E. (1984) An estimate of the remainder in the combinatorial central limit theorem. Zeitschrift fur Wahrscheinlichkeitstheorie und Verwandte Gebiete, 66, 379-386. Brightwell, G. and Winkler P. (1991), Counting linear extensions, Order, 8, 225-242. Caussinus H. (1966) Contribution a l'analyse statistique des tableaux de correlation, Annales de la Faculte des Sciences de Toulouse(annee 1965), 29, 77-182. Chung F., Diaconis P., Graham R. and Mallows C. L. (1981) On the permanents of the complements of the direct sum of Identity matrices, Advances in Applied Maths., 2, 121-137. Clogg, C.C. (1986) Quasi-independence, in Encyclopedia of Statistical Sciences (9 vols. plus Supplement) 7, 460- 464. Cook, W. (1998) Combinatorial Optimization, Wiley, NY. Daley, D. J. and Vere-Jones, D. (1988) An Introduction to the Theory of Point Processes, Springer-Verlag,NY. Diaconis, P. (1988) Group Representations in Probability and Statistics, Institute of Mathematical Statistics, Hayward, California. Diaconis, P. and Graham R. (1977), Spearman's footrule as a measure of disarray Journal of the Royal Statistical Society, Series B, 39, 262268. Diaconis, P. and Graham R. (1981) The analysis of sequential experiments with feedback to the subjects. Annals of Statistics, 9, 3-23. Diaconis P. and Sturmfels B. (1998), Algebraic algorithms for sampling conditional distributions, Annals of Statistics, 26, 363-397.

220

Persi Diaconis, Ronald Graham and Susan P. Holmes

Efron B. and Petrosian V. (1992), A simple test of independence for truncated data with applications to red-shift surveys, Astrophysicαl Journal, 399,345-352. Efron B. and Petrosian V. (1999), Non parametric methods for doubly truncated data, Journal of the American Statistical Association, 94, 824834. Elderton E.M., Barrington A., Jones H.G., Lamotte E.M., Laski H.J., Pearson, K. (1913) On the correlation of fertility with social value: A cooperative study, Eugenics Laboratory Memoirs XVIII, University of London. Feller W. (1968) Introduction to Probability Theory and its applications, vol I, 3rd edition, Wiley, New York. Goodman L.A. (1968), The analysis of cross-classified data: Independence, quasi-independence, and interactions in contingency tables with or without missing entries, Journal of the American Statistical Association, 63, 1091-1131. Garey M.R. and Johnson D.S. (1979) Computers and intractability, a guide to the theory of NP-completeness, Freeman and co., San Francisco. Godsil C D . (1981) Matching behavior is asymptotically normal. Combinatorica, 1,369-376. Graham R.L., Knuth D., Patashnik, (1989) Concrete Mathematics, Addison Welsley, Reading, MA. Guo, S. W., Thompson, E. A.,(1992), Performing the exact test of HardyWeinberg proportion for multiple alleles, Biometrics, 48, 361-372 Hanlon P. (1996), A random walk on the rook placements on a Ferrers board. Electronic Journal of Combinatorics, vol 3,

http://math34.gatech.edu:8080/Journal/journalhome.html Hoeffding, W. (1948), A non-parametric test of independence, Annals of Mathematical Statistics, 19, 546-557. Jerrum, M.R. and Sinclair, A. J.(1989), Approximating the permanent. SI AM Journal on Computing 18 , 1149-1178. Jerrum, M.R. and Vazirani,U. (1992) A mildly exponential approximation algorithm for the permanent. Proceedings of the 33rd Annual IEEE Conference on Foundations of Computer Science, 320-326. Karmarkar,N.,Karp, R.,Lipton,R., Lovasz,L. and Luby,M., (1993) A MonteCarlo algorithm for estimating the permanent. SI AM Journal on Computing 22, 284-293. Lai, T.L. Ying, Z. (1991), Rank regression methods for left-truncated and right-censored data, Annals of Statistics, 19, 531-556. Lazeronni L. and Lange K. (1997), Markov chains for Monte Carlo tests of geneteic equilibrium in multidimensional contingency tables, Annals of Statistics, 138-168. Lehmer D.H. (1970) Permutations with strongly restricted displacements, in Combinatorial Theory and its Applications II, Eds. Erdos P., Renyi A., Sόs V., 755-770 North Holland, Amsterdam.

Independence with Truncation

221

Lieb E.H. (1966) Proofs of some conjectures on Permanents, Journal of Mathematics and Mechanics, 16, 127-134. Lovasz L. and Plummer (1986) Matching Theory, North Holland, Amsterdam. Lynden-Bell, D. (1971) A method of allowing for a known observational selection in small samples applied to 3CR quasars, Monographs of the National Royal Astrophysical Society, 155, 95-118. Lynden-Bell, D. (1993) Eddington-Malmquist bias, streaming motions, and the distribution of galaxies (Disc: p217-220) in Statistical Challenges in Modern Astronomy, ed. Babu J. and Feigelson E., 201- 216. Mallows,C. (1957) Non-null ranking models I, Biometrika, 44, 114-130. Mann B.(1995) Ph D thesis, Harvard University, Cambridge,MA. Pitman J. (1997) Probabilistic bounds on the coefficients of polynomials with only real zeros, Journal of Combinatorial Theory,k, 77, 279-303. Rasmussen,L.E.(1998) On approximating the permanent and other #Pcomplete problems. PhD Thesis, University of California at Berkeley. Rasmussen,L.E.(1994) Approximating the permanent: a simple approach. Random Structures and Algorithms 5, 349-361. Read R. (1962) Card guessing with information-a problem in probability. American Mathematics Monthly, 69, 506-511. Reinert G. (1998) Stein's method for the Chi-2 statistic, Technical Report Riordan J. (1958) An introduction to combinatorial theory, Wiley, N.Y. Romano, J. P. (1989) Bootstrap and randomization tests of some nonparametric hypotheses, Annals of Statistics, 17, 141-159 Schneller,W. (1989), Edgeworth expansions for linear rank statistics, Annals of Statistics, 17, 1103-1123. Sinclair A. (1992) Algorithms for random generation and counting. A Markov chain approach. Birkhaeuser. Stein C. (1986) Approximate computation of expectations Institute of Mathematical Statistics, Lecture Notes and Monographs, Hayward, California. Stembridge J. (1991) Immanants of totally positive matrices are nonnegative, Bulletin of the London Mathematics society,23, 422-428. Stembridge J. (1992) Some conjectures for Immanants, Canadian Journal of Mathematics, 44, 1079-1099. Stanley R. (1986) Enumeratiυe Combinatorics, volume I, Wadsworth/Brooks, Cole, Monterey, CA. Stigler S. (1992) Studies in the history of probability and statistics XLIII. Karl Pearson and quasi-independence, Biometrika,79,563-575. Tsai,W.Y. (1990), Testing the assumption of independence of truncation time and failure time, Biometrika, 77, 169-177.

222

Persi Diaconis, Ronald Graham and Susan P. Holmes

Tsui, K.L., Jewell, N. P. Wu, C. F. J. (1988) A nonparametric approach to the truncated regression problem, Journal of the American Statistical Association^, 785-792. Valiant L.,(1979) The complexity of computing the permanent, Theoretical Computer Science, 8, 189-201. Van Lint J. and Wilson,R. (1992) A course in combinatorics , Cambridge University Press, Cambridge. Zeckendorf,E. (1972) Representation des nombres naturels par une somme de nombres de Fibonacci ou de nombres de Lucas, Bulletin de la Societe Royale Scientifique de Liege, 41,179-182.

PERSI DIACONIS MATHEMATICS AND STATISTICS SEQUOIA HALL STANFORD UNIVERSITY

CA 94305-4065 RONALD GRAHAM COMPUTER SCIENCE UNIVERSITY OF CALIFORNIA AT SAN DIEGO AND ATT, FLORHAM PARK, NJ.

graham @ucsd. edu

SUSAN HOLMES STATISTICS STANFORD UNIVERSITY AND UNITE DE BIOMETRIE INRA-MONTPELLIER, FRANCE

susan @stat. Stanford, edu

MARKOV CHAIN CONDITIONS FOR ADMISSIBILITY IN ESTIMATION PROBLEMS WITH QUADRATIC LOSS

MORRIS L. EATON

1

University of Minnesota Consider the problem of estimating a parametric function when the loss is quadratic. Given an improper prior distribution, there is a formal Bayes estimator for the parametric function. Associated with the estimation problem and the improper prior is a symmetric Markov chain. It is shown that if the Markov chain is recurrent, then the formal Bayes estimator is admissible. This result is used to provide a new proof of the admissibility of Pitman's estimator of a location parameter in one and two dimensions. AMS subject classifications: K1232, H3789. Keywords and phrases: estimation, quadratic loss, admissibility, Markov chain, recurrence.

1

Introduction

In this paper we consider a classical parametric estimation problem when the loss is quadratic. Here attention is restricted to the so-called formal Bayes estimators - that is, estimators obtained as minimizers of the posterior risk calculated via a formal posterior distribution. Because the loss is quadratic, admissibility questions regarding such estimators are typically attacked using the explicit representation of the estimator as the posterior mean of the function to be estimated. Examples can be found in KARLIN (1958), STEIN (1959), ZIDEK (1970), PORTNOY (1971), BERGERand SRINIVASAN (1978), BROWN and HWANG (1982), EATON (1992), and HOBERT and ROBERT (1999). To describe the problem of interest here, let P(dx\θ) be a statistical model on a sample space X where the parameter θ E Θ is unknown. That is, for each 0, P( \θ) is a probability measure on the Borel sets of X. Both X and Θ are assumed to be Polish spaces with the natural σ-algebra. Given a real valued function φ(θ) that is to be estimated, consider the loss function (1.1) 1

L{α,θ) = (α

Research was supported in part by National Science Foundation Grant DMS 96-26601. Research was supported in part by grants from CWI and NWO in the Netherlands.

224

Morris L. Eaton

In order to define a formal Bayes estimator of φ(θ), let v be a σ-finite improper prior distribution defined on the Borel sets of Θ, so i/(θ) = +00. The marginal measure on X is defined by (1.2)

M(B)=

(P{B\θ)v(dθ) Θ

for Borel subsets of X. When M is σ-finite (assumed throughout this paper), then a formal posterior Q{dθ\x) exists and is characterized by (1.3)

P(dx\θ)v{dθ) = Q{dθ\x)M{dx).

The equality in (1.3) means that the measures o n ί x θ defined by the left and right side of (1.3) are equal. The formal posterior Q(-\x) is a probability measure for each x £ X, For a discussion of the existence of Q and uniqueness (up to sets of M-measure zero), see JOHNSON (1991). When the loss is (1.1) and the improper prior is z/, the formal Bayes estimator of φ(θ) is defined to be the point α(x) which miminizes (over α's)

J(α-φ(θ))2Q(dθ\x).

(1.4) Of course, the minimizer is

(1.5)

φ(x) = Iφ(θ)Q(dθ\x).

For the present, questions concerning the existence of integrals will be ignored. The risk function of this estimator is (1.6)

R(φ,θ) =

Eθ(φ(X)-φ(θ))2

where EQ denotes expectation under P{ \θ). The main focus of this paper concerns the admissibility of φ and the relationship of this admissibility to a Markov chain associated with the estimation problem. For our purposes, the relevant notion of admissibility is the following (STEIN (1965)). Definition 1.1 For any estimator t(X) of φ(θ), let R{t,θ) = Eθ(t(X) φ(θ))2 be the risk function of t. The estimator φ is almost-v-admissible (α — v — a) if for every estimator t which satisfies (1.7)

i?(ί,0)
the set (1.8)

B =

has v-measure zero.

{θ\R(t,θ)
Markov Chains and Admissibility

225

In other words, φ is α — v — α if there is no estimator t which is at least as good as φ everywhere (i.e., (1.7) holds) and which beats φ on a set of positive z/-measure. An important technical tool for establishing α — v — α is the so-called Blyth-Stein condition (BLYTH (1951), STEIN (1955)). To describe this condition, let C be a measureable subset of Θ with 0 < i/(C) < +oo. Consider the following class of real valued functions defined on Θ: U(C) = {g\g > 0, g is bounded , (1.9) g(θ) > 1 for θ e C,

r / g(θ)u(dθ) < +oo}.

For g £ U(C), think of g{θ)v(dθ) as defining a proper prior distribution (it has not been normalized to integrate to one) and consider the marginal measure on X given by

(1.10)

Mg(B) = J P(B\θ)g(θ)u(dθ).

Because the measure Mg is finite, we can write (as in (1.3)), (1.11)

P{dx\θ)g(θ)u{dθ)

=

Qg{dθ\x)Mg{dx)

where Qg(dθ\x) now is a proper posterior distribution corresponding to the proper prior cg{θ)v(dθ) where c is the normalizing constant. Thus, the Bayes solution to the estimation problem is the Bayes estimator

(1.12)

φg(x) = Jφ(θ)Qg(dθ\x)

which is the posterior mean of φ(θ). Next, consider the integrated risk difference (1.13)

IRD(g) = J[R(φ,θ) - R(φg,θ)]g(θ)v(dθ).

Roughly (subject to some regularity described precisely in later sections), one version of the Blyth-Stein condition is: For sufficiently many sets C, (1.14)

inf IRD{g)=0. geu(C)

When (1.14) holds, then φ is a α - v - a (for example, see STEIN (1965)). In typical examples, a direct verification of (1.14) is not routine. A main result in this paper provides an upper bound for IRD(g) which allows us to use results from Markov chain theory to establish a sufficient

226

Morris L. Eaton

condition for (1.14). This result, established in Section 3 under regularity conditions, is the following: For g E U{C), IRD(g) < Δ(y/g) where Δ(Λ) =

" Φ(v))2Q(dθ\x)Q(dη\x)M(dx)

I θ θ X

is defined for real valued functions h. Although the function Δ(h) looks rather complicated, there is a Markov chain associated with Δ lurking in the background. To see this, recall (1.3) and let (1.16)

R{dθ\η) =

ίQ(dθ\x)P(dx\η).

x Then R{-\η) is the expected value of the formal posterior Q(-\x) when the model is P( \η). Obviously, R( \η) is a transition function (see EATON (1992, 1997) for further discussion; see HOBERT and ROBERT (1999) for some related material) and we can write

(1.17)

Δ(Λ) = J J(h(θ) - h(η))2(φ(θ) - φ(η))2R(dθ\η)v(dη). Θ

e

Then, with ' (1.18)

Φ(jl) =

< T(dθ\η) = ξ(dη)

=

J(Φ(θ)-φ(η))2R(dθ\η) 2

%-\η){φ{θ)-φ{η)) R{dθ\η) φ(η)u(dη)

it follows that

(1.19)

Δ(Λ) = I J(h(θ) - h(η))2T{dθ\η)ξ(dη). θ θ

By definition, T(dθ\η) is a transition function and hence defines a discrete time Markov chain, W = (WQ = 77, W\, W2,...) whose state space is Θ and whose path space is Θ°°. That is, under T( |77), the chain starts at Wo — η and the successive states of the chain Wi+i have distribution Γ( |Wi), i = 0,1,2, Under some regularity conditions to be specified later, when

Markov Chains and Admissibility

227

the chain W is "recurrent", it follows from results in EATON (1992, Appendix 2) that

{

for each set C with 0 < i/(C) < +oo, geu(C)

Therefore, the recurrence of the chain W implies that (1.14) holds and hence α — v — α for φ obtains. In summary, the above argument runs as follows: (i) The Blyth-Stein condition (1.14) is sufficient for α — v — α. (ii) The integrated risk difference is bounded above by Δ(y/g) as in (1.15). (iii) When the Markov chain associated with Δ is recurrent, then (1.20) implies (1.14) holds and we have α — v — α. Step (i) is a well known technique in decision theory and has appeared in many application such as those listed at the beginning of this section. Step (iii) was used in EATON (1992) and is a direct consequence of general results concerning symmetric Markov chains. What is new in this paper is step (ii) as expressed in (1.15). Inequalities like (1.15) were used in EATON (1992) but only for bounded functions φ. Thus the advance here is the extension of the Markov chain arguments to cover cases of estimating unbounded functions such as mean values. The following is a simple, but not so trivial, example which shows how the results described above can be applied. E x a m p l e 1.1 Let / be a symmetric density with an absolute third moment on JR1 and assume one observation X is made from f(x — θ)dx where θ is an unknown translation parameter, θ £ R1. The loss function is (α — θ)2 so the parameter θ is to be estimated. Consider the improper prior distribution dθ so the formal posterior is Q(dθ\x) — f(x — θ)dθ. Thus the formal Bayes estimator is f

ΘQ{dθ\x)

= x 2

and the risk function is just the constant E^X where EQ denotes expectation when 0 = 0. The Markov chain associated with this problem has transition function T given in (1.18). A routine calculation shows that the transition function R(dθ\η) of (1.16) is R(dθ\η) = r(θ-η)dθ where

(u) = r(-u) = / f(x-u)f(x)dx

228

Morris L. Eaton

is a density on R1. Thus 2

ψ(η) = f(θ-η) R(dθ\η)

2

= ίθ r{θ)dθ = c2

is constant. From (1.18) we have T(dθ\η) = (fl"7?) r^~^de

= t{θ-η)dθ

C2

and ξ{dη) = c2dη. Therefore T is a translation kernel with density t, so the Markov chain associated with T is a random walk on R1. Thus the existence of a first moment for t implies this random walk is recurrent (CHUNG-FUCHS (1951)). Using the definition of t and the third moment assumption for / yields

/

-

I ί \u\t{u)du = — / \uΓr(u)du = C2 J

3

ί ί \u\ f{x - u)f{x)dudx < -E0\X\* < +oo.

02 J J

C2

Hence the random walk is recurrent and the estimator x is almost admissible (relative to Lebesque measure). Of course, this example is just a very special case of the admissibility of Pitman's estimator on R1 when third moments exist. This was first established by STEIN (1959) using the Blyth-Stein method directly. Here is a brief summary of this paper. Section 2 contains the formal problem statement, basic assumptions, and a statement of the Blyth-Stein condition. The basic inequality is proved in Section 3 while Section 4 contains some background material on symmetric Markov chains. The main theorem connecting recurrence and admissibility is proved in Section 5, while some useful extensions are described in Section 6. The results are then applied in Section 7 to provide an alternative proof of the admissibility of the Pitman estimator of a location parameter in one and two dimensions. BROWN (1971) considered the problem of estimating the mean vector of a multivariate normal distribution when the loss is quadratic. Under regularity conditions, he established a close connection between admissibility and the recurrence of an associated diffusion process defined on the sample space. The relationship between Brown's work and the results here remains quite obscure. For further discussion, see EATON (1992, 1997).

Markov Chains and Admissibility 2

229

Notation and Assumptions

Certain integrability assumptions are needed to justify the arguments that are sketched in Section 1. Some of these assumption are stated here. The two spaces X and Θ are assumed to be Polish spaces with the natural σ-algebras. The model P(dx\θ) is a Markov kernel and the improper prior distribution v is σ-finite. The marginal measure M(dx) defined in (1.2) is assumed to be σ-finite so that equation (1.3) holds for the formal posterior Q{dθ\x). Let φ be a real valued function defined on Θ such that

(A.I)

I φ2(θ)Q(dθ\x) < +oo for all x..

Then the formal Bayers estimator φ(x) given in (1.5) is well defined. The risk function defined by (1.6) is assumed to satisfy the following local integrability condition: There exists an increasing sequence of (A.2)

>

sets {Ki} such that (wJ K{ = Θ, 0 < u(Ki) < oo, / R(φ, θ)u(dθ) < oo, for each i.

Observe that if g E U(Ki) (as defined in (1.9)) and g vanishes outside some Kj with j > ΐ, then the integrated risk

(2.1)

j R{φ,θ)g{θ)v{dθ).

is finite. Now, recalling (1.9), let g E U(C) and consider

(2.2)

g{x) = jg(θ)Q(dθ\x).

Recall that the marginal measure Mg is

(2.3)

Mg(B) = J J IB(x)P(dx\θ)g(θ)v(dθ). Θ X

Using (1.3), we see (2.4)

Mg(dx) = g(x)M(dx)

so that g is the Radon-Nikodym derivative of Mg with respect to M. Hence the set Ao = {x\g(x) = 0} has Mg measure zero. Now, define Qg(dθ\x) as follows: Q(dθ\x)

230

Morris L. Eaton

It is then easy to verify that (2.6)

P(dx\θ)g{θ)v(dθ) =

Qg(dθ\x)Mg(dx).

Therefore the Bayes estimator

(2.7)

φg(x) = I φ(θ)Qg(dθ\x)

is well defined because (A.I) and the boundedness of g imply ί φ2(θ)Qg{dθ\x)

(2.8)

< +oo for all x.

A rigorous statement of the Blyth-Stein Lemma follows. Given a K% in (A.2), let (2.9)

Ό\Ki)

= {g\g E U(Ki), jR(φ,θ)g(θ)v(dθ)

< +oo}.

Theorem 2.1 (Blyth-Stein Lemma). For each i, assume that (2.10)

inf

IRD{g)

= 0.

geU*(Ki)

Then φ is a — v — a. Proof The proof of this well known condition is by contradiction. The details are left to the reader. • Theorem 2.2 For g

(2.11)

eU*(Ki),

IRD(g) = J(φ(x) - φg(x))2g(x)M(dx).

Proof The proof of (2.11) is routine algebra coupled with the earlier observation that AQ has Mg measure zero. • 3

The Basic Inequality

In this section, the inequality described in (1.15) is established for g G U*(K{),i = 1,2, Here is a basic lemma which may be of independent interest. 2

Lemma 3.1 Let W and Y be real valued random variables such that EW < +oo,y > 0, and μ = EY < +oo. Also let (W,Ϋ) be an independent and identically distributed copy of (W,Y). Then (3.1)

[Cov(Wi Y)]2 < μE(W - W)2(VΫ

-

Markov Chains and Admissibility Proof

231

A direct calculation shows that

Cov{W,Y) = \E(W -W)(Y

-Ϋ).

Writing

= (y/Ϋ - y/γ)(y/Ϋ + y/Ϋ)

(Y-Ϋ)

and using the Cauchy-Schwarz inequality yields

[Cσv(W,Y)}2 < ].E(VΫ + VΫ)2E{W -

W)2(VΫ-

But (VΫ+ y/Ϋ)2 < 2(Y + Ϋ) so that \E{VΫ + VY)2 < μ. This completes the proof. • Theorem 3.1 For g E U*{Ki), (3.2)

IRD(g) < A(y/g)

where Δ is defined in (1.15). Proof For each x E Ag = {x\g(x) > 0},

φ{x) - φg(x) = I φ(θ)Q(dθ\x) - I φ(θ)Qg(dθ\x) (3.3)

= =

^Jφ(θ)(g(χ)-g(θ))Q(dθ\x) -TΠΛCOVΛΦ,9)

where Covx denotes covariance under the probability measure Q( \x). The last equality follows since g(x) is the mean of g(θ) under Q( \x). Applying inequality (3.1) with W = φ and Y = g, we have

(φ(χ)-φg(χ)f (3.4)

= -

1 θ

θ

Subsituting this inequality into the rightside of (2.11) clearly yields (3.2). This completes the proof. • The upper bound A(y/g) in (3.2) depends only on the three essential components of the original problem - namely the model, the improper prior and the function φ to be estimated. Of course this statement assumes that

232

Morris L. Eaton

the loss is quadratic. When the function φ is bounded, say \φ(θ\ < c, then obviously (3.5) where

The function Δi appeared first in EATON (1992) and was used to relate Markov chain recurrence to admissibility questions regarding the estimation of bounded functions. Not only is the argument here more general, it is far more transparent than the original in the case when φ is bounded.

4

Symmetric Markov chains

Some basic theory concerning symmetric Markov chains with values in a Polish space is described here. Of course, the emphasis is on those aspects of the theory which are most directly related to the admissibility questions under consideration here. The discussion follows EATON (1992, Appendix 2) quite closely. Let (y, B) be a measurable space where y is Polish and B is the usual Borel σ-algebra. Consider a Markov kernel S(du\υ) defined on B x y so that S( \v) is a probability measure for each v £ y and S(B\ ) is #-measureable for each B E B. Let ξ be a σ-finite measure defined on B with ζ(y) > 0. Definition 4.1 The Markov kernel S(du\v) is ξ-symmetric if the measure (4.1)

m{du,dv)

= S{du\v)ξ{dv)

defined on B x B is a symmetric measure. In all that follows, S(du\v) is assumed to be ξ-symmetric. The assumption that ξ is σ-finite is important (see the development in Appendix 2 in EATON (1992)). The symmetry of m implies that m has marginal measures ξ-that is, (4.2) m(y x B) = m(B x y) = ξ(B). Of course, (4.2) implies that ξ is a stationary measure for S(du\υ) since

(4.3)

Js(B\v)ξ(dv) = ζ(B). y

Now, each Markov kernel defines a Markov chain, and conversely, to specify a Markov chain one needs, at least implicity, a Markov kernel. A Markov chain is called symmetric if this Markov kernel is symmetric with respect

Markov Chains and Admissibility

233

to some σ-finite measure. For finite and countable state spaces, symmeteric Markov chains are also called reversible chains, but that terminology is not used here (see KELLY (1979) or LAWLER (1995)). According to the above terminology, a symmetric Markov chain on y gives rise to a symmetric measure (as in (4.1)) on B x B and this symmetric measure has a σ-finite marginal measure as defined in (4.2). Conversely, suppose n(du, dv) is a symmetric measure on B x B and suppose its marginal measure (4.4) μ{B) = n{B x y) is σ-finite. This implies that there is a unique (up to sets of n-measure zero) Markov kernel T(du\v) such that (4.5)

n(du,dυ) = T(du\υ)μ(dυ).

This result seems to be well known but I do not know a reference with an explicit statement. A slightly more general result can be found in JOHNSON (1991). The above discussion shows there is a one to one correspondence between symmetric Markov chains and symmetric measures with σ-finite marginals. This observation is what allows us to associate a Markov chain with the function Δ appearing in (1.15). More about this in the next section. Now, let S(du\v) be ^-symmetric and let Y = (YQ = v, Yί, I2? •) be the corresponding Markov chain with values in y. The notation means the chain starts at v and the succesive Y +i have distribution S(-\Yi) for i = 0,1, — The joint measure of the chain on y°° is denoted by Prob( |?;) where YQ = v is the initial state of the chain. Next, we turn to a discussion of recurrence when S(du\v) is ^-symmetric. Definition 4.2 Let B E B satisfy 0 < ξ(B) < +00. The set B is locally-ξrecurrent (l — ξ — r) if the set (4.6)

{υ\v e £,Prob(lj G B for some j > l\v) < 1}

has ξ measure zero. In other words, B is I — ξ — r if except for a set of starting values of ξmeasure zero, the chain returns to B with probability one when it starts in B. A characterization of local-ξ-recurrence can be given in terms of a quadratic form. For h G L2(ξ), the linear space of ξ square integrable functions, define D(h) by

(4.7)

D(h) = ί

f(h(u)-h(v))2m(du,dv).

where m is the symmetric measure given by (4.1). For B such that 0 < ξ(B) < +00, let (4.8)

V(B) = {h\h > 0, h € L2{ξ), h{u) > 1 for u G B}.

234

Morris L. Eaton

Theorem 4.1 The following are equivalent: (i) B (ii)

isl-ξ-r

inf Dlh) = 0 hev(B) v J

Proof This is a direct consequence of Theorem A.2 in

EATON

(1992). •

For our applications, a slight strengthening of Theorem 4.1 is needed. Let C G B satisfy C D B and ξ{C) < +oo. Then set (4.9) V{B, C) = {h\h e V{B), h is bounded , h{u) = 0 for u e Cc}. Theorem 4.2 Consider Cγ C C 2 C following are equivalent (i) B

isl-ξ-r

(ii) lim

inf

«Λίή B C C i αnrf limC< = y. The

Dlh) = 0.

i^ooheV{Bd)

Proof This is a consequence of results in

EATON

(1992, Appendix 2). •

It is Theorem 4.2 which will be used to establish a connection between the Blyth-Stein condition and recurrence. Definition 4.3 The chain Y is locally-ξ-recurrent if for each set B with 0 < ξ(B) < +oo, B isl-ξ-r. It is not too hard to show that Y is locally-ξ-recurrent iff there exists an increasing sequence of sets C\ C C 2 C with 0 < ξ(Ci) < +oo and lim d = y such that each d is / — ξ — r. In applications one can often choose a convenient sequence of sets d m order to check I — ξ — r. The quadratic form D(h) in (4.7) is well known in the theory and applications of symmetric Markov chains. In the probability literature \D{h) is known as the Dirichlet form associated with the symmetric measure m, or the symmetric transition S in (4.1). It is typical to write \D(h) in terms of the linear transformation S* defined on L2(ξ) as follows: (4.10)

(S*h){v) =

ίh{u)S{du\v). 2

Let (Λi,/i2) denote the standard inner product on L (ξ) given by

(4.11)

(huh2)

= j hλ{u)h2{u)ξ(du).

Markov Chains and Admissibility

235

A routine calculation shows that (4.12)

l

=

-D{h)

(h,(I-S*)h)

where / is the identity. The operator / — S* is commonly called the LaPlacian. Further discussion and some applications can be found in DlACONlS and STROOK (1991) and LAWLER (1995). 5

Recurrence implies admissibility

It is argued here that, under an additional assumption, recurrence of the Markov chain associated with the quadratic form

(5.1) Δ(Λ) = JJ J(h(θ) - h(η))2(φ(θ) - φ(η))2Q(dθ\x)Q(dη\x)M(dx) Θ

Θ X

will imply that the Blyth-Stein condition of Theorem 2.1 holds, so that φ is α — v — α. To carry out this argument, first observe that the measure on Θ x Θ given by (5.2)

α(dθ,dη) = ί(φ(θ) -

φ(η))2Q(dθ\x)Q(dη\x)M(dx)

x is, by inspection, symmetric. Using (1.3) and (1.16), the measure α can be written (5.3) α(dθ,dη) = (φ(θ) - φ{η))2R{dθ\η)v{dη) where R(dθ\η) is a transition function and v is the improper prior used to defined the estimator φ[x) in (1.5). Next, for r E θ , let

(5.4)

φ(η) =

j{φ(θ)-φ{η))2R{dθ\η).

The following assumption controls the behavior of φ and is expressed in terms of the sets K{ appearing in assumption (A.2) of Section 2. 0 < φ(η) < +oo for all η E θ , and (A.3)

/ ψ(η)v(dη) < +oo for all i.

Theorem 5.1 Assume (A.3) holds. Then the symmetric measure a has a σ-finite marginal measure (5.5)

ξ(dη) =

236

Morris L. Eaton

Further, with (5.6)

T(dθ\η) = φ-\η)(φ(θ) - φ(η))2R(dθ\η),

The measure a is given by (5.7)

a(dθ,dη) = T(dθ\η)ξ(dη).

Proof That (5.7) holds is immediate from (5.3) and the definition of ξ and T. Since T(dθ\η) is a transition function by definition, integration of (5.7) over Θ shows that a has ξ as a marginal measure. The σ-finiteness of ξ is immediate from assumption (A.3). This completes the proof. • Now, let W = (Wo = r?, Wi, W 2 ,...) be the Markov chain on Θ with transition function T. The above discussion shows that T is ^-symmetric (i.e. W is a symmetric Markov chain). Observe that the quadratic form associated with this chain as defined in (4.7) is exactly Δ given in (5.1). In other words, for h G L2(ξ),

(5.8)

Δ(Λ) = I J(h(θ) - h(η))2a(dθ, dη)

so that the results described in Section 4 are directly applicable. Here is the main result of this paper. Theorem 5.2 Assume (A.I), (A.2) and (A.3) hold. If the Markov chain W associated with the quadratic form Δ is locally-ξ-recurrent, then the formal Bayes estimator φ(x) is almost-v-admissible. Proof It suffices to show that condition (2.10) holds for each ΐ, ί = 1,2, — Fix an index j > i and consider the set V(K^ Kj) defined in (4.9). Assumptions (A.2) and (A.3) show that if y/g <Ξ V{KuKj) then g E U*(Ki). This observation together with the basic inequality (3.2) yields (5.9)

inf IRD(g)< K eu*{K)

inf Δ{y/g). /ev(κK)

By assumption the chain W is / - ξ - r so the limit of the right side of (5.9) as j ; -» oo is zero. Thus for each i, (2.10) holds and the proof is complete. •

6

An Extension

In this section, we extend the results of the previous sections to cover the case of estimating a vector valued function φ(θ, #), θ G Θ, x G X. The model fc P(dx\θ) and the improper prior are as in Section 2. For vectors w G i2 , \\w\\

Markov Chains and Admissibility

237

denotes the usual Euclidean norm. The loss function for the estimation problem is L{α,θ,x) = \\α-φ(θ,x)\\2,αeRk

(6.1)

so φ(θ, x) is a fc-dimensional vector and the loss function now depends on x £ X. The following assumption is the appropriate analogue of (A.I) given in Section 2. Assume (B.I)

ί\\φ(θ,x)\\2Q(dθ\x) < +oo for all x

where Q(dθ\x) is the formal posterior. Thus, the formal Bayes estimator is now the vector function

(6.2)

φ(x) = J φ(θ,x)Q(dθ\x).

Of course, the risk function is

R(φ,θ) = J\\φ(x)-φ(θ,x)\\2P(dx\θ).

(6.3)

Assumption (B.2) is that the risk function satisfies the local integrability condition (A.2) given in Section 2. Now, the Blyth-Stein Lemma given in Theorem 2.1 remains valid and the analogue of Theorem 2.2 is Theorem 6.1 For g E U*(Ki),

IRD(g) = J \\φ(x) - φg(x)\\2g(x)M(dx).

(6.4)

Proof Apply Theorem 2.2 one coordinate at a time to the problem of estimating φj(θ, x) where φj(θ, x) is the j t h coordinate of 0(0,x). Then sum on j to obtain (6.4). This completes the proof. • The next step is to extend Theorem 3.1 to the case at hand. To this end, define Δ 2 for real valued functions h(θ) by Δ2(Λ) (6 5)

'

-

///w f l )- f c M) 2 iw^^ Θ Θ X

Theorem 6.2 For g E U*(Ki), (6.6)

IRD{g) < Δ2(y/g)

where Δ 2 is defined by (6.5).

238

Morris L. Eaton

Proof The argument used to prove Theorem 3.1 shows that for each C x € A Q = {x\g(x) > 0},

(6.7)

i f f , 9 t 92 77-7 / / {φj{θ,x) - Φj{η,x)) Wg{θ) - y/g(η)) Q(dθ\x)Q(dη\x). x 9\ ) J J

Summing this on j , integrating with respect to g{x)M{dx), and using (6.4) shows that (6.6) holds. This completes the proof. • The final step in the argument here is to associate a symmetric Markov chain with Δ2. To this end, define the measure a2 on Θ x Θ by (6.8)

a2{dθ,dη) = J \\φ{θ,x) - φ(η,x)\\2Q(dθ\x)Q(dη\x)M(dx).

x Obviously, OLΊ is symmetric. To formulate the analogue of assumption (A.3) of Section 5, define ^(rj) by

(6.9)

φ2(η) = J J\\φ(θ,x)-φ(η,x)\\2Q(dθ\x)P(dx\η). x θ

Now, make the following assumption: 0 < ^2(77) (B.3)

/

< +00 for all η G θ and < +00 for all i.

Setting (6.10)

T2(dθ\η) =

ψϊι(η)J\\φ(θ,x)-φ(η,x)\\2Q(dθ\x)P(dx\η), x

and (6.11)

ζ2{dη) = φ2{η)ιy(dη)^

it is clear that T2 is a transition function, the measure ξ2 is cr-finite since (B.3) holds, and (6.12)

a2(dθ,dη) = T2{dθ\η)ξ2{dη).

Thus, a2 has a σ-finite marginal measure ζ2 and the results described in Section 5 apply directly to the Markov chain with transition function T2. The extension of Theorem 5.2 is now immediate.

Markov Chains and Admissibility

239

Theorem 6.3 Assume (B.I), (B.2) and (B.3) hold. If the Markov chain associated with the quadratic form Δ2 is locally-ξ-recurrent, then the formal Bayes estimator φ(x) is almost-v-admissble. Proof

7

The proof of Theorem 5.2 applies directly. •

Admissibility of the Pitman Estimator

Here we provide an alternative proof of the almost admissibility of the Pitman estimator of a location parameter in one and two dimensions. The original proofs (STEIN (1959) for one dimension and JAMES and STEIN (1961) for two dimensions) are based on a direct verification of the Blyth-Stein condition. The proof given here uses Markov chain arguments via Theorem 6.3. For notational convenience, we consider a model in the so-called invariant Pitman form. A random quantity X = (Y, Z) is to be observed where Y is a k-vector and Z takes value in a Polish space Z. The parametric model for X is assumed to have the form (7.1)

P{dx\θ) = f(y-θ,z)dyλ(dz)

where dy is Lebesque measure onfl f e , λ is a σ-finite measure on the Borel sets of Z, θ is an unknown vector in Rk and / is a density with respect to the product measure dyλ(dz). The function to be estimated is the vector function φ(θ) = θ, the loss is quadratic and the improper prior distribution is Lebesque measure dθ on Rk. It is clear that

(7.2)

m(z) = j f(y-θ,z)dy = J f(y,z)dy

is the marginal density of Z with respect to λ. It is easy to see that the marginal measure on Rk x Z is given by (7.3)

M{dy,dz) = m(z)dyλ{dz)

and is σ-finite. Define q(θ\y,z) by (7.4)

q(θ\y,z) =

f fJ^r

if 0 < m(z) < oo . II Qo(y-( Qo{y ~ θ) otherwise

where go is a density on Rk with finite second moments. A routine argument shows that (7.5) Q{dθ\y,z) = q(θ\y,z)dθ

240

Morris L. Eaton

serves as a formal posterior so (1.3) holds. The Pitman estimator for θ is

(7.6)

φ(y,z) = JθQ(dθ\y,z),

which is the formal Bayes estimator for θ. The formal statement regarding the almost admissibility of φ is the following. Theorem 7.1 For k = 1 or k — 2 assume that

(7.7)

J J\\y\\'2+kf(y,z)dyX(dz) < +00.

Then φ in (7.6) is an almost admissible estimator for θ £ Rk, k = 1,2. Proof The arguments for k = 1 and k = 2 are essentially the same. The details are given for the case of k = 1. Assumption (7.7) implies that the set

(7.8)

N = {z\J\yff(y,z)dy = +00}

has λ-measure zero. Thus the density / can be set equal to zero on this set without changing the problem,. In what follows, assume this has been done. Now, assumption (A.I) follows immediately. Since the estimator φ is translation invariant, i.e. (7.9)

Φ(y-c,z) = c

and the model is invariant under translation, it follows that the risk function R(φ, θ) is a constant given by

(7.10)

co = I J(φ(y,z))2f(y,z)dyλ(dz).

That co < +00 follows from (7.7). Therefore assumption (A.2) holds with Ki = [ - M ] C Λ \ i = l,2,.... For the verification of (A.3), first observe that the transitition function defined in (1.16) is, in the present context, given by (7.11)

R{dθ\η) = r(θ-η)dθ

where

(7.12)

r(u) = r(-ti) = j Jq(u\y,z)f(y,z)dyλ(dz)

is a density on R1. Therefore the function φ(η) defined in (5.4) is (7.13)

ψ{η) = ί(θ - ηfr{θ - η)dθ = ί u2r{u)du

Markov Chains and Admissibility

241

which is a constant, say c\. Again (7.7) implies that c\ < +00 so (A.3) holds. The final step in the proof requires us to show that the Markov chain with the transition function (7.14)

c^(θ-η)2r(θ-η)dθ

T(dθ\η) =

is almost-ξ-recurrent. Since the transition function T has the form

(7.15)

T{dθ\η) = t{θ-η)dθ

where t is a symmetric density on i? 1 , a sufficient condition for recurrence is (7.16)

ί \u\t(u)du < +00.

(see CHUNG-FUCHS (1951)). Substituting the expressions for t and r into (7.16) shows that / \u\t{u)du = c[ι / \u\3r(u)du

= cϊλ J j J \u\3q(u\y, z)f(y, z)dyλ{dz)du

= cϊι j J J \u\3f(y - u, z)f{y, z)~-dy\{dz)du

(7 17 )

-ι 11 J\y_ wff{w^ z)f{y^ z)-l^dydwλ(dz)

=c

< 4CΓ11 I I I \y\33f(w, z)f(y, z) ^dydw\{dz) / / Iw\ f{w,z)f(y,z)—rn{z) J J J 3

Γ

=

8cϊ1f\y\3f(y,z)dyλ(dz).

The final expression is finite by assumption (7.7) so the random walk associated with T in (7.14) is recurrent. By Theorem 5.2, φ is almost admissible. This completes the proof for dimension k = 1. When k = 2, the argument proceeds as above until the final step. On 2 i? , the existence of a first moment for the transition density t in (7.15) is not sufficient for recurrence. However the existence of second moments is sufficient (see REVUZ (1984, Chapter 3) for example). This is the reason condition (7.7) depends on the dimension parameter k. The details of the argument are left to the reader. This completes the proof. • Of course the above argument fails completely for k > 3 since Rk(k > 3) does not support any non-trivial recurrent random walks (see GuivARCH'H, KEANE and ROYNETTE (1977)). Appropriate shrinkage estimators

242

Morris L. Eaton

on Rk,k > 3, provide explicit dominators of Pitman estimators in many translation problems. The results in PERNG (1970) show that in the case of k = 1, failure of the third moment assumption can lead to inadmissibility of the Pitman estimator. It is encouraging that the Markov chain arguments used here reproduce results which are known to be fairly sharp. At present, very little more is known concerning the sharpness of the Markov chain argument in Theorem 6.3. Work in this direction is underway.

REFERENCES Berger, J.O. and Srinivasan, C. (1978). Generalized Bayes estimators in multivariate problems. Ann. Statist 6. 783-801. Blyth, C.R. (1951). On minimax statistical procedures and their admissibility. Ann. Math. Statist. 22, 22-42. Brown, L. (1971). Admissible estimators, recurrent diffusions, and insolvable boundary value problems. Ann. Math. Statist. 42. 855-904. Brown, L. and Hwang, J.T. (1982). A unified admissibility proof. In Statistical Decision Theory and Related Topics IV. (S.S. Gupta and J.O. Berger, eds.). 1, 299-324. Academic Press, New York. Chung, K.L. and Fuchs, W.H. (1951). On the distribution of values of sums of random variables. Mem. Amer. Math. Soc. 6 1-12. Diaconis, P. and Strook, D. (1991). Geometric bounds for eigenvalues of Markov chains. Ann. Appl. Probab. 1, 36-61. Eaton, M.L. (1992). A statistical diptych: Admissible inferences-Recurrence of symmetric Markov chains. Ann. Statist. 20, 1147-1179. Eaton, M.L. (1997). Admissibility in quadratically regular problems and recurrence of symmeteric Markov chains: Why the connection? Jour. Statist. Planning and Inference. 64, 231-247. Guivarc'h, Y. Keane, M. and Roynette, B. (1977). Marches aleatoires sur les Groupes de Lie. Lecture Notes in Math. 624. Springer Verlag, New York. Hobert, J.P. and Robert, C.P. (1999). Eaton's Markov chain, its conjugate partner and P-admissibility. To appear in Ann. Statist. James, W. and Stein, C. (1961). Estimation with quadratic loss. In Proc. Fourth Berkeley Symp. Math. Statist. Probab. 1, 361-380. University of California Press, Berkeley. Johnson, B. (1991). On the admissibility of improper Bayes inferences in fair Bayes decision problems. Ph.D. Thesis, University of Minnesota. Karlin, S. (1958). Admissibility for estimation with quadratic loss. Ann. Math. Statist 29, 406-436. Kelly, F.P. (1979). Reversibility and Stochastic Networks. Wiley, New York.

Markov Chains and Admissibility

243

Lawler, G.F. (1995). Introduction to Stochastic Processes. Chapman-Hall, London. Perng, S.K. (1970). Inadmissibility of various "good" statistical procedures which are translation invariant. Ann. Math. Statist. 4 1 , 1311— 1321. Portnoy, S. (1971). Formal Bayes estimation with applications to a random effects model. Ann. Math. Statist. 42, 1379-1402. Revuz, D. (1984). Markov chains, Second Edition. North-Holland, Amsterdam. Stein, C. (1955). A necessary and sufficient condition for admissibility. Ann. Math. Statist. 30, 970-979. Stein, C. (1959). The admissiblity of Pitman's estimator of a single location parameter. Ann. Math. Statist. 30, 970-979. Stein, C. (1965). Lecture notes on decision theory. Unpublished manuscript. Zidek, J.V. (1970). Sufficient conditions for admissibility under squared error loss of formal Bayes estimators. Ann Math. Statist. 41, 446-456.

SCHOOL OF STATISTICS UNIVERSITY OF MINNESOTA 224

CHURCH S T R E E T

SE

MN 55455 USA eaton@stat. umn. edu MINNEAPOLIS

THE SCIENTIFIC FAMILY TREE OF WILLEM R. VAN ZWET

CONSTANCE VAN EEDEN

University of British Columbia Willem van Zwet had 16 PhD students, 15 as supervisor (promotor in Dutch) and one as co-supervisor, and what I have tried to do is to get a complete list of the members of his scientific family up to 20 February 2000. By "A is a member of Willem van Zwet's scientific family", I mean that A can trace his "scientific ancestry" back to Willem van Zwet through (co-)promotors.

1

Introduction

The family members are listed by generation (with the PhDs of W.R. van Zwet as the first generation), within each generation by PhD-thesis supervisor (promotor in Dutch) and, for the PhDs of a given supervisor, in decreasing order of age. Further, for a given generation, the supervisors are listed in decreasing order of their own supervisors age and, in case two of them have the same supervisor, in decreasing order of their own age. In determining the age of a PhD, the date (year) of obtaining his/her PhD degree is taken as his/her birthday (birthyear). For each person in the list, his/her thesis title as well as the date on which he/she defended his/her thesis (or the year in which the degree was granted) is given. The university mentioned in the heading of each sub-list is, unless otherwise mentioned, the one at which the students in that sub-list received their degrees. Note that, because students may have more than one tree-member as (co-)promotor, some PhDs are listed several times - once for each of their (co-)promotors who is a tree-member. In case the several (co-)promotors are not of the same generation, these multiply-listed PhDs show up in more than one generation. For such multiply-listed PhDs who are themselves (co)promotors, the generation number of their PhDs is taken as "the lowest generation number among their (co-)promotors +1". The table of contents lists all those in the tree who have been (co)promotors, with mention of the generation number of their PhDs (i.e., their own number + 1), the number of PhD students they had up to now, as well as the numbers of the sections in which their PhDs are listed. The total number of tree members (inclusive van Zwet) is 145 and of those 23 have been (co-)promotors of members of van Zwet's tree. My thanks to all who helped me assemble the data.

Family Tree of Willem van Zwet

2

Overview

Generation

Promotor

1 2

W.R. van Zwet J. Oosterhoff W. Molenaar F.H. Ruymgaart W. Albers M.C.A. van Zuijlen R. Helmers C.A.J. Klaassen R.J.M.M. Does S.A. van de Geer W.C.M. Kallenberg P. Groeneboom R.D. Gill P.D. Bezemer F.C. Drost M.N.M. van Lieshout K. Sijtsma J.A.M. van Druten J.H.H. Einmahl J.A. Beirlant Tj. Imbos L.C. van der Gaag M.J. van der Laan

3

4

3 3.1

245

# of PhDs

section

16 8 60 6 6 3 1 5 5 1 6 8 14 3 1 1 4 1 3 2 1 1 2

3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

First Generation: PHD Students of W.R. van Zwet Rijksuniversiteit Leiden, The Netherlands

J. Oosterhoff W. Molenaar F.H. Ruymgaart W. Albers J.L. Mijnheer M.C.A. van Zuijlen R. Helmers C.A.J. Klaassen R.J.M.M. Does A.W. van der Vaart S.A. van de Geer M.C.M. de Gunst

Combination of one-sided statistical tests Approximations to the Poisson, binomial and hypergeometric distribution functions (co-promotor with J. Hemelrijk as promotor) 30/5/1973 Asymptotic theory of rank tests for independence 27/6/1974 Asymptotic expansions and the deficiency concept in statistics Sample path properties of stable processes (promotors: 2/9/1974 J. Fabius and W.R. van Zwet) 15/12/1976 Empirical distributions and rank statistics 13/12/1978 Edgeworth expansions for linear combinations of order statistics 22/10/1980 Statistical performance of location estimators 16/6/1982 Higher order asymptotics for simple linear rank statistics Statistical estimation in large parameter spaces (co4/6/1987 promotor: C.A.J. Klaassen) 30/9/1987 Regression analysis and empirical processes (promotors: R.D. Gill and W.R. van Zwet) A random model for plant cell population growth (pro2/3/1988 motors: K.R. Libbenga and W.R. van Zwet)

26/6/1969 6/5/1970

246

Constance van Eeden

H. Putter M. Wegkamp C.G.H. Diks M. Fiocco

4 4.1

23/11/1994 Consistency of resampling methods 26/6/1996 Entropy methods in statistical estimation (copromotor: S.A. van de Geer) 25/9/1996 On nonlinear time series analysis (promotors: F. Takens and W.R. van Zwet) 15/10/1997 Statistical estimation for the supercritical contact process

Second Generation: PHD Students of J. Oosterhoff Vrije Universiteit Amsterdam, The Netherlands

W.C.M. Kallenberg

14/12/1977 Asymptotic optimality of likelihood ratio tests in exponential families P. Groeneboom 14/6/1979 Large deviations and asymptotic efficiencies R.D. Gill 20/12/1979 Censoring and stochastic integrals (co-promotor: C.L. Scheffer) P.D. Bezemer 8/5/1981 Referentiewaarden (co-promotor with Chr.L. Rϋmke as promotor) A.D.M. Kester 22/6/1983 Some large deviation results in statistics (co-promotor: W.C.M. Kallenberg) B.F. Schriever 21/6/1985 Order dependence (co-promotor: R.D. Gill) F.C. Drost 7/10/1987 Asymptotics for generalized chi-square goodness-of-fίt tests (co-promotor: W.C.M. Kallenberg) M.N.M. van Lieshout 7/4/1994 Stochastic geometry models in image analysis and spatial statistics (co-promotor with A.J. Baddeley as promotor)

5 5.1

Second Generation: PHD Students of W. Molenaar Rijksuniversiteit Groningen, The Netherlands

F.P.H. Dijksterhuis

21/6/1973

H. van der Laan

13/12/1973

H.L.W. Angenent

20/6/1974

R. van der Kooy

14/10/1974

G.J.P. Visser

8/10/1975

H. Cohen

9/10/1975

N.A.J. Lagerweij

9/12/1976

G.G.H. Jansen

24/11/1977

J.M.F. ten Berge

15/12/1977

De gevangenis Bankenbos II (co-promotor with W. Buikhuisen as promotor) Leren lezen, schrijven en rekenen (co-promotor with W.J. Bladergroen as promotor) Opvoeding, persoonlijkheid en gezinsverhoudingen in verband met kriminaliteit (co-promotor with W. Buikhuisen as promotor) Spelen met Spel (co-promotor with W.J. Bladergroen as promotor) Luxatie-fracturen van de enkel (promotors: P.J. Kuijjer and W. Molenaar) Drugs, druggebruikers en drug-scene (co-promotor with W. Buikhuisen as promotor) Handleidingen in het onderwijs (co-promotor with L. van Gelder as promotor) An application of Bayesian statistical methods to a problem in educational measurement (promotors: W. Molenaar and W.K.B. Hofstee) Optimizing factorial invariance (promotor: J.P. van de Geer; co-promotors: W.K.B. Hofstee and W. Molenaar)

Family Tree of Willem van Zwet L. Dijkstra

19/1/1978

F.B. Brokken

22/6/1978

K.M. Stokking P. Vijn H.H. de Vos

21/6/1979 10/1/1980 13/3/1980

Tj. Tijmstra

24/4/1980

H. Kuyper

29/5/1980

T.P.B.M. Suurmeijer 16/12/1980 R. de Groot

27/8/1981

R. Nauta

24/9/1981

H.H. van der Molen

28/4/1983

A. Boomsma

23/6/1983

T.J. Euverman & A.A. Vermulst R. Popping

3/11/1983

247

Ontvouwing. Over het afbeelden van rangordes van voorkeur in ruimtelijke modellen (promotors: I. Gadourek and W. Molenaar) The language of personality (co-promotor with W.K.B. Hofstee as promotor) Toetsend onderzoek (co-promotor: L.W. Nauta) Prior information in linear models Het met en van werkorientaties, een vergelijking van verschillende technieken voor het meten van houdingen (co-promotor: I. Gadourek) Sociologie en tandheelkunde (co-promotor with I. Gadourek as promotor) About the saliency of social comparison dimensions (promotors: H.A.M. Wilke and W. Molenaar) Kinderen met epilepsie (promotors: Th.J. IJzerman and W. Molenaar) Adolescenten met leermoeilijkheden in het LBO (copromotor with W.J. Bladergroen as promotor) Studie politiek cultuur (promotors: P.J. van Strien and W. Molenaar) Pedestrian ethology (co-promotor with J.A. Michon as promotor) On the robustness of LISREL (maximum likelihood estimation) against small sample size and non-normality Bayesian factor analysis

15/12/1983 Overeenstemmingsmaten voor nominale data (promotors: F.N. Stokman and W. Molenaar) B.F. van der Meulen 23/2/1984 Bayley ontwikkelingsschalen (promotors: A.F. Kalverboer and W. Molenaar) & M.R. Smrkovsky P.F. Lour ens 24/5/1984 The formalization of knowledge by specification of subjective probability distributions (promotors: W. Molenaar and W.K.B. Hofstee) 13/9/1984 Over psychiatrische invaliditeit (promotors: R. Giel A. de Jong and W. Molenaar) 4/10/1984 Leiding & Participate in het werk (promotors: P.J. J. Geersing van Strien, W. Molenaar and P. Veen) 13/12/1984 Structure in political beliefs. A new model for stochasH. van Schuur tic unfolding with application to European Party activists (promotors: F.N. Stokman and W. Molenaar) Planningsgedrag van leraren. Een empirisch onder9/5/1985 J. Ax zoek naar de onderwijsplanning door leraren in het voortgezet onderwijs (promotors: H.P.M. Creemers, W. Molenaar and N.A.J. Lagerweij) H.J.A. Schouten 20/11/1985 Statistical measurement of interobserver agreement. Analysis of agreements and disagreements between observers (promotors: R. van Strik and W. Molenaar) 11/12/1986 A general family of association coefficients (promotors: F.E. Zegers W. Molenaar and W.K.B. Hofstee) 31/3/1988 Arbeid en gezondheid van stadsbuschauffeurs (promoM. Kompier tors: G. Mulder, F.J.H. van Dijk and W. Molenaar)

Constance van Eeden

248

Contributions to Mokken's nonparametric item response theory (co-promotor: P.J.D. Drenth) 20/4/1989 Subjective probability distributions, a psychometric P. Terlouw approach (promotors: W. Molenaar and W.K.B. Hofstee) Y.J. Pijl 31/8/1989 Het toelatingsonderzoek in het LOM- en MLKonderwijs (promotors: B.P.M. Creemers, W. Molenaar and J. Rispens) T.C.W. Luijben 30/11/1989 Statistical guidance for model modification in covariance structure analysis W. Schoonman 21/12/1989 An applied study on computerized adaptive testing (promotors: W. Molenaar and W.K.B. Hofstee) PARELLA, Measurements of latent traits by proximH.J.A. Hoijtink 7/6/1990 ity items I. van Kamp 6/9/1990 Coping with noise and its health consequences (promotors: G. Mulder and W. Molenaar) G.I.J.M. Kempen 25/10/1990 Thuiszorg voor ouderen (promotors: W.J.A. van den Heuvel and W. Molenaar) Construction and validation of the SON-R 5 \ J.A. Laros &; 2/1991 -17, the Snijders-Oomen non-verbal intelligence test P.J. Tellegen (promotors: W.K.B. Hofstee, T.A.B. Snijders and W. Molenaar) T.F. Meijman 25/4/1991 Over vermoeidheid (promotors: G. Mulder, W. Molenaar and Hk. Thierry) H. Meurs 19/9/1991 A panel data analysis of travel demand (promotors: T.J. Wansbeek, G. Ridder and W. Molenaar) W.J. Post 7/5/1992 Nonparametric unfolding models. A latent structure approach (promotors: T.A.B. Snijders and W. Molenaar) P. Cavalini 22/10/1992 It is an ill wind that brings no good (promotors: C. A. J. Vlek and W. Molenaar) R. Festa 12/11/1992 Optimum inductive methods. A study in inductive probability, Bayesian statistics, and verisimilitude (first promotor T.A.F. Kuipers, second promotor promotor W. Molenaar) M.A.J. van Duijn 4/2/1993 Mixed models for repeated count data (promotors: T.A.B. Snijders and W. Molenaar) J. van Lenthe 16/9/1993 ELI. The use of proper scoring rules for eliciting subjective probability distributions (promotors: W. Molenaar and W.K.B. Hofstee) R.R. Meijer1 17/1/1994 Nonparametric Person Fit Analysis (promotors: W. Molenaar and P.J.D. Drenth, co-promotor: K. Sijtsma) W.R.E.H. Mulder2/11/1994 Categories or dimensions ? The use of diagHajonides van der nostic models in psychiatry (promotors: W. Meulen Molenaar and R.H. van den Hoofdakker) P.M. van der Lubbe 11/12/1995 De ontwikkeling van de Groningse vragenlijst over sociaal gedrag (GVSG) (promotors: R.Giel and W. Molenaar) E.P.W.A. Jansen Curriculumorganisatie en studievoortgang (promo4/4/1996 tors: W.T.J.G. Hoeben and W. Molenaar)

K. Sijtsma

2/6/1988

Family Tree of Willem van Zwet P. de Kort

22/5/1996

B.T. Hemker2

27/9/1996

R.I. Gal

29/5/1997

A. Camstra J.M.E. Huisman

7/5/1998 4/2/1999

6

249

Neglect (promoters: J.M. Minderhoud, B.G. Deelman and W. Molenaar) Unidimensional IRT models for polytomous items, with results for Mokken scale analysis (promotors: P.G.M. van der Heijden and W. Molenaar; copromotor: K. Sijtsma) Unreliability. Contract discipline and contract governance under economic transition (promotors: S.M. Lindenberg and W. Molenaar) Cross-validation in covariance structure analysis Item nonresponse: Occurence, causes, and imputation of missing answers to test items

Second Generation: PHD Students of F.H. Ruymgaart

6.1

Katholieke Universiteit Nijmegen, The Netherlands

M.J.M. Jansen

25/6/1981

J.A.M. van Druten

1/10/1981

J.H.J. Einmahl

16/5/1986

G. Nieuwenhuis

12/6/1989

6.2

Equilibria and optimal threat strategies in two-person games (co-promotor: T.E.S. Raghavan) A mathematical-statistical model for the analysis of cross-sectional serological data with special reference to the epidemiology of malaria (co-promotor with J.H.E.Th. Meuwissen as promotor) Multivariate empirical processes (co-promotor: D.M. Mason) Asymptotics for point processes and general linear processes (co-promotor: W. Vervaat)

Texas Technical University, Lubbock, USA

A.K. Dey

8/1994

K. Chandrawansa

12/1997

7

Cross-validation for parameter selection in statistical inverse estimation problems Statistical inverse estimation of irregular input signals

Second Generation: PHD Students of W. Albers

7.1

Rijksuniversiteit Limburg, Maastricht, The Netherlands

L.W.G. Strijbosch

7.2

Experimental design and statistical evaluation of limiting dilution assays (co-promotor: R.J.J.M. Does)

Universiteit Twente, Enschede, The Netherlands

C.A.W. Glas3 A.J. Koning G.D. Otten G.R.J. Arts

1 2

21/4/1989

12/10/1989 Contributions to estimation and testing Rasch models (promotors: W.J. van der Linden and W. Albers) 8/3/1991 Stochastic integrals and goodness-of-fit tests (assistant-promotor: J.H.J. Einmahl) 23/6/1995 Statistical test limits in quality control (assistantpromotor: W.C.M. Kallenberg) 12/6/1998 Test limits in quality control using correlated product characteristics (assistant-promotor: W.C.M. Kallenberg)

Degree obtained at the Vrije Universiteit van Amsterdam, The Netherlands. Degree obtained at the Rijksuniversiteit Utrecht, The Netherlands. The degree was

awarded cum lαude.

250

Constance van Eeden

P.C. Boon 8

17/12/1999 Asymptotic behavior of pre-test procedures (assistantpromotor: W.C.M. Kallenberg)

Second Generation: PHD Students of M.C.A. van Zuijlen

8.1

Katholieke Universiteit Leuven, Belgium

J.A. Beirlant

8.2

1984

Eigenschappen van de empirische verdelingfunctie en het empirisch proces van toevalsafstanden en asymptotische theorie van hierop gebaseerde statistische functionalen (co-promotor with E.C. van der Meulen as promotor)

Katholieke Universiteit Nijmegen, The Netherlands

T. van der Meer

12/6/1995

Applications of operators in nearly unstable models (co-promotor with W.Th.F. den Hollander as promotor)

I. Alberink

26/1/2000

Berry-Esseen bounds for arbitrary statistics (copromotor with W.Th.F. den Hollander as promotor)

9

Second Generation: PHD Students of R. Helmers

9.1

Erasmus Universiteit Rotterdam, The Netherlands

A.L.M. Dekkers 10

14/11/1991 On extreme-value estimation (co-promotor with L. de Haan as promotor)

Second Generation: PHD Students of C.A.J. Klaassen

10.1

Rijksuniversiteit Leiden, The Netherlands

A.W. van der Vaart 10.2

4/6/1987

Universiteit van Amsterdam, The Netherlands

S.A. Venetiaan E.R. van den Heuvel

6/12/1994 21/5/1996

W.J.H. Stortelder

12/3/1998

A.J. Lenstra

26/3/1998

11

Statistical estimation in large parameter spaces (copromotor with W.R. van Zwet as promotor) Bootstrap bounds Bounds for statistical estimation in semiparametric models Parameter estimation in nonlinear dynamic systems (co-promotor with P.W. Hemker as promotor) Analyses of the nonparametric mixed proportional hazards model (promotors: C.A.J. Klaassen, G. Ridder and A.CM. van Rooij)

Second Generation: PHD Students of R.J.M.M. Does

11.1

Rijksuniversiteit Limburg, Maastricht, The Netherlands

Tj. Imbos

17/2/1989

L.W.G. Strijbosch

21/4/1989

Degree awarded cum lαude.

Het gebruik van einddoeltoetsen bij aanvang van de studie (co-promotor with W.H.F.W. Wijnen as promotor) Experimental design and statistical evaluation of limiting dilution assays (co-promotor with W. Albers as promotor)

Family Tree of Willem van Zwet 11.2

Universiteit van Amsterdam, The Netherlands

E.S. Tan C.B. Roes A.F. Huele

12 12.1

13.1

16/12/1994 A stochastic growth model for the longitudinal measurement of ability (co-promotor: Tj. Imbos) 27/11/1995 Shewhart-type charts in statistical process control 5/11/1998 Statistical robust design (co-promotor: J. Engel)

Second Generation: PHD Students of S.A. van de Geer Rijksuniversiteit Leiden, The Netherlands

M. Wegkamp

13

26/6/1996

Vrije Universiteit Amsterdam, The Netherlands 22/6/1983

F.C. Drost

7/10/1987

G.D. Otten G.R.J. Arts P.C. Boon

14.1

22/6/1990

Stochastic models in reliability (assistant-promotor with J.H.A. de Smit as promotor) 23/6/1995 Statistical test limits in quality control (assistantpromotor with W. Albers as promotor) 12/6/1998 Test limits in quality control using correlated product characteristics (assistent-promotor with W. Albers as promotor) 17/12/1999 Asymptotic behavior of pre-test procedures (assistentpromotor with W. Albers as promotor)

Third Generation: PHD Students of P. Groeneboom Universiteit van Amsterdam, The Netherlands

J. Praagman4

27/6/1986

A.J. van Es

2/11/1988

F.A.G. Windmeyer

9/6/1992

14.2

Some large deviation results in statistics (co-promotor with J. Oosterhoff as promotor) Asymptotics for generalized chi-squared goodness-of-fit tests (co-promotor with J. Oosterhoff as promotor)

Universiteit Twente, Enschede, The Netherlands

D.P. Kroese

14

Entropy methods in statistical estimation (copromotor with W.R. van Zwet as promotor)

Third Generation: PHD Students of W.C.M. Kallenberg

A.D.M. Kester

13.2

251

Efficiency of change-point tests (promotors: P. Groeneboom and P.C. Sander) Aspects of nonparametric density estimation (copromotor: P.L. Janssen) Goodness-of-fit in linear and qualitative-choice models (promotors: P. Groeneboom and H. Neudecker)

Technische Universiteit Delft, The Netherlands

E.A.G. Weits A.J. Cabo

17/5/1990 28/6/1994

A stochastic heat equation for freeway traffic flow Set functionals in stochastic geometry (promotors: A.J. Baddeley and P. Groeneboom) G. Jongbloed 9/10/1995 Three statistical inverse problems - estimators, algorithms, asymptotics R.B. Geskus 11/2/1997 Estimation of smooth functionals with interval censored data and something completely different P.P. de Wolf 5/10/1999 Estimating the extreme value index - tales of tails 4 Degree obtained at the Technische Universiteit Eindhoven, The Netherlands.

252

15 15.1

Constance van Eeden

Third Generation: PHD Students of R.D. Gill Vrije Universiteit Amsterdam, The Netherlands

B.F. Schriever

21/6/1985

R.H. Baayen

29/5/1989

15.2

Rijksuniversiteit Leiden, The Netherlands

S.A. van de Geer

15.3

30/9/1987

Regression analysis and empirical processes (promotors: W.R. van Zwet and R.D. Gill)

Universiteit van Amsterdam, The Netherlands

L.C. van der Gaag

15.4

Order dependence (co-promotor with J. Oosterhoff as promotor) A corpus-based approach to morphological productivity (promotors: G.E. Booij and R.D. Gill)

26/9/1990

Probability-based models for plausible reasoning (promotors: J.A. Bergstra and R.D. Gill)

Rijksuniversiteit Utrecht, The Netherlands

M.C.J. van Pul

Statistical analysis of software reliability (copromotor: K. Dzhaparidze) M. J. van der Laan 13/12/1993 Efficient and inefficient estimation in semiparametric models (promotors: R.D. Gill and P.J. Bickel) B.J. Wijers 19/1/1995 Nonparametric estimation for a windowed linesegment process C.G.M. Oudshoorn 7/11/1996 Optimality and adaptivity in nonparametric regression (co-promotor: B.Y. Levit) C.M.A. Schipper 15/1/1997 Sharp asymptotics in nonparametric estimation (copromotor: B.Y. Levit) R.W. van der Hofstad 16/6/1997 One-dimensional random polymers (promotors: W.Th.F. den Hollander and R.D. Gill) E.N. Belitser 17/6/1997 Minimax estimation in regression and random censorship models (co-promotor: B.Y. Levit) Y. Nishiyama 25/5/1998 Entropy methods for martingales G.E. Krupa 2/11/1998 Limit theorems for random sets (co-promotor: E.J. Balder) E.W. van Zwet 3/9/1999 Likelihood devices in spatial statistics

16 16.1

Third Generation: PHD Students of P.D. Bezemer Universiteit van Amsterdam, The Netherlands

H. Walinga

16.2

24/2/1993

30/5/1985

Varioliforme erosies - Een onderzoek naar de etiologie (promotor: G.N.J. Tijtgat with W. Dekker, P.J. Kostense and P.D. Bezemer as co-promotors)

Vrije Universiteit Amsterdam, The Netherlands

E.S.M. de Lange-de Klerk L. van den Berg

19/5/1993

Effects of homoeopathic medicines on children with recurrent upper respiratory tract infections (copromotor with L.Feenstra and O.S. Miettinen as promotors) 31/10/1997 Postoperatieve pijnbestrijding: Een prospectieve studie naar de kosteneffectiviteit van twee methoden (promotor: J.J. de Lange with W.W.A. Zuurmond and P.D. Bezemer as co-promotors)

Family Tree of Willem van Zwet 17

253

Third Generation: PHD Students of F.C. Drost

17.1

Katholieke Universiteit Brabant, Tilburg, The Netherlands

B.J.M. Werker

18

14/9/1995

Statistical methods in financial econometrics (copromotor with B.B. van der Genugten and Th.E. Nijman as promoters)

Third Generation: PHD Students of M.N.M. van Lieshout

18.1

University of Warwick, Coventry, United Kingdom

E. Thδnnes

19

24/11/1998 Perfect and imperfect simulations in stochastic geometry in image analysis (co-promotor with W.S. Kendall as promotor)

Third Generation: PHD Students of K. Sijtsma

19.1

Vrije Universiteit Amsterdam, The Netherlands

R.R. Meijer A.C. Verweij

19.2

28/2/1996

Dansende beren. Beoordelingsprocessen bij personeelsselectie (co-promotor with G.J. Mellenbergh as first and W.M.M. Alt ink as second promotor)

Rijksuniversiteit Utrecht, The Netherlands

B.T. Hemker6

20

Nonparametric person fit analysis (co-promotor with W. Molenaar and P.J.D. Drenth as promotors) 20/12/1994 Scaling transitive inference in 7-12 year old children (co-promotor with W. Koops as promotor)

Universiteit van Amsterdam, The Netherlands

K. van Dam 5

19.3

17/1/1994

27/9/1996

Unidimensional IRT models for polytomous items, with results for Mokken scale analysis (co-promotor with P.G.M. van der Heijden and W. Molenaar as promotors)

Third Generation: PHD Students of J.A.M. van Druten

20.1

Universiteit van Amsterdam, The Netherlands

J.C.M. Hendriks

21

7/5/1999

The incubation period of AIDS (co-promotor with R.A. Coutinho and G.J.F. van Griensven as promotors)

Third Generation: PHD Students of J.H.J. Einmahl

21.1

Universiteit Twente, Enschede, The Netherlands

A.J. Koning 5 6

8/3/1991

Stochastic integrals and goodness-of-fit tests (assistant-promotor with W. Albers as promotor)

K. van Dam received the 1996 NITPB prize for his PhD thesis. Degree awarded cum lαude.

254

Constance van Eeden

21.2

Erasmus Universiteit Rotterdam, The Netherlands

H. Xin

15/1/1992

A.K. Sinha

2/10/1997

22 22.1

Third Generation: PHD Students of J.A. Beirlant Katholieke Universiteit Leuven, Belgium

P. Vynckier

1996

C. Vynckier

1997

23 23.1

24.1

Universiteit van Amsterdam, The Netherlands 16/12/1994 A stochastic growth model for the longitudinal measurement of ability (co-promotor with R.J.M.M. Does as promotor)

Fourth Generation: PHD Students of L.C. van der Gaag Rijksuniversiteit Utrecht, The Netherlands

R.R. Bouckaert

25 25.1

Tail estimation, quantile plots and regression diagnostics Applications of generalized quantiles in multivariate statistics

Third Generation: PHD Students of Tj. Imbos

E.S. Tan

24

Statistics of bivariate extreme values (copromotor with L. de Haan as promotor) Estimating failure probability when failure is rare: multidimensional case (co-promotor with L. de Haan as promotor)

13/6/1995

Bayesian belief networks: Prom construction to inference (co-promotor with J. van Leeuwen as promotor)

Fourth Generation: PHD Students of M.J. van der Laan University of California, Berkeley, USA

A.E. Hubbard

1998

D.R. Peterson

1998

Applications of locally efficient estimation to censored data models On nonparametric estimation and inference with censored data, bandwidth selection for local polynomial regression, and subset selection in explanatory regression analysis

Acknowledgements. The information of this paper is taken from the larger family tree in "The Scientific Family Tree of David van Dantzig", compiled by Constance van Eeden, published by the Stichting Mathematisch Centrum, September 2000. DEPARTMENT OF STATISTICS THE UNIVERSITY OF BRITISH COLUMBIA

333-6356

AGRICULTURAL ROAD

VANCOUVER, B.C.,

cυe@xs4αll.nl

CANADA, V6T

1Z2

ASYMPTOTICS IN QUANTUM STATISTICS

RICHARD D. GILL

University of Utrecht and Eurandom Observations or measurements taken of a quantum system (a small number of fundamental particles) are inherently random. If the state of the system depends on unknown parameters, then the distribution of the outcome depends on these parameters too, and statistical inference problems result. Often one has a choice of what measurement to take, corresponding to different experimental set-ups or settings of measurement apparatus. This leads to a design problem—which measurement is best for a given statistical problem. This paper gives an introduction to this field in the most simple of settings, that of estimating the state of a spin-half particle given n independent copies of the particle. We show how in some cases asymptotically optimal measurements can be constructed. Other cases present interesting open problems, connected to the fact that for some models, quantum Fisher information is in some sense non-additive. In physical terms, we have non-locality without entanglement. AMS subject classifications: 62F12, 62P35.

Keywords and phrases: quantum statistics, information, spin half, qubit, two level system.

1

Introduction

The fields of quantum statistics and quantum probability have a reputation for being esoteric. However, in our opinion, quantum mechanics, from which they surely derive, is a fascinating source of probabilistic and statistical models, unjustly little known to 'ordinary' statisticians and probabilists. Quantum mechanics has two main ingredients: one deterministic, one random. In isolation from the outside world a quantum system evolves deterministically according to Schrόdinger's equation. That is to say, it is described by a state or wave-function whose time evolution is the (reversible) solution of a differential equation. On the other hand, when this system comes into interaction with the outside world, as when for instance measurements are made of it (photons are counted by a photo-detector, tracks of particles observed in a cloud chamber, etc.) something random and irreversible takes place. The state of the system makes a random jump and the outside world contains a record of the jump. Prom the state of the system at the time of the interaction one can read off, according to certain rules, the probability distribution of the macroscopic outcomes and the new state of the system. See Penrose (1994) for an eloquent discussion of why there is something paradoxical in the peaceful coexistence of these two principles;

256

Richard D. Gill

see Percival (1998) for interesting stochastic modifications to Schrδdinger's equation which might offer some reconciliation.1 Till recently, most predictions made from quantum theory involved such large numbers of particles that the law of large numbers takes over and predictions are deterministic. However, technology is rapidly advancing to the situation that really small quantum systems can be manipulated and measured (e.g., a single ion in a vacuum-chamber or a small number of photons transmitted through an optical communication system). Then the outcomes definitely are random. The fields of quantum computing, quantum communication, and quantum cryptography are rapidly developing and depend on the ability to manipulate individual quantum systems. Especially of interest are assemblages of two-level systems, known as qubits or spin half systems in quantum computing and information. Theory and conjecture are much further than experiment and technology, but the latter are following steadily. In this paper we will introduce as simply as possible the model of quantum statistics and consider the problem of how to best measure the state of an unknown spin-half system. We will survey some recent results, in particular, from joint work with O.E. Barndorff-Nielsen and with S. Massar (Barndorff-Nielsen and Gill, 2000; Gill and Massar, 2000). This work has been concerned with the problem posed by Peres and Wootters (1991): can more information be obtained about the common state of n identical quantum systems from a single measurement on the joint system formed by bringing the n systems together, or does it suffice to combine separate measurements on the separate systems? A useful tool for our studies is the quantum Cramer-Rao bound with its companion notion of quantum information, introduced by Helstrom (1967). Actually there are several ways to define quantum information with different resulting Cramer-Rao type bounds (Yuen and Lax, 1973, Stratonovich, 1973, Belavkin, 1976, Holevo, 1982). Quantum statistics mainly consists of exact results in various rather special models; see the books of Helstrom (1976) and Holevo (1982). Just as in ordinary statistics, the Cramer-Rao bound on the variance of an unbiased estimator is rarely achieved exactly (only in so-called quantum exponential models). In any case, one would not want in practice to restrict attention to unbiased estimators only. There are results on optimal invariant methods, but again, not many models have the structure that these results are applicable and even then the restriction to invariant statistical methods is not entirely compelling. One might hope that asymptotically it would be possible to achieve the highly recommended: Sheldon Goldstein, 'Quantum mechanics without observers', Physics Today, March, April 1998; letters to the editor, Physics Today, February 1999.

Quantum Asymptotics

257

Cramer-Rao bound. However, asymptotic theory is so far very little developed in the theory of quantum statistics, one reason being that the powerful modern tools of asymptotic statistics (contiguity, local asymptotic normality, and so on) are just not available2 since even if we are considering measurements of n identical quantum systems, there is no a priori reason to suppose that a particular sequence of measurements on n quantum systems together will satisfy these conditions. Here, we make a little progress through use of the van Trees inequality (see Gill and Levit, 1995), a Bayesian CramerRao bound, which will allow us to make asymptotic optimality statements without assuming or proving local asymptotic normality. Another useful ingredient will be the recent derivation of the quantum Cramer-Rao bound by Braunstein and Caves (1994), linking quantum information to classical expected Fisher information in a particularly neat way. We will show that, for certain problems, a new Cramer-Rao type inequality of Gill and Massar (2000) does provide an asymptotically achievable bound to the quality of an estimator of unknown parameters. For some other problems the issue remains largely open and we identify situations where Peres and Wootter's question has an affirmative answer: there can be appreciably more information in a joint measurement of several particles than in combining separate measurements on separate particles. This clarifies an earlier affirmative answer of Massar and Popescu (1995), which turned out only for small samples to improve on separate measurements. It also clarifies the recent findings of Vidal et al. (1998). Helstrom wrote in the epilogue to his (1976) book: "Mathematical statisticians are concerned with asymptotic properties of estimators. When the parameters of a quantum density operator are estimated on the basis of many independent observations, how does the accuracy of the estimates depend on the number of the observations as that number grows very large? Under what conditions have the estimators asymptotically normal distributions? Problems such as these, and still others that doubtless will occur to physicists and mathematicians, remain to be solved within the framework of the quantum mechanical theory." More than twenty years later this programme is still hardly touched (some of the few contributions are by Brody and Hughston, 1998, and earlier papers, and Holevo, 1983) but we feel we have made a start here. In 20 ± e pages (even when ±e = +12) it is difficult to give a complete introduction to the topic, as well as a clear picture of recent results. The classic books by Helstrom and Holevo mentioned above are still the only books on quantum statistics and they are very difficult indeed to read for a beginner. A useful resource is the survey paper by Malley and Hornstein (1993). However the latter authors, among many distinguished writers both 2

though R. Rebolledo is working on a notion of quantum contiguity

258

Richard D. Gill

from physics and from mathematics, take the stance that the randomness occurring in quantum physics cannot be caught in a standard Kolmogorovian framework. We argue elsewhere (Gill, 1998), in a critique of an otherwise excellent introduction to the related field of quantum probability (Kummerer and Maassen, 1998), that depending on what you mean by such a statement, this is at best misleading, and at worst simply untrue. With more space at our disposal we would have included extensive worked examples; however they have been replaced by exercises so that the reader can supply some of the extra pages (but—unless you are Willem van Zwet—leave the starred exercises for later). Some references which we found specially useful in getting to grips with the mathematical modelling of quantum phenomena are the books by Peres (1995), and Isham (1995). To get into quantum probability, we recommend Biane (1995) or Meyer (1986). Also highly recommended are the lecture notes of Preskill (1997), Werner (1997) and Holevo (1999). This introductory section continues with three subsections summarizing the basic theory: first the mathematical model of states and measurements; secondly the basic facts about the most simple model, namely of a two-state system; and thirdly the basic quantum Cramer-Rao bound. That third subsection finishes with a glimpse of how one might do asymptotically optimal estimation in one-parameter models: in a preliminary stage obtain a rough estimate of the parameter from a small number of our n particles. Estimate the so-called quantum score at this point, and then go on to measure it in the second stage on the remaining particles. Section 2 states a recent new version of the quantum Cramer-Rao bound which makes precise how one might trade information between different components of a parameter vector. Section 3 outlines the procedure for asymptotically optimal estimation of more than one parameter, again a two-stage procedure. This is work 'in progress', so some results are conjectural, imprecise, or improvable. In a final short section we try to explain how some of our results are connected to the strange phenomenon of non-locαlity without entanglement, a hot topic in the theory of quantum information and computation. 1.1

The basic set-up

Quantum statistics has two basic building blocks: the mathematical specification of the state of a quantum system, to be denoted by p = p(θ) as it possibly depends on an unknown parameter 0, and the mathematical specification of the measurement, denoted by M, to be carried out on that system. We will give the recipe for the probability distribution of the observable outcome (a value a: of a random variable X say) when measurement M is carried out on a system in state p. Since the state p depends on an unknown parameter 0, the distribution of X depends on 0 too, thereby setting a sta-

Quantum Asymptotics

259

tistical problem of how best to estimate or test the value of θ. Since we may in practice have a choice of which measurement M to take, we have a design problem of choosing the best measurement for our purposes. There is also a recipe for the state of the system after measurement, depending on the outcome, and depending on some further specification of the measurement; see Preskill (1997), Werner (1997), Bennett et al. (1998), or Holevo (1999). We do not need it here. For simplicity we restrict attention to finite-dimensional quantum systems. The state of a d-dimensional quantum system can be summarized or specified by a d x d complex matrix p called the density matrix of the system. For instance, when we measure the spin of an electron in a particular direction only two different values can occur, conventionally called 'up' and 'down'. This is just one example of a two level system, requiring a d — two-dimensional state space for its description. Similarly if we measure whether a photon is polarized in a particular direction by passing it through a polarization filter, it either passes or does not pass the filter. Again, polarization measurements on a single photon can be discussed in terms of a two-dimensional system. If we consider the spins of n electrons, then 2n different outcomes are possible and the system of n electrons together (or rather, their spins) is described by a d x d matrix p with d = 2n. The future quantum computer might consist of an assemblage of n atoms at very low temperature, each of which could be found in its ground state or in an excited state; interacting together they have a 2n-dimensional state space. Definition 1.1 (Density matrix) The density matrix p of a d-dimensional quantum system is a d x d self-adjoint, nonnegative matrix of trace 1. 'Self-adjoint' means that p* = p where * denotes the complex conjugate and transpose of the matrix. That p is nonnegative means that ψ*pψ > 0 for all column vectors ψ (since p is self-adjoint this quadratic form is a real number). We often use the Dirac bra-ket notation whereby \φ) (called a ket) is written for the complex column vector ψ and (ψ\ (a bra) is written for its adjoint, the row vector containing the complex conjugates of its elements. The quadratic form ψ*pψ is then written (ψ \ p\ψ). The notation allows one to graphically distinguish numbers from matrices, as in (ip \ φ) versus \ψ) (ψ\, and to specify vectors through labels or descriptions as in | lable) or I description) The diagonal elements of a density matrix must be nonnegative reals adding up to one. Moreover by the eigenvalue-eigenvector decomposition of self-adjoint matrices we can write p = Σ
260

Richard D. Gill

a particular eigenvalue are an arbitrary orthonormal basis of the eigenspace. Even if the eigenvalue is simple (the eigenspace one-dimensional), the eigenvector is only uniquely defined up to multiplication by an arbitrary complex number of modulus one. One says that the density matrix p represents the mixed state obtained by taking with probability pi the system in the pure state described by the density matrix \i) (i\. A pure state \φ) (ψ\ is often referred to through its state vector \ψ), also called the wave-function. One may multiply \φ) by an arbitrary complex number of modulus one, and it still represents the same state. Definition 1.2 (Measurement) A measurement M on a d-dimensional quantum system taking values x in a measurable space (X,A) is described by an operator-valued probability measure (oprom), that is, a collection of self-adjoint matrices M(A) : A G A such that 1. M(X) = 1, the identity matrix, 2. Each M(A) is non-negative. 3. For disjoint A*, Note that these three rules are the ordinary axioms of a probability measure on (X,A), except that the measure takes values in the self-adjoint matrices instead of the real numbers. The sample space X might be the real numbers or a subset thereof, with the Borel sigma algebra, but it could also be anything else. Now we can give the so-called trace-rule telling us the probability distribution of the random outcome X when M is used to measure p: Definition 1.3 (Trace rule) The probability distribution of the outcome X is given by (1)

Pτ{XeA}

= trace(pM(A)),

AeA

Exercise 1.1 (Legitimacy of trace rule) Prove that (1) indeeds defines a probability measure on X, A.

One can argue from basic principles of quantum mechanics that however one measures a quantum system, the result must be an affine mapping from density matrices to the space of probability distributions on the outcome space. It is a theorem that any such mapping can be represented by an oprom. Thus the class of oproms contains all conceivable measurements. On the other hand, as we will see later, any oprom can be realised by some concrete experimental set-up, at least in principle, so the definition

Quantum Asymptotics

261

captures exactly what it should. The inaccurate abbreviations 'povm' (positive operator valued measure) and 'pom' (probability operator matrices) are widespread. Oproms are often called generalised measurements, to contrast them with a special subclass of measurements called simple measurements, defined as follows: Definition 1.4 (Simple measurement) A simple measurement Π on a d-dimensional quantum system is a real-valued measurement such that each Tl(A) is idempotent, i.e., is an orthogonal projector onto a subspace One can now show that the measurement takes on at most d different values, i.e., there exist x\,...,Xk G X = M with k < d such that Π({xi,... ,Xk}) = 1. With U(xi) as abbreviation for Π({xi}), the matrices U(x{) project onto k orthogonal subspaces of C* together spanning the whole space. Define a self-adjoint matrix X (to be distinguished from the random variable X representing the outcome of the measurement) by X = Σ{ XiU(xi). Then the X{ are the eigenvalues of X and the U(xi) project onto the eigenspaces. Conversely, given a self-adjoint matrix X one can construct a corresponding simple measurement or projector-valued probability measure. In this role we call X an observable. It follows that the expected value of the outcome of a measurement of X is given by trace (pX). For an ordinary real function / (e.g., square, inverse, logarithm, . . . ) one defines the same function of the observable X by f(X) = Σif(xi)ϊl(xi), and the expected value of the outcome of a measurement of the observable f(X) is trace(p/(X)). Simple measurements are often called von Neumann measurements. We will occasionally use the term 'proprom' (projector-valued probability measure). Physicists generally agree that any simple measurement could in principle be implemented in practice. 'Between measurements' a quantum system evolves deterministically according to the Schrόdinger equation, a differential equation for the component pure states \i) of a given mixed system. One thinks of a measurement as taking place instantaneously. After the measurement, the quantum system jumps to a new state (depending on the outcome x)\ this is called 'the collapse of the wave function'. Some simple rules specify what happens, but we will not give them here. If we bring two separate quantum systems together into some kind of interaction, then their future evolutions will be linked together. Measurements can be made on the 'joint system', including all the separate measurements on each of the separate systems but many more besides. Mathematically this is modelled as follows:

262

Richard D. Gill

Definition 1.5 (Product system) Consider two quantum systems, of dimension d and d1, in states p and p' respectively. Together the two are in the state p ® pι in C? ® C*' = C r f x d ' where ® denotes tensor product (of matrices, vectors, or spaces as appropriate). For the reader who is not familiar with tensor products, the tensor product of C* with C*' has as basis the tensor product of each element of a basis of C* with each element of a basis of C*'. One can take linear combinations of tensor products φ ® φ1 by expanding bilinearly in chosen bases of the two spaces. Tensor products of matrices are defined in the natural way by how they operate on products of vectors: X ® X' φ ® φ1 = Xφ ® X'φ1. The trace of a tensor product of two matrices is the product of the traces. Suppose M and M' are measurements on two separate quantum systems p and p'. Then we can define a joint measurement M ® M' on the combined system in the obvious way, taking values in the product of the outcome spaces of M and M1. Exercise 1.2 (Product measurement) Show that the outcome of measurement of M®M' on a system in state p®pι is distributed as independent realisations of measurement of M and M1 on p and p1 respectively. However, the important point is that bringing two quantum systems together allows many more measurements than just product measurements (which, as we saw from exercise 1.2, are not very interesting). Product systems are important for two main reasons. Firstly, one of the main themes of this paper is going to be: if we have n independent systems each in the same state p(θ) (i.e., in identical states all depending on the same unknown parameter 0), can we learn more about θ from a joint measurement on the cP dimensional combined system p®n(0)? In the next section we will discuss some of the history and other background to this question, which has been the subject of a series of papers in recent years. Secondly, product systems play a role in the realisation of generalised measurements. It is a theorem (due to Naimark) that any generalised measurement whatever can be realised by a simple measurement after a 'quantum randomisation'. That is to say, given any measurement M there exists a so-called ancillary system and a fixed and known state p' of that system, and a simple measurement Π on the joint system p ® p' such that trace(pM(A)) = trace(p ® p'Π(A)) for all A and p. 1.2

Spin half

In order to make the above rather abstract concepts a little more concrete, let us go to the most simple special case, d = 2, called a spin-half system, a two-level system, or a qubit. We will see that we can associate the state

263

Quantum Asymptotics

of a spin-half system with a real vector α of length less than or equal to 1 in ordinary three dimensional space, and a simple measurement—which can take on at most two different values—with a direction in space, or a unit vector u. The trace rule (1) reduces to a simple formula involving α and u. The model applies to the famous Stern-Gerlach experiment, featuring in many introductory textbooks on quantum physics. In that experiment silver atoms were made to pass through a magnetic field with large gradient in a certain direction. Each atom was either deflected upwards or downwards with respect to this direction. The deflection is due to the spin of the outermost electron in the silver atom, which can be characterized by a vector α. The orientation of the magnet determines which measurement is being taken, i.e., the value of u. First we take some time to study some special features of the 2 x 2 selfadjoint matrices. The properties we find will greatly simplify calculations. Define the Pαuli spin matrices as follows: Definition 1.6 (Pauli spin matrices)

t*\

( °

1

/ o -« Λ

\

/ l

o λ

These three matrices are self adjoint, each has trace zero and determinant minus one, hence each has eigenvalues ±1. They satisfy (check this yourself!) o'xO'y = —σyσx /QX

σyσx

=

iσz.

σzσx = -σxσz2

=

iσy,

σ

σ

y z

l

=

—

l

A n a r b i t r a r y self-adjoint 2 x 2 c o m p l e x m a t r i x h a s t o b e of t h e form (4)

X=(

U +

Z

\ x + ιy

X

~iy) u —z J

where a:, y, z, u are uniquely determined real numbers. Thus we can write (5)

X = ul + xσx + yσy + zσz.

Specializing to density matrices, the requirement that trace p = 1 imposes the condition that u = \. The requirement that p is nonnegative is 2 2 2 2 equivalent being nonnegative, or u — z — x — y > 0, ivalent to its determinant det 2 2 2 or x + y + z < \ . It is convenient to write (6)

264

Richard D. Gill

where α = (α x , αy,αz) E i 3 and satisfies

(7)

\\α\\2 = αl + α2y + α2z<ί

while σ = (σ x , σy, σ z ), a vector of matrices, and denotes the inner-product. Thus the space of density matrices of a two-dimensional quantum system can be represented by the closed unit ball B in three-dimensional Euclidean space. The sphere 5, or surface of the unit ball, corresponds to density matrices ^(1 + α σ) with ||α|| 2 = 1, which are singular since their determinant is zero. Such a density matrix has therefore eigenvalues 0 and 1 and represents a pure state. The density matrix of a pure state is a projector matrix, projecting onto a one-dimensional subspace of C 2 . Letting u denote a unit vector in M3, let us write U(u) = p(u) = | ( 1 + u σ) for this matrix. We can also write Π(£t) = \u) (u\ for a certain state vector \u) which we will determine in a moment. Check using (3) that ΐl(u) is idempotent, and that U(u) and Π(—u) commute (in fact, their product is the zero matrix) and add to the identity matrix. Thus the projectors ΐl(u) and H(—u) project onto two orthogonal one-dimensional subspaces of C 2 , the spaces generated by \u) and \—u) respectively. The only other projector matrices are 0 and 1, projecting onto the zero subspace and the whole space of C 2 respectively. Given a density matrix p = p(α), define the unit vector u = α/||α|| and probabilities α = \{l + ||α||), β = ±(1 - ||α||). If ||α|| = 0, u can be chosen arbitrarily. Then we can write (8)

p(α) = αp(u) + βp(-u).

This shows that p(α) has eigenvalues α and /?, and its eigenvectors are \u) and \—u) respectively. One may consider the state p(α) as the mixture, with probabilities α and /?, of the pure states with state vectors \u) and \—u) (though this is only one of many representations of p as a mixture of pure states). The vector u is a point on the unit sphere in E 3 . Let θ and φ denote its polar coordinates, where θ G [0, π] is the latitude measured from the North pole (z-axis) and φ G [0,2π) is the longitude, measured from the xaxis. (We should really say co-latitude rather than latitude). Thus u = (sin0cos,sin0sin<£,cos0). We are going to show that \u) = \θ,φ), the vector in (C2 defined by

265

Quantum Asymptotics Note that (0, φ\θ,φ)

= 1 while

(10)

\O,Φ)<Θ,Φ\ =

) 2

_

( cos (0/2) e-^cos(0/2)sin(0/2) V e^cos(0/2)sin(0/2) sin2(0/2) i ί l + cos(0) (cos φ — i sin φ) sin 0 2 \ (cos φ + i sin 0) sin 0 1 — cos 0 ί

\ ) \ )

complex vector |£) of length 1 can be written as e zα |0, φ) for some α E [0,2π) and polar coordinates θ,φ and \ξ) (ξ\ is the same for all α. We summarize our findings so far:

Rule 1.1 (Spin-half density matrices, projectors) The density matrix p(a), where a is a point in the unit ball in R 3 , has eigenvalues ^(l±||α||) and and normalized eigenvectors \u) = |0, φ), \—u) = |τr — 0, φ + π), where θ and φ are the polar coordinates ofu = α/||α||. The projector matrix ΐl(u) projects onto the one-dimensional subspace o/C2 spanned by \ΰ). The projector onto the space orthogonal to this, spanned by \—u), is Π(—u). Let u and v be two unit vectors in M3. Since ΐl(u) = \u) {u\ we see t h a t t r a c e U ( u ) U ( v ) = t r a c e \u) (u\ \ΰ) (v\ = ( v \ u ) { u \ v ) = \ ( u \ ΰ ) | 2 . O n the other hand, using the properties (3) of the Pauli matrices, one readily computes trace U(u)U(v) = | ( 1 + u- v). Now u v is the cosine of the angle between the vectors u and v, hence ^(1 + u v) is the squared cosine of half the angle between u and v. Rule 1.2 (Calculation rule) The absolute value of the squared inner product between the complex vectors \u) and \v) in C2 is the squared cosine of half the angle between the corresponding unit vectors u and v in R 3 . In particular, opposite points on the unit sphere correspond to orthogonal vectors mC2. We can now describe the probability distributions of all simple measurements of a spin-half system. The state of the system is modelled by a 2 x 2 density matrix of the form p(a) = i ( l + a σ) where a is a point in the closed unit ball in R 3 . The non-trivial simple measurements take on just two different values. Consider a simple measurement M = Π taking values in a set X consisting of just two elements, let's call these elements ± 1 . The measurement is determined by the two projectors Π(±), which should project onto orthogonal one-dimensional subspaces of C 2 . Each subspace is generated by a vector of

266

Richard D. Gill

the form \u) for some u on the unit sphere, and the associated projectors are Tl(u). Recall that opposite points ±u on the unit sphere correspond to orthogonal vectors \±u) in C 2 , and hence to orthogonal projectors Tί(±u). Thus a projector-valued probability measure for a simple measurement with values in X is given by M(±l) = ΐl(±u) = ^(1 ± u σ) for some u. We apply the trace rule (1) to compute the probabilities of the two outcomes ±1 when the simple measurement M(±l) = Π(±ΐϊ) is carried out on a system in the state p(a) = ^(1 + a σ). Extending Rule 1.2, the reader should verify that these probabilities are (11)

trace p(α)Π(±w) = \{l±a

u).

Using further rules for the state of the system after measurement, it turns out that after measurement the system is in the pure state p(±u) according to the outcome ±1. One can therefore go on to compute probabilities of the series of outcomes of a series of simple measurements carried out on one particle. In the Stern-Gerlach experiment, the initial state of the silver atom is described by the density matrix p(6) = ^ 1 . One can think of this state as corresponding to an electron having spin in a random direction u uniformly distributed over the unit sphere. Indeed, if one takes the mean of p(u) = \{1 + u - σ) with u uniformly distributed over the sphere, the matrix \l results, though this representation of the 'completely random' state p(0) as a mixture of pure states is not unique. Exercise 1.3 (A generalised measurement of spin-half system) Let M(A) = JA Tl(u)du/2π where du denotes integration with respect to Lebesgue surface measure on 5. Show that M is a generalized measurement on a spinhalf system with values in S, and compute the distribution of the outcome of this measurement on the system p(a). This measurement would be physically realised by somehow coupling the spin-half system with a particle moving on the sphere and measuring the position of that particle.

Exercise 1.4 (A generalized measurement of n spin-half systems*) For the state-space (C2)®71 define \u)n = \u) (8) \u) and define Un(u) = \u)n(ύ\n. Define M(A) = (n + 1) / A Π n (ί?)d#/4π and show that M{S) is the projector onto the n + 1 dimensional subspace of vectors, invariant under permutation of the n components of (C 2 )® 71 . Call this subspace Sn and note that trace p®nIlsn — 1. Show that M defines a generalized measurement on n identical copies of a spin-half system with values in S, and compute the distribution of the outcome of this measurement on the system p(v).

A Stern-Gerlach magnet oriented in the direction u implements the simple measurement M(±l) = U(±u). Since for a = 0 the probabilities (11)

Quantum Asymptotics

267

both equal ^, one will find electrons with spin in the directions ±u with equal probabilities. Electrons in the emerging ' + ' beam are in the pure state p(u). Sending them through a Stern-Gerlach device with orientation v splits them again, now with probabilities \(l±u v) (the squared cosine of half the angle between the directions u and v) into two beams of electrons in the states p(±v), and so on. If the electrons started out in the arbitrary mixed state p(α), then the first Stern-Gerlach magnet splits them into two output beams in the pure states p(±u) in the proportions \{l±α u). So, if α was unknown, we do learn something about it from counting the numbers of electrons in each beam. Further operations on the output beams however will not teach us any more as the state of the electrons in either output beam no longer depends on α. If we are allowed to measure a large number of electrons each in the same mixed state p(α), we see that a large number of Stern-Gerlach measurements in three linearly independent directions will enable us to determine α. The question we will study in the rest of the paper is: what is the best way to do this? Will it suffice to use simple measurements on separate particles or can we do better by using more sophisticated measurements, in particular, joint measurements on several particles simultaneously? One can consider rotating a given coordinate system in M3 in such a way as to transform the vectors α and u representing a state or a simple measurement into convenient choices, e.g., we will in the future claim that 'without loss of generality α = (0,0, α^Y which makes p(α) a diagonal matrix. How to do this is given by the following (more difficult) exercise: Exercise 1.5 (Rotation of coordinate system*) For given unit-vector u and angle θ define U = exp(-iθu - σ/2). Then UU* = U*U_^= 1, i.e., U is a unitary transformation o/C 2 , and Up{a)U* = p(b) where &GM3 results from a by rotation about u through an angle θ. This result belongs to the representation theory of groups, an important topic having deep connections with quantum theory. It is a curious fact that with θ — 2π the operator U is equal to —1. Though, in this case, U works on a density matrix by a rotation through 360°, it does not transform a state vector to itself but to its negative. A rotation through 720° or the angle 4π is needed to transform a vector to itself. The fact that one complete revolution does not transform a vector into itself has experimentally observable consequences, namely certain interference effects. 1.3

Quantum Cramer-Rao inequality

Consider a quantum statistical model whereby the density matrix p depends on an unknown parameter θ. Possibly θ is a vector but we will not emphasize that fact in the notation. In particular, a spin-half system has a density

268

Richard D. Gill

matrix p = p(α) depending on a point α in the closed unit ball B. By S we denote the unit sphere, the boundary of B. Interesting statistical models could therefore have a one-, two- or three-dimensional parameter 0, specifying a curve, a surface, or an open region of B. Of particular interest are one- and two-dimensional pure-state models, specifying a curve in S and the whole of S respectively. Results are strikingly different according to whether the true value of θ corresponds to a point in S or in the interior of B. By a mixed-state model we mean a model in the interior of B. By the full model, pure or mixed, we mean the model: 4p is in S\ and 'p is in the interior of fΓ respectively. By the natural parametrization of these models we mean the parametrization p = p(u), p = p(a) respectively.3 The quantum Cramer-Rao bound involves a collection of self-adjoint matrices λi called the quantum score matrices, one for each component of θ, and a quantum information matrix.These are defined as follows. Definition 1.7 (Quantum score matrices) Suppose p = p(θ) depends on parameters θ = (0i,...,0fc). Suppose that p is differentiate with respect to θ. Define the quantum scores Xi = Xi(θ) as the self-adjoint matrices which solve the equations

(12)

* = ΊSΓ =

Ϊ ^

+ PV-

When θ is one-dimensional, we drop the index and write λ for the quantum score. Note that the Xi = λ%{θ) typically depends on θ. The quantum score is also known as the symmetric logarithmic derivative of p with respect to θ{. If p and its derivative pi with respect to θ{ commute, then Xi is nothing else than the derivative of log p. By using a basis of C* making p diagonal, p = Σ,Pj \j) ( j | , one can solve (12) to obtain

If some pj are zero, the corresponding elements of Xi may be chosen arbitrarily (subject to self-adjointness) without effect on subsequent calculations. If 2 p is a pure state, then p = p and it follows from differentiating this equation with respect to 0; that in this case λ^ = 2p^. Exercise 1.6 (Mean quantum score zero) Show that the quantum score has expectation zero, that is, the distribution of a measurement of the observable Xi has mean zero, or trace(pλ ) = 0. 3

It would be nice to express conditions and results in the language of differential geometry, i.e., independent of the specific parametrizations of the models under consideration.

269

Quantum Asymptotics

Exercise 1.7 (Spin half, mixed) Consider the full mixed-state spin-half model d = 2, p — \{\ + θ σ), where θ is three-dimensional and satisfies Σ ά 0 ? < 1. Then p{ = σ{ for each i. At the point θ = (0,0, ξ) the density matrix is diagonal with diagonal elements ^(1 ±ξ) and the quantum scores are found from (13) to be σx, σy and (1 — ξ)~2(—ξl + σz). Exercise 1.8 (Spin half, pure) The full pure-state spin-half model has everything as in the previous exercise but now with Σ) 2 0f = 1. A twodimensional parametrization is called for, using, e.g., the polar coordinates of the unit vector θ. However, on the Northern hemisphere we can stick to θ = (0i, 0 2 ) with 0 3 = +(1 - θ\ - θl)ιl2 and we find that at θ = (0,0) the quantum scores are σx and σy. Exercise 1.9 (n copies) Suppose p^n\θ) scores are given by (14)

λ j 1 } <8>l®

— p®n(θ).

® l + ... + l ®

Then the quantum

® l ® λί 1 } .

Now we can define the quantum information matrix and state the quantum Cramer-Rao bound.

Definition 1.8 (Quantum information matrix) The quantum information matrix IQ is defined by (15)

(IQ)U>

= -trace(p(λ<λi/+λ
Check that this defines a real, positive semi-definite matrix! Exercise 1.10 (n copies, continued) Show from (14) and Exercise 1.6 n that the quantum information IQ for a parameter θ in the system p® is just n times the quantum information for θ in a single copy of the system.

Theorem 1.1 (Quantum Cramer-Rao bound) Define IM{Θ) to be the Fisher information matrix for the parameter θ in the distribution of the outcome of a measurement M on the quantum system p(θ). Then (with respect to the usual ordering of symmetric positive semi-definite matrices) we have IM(Θ)
270

Richard D. Gill

based on the outcome of an arbitrary measurement M can not be smaller than J Q ^ ) - 1 . Both Braunstein and Caves' (1994) and Helstrom's (1967) proofs involve little more than the Cauchy-Schwarz inequality for the complex innerproduct trace X*Y between two self-adjoint matrices. And just as in the usual proof of the Cramer-Rao inequality, as a by-product one can read off from the proof necessary and sufficient conditions for equality to hold. For a one-dimensional parameter, an attractive sufficient condition is that M should be a simple measurement of the observable λ: Exercise 1.11 (Optimal M for 1-d 0) Show for one-dimensional θ that if M is the simple measurement of the observable \, i.e., its values are in one-to-one correspondence with the eigenvalues of λ and each M(x) is the projection onto the corresponding eigenspace, then IM = IQ Life would be easy (but less interesting) if this provided a practical solution. However, typically λ will depend on 0, and typically in such a strong way that the eigenspaces of λ (and not just eigenvalues) depend on 0. Thus the best measurement of 0 in terms of Fisher information depends on the true value of 0. However things are very simple in the following example: Exercise 1.12 Suppose all p(0) commute, i.e., have common eigenspaces. Show that the Pi(θ) then also commute for all i and 0. Show that a simple measurement of the common eigenspaces of all these matrices has Fisher information equal to the quantum information for all values of θ. In this example, we have p = ΣiPiiβ) \i) (z|, a mixture with mixing distribution depending on 0 over the fixed pure states \i). The optimal measurement asks which of these pure states the system is in, in other words it is represented by the projector-valued probability measure with elements \i) (i\. This measurement has outcome i with probability Pi(θ). The quantum information matrix is the Fisher information matrix for this distribution. The result of Exercise 1.11 does give hope for a clear solution to the problem of estimating a one-dimensional parameter, for large n, for the system p Θ n (α(0)), as was first pointed out by Barndorff-Nielsen and Gill (2000). Suppose the parameter 0 is identified, i.e., the mapping from 0 to p(0) is one to one. Then there are a finite number of simple measurements, the distributions of whose outcomes identify 0. For example, in the spin-half case p = ^(1 + 0 σ), measurements of σi, σ2, σ^ result in Bernoulli trials with probabilities ^(1 ± 0;). Suppose that from consistent estimators of the ai(θ) we can construct a consistent estimator of 0. Use a growing number but vanishing proportion of copies of our quantum system with which to 'pre5 estimate 0 consistently. Call this preliminary estimator 0. Next, compute the quantum score for 0 at 0, determine its eigenspaces, and implement the

Quantum Asymptotics

271

corresponding simple measurement on all remaining copies of the system. This gives us an i.i.d. sample from some distribution p(-\θ; θ). Estimate θ by maximum likelihood on these observations conditional on the observed value of θ. The result θ will be an estimator approximately normally distributed about θ with variance approximately l/nl(θ\ θ) where 7(0; θ) is the Fisher information for θ in one of these observations, given θ. Now for n large we have arranged that θ is close to the true value of θ. We may hope that the eigenspaces of \{θ) are close to the eigenspaces of λ(θ) and hence that the Fisher information in one observation (one simple measurement) of λ(θ) is close to that in one observation of λ(0). But the latter achieves the quantum Cramer-Rao bound at θ. Thus under suitable smoothness conditions J(0; θ) will be close to IQ{Θ) and hence the asymptotic distribution of our final estimator close to normal about θ with variance 1/ΠIQ(Θ). This is coming close to saying that θ is asymptotically optimal. We know that no unbiased estimator of θ can have smaller variance. However that does not tell us no estimator whatever can do better, e.g., in terms of mean square error. Indeed the phenomenon of super-efficiency is just as present here as in ordinary statistics. In order to make a compelling optimality statement about our estimator we must either restrict attention to a sub-class of nicely behaved estimators, or make optimality statements which are of a Bayesian or a minimax nature. A very useful tool, which can be used in any of these approaches, is the van Trees inequality which says for a one-dimensional parameter θ with prior distribution π(d0), under some regularity conditions, that the expected (with respect to the prior) mean square error of a completely arbitrary estimator of θ is bounded by one divided by the expected Fisher information for the parameter plus the information, with respect to location, in the prior distribution. This writer prefers to restrict the class of estimators according to some regularity condition. We will go into this in more detail in the next section, but before that, let us consider the multiparameter case. We will see that a more fundamental complication arises: at a fixed parameter value, quantum scores for different components of the parameter may not commute. Exercise 1.13 (Quantum information for spin-half models) In exercises 1.8 and 1.7 we noted the score matrices for the full pure-state model p = p{u) and for the full mixed-state model p = p{a). Show that, at u = (0,0,1) in the first case and at a = (0,0, £) in the second case, the quantum information matrices for θ = (1x1,^2) and for θ = a are respectively

(16)

IQ = (

I I) ,

IQ =

I

1 0 0 0 1 0 00 2 0 0 (1-ξ)-

272

Richard D. Gill

Now the approach just sketched for the one-parameter case breaks down. Certainly we can form a preliminary estimator of 0 and thereby 'estimate' the quantum score matrices. Next, in the full pure- and mixed-state models, one can rotate the coordinate system and reparametrize so that the quantum scores become σx,σy (pure-state model), and σx^σy^α + bσz (mixed state model). There is no way, in either model, we can simultaneously measure these observables since they do not commute. Thus no measurement on a single particle has an information matrix equal to IQ. The big question is, what is the class of information matrices IM which are available? And if we can perform measurements on the system obtained by combining particles, what scaled information matrices 1$ ln become available? The latter class includes all of the former class, since the joint measurements include n i.i.d. copies of measurements on separate particles; moreover, these classes are convex and bounded. Though all scaled information matrices 1^\ /n are bounded by /Q, we cannot expect them, for given n, to contain a single 'best' information. Which measurement we should choose, will depend on the relative accuracy with which we want to estimate the different components of 0. For instance, if in the pure-state case, close to 0 = (0,0), we are only interested in 0i, we should simply measure σx on each of our n particles yielding the maximum information on θ\ but no information at all on 02- After we have characterized the class of all information matrices available, we must specify through some loss function the relative importance of the different parameters and solve some optimization problem. 2

A new Cramer-Rao type bound

In this section we report on recent results of Gill and Massar (2000), concentrating on the spin-half situation, and within that case, emphasizing the full pure-state model and the full mixed-state model. There turns out to be a striking difference between these two cases. For pure states, there is asymptotically no advantage in joint measurements on many particles. However for mixed states there typically is an advantage. How much is still an open question. The following result should be called a 'Theorem' (in quotes) since we do not specify regularity conditions and indeed only a 'Proof exists, not yet a Proof. Theorem 2.1 (Achievable information matrices, n = 1) The set of all Fisher information matrices of outcomes of measurements of one spin-half particle for a smooth model p{θ) is {F : trace{IQ1F) < 1}. The parameter 0 could be one-, two- or three-dimensional. We suppose either that we have a pure-state model, or a strictly mixed-state model. The argument, in Gill and Massar (2000), has two main parts. In the first part

273

Quantum Asymptotics

we show that for all M, F = IM satisfies trace (IQ F) < d - 1 (we do not yet need that d = 2). In the second part we show that, when d = 2, for any F satisfying this inequality one can construct a measurement M for which IM = F. For d > 2, not all F satisfying trace(/g 1 F) < 1 are achievable, and it remains open to characterize exactly the class of achievable information matrices. 1

For the first part a series of preparatory steps are taken to bring us, 'without loss of generality', to a situation that allows exact computations. For simplicity take d = 2. If p(θ) lies in the interior of the unit ball, and θ has dimension one or two, one can augment θ with other parameters, raising its dimension to 3. This can be done in such a way that the crossinformation elements in the augmented IQ{Θ) are all zero. It then suffices to prove the inequality for θ of dimension 3, and then we may as well use the natural parametrization p(θ) = ^(1 + θ σ) with ||β|| < 1 since the quantity trace{IQ 1 F) is invariant under smooth reparametrization. If, on the other hand, p(θ) is a pure state model we can in the same way after augmenting θ assume that θ has dimension 2 and after reparametrization the model is p{θ) = \(l + θ σ) w i t h | | 0 | | = l .

For the next preparatory step we need the concepts of refinement and coarsening of a measurement. Definition 2.1 (Coarsening and refinement) A measurement M with sample space X is a refinement of Mι with sample space y, and M' is a coarsening of M, if a measurable function f : X -ϊ y exists with M'(B) —

The result of measurement of M1 then has the same distribution as taking / of the outcome of measurement of M. It follows that the Fisher information in the outcome of M' is less than or equal to that in M since under coarsening of data, Fisher information can only decrease. Now we show that any measurement M' has a refinement M for which M(A) = JA M(x)μ(dx) for some nonnegative operator-valued function M and bounded measure μ and for which M(x) has rank one for all x, thus M(x) = \x) (x\ for some (not necessarily normalised) vector function \x). Consequently it will suffice to prove the result for such maximally refined measurements M. Start with the measurement Mf with sample space y. r Define a probability measure v on 3^ by u(B) = tmce(M (B))/d; by taking Radon-Nikodym derivatives one can define Mf(y) such that M'(B) = f M w h e r e fBM'{y)v(dy). Since the rank of M (y) is finite, M'(y) = Σi i(y) each Mi(y) has rank one. Now refine the original sample space y to X = y x {1,..., G?}, defining M(A x {*}) = fA Mi(y)v{άy). Equivalently M(A) = JAMi(x)μ(dx,di) where μ is the product of v with counting measure.

274

Richard D. Gill

This brings us to the situation where the model is either full pure-state or full mixed-state, and where the measurement is maximally refined. We take the natural parametrization of either of these models, and without loss of generality work at a point θ where θ = (0,0) or (0,0,ξ). This is possible by the result of Exercise 1.5. Now we have a formula for IQ and for the derivatives of p with respect to the components of 0, both in the pure and the mixed case, and we have a representation for M in terms of a collection of vectors \x) which must satisfy the normalization constraint fχ \x) {x\ μ(dx) = 1 but which are otherwise arbitrary. Both p and IQ are diagonal. We simply compute trace IQ IM and show that it equals 1 in the case d = 2. We leave the details as an exercise for the diligent reader—the computation is not difficult but does not seem all that illuminating either. We would dearly like to know if there is a more insightful way to get this result! The same arguments work for arbitrary d though the details are more complicated; a full mixed-state model has ^d(d +1) parameters, a full purestate model \d(d + 1) — (d — 1) parameters, and a careful parametrization is needed to make IQ diagonal. In the second part (for d = 2 only) it is shown that for any F satisfying trace(/g 1 F) < 1, one can construct a measurement M for which IM = F. This measurement will be described in the next section. It typically depends on the point θ so a multi-stage procedure is going to be necessary to achieve asymptotically this information bound. That will be the main content of the next section, where we do some quantum asymptotics proving asymptotic optimality results for n —> oo of the resulting two-stage procedure. We only have partial results for n > 1. In two special cases the available scaled information matrices do not increase as n increases. One of these cases is the case of pure-state models. This case has been much studied in the literature and is of great practical importance. The other case is when we make a restriction on the class of measurements to measurements of product form (in the literature also sometimes called an unentangled measurement). We first define this notion and then explain its significance. 1

Definition 2.2 (Product-form measurements) We say that a measuren ment on n copies of a given quantum system is of product form if M^ \A) = JAM^n\x)μ(dx) for a real measure μ and matrix-valued function M^n\x) n where M^ \x) is of the form M\{x) ® ® Λίn(x), with nonnegative components. We described in the previous section a measurement procedure whereby we first carried out measurements on some of our n particles, and then, depending on the outcome, carried out other measurements on the remaining particles. Altogether this procedure constitutes one measurement on the

Quantum Asymptotics

275

joint system of n particles taking values in some n-fold product space. One can conceive of more elaborate schemes where, depending on the results at any stage, one decides, possibly with the help of some classical randomisation, which particle to measure next and how. It would be allowed to measure again a particle which had previously been subject to measurement. There exists a general description of the state of a quantum system after measurement, allowing one to piece all the ingredients together into one measurement of the combined system. A measurement which can be decomposed into separate steps consisting of measurements on separate particles only, is called a separable measurement. It turns out that all separable measurements, provided all outcomes of the component steps are encoded in the overall outcome x, have productform. On the other hand, product-form measurements exist which are not separable; see Bennett et al. (1998). The product-form measurements form a large and interesting class, including all measurements which can be carried out sequentially on separate particles as well as more besides. In the notion of separable measurement it is insisted that all intermediate outcomes are included in the final outcome. If one throws away some of the data, one gets an outcome whose distribution is the same as the distribution of a coarsening of the original measurement. Coarsening of a measurement can obviously destroy the product form, replacing products of nonnegative components by integrals of such products. Theorem 2.2 (Achievable information matrices n > 1) The scaled information matrices of measurements on a smooth model p®n(θ) remain {F : ^ F ) < 1} 1. when θ is one-dimensional; 2. in a pure-state spin-half model; 3. in a mixed-state spin-half model with the class of measurements restricted to measurements which can be refined to product-form. The theorem is proved exactly as before, again finishing in an unilluminating calculation. Since coarsening of a measurement can only decrease Fisher information, one need only consider product-form measurements in the mixed-state case. We have a counterexample to the conjecture that, for mixed states, the bound holds for all measurements. In the case n = 2, at the point p = ^1, 1 there is a measurement for which trace(IQ 1^/2) = 3/2, thus 50% more information in an appropriate measurement of two identical particles than any combination of separate measurements of the two. What the set of

276

Richard D. Gill

achievable scaled information matrices looks like and whether it continues to grow (and to what limit) as n grows, is completely unknown. The measurement has seven elements, the first six of the form gΠ[ψ], and the seventh Π^,], where Π^j denotes the projector onto the one-dimensional sub-space spanned by the vector | ^ ) . The various \φ) are \+z + z), \—z — z), \+x + x), \-x - x), \+y + y), \-y - y), \S). By \+z + z) we mean \+z) ® \+z) = \ez) ® \ez) and similarly for the next five. The last φ is the socalled singlet state jξ(\+z) ® \—z) — \—z) ® \+z)). As a pure state of two interacting spin-half particles, this is the famous entangled state resulting in the violation of the Bell inequalities, and hence of locality (according to some interpretations). Here it arises as part of a measurement of two completely non-interacting particles; however this measurement can never be implemented by doing separate operations on the separate particles. Similar examples occur in the paper of Vidal et al. (1998), extending the pure-state results of Massar and Popescu (1995) to mixed states. 3

Quantum asymptotics

The results of the previous section are in the form of a bound on the information matrix based on the outcome of any measurement (perhaps restricted to the class of product-form measurements) on n identical copies of a given spin-half quantum system with state depending on an unknown parameter θ. We will now explain how such a bound can be used to give asymptotic bounds on the quality of estimators based on those measurements. Furthermore, we show how the bounds can be achieved by a two-stage procedure based on simple measurements on separate particles only. As far as achieving the bounds is concerned, only for the full mixed-state model under the natural parametrization is the problem completely solved. For the other models, the results are conjectural. We will discuss two kinds of bounds: firstly, a bound on the limiting scaled mean quadratic error matrix of a well-behaved sequence of estimators, and secondly, a bound on the mean quadratic error matrix of the limiting distribution of a well-behaved sequence of estimators. Each has its advantages and disadvantages. In particular, since the delta-method works for (the variance of) limiting distributions but not for limiting mean square errors, stronger conditions are needed to prove optimality of some procedure in the first sense than in the second sense. 3.1

Two asymptotic bounds

Obviously a bound on the information matrix, by the ordinary CramerRao inequality, immediately implies a bound on the covariance matrix of an unbiased estimator. However, this is not a restriction we want to make. It turns out much more convenient to work via a Bayesian version of the

Quantum Asymptotics

277

Cramer-Rao inequality due to van Trees (1968), as generalised to the multiparameter case by Gill and Levit (1995). For a one-dimensional parameter the van Trees inequality is easy to state: the Bayes quadratic risk is bounded by one over expected information plus information in the prior. In the multiparameter case one has a whole collection of inequalities corresponding to different choices of quadratic loss function and some other parameters, more difficult to interpret. Let π(0) be a prior density for the p-dimensional parameter 0, which we suppose to be sufficiently smooth and supported by a compact and smoothly bounded region of the parameter space; see Gill and Levit (1995) for the precise requirements. Let C(θ) be&pxp symmetric positive-definite matrix (C stands for cost function) and let Vj£'(θ) be the mean quadratic error matrix of a chosen estimator of θ based on a measurement of n copies of the quantum system. Letting Θ denote a random drawing from the prior distribution π, it follows that E trace C(Θ)V^(Θ) is the Bayes risk of the estimator with respect to the loss function (0W - θ)τC{θ)(θ^ - θ). Let D(θ) be another p x p matrix function of θ. Let jj£\θ) denote the Fisher information matrix in the measurement. Then the multivariate van Trees inequality reads (17) (EtrBcei?(Θ))» 1

E trace C(Θ)- D(Θ)(/g)(Θ)/n)JD(Θ)τ +I(π)/n where

(is)

ϊw = /^Σ^)^(^(ίw»))^^

From Theorem 2.2 we have the bound trace IQ1(Θ){I^){Θ)/Π) < 1, where, in the mixed case, we restrict attention to measurements refinable to productform. We are going to assume that our sequence of measurements and estimators is such that the normalized mean quadratic error matrix V^\θ) converges sufficiently regularly to a limit V(θ). Our aim is to transfer the inequality of Theorem (2.2) to V obtaining the bound trace IQ1{Θ)V(Θ)~1 < 1. We will do this by making appropriate choices of C and D. We will need regularity conditions both on the sequence of estimators and on the model p(θ) in order to carry over (17) to the limit. Theorem 3.1 (Asymptotic Cramer-Rao 1) Suppose that on some open set of parameter values θ: 1. nVίn\ converges uniformly to a continuous limit V;

278

Richard D. Gill

2. IQ(Θ) is continuous with bounded partial derivatives; 3. V and IQ are non-singular. Then the limiting normalised mean quadratic error matrix satisfies (19)

trace IQ {Θ)V{Θ)1

< 1.

1

We outline the proof of the theorem as follows. First of all, we pick a point 0o and define VQ = V(ΘQ). Next we define (20)

C(Θ)

(21)

1

=

1

V0- IQ (Θ)V0-\

D(θ) = V^I^iθ).

With these choices (17) becomes (22) ίni E trace V^1 K1 (Θ)VQ-1 {nV^ (θ)) >

(E trace V Γ 1 /Q ^V© ) ) 2 ° " „

.

We can bound the first term in the denominator of the right hand side by 1, by the results of the last section. The second term in the denominator of the right hand side is finite, by our third assumption, and for n —> oo it converges to zero. By our first assumption (22) converges to (23)

Etrace V o - ^ g H θ ) ^ " 1 ^ © ) > ( E t r a c e V ^ J Q ^ Θ ) ) 2 .

Now replace the prior density π by one in a sequence of priors, concentrating on smaller and smaller neighbourhoods of ΘQ. Using the continuity assumptions on V and IQ, we obtain from (23) the inequality

trace V^I^iflύV^V* > or in other words, with θ = ΘQ, the required (24)

trace IQ {Θ)V'\Θ) 1

< 1.

In some situations it might be more convenient to have a bound on the mean quadratic error of a limiting distribution, assuming one to exist. At the moment of writing we believe the following: Theorem 3.2 (Asymptotic Cramer-Rao 2) Let Z be distributed with the limiting distribution of y/n{θ — θ). Suppose that: 1. θn is Hάjek regular at θ at root n rate;

Quantum Asymptotics

279

2. the asymptotic mean quadratic error matrix V — Έ(Z Zτ) is nonsingular; 3. IQ is non-singular. Then V satisfies trace/g1^)^)"1^!.

(25)

The proof should follow the lines of the similar result in Gill and Levit (1995), with a prior distribution concentrating on a root n neighbourhood of the truth. We will need similar choices of C and D as in the proof of Theorem 3.1, though the dependence of D on θ can now be suppressed. 3.2

Achieving the asymptotic bounds

At present we have essentially complete results in the full mixed-state spinhalf model with the natural parametrization. We believe they can be extended to smooth (C 1 ) pure- and mixed-state models. Give yourself a target mean quadratic error matrix W(θ) satisfying (26)

trace IQ(β)-ιW(θ)-1

< 1.

Is there a sequence of measurements M^ satisfying the conditions of Theorems 3.1 or 3.2 with limiting mean quadratic error matrix V(θ) equal to the target? Possibly we do not start with a target W but with a step earlier, with a quadratic cost function. For given C(θ) it is straightforward to compute the matrix W(θ) which minimizes trace C(Θ)W(Θ) subject to the constraint (26); the solution is W = t r a c e ( ( J ~ ^ C / ^ ) l ) J ^ ( / | c / | ) ^ / ^ . Now we pose the same question again, with the W we have just calculated as target. ι Let us call F = W~ the target information matrix. First we pretend θ is known and exhibit a measurement M on a single particle with the target information matrix at the given parameter value. In the previous section we omitted explaining how the bound of Theorem 2.1 can be attained. That theorem stated that, at a given parameter value, for any positive-semidefinite symmetric F satisfying trace IQ F < 1 there is a measurement M on a single spin-half particle with IM = F. What is that measurement? We describe it in the case of a full mixed-state spinhalf model with the natural parametrization, thus p(θ) = ^(1 + θ σ). The matrices IQ and F axe 3 x 3 . To start with, we compute the eigenvector-eigenvalue decomposition of IO2FIO2,

obtaining eigenvectors hi and nonnegative eigenvalues 7i, say.

The condition on F translates to ^ 7 ^ < 1. Now define gι = Ighi and

280

Richard D. Gill

three unit vectors Ui = gi/\\gi\\, and finally consider the measurement M taking seven different values, whose elements are 7iΠ(±ί?i), i = 1,2,3, and (1-Σ7ί)l It turns out by a straightforward computation (carried out, without loss of generality, at θ = (0,0, £)) that the information matrix for the measurement with the two elements U(±Ui) has information matrix gi®gi and hence the measurement M has information matrix Σ{ ηigi ®gι = F. This seven-outcome measurement can be implemented as a randomized choice between three simple measurements: with probability 7$ measure spin in the direction Ui, with probability 1 — ΣΊi do nothing. However, in practice this measurement is not available, since the directions Ui and probabilities 7; depend on the unknown θ. We therefore take recourse to the following two-stage measurement procedure. First measure spin in the x, y and z directions on \nα each of the particles, where 0 < α < 1 is fixed and the numbers are rounded to whole numbers. The expected relative frequency of 'up' particles in each direction is | ( 1 + 0;), i = 1,2,3, so solving 'observed equals expected' yields a consistent preliminary estimator θ of θ. If the estimate lies outside the unitball, project onto the ball, and stop. With large probability no projection is necessary. We can compute the eigenvalue-eigenvector decomposition of Iq2(Θ)F(Θ)IQ2(0), leading to fractions ηι and directions Ui as above. Measure the spin of a fraction ηι of the remaining particles in the direction iϊi. Solve again the three (linear) equations 'observed relative frequency equals expected', treating the ΰi as fixed. Project onto the unit ball if necessary. Call the resulting estimator 0. Our claim is that this procedure exhibits a measurement M^ on the n particles, and an estimator 0(n) based on its outcome, which satisfies the conditions of Theorem 3.1, with V(θ) equal to the target W(θ). Thus the bound of Theorem 3.1 is also achievable, and a measurement which does this has been explicitly described above. Apart from projecting onto the unit ball, the estimator involves only linear operations on binomial variables, so is not difficult to analyse explicitly. We need a preliminary sample size n of order nα and not, for example, of order logn, in order to control the scaled mean quadratic error of the estimator. There is an exponentially small probability—in n, not in n—that the preliminary estimate is outside of a given neighbourhood of the truth, and hence that the scaled quadratic error is of order n. One can further check that the estimator we have described also satisfies the conditions of Theorem 3.2. Possibly one is interested in a different parametrization of the model. Under a smooth (C 1 ) reparametrization, the delta method allows us to maintain optimality in the sense of Theorem 3.2. However optimality in the sense

Quantum Asymptotics

281

of Theorem 3.1 could be destroyed; in order for it to be maintained the reparametrization should also be bounded. Alternatively one must modify the estimator by a truncation at a level increasing slowly enough to infinity with n, cf. Schipper (1997; section 4.4) or Levit and Oudshoorn (1993) for examples of the technique. This approach can be extended to other spin-half models. The difficulties are exemplified by the case of the two-parameter full pure-state spin-half model. Locally, consider the natural parametrization θ = (#i,#2), #3 = (1 - θ\ - 0f) 1 / 2 , p = p(θ) at the point θ = (0,0). The quantum information matrix for three parameters 0χ, 0 2 , #3 contains an infinite element. However, the recipe outlined above continues to work if we add to a given 2 x 2 target information matrix a third zero row and column—infinities always get multiplied by zero. The third fraction 73 equals zero so simple measurements in just two directions suffice. The resulting procedure involves linear operations on binomial counts, projecting onto 5, and reparametrization. Under some smoothness we should finish with an estimator optimal in the sense of Theorem 3.2; under further smoothness, boundedness, and a sufficiently large preliminary sample also optimality in the sense of Theorem 3.1 should hold. If the target information matrix includes some zeros, i.e., one is not interested at all in certain parameters, the results should still go through; the preliminary sample should be of size of order n α , \ < α < 1, in order that the uncertainty in the initial estimate of the 'nuisance parameters' does not contaminate the final result.

4

Non-locality without entanglement

It would take us too far afield here to explain the notions of entanglement and of non-locality. For some kind of introduction see Kϋmmerer and Maassen (1998) and Gill (1998), and Gill (2000); see also the books of Peres (1995), Isham (1995), Penrose (1994), Maudlin (1994). However, we would like to discuss whether or not our finding that non-separable joint measurements on several independent (non-entangled) quantum particles can yield more information than any separate measurements on the separate particles, should be considered surprising or not. Recall that separable measurements, cf. Bennett et al. (1998), are measurements which can be decomposed into a sequence of measurements on separate particles, each measurement possibly depending on the outcome of the preceding ones, and whereby it is allowed to measure further a particle which has already been measured (and hence its state has been altered in a particular way) at an earlier step. Prom a mathematical point of view there should not be much surprise. The class of separable measurements is contained in the class of productform measurements, which is clearly a very small part of the space of all

282

Richard D. Gill

measurements whatsoever. The optimisation problem of maximising Fisher information (more precisely, some scalar functional thereof) must only be expected to have a larger outcome when we optimise over a larger space. The surprise for the mathematician is rather that for pure states, and for one dimensional parameters, there is no gain in joint measurements. And it is strange that mixed states should exhibit this phenomenon, whereas pure states do not: the difference is classical probabilistic mixing, which should not lead to nonclassical behaviour. However, physicists are and should be surprised. The reason is connected to the feeling of many physicists that the randomness in measurement of a quantum system should have a deterministic explanation (Einstein: "God does not throw dice") . We appreciate very well that tossing a coin is essentially a completely deterministic process. It is only uncontrolled variability in initial conditions which lead to the outcome appearing to be completely random. Might it be the case also that the randomness in the outcome of a measurement of a quantum system might be 'merely' the reflection of statistical variability in some initial conditions? Hidden variables, so called because at present no physicist is aware what these lower level variables are, and there is no known way directly to measure them. In fact there already exist arguments aplenty that if there is a deterministic hidden layer beneath quantum theory, it violates other cherished physical intuitions, in particular the principle of locality; see again Kϋmmerer and Maassen (1998), Gill (1998) for some introduction to the phenomenon of entanglement, and further references. But let us ignore that evidence and consider the new evidence from the present results. Consider two identical copies of a given quantum state. Suppose there were a hidden deterministic explanation for the randomness in the outcome of any measurement on either or both of these particles. Such an explanation would involve hidden variables CJI, CJ2 specifying the hidden state of the two particles. Since applying separate measurements to the two systems produces independent outcomes, and since the outcomes of the same measurements are identically distributed, one would naturally suppose that these two variables are independent and identically distributed. Their distributions would of course depend on the unknown parameter θ. Now when we measure the joint system, there could be other sources of randomness in our experiment, possibly even quantum randomness, but still it would not have a distribution depending on 0. So let us assume there is a third random element UM such that the outcome of the measurement M on the system p{θ) ® p(θ) is a deterministic function of CJI, CJ2 and UMΊ the first two are independent and identically distributed, with marginal distributions depending on 0, while the distribution of LJM given the other two is independent of θ. Thus the random outcome X of the measurement of M is just X(U;I,CU2,^M) 5 a random variable on the prob-

Quantum Asymptotics

283

ability space (Ω x Ω x ΩM), ((P# X P#) * P M ) , where P M is some Markov kernel from Ω x Ω to Ω ^ Now it is well-known from ordinary statistics that the Fisher information in θ from the distribution of any random variable defined on this space is less than twice the information in one observation of ωι itself seen as a random variable defined on (Ω,P^). Thus if one could realise any ΩM> P M and any X whatsover by suitable choice of measurement M, achievable Fisher information would be additive! What can we conclude from the fact that achievable Fisher information is not additive? If they exist, the hidden variables are so well hidden that we cannot uncover them from any measurements on single particles, i.e., it is not possible to realise any ( Ω M , P M ) and any X whatever by appropriate choice of experimental set-up. However, we can uncover the hidden variables better, apparently, from appropriate measurements on several particles brought together, even though these particles have nothing whatever to do with one another—their hidden variables are independent and identically distributed. Alternatively the explanation must be found in some pathological non-measurability or non-regularity of the statistical model we have just introduced. Whatever escape-route one chooses, it is clear that if there is a deterministic explanation for quantum randomness, it has to be a very weird explanation. God throws rather peculiar dice.

Acknowledgements. This paper is based on work in progress together with O.E. Barndorff-Nielsen and with S. Massar. I am grateful for the hospitality of the Department of Mathematics and Statistics, University of Western Australia. I would like to thank Boris Levit for his patient advice.

REFERENCES Barndorff-Nielsen, O.E. and Gill, R.D. (2000). Fisher information in quantum statistics. To appear in J. Phys. A. Preprint quant-ph/9802063, http://xxx.lanl.gov Belavkin, V. P. (1976). Generalized uncertainty relations and efficient measurements in quantum systems. Teoreticheskαyα i Mαtemαtiskαyα Fizikα 26, 316-329; English translation, 213-222. Based on author's candidate's dissertation, Moscow State University, 1972. Bennett, C.H., DiVincenzo, D.P., Fuchs, C.A., Mor, T., Rains, E., Shor, P.W., Smolin, J.A., and Wootters, W.K. (1998). Quantum nonlocality without entanglement. To appear in Phys. Rev. A. Preprint quant-ph/9804053, http://xxx.lanl.gov

284

Richard D. Gill

Biane, P. (1995). Calcul stochastique non-commutatief. pp. 4-96 in: Lectures on Probability Theory: Ecole deVe de Saint Flour XXIII-1993, P. Biane and R. Durrett, Springer Lecture Notes in Mathematics 1608. Braunstein, S.L. and Caves, C M . (1994). Statistical distance and the geometry of quantum states. Physical Review Letters 72, 3439-3443. Brody, D.C. and Hughston, L.P. (1998), Statistical geometry in quantum mechanics, Proceedings of the Royal Society of London Series A 454, 2445-2475. Gill, R.D. (2000). Lecture Notes on Quantum Statistics, h t t p :

//www.math.uu.nl/people/gill/Preprints/book.ps.gz Gill, R.D. (1998). Critique of'Elements of quantum probability'. Quantum Probability Communications 10, 351-361. Gill, R.D. and Levit, B.Y. (1995). Applications of the van Trees inequality: a Bayesian Cramer-Rao bound. Bernoulli 1 59-79. Gill, R.D. and Massar, S. (2000). State estimation for large ensembles. To appear in Phys. Rev. A. Preprint quant-ph/9902063, h t t p : / / x x x . l a n l . g o v Helstrom, C.W. (1967). Minimum mean-square error of estimates in quantum statistics. Phys. Letters 25 A, 101-102. Helstrom, C.W. (1976). Quantum Detection and Estimation Theory. Academic, New York. Holevo, A.S. (1982). Probabilistic and Statistical Aspects of Quantum Theory. North Holland, Amsterdam. Holevo, A.S. (1983). Bounds for generalized uncertainty of the shift parameter. Springer Lecture Notes in Mathematics 1021, 243-251. Holevo, A.S. (1999). Lectures on Statistical Structure of Quantum Theory. h t t p : //www. imaph. t u - b s . d e / s k r i p t e_. html Isham, C. (1995). Quantum Theory. World Scientific, Singapore. Kύmmerer, B. and Maassen, H. (1998). Elements of quantum probability. Quantum Probability Communications 10, 73-100. Levit, B.Y. and Oudshoorn, C.G.M. (1993). Second order admissible estimation of variance. Statistics and Decisions, supplement issue 3, 17-29. Malley, J.D. and Hornstein, J. (1993). Quantum statistical inference. Statistical Science 8, 433-457. Massar, S. and Popescu, S. (1995). Optimal extraction of information from finite quantum ensembles. Physical Review Letters 74, 1259-1263. Maudlin, T. (1994). Quantum Non-locality and Relativity. Blackwell, Oxford. Meyer, P.A. (1986). Elements de probabilites quantiques. pp. 186-312 in: 7 S eminaire de Probabilites XX, ed. J. Azema and M. Yor, Springer Lecture Notes in Mathematics 1204. Penrose, R. (1994). Shadows of the Mind: a Search for the Missing Science of Consciousness. Oxford University Press.

Quantum Asymptotics

285

Percival, I. (1998). Quantum State Diffusion. Cambridge University Press. Peres, A. (1995). Quantum Theory: Concepts and Methods. Kluwer, Dordrecht. Peres, A. and Wootters, W.K. (1991). Optimal detection of quantum information. Physical Review Letters 66, 1119-1122. Preskill (1997). Quantum Information and Computation. http://www.theory.caltech.edu/people/preskill/ph299

Schipper, CM.A. (1997). Sharp Asymptotics in Nonparametric Estimation. PhD thesis, University Utrecht, ISBN 90-393-1208-7. Stratonovich, R.L. (1973). The quantum generalization of optimal statistical estimation and testing hypotheses. Stochastics 1, 87-126. van Trees, H.L. (1968). Detection, Estimation and Modulation Theory (Part 1). Wiley, New York. Vidal, G., Latorre, J.I., Pascual, P., and Tarrach, R. (1998). Optimal minimal measurements of mixed states.

Preprint quant-ph/9812068, http: //xxx. lanl. gov. R.F. Werner (1997). Quantum information and Quantum Computing.

h t t p : //www. imaph. tu-bs. de/skripte_. html MATHEMATICAL INSTITUTE UNIVERSITY OF UTRECHT

Box 80010 3508 TA UTRECHT NETHERLANDS

[email protected]

ADAPTIVE CHOICE OF BOOTSTRAP SAMPLE SIZES

1

FRIEDRICH GOTZE AND ALFREDAS RACKAUSKAS

2

University of Bielefeld and University of Vilnius Consider sequences of statistics Tn(Pn,P) of a sample of size n and the underlying distribution. We analyze a simple data-based procedure proposed by Bickel, Gotze and van Zwet (personal communication) to select the sample size m = mn < n for the bootstrap sample of type "m out of n" such that the bootstrap sequence T^ for these statistics is consistent and the error is comparable to the minimal error in that selection knowing the distribution P. The procedure is based on minimizing the distance between L m (P n ) and Pn), where Lm{Pn) denotes the distribution of T^. AMS subject classiήcations: 62D05.

Keywords and phrases: Bootstrap, m out of n bootstrap, Edgeworth expansions, model selection.

1

Introduction

In this paper, we investigate an adaptive choice of the bootstrap sample size m in sampling from an i.i.d. sample of size n m-times independently and with (resp., without) replacement. To simplify the writing we shall abbreviate the notion of m out of n sampling as moon bootstrap. Assume that the random elements ΛΊ,..., X n ,... are independent and identically distributed from a distribution P on a measurable space (S, *A). Let Pn denote the empirical measure of the first n observations XL, ..., Xn. Throughout we assume that P G Vo C V, where Vo is a set of probability measures on (5, *4) containing all empirical measures Pn. Let Tn = T n (Xi,... ,X n ;P) denote a sequence of statistics, possibly dependent on the unknown distribution P in order to ensure that Tn is weakly convergent to some limiting distribution as n tends to infinity. A typical example is given by Tn = nα(F(Pn) - F(P)), where F : VQ -> R denotes a functional on VQ and α > 0 is an appropriate normalization rate. We are interested in the estimation of the distribution function (d.f.) Ln(P; α) of Tn by means of resampling methods. 1 2

This research was supported by the Deutsche Forshungsgemeinshaft SFB 343.

This research was supported by the Deutsche Forshungsgemeinshaft SFB 343 and Institute of Mathematics and Informatics, Lithuania.

287

Bootstrap Sample Sizes

The nonparametric bootstrap estimates the d.f. L n (P, α) by the plug-in method, that is, by the conditional d.f.

where X*,... , X* is a bootstrap sample from the empirical distribution P n . One of the major problems for the nonparametric bootstrap estimate Ln is its consistency. Various types of consistency can be considered. Usually, if d denotes a certain distance on the set of all distribution functions then Ln is said to be d-consistent (or, simply consistent, when d is fixed) in probability (resp., a.s.) provided that d(Ln,Ln(P)) —> 0 in probability 71—>OO

(resp., a.s.). Conditions ensuring the consistency were considered, e.g., by Bickel and Preedman (1981), Bretagnolle (1983), Athreya (1987) and Beran (1982, 1997). Extensive references and details on various bootstrap methods can be found in the recent monograph by Davison and Hinkley (1998). A number of examples, where the bootstrap fails to be consistent together with positive results suggest that the consistency of the bootstrap estimate Ln requires the following conditions: 1) for any Q from a neighborhood V(P) of P, Ln(Q) has to converge weakly to a limit I/(Q), say, and the convergence has to be uniform on V(P); 2) the function Q —> L(Q) has to be continuous. The moon bootstrap with replacements (shortly, ra/n-bootstrap) estimates the d.f. Ln(P,α) by L m (P n ,α), whereas the moon bootstrap without replacements (shortly, Q)-bootstrap) estimates Ln(P,ά) by

L*m(Pn,a) = I

V

l{Tm(Xh,...,Xim;Pn)

< a}.

Under very weak conditions, the moon bootstrap resolves problems of consistency of the classical bootstrap by choosing m = o(n) bootstrap samples. It was first suggested by Bickel and Preedman (1981) and investigated in Bretagnolle (1983), Gόtze (1993), resp. Bickel, Gόtze and Zwet (1997), Shao (1994), Politis and Romano (1994) (examples of nonregular statistics), Swanepoel (1986), Deheuvels, Mason and Shorack (1990) (extreme value statistics), Shao (1996) (model selection), Datta and McCormik (1995), Datta (1996), Heimann and Kreiss (1996) (first order autoregression models), Athreya (1987), Arcones and Gine (1989) (heavy tailed population distributions). If d denotes a distance between distribution functions (e.g., Kolmogorv, Levy-Prokhorov, or bounded Lipschitz distance) then a measure of risk in estimating Ln(P) by some estimator Ln is given by

EPd(Ln,Ln(P)). For Ln being the moon bootstrap estimator Lm(Pn) the 'generic' nonparametric case is described by the nonconsistency of this estimator for m ~ n

288

Friedrich Gόtze and Alfredas Rackauskas

due to the essential randomness of its limit distribution under Pn for such m. Introducing h ~ n/m as a 'bandwidth' type parameter in this nonparametric estimation problem, the case h ~ 1 is characterized by the fact that the variance of the bootstrap estimate may not tend to zero as n tends to infinity. On the other hand for large values of h the variance decreases in 1 many cases of order ©(/i" ). Since the moon bootstrap actually estimates I/ m (P), the difference d(Lm(P),Ln(P)) will be significant for m/n small (or h large) and contributes a bias term which dominates the estimation error in this case. Thus, as in most nonparametric problems one has to look for a tradeoff choice of m minimizing the estimation error. On the other hand for 'parametric' problems where the bootstrap works, like in the estimation of the distribution of Student's test statistic under the hypothesis, one can show by higher order approximations that the bias as well as the variance of Lm(Pn) essentially decrease as m grows up m ~ n, see Hall (1992). One would like to find a common recipe for choosing m effectively for both nonparametric as well as parametric situations in order to obtain a uniformly consistent and effective estimate for the distribution Ln(P). One way would be to look for a sample size m minimizing some crossvalidation measure by a jackknife estimate related to the risk under the unknown distribution. This has been suggested by Datta and McCormik (1995) in a first order regression problem. Unfortunately, this method is computationally rather involved and the performance of this scheme is difficult to analyze. Bickel, Gόtze and van Zwet (personal communication) suggested to base the choice of m on the discrepancy between Lm(Pn) and Lrnj2{Pn) We motivate this idea by showing that the (random) distance between Lm(Pn) and Ln(P) as a function on m is stochastically equivalent to the (random) distance between Lm(Pn) and Lm/2(Pn) as n -> oo and m = m(n) —> oo. More precisely, consider for some distance d between distributions (like e.g. Kolmogorov's distance) (1.1)

and Δ m :=

Am~d(Lm(Pn),Ln(P))

d(Lm(Pn),Lm/2(Pn)).

In Theorem 2.3 we prove that, under certain conditions, for some model dependent rate 0 < α < 1 (like = 1/2,1/3,1/4 etc. ) and sequences ra(rc) such that the limit limn m(n)/nα, say 7 G [0,00] exists, we have

(1-2)

|^

A

where ξo and ξoo axe constants depending on P and, for 0 < 7 < 00, ξΊ denotes a random variable depending on P. Here, —> denotes as usual convergence in distribution. Typically, we find that EpAm/EpA where cΊ(P) is a constant depending on 7 and P.

289

Bootstrap Sample Sizes

Based on these observations, we suggest m* = argmin 2 < m < n Δ m as a random choice of m for the ra/rc-bootstrap. We will show that this choice is as good as choosing the optimal m when knowing the unknown distribution P as long as m/n —> 0 holds. Simulations show that the method works in the region m ~ n as well but behavior in this region is difficult to analyze for general models of distributions. The reasons for such a choice are illustrated by the following simple example. Example 1.1 Consider the statistic Tn = Tn(Xu . . . ,Xn\P) = n(Xn)2, where A Ί , . . . , Xn is an i.i.d. sample from a distribution P on the real line with zero mean and let Xn denote the sample mean. Let Ln(P;r) denote the d.f. of Tn. The corresponding m/n-bootstrap approximation is the d.f. Lm{Pn\ r) of the statistic T* = m(X%), where X*,..., X^ is a sample from the empirical distribution P n , and X™ denotes the corresponding sample mean. Assume that EXf < oo and that P satisfies Cramer's condition of smoothness. Consider the uniform errors introduced in (1.1), based on the Kolmogorov distance d. Let Y denote a standard normal random variable and assume that the sequence m = m(n) is chosen such that m —> oo and m/n -» 0. In Section 4 we prove the following. If m/nιl2 -» oo, then (1.3)

(n/m)(Δm,Δm)

A

(cλY2\cλY2!/2),

where c\ denotes an absolute constant. If m/n 1 / 2 -> 0, then (1.4)

mΔm

A

co(P), and m Δ m

A

where co(P) is a constant depending on P. If m/n1/2

n^2(Am,Άm)

(1.5)

A

71—>OO

c o (P), -> c = const, φ 0, then

(fo(Y)Ji(Y)),

where /o, /i are certain measurable functions of Y (see Section 4 for details). Thus, if limm/n 1 / 2 = 7 G [0,oo] exists, then (1.3)-(1.5) imply (1.2). Moreover, ξo = 1 and £oo = 1/2. Under the same conditions we obtain as well EpAm/EpAm — 71—>

Note that the value of m obtained in this way by minimizing Δ m strongly depends on the particular sample. For instance if by chance in this example ~Xn approximates the true value EX\ — 0 very accurately, that is nιl2Xn = o(l) (which happens rarely), the approximation of Ln(P) by the random bootstrap distribution Ln(Pn) might be accidentally precise as well (compare (3.1) and (3.2)). In this case the bias as well as the variance of the estimate Lm(Pn) will decrease with 7τι which leads to a choice of a large sample size

290

Friedrich Gόtze and Alfredas

Rackauskas

m ~ n. Such a case cannot be detected by an average criterion for the choice of m like say Ep d ίL m (P n ), Ln(P)J, which would lead to a much less adaptive and accurate choice of m for such an exceptional sample. Similar arguments apply to the moon bootstrap without replacements. We will show under certain conditions (see Theorems 2.1, 2.3 and Remark 2.1) that, for L2-distances of distribution functions say d, the random distance Δ£j = d(L^(P n ),L n (P)) is stochastically equivalent to the distance Δ ^ = d(L^(P n ), L^2(Pn)). More precisely, for some model dependent rate a 0 < a < 1 and sequences m(n) such that 7 = limn m(n)/n € [0,oo] exists we have Δ* 771

T) ί y

.

where 770 is a constant depending on P and, for 0 < 7 < 00, ηΊ is a random variable depending on P. Typically wejξet EpA^/EpA^ -» c 7 (see Remark j 21) Thi i * Δ 2.1). This motivates m* = argminΔ^ as a random choice of m for the (m) "bootstrap.

Figure 1. Smoothed graphs of the functions m -» Δ m (dot line) and m -¥ Δ m (solid line) from Example 1.1, where P is a centered χl distribution. Sample size n = 400.

In Figure 1 we consider Example 1.1. It shows (smoothed) graphs of the functions m -» Δ m (dot line) and m -> Δ m (solid line), where P is a centered χ1 distribution. The simulations were done for a sample size of n = 400. In Fig. 2 the true and estimated Kolmogorov distances m -> Δ ^ and m -> ΔJ^ are smoothed and plotted based on an individual sample of size n = 400 and m < n/2 when sampling without replacement. The first plot shows the behavior of sampling without replacement in the setup of Fig. 1 with P = χι. The second and third plot represents a parametric case: Student's t-test with P = 7V(0,1) and sampling with/without replacement. The third one represents a nonparametric case: distances for the normalized distribution of the largest order statistic for P = Uniform(0,1) and sampling

Bootstrap Sample Sizes

291

Sum Λ 2, without repl.

2

Student's T, with repl.

exact estimated

exact estimated

8

1 !

I I

60

80

Student's T, without repl.

100

200

40

60

400

Minimum uniform, without repl.

exact estimated

20

300

exact estimated

80

20

40

60

80

Figure 2. True and estimated Kolmogorov distances m —> Δ ^ and m -> Δ™ smoothed and plotted based on an individual sample of size n = 400 and m < n/2 when sampling without replacement.

without replacement, see also Example 2.2 below. The paper is organized as follows. In Section 2 we investigate the moon bootstrap without replacements. To this aim Hoeffding expansions for Ustatistics are used for m/n = o(l) in order to evaluate the error of the random approximations. In Section 3 we investigate as well the moon bootstrap with replacements representing our statistics in terms of empirical processes. Here, following Beran (1997), we require that the sequence of statistics should be locally asymptotically weakly convergent. Furthermore, we shall use Edgeworth expansions to prove the stochastic equivalence of the random distances in the examples studied in this paper. Finally, Section 3 contains the proofs of our results. Throughout the paper we write m G n(α^) to indicate that m = m(n) α is a sequence such that m -> oo, m/n ->• 0 and limn m/n = 7 exists allowing 7 £ [0,00]. 2

Moon bootstrap without replacements

In this section we let Xi,

, Xn denote a sample of i.i.d. random elements

292

Friedήch Gδtze and Alfredas Rackauskas

from an unknown distribution P on a measurable space (S,A), and let Tn = Γ n (-XΊ,... ,X n ; P ) denote a sequence of statistics with distribution Ln{P). We assume that Tn converges in distribution to a random variable T ^ . Let θn(P;α) = Eh(Tn;α), where h : R x R -> R is a real measurable bounded function, denote a family of parameters indexed by α G JR. The moon bootstrap without replacements estimates θn(P\α) by

θmn(Pn;α)

= L*m(Pn)h(;α) = J L

]Γ

h{Tm(Xil,...,Xin;Pn);α).

As distance d between Ln(P) and L^(Pn) we choose the Z^-distance between θn{P;α) and β m n ( P n ; α ) writing

Δ^ = (jR{θmn{Pn α)-θn{P ,

r

A

*rn=(JR

,α))\{dα))l'\

^

9

{Omn(Pn\α) - θMn(Pn]

vi/2

α)) μ(rfα))

, M = m/2,

where μ is a probability measure. For indicator functions h(x; ά) = l{x < α}, ΔJ^ reduces to the integrated square error between the distribution function L n ( P ; α) and its Q ) -bootstrap estimator L ^ ( P n ; α). For special discrete measures μ, ΔJ^ may then be written as

^ shall give conditions which ensure the stochastic equivalence of ΔJ^ and A^. First, we impose some restrictions on the sequence of statistics Tn. Assumption (I). There exist measurable functions «, ξ m : S x R —> R such that E (Λ(Tm; α)|Xχ) = ^ ( Γ ^ ; α) + πΓλl2κ{X^

α) + ξ m ( X i ; α),

where / β E κ 2 (Xi; α)μ(dα) < oc and / β E ^(-XΊ; α)μ(dα) = Assumption where

(J).

$R{θmn{Pn\ά)

- θmn{P;α))2μ{dα)

(Λ

Σ

KmJ

l<*i<...
o(m~ι).

= oP(m/n

+ 1/m),

h(Tm(Xh,...,Xim;P);α).

To establish stochastic equivalence of ΔJ^ and ΔJ^ we shall jxmsider stochastic processes^ζ^n{α) = θmn{Pn; α) - θn(P; α), α e R, and Cmn(α) = θmn{Pn\o) — θMn{Pn\o), a E R, M — m/2, as random elements in the space 1/2 (i?, μ). Let ζp denote a mean zero Gaussian process with covariance given by EζP(α)ζp(b) = cov(κ(Xi;α),/ς(Xi;6)), α,6 G Λ

Bootstrap Sample Sizes

293

Theorem 2.1 Suppose that assumptions (I) and (J) are satisfied and αp( ) = E κ(-XΊ ) / 0. Choose a sequence m G n(l/2,7). Depending on 7 = 00 or 0 < 7 < oo choose as norming sequence τnτn = (n/ra) 1 / 2 or ra1/2 respectively. Then (2 i)

n—ϊoc

in the space L 2 (i2, μ) x £ 2 (fl, μ), where ("writing c := 1-2" 1 / 2 , d := 1 —21/2>) (ξoo,tyx>)= (1, c)ζp and (ξ 7 , r/7) = (7<> + α P , C7<> + dα P ) for 0 < 7 < 00. Noting that Δ ^ = ||C£ n || 2 and A^ = HCUh, where || - || 2 denotes the norm in L 2 (μ) we have by Theorem 2.1 the following corollary. Corollary 2.2 Suppose that assumptions (I) and (J) are satisfied and the sequences m = m(n) and τnπι axe chosen as in Theorem 2.1. Then

(2.2)

S

A

where r ^ = 1 — 2" 1 / 2 , TO = 2 1 / 2 — 1 and τ 7 is a random variable for 0 < 7 < 00.

Remark 2.1 Prom the proof of Theorem 2.1 it is clear, that if assumption (J) is substituted by

L

R

E(θmn{Pn', α) - θmn{P; α))2μ(dα) = o{m/n + 1/m),

then, in addition to (2.2), we also have EpA^/EpA^ —> c 7 (P). A similar remark applies to other results in this section. Furthermore, similar results hold with different c = c(λ) when comparing sample sizes m and λm, 0 < λ < 1 in Δ ^ . In the following we shall discuss the nature of the assumptions of Theorem 2.1. Assumption (J) allows to reduce the analysis of the (^)-bootstrap approximation 0 m 7 l (^n) to that of [/-statistics emn(P) with increasing degree m = m(n) and values in the space L2(/x). This assumption is satisfied for a large class of statistics Tn. For example, we consider the estimation of a parameter Θ(P) of an unknown distribution P by means of a plug-in estimator θ(Pn). Introduce the statistic Tn = rn(θ(Pn) — 0(P)), where the normalization τn is chosen such that Tn converges in distribution. If h is a 2 2 Lipschitz function, e.g., /(Λ(t; α) - Λ(s; α)) dμ <\t- s| , we obtain R

{n{Pn',α)

- θmn(P;α))2μ(dα)

< τm\θ(Pn) - Θ(P)\

294

Friedrich Gδtze and Alfredas Rackauskas

and assumption (J) is satisfied provided that τm/τn = o^m/ In case h(x; ά) = I{x < α} is an indicator function, we obtain on the set τm\θ{Pn) - Θ{P)\ < ε, that \θmn{Pn) - θmn(P)\ < Um,nhε, where Um,nhε is a [/-statistic with kernel /i ε (Xi,..., X m ) = I { α ~ ε < Tm(X\,..., X m ; P) < α + ε}. Hence, E\θmn{Pn) — θmn(P)\ does not exceed the quantity

inf {P{α -ε
+ ε) + P{τm\θ(Pn) - Θ{P)\ > ε))

which, under the convergence assumption, is typically, of the required order. Edgeworth expansions are well suited to estimating the accuracy of a bootstrap approximation (see, e.g., the monograph by Hall (1992)). Assumption (ί) requires certain expansions for the distribution Lm(P) as well as the conditional distribution Lm{P)\X\. Moreover, by conditioning the statistic T m on a random sample, we implicitly assume some kind of stochastic expansion for the function Tm in the sense that assumption (I) may be checked by studying the function y -> g(y) = Eh(Tm(y,X2,... , X m ; P ) ; α ) . Consider, for example, an i.i.d. sample XL, . . . ,Xn from a distribution P on Rd. Let EX\ = 0. For a given symmetric d x d matrix Q, consider the statistic Tn = nQ(Xn), where Q denotes a quadratic form with Q(x) = (x, Qx) for x G Rd, where ( , ) denotes the Euclidean scalar product. Set Xn\\ = n~lY%=2Xk and T n |i = nQ(Xn\ι). If the function h has two bounded derivatives with respect to the first argument, we have E (Λ(Γro; α)\Xλ) = Eh{Tm^

α) + 2Eh\Tm^

α)(X

where ER^Xi α) < cm~2 provided £7|Xi| 4 < oo. Since m 1 / 2 X m | 1 converges in distribution to a mean zero Gaussian random vector Yp with cov Yp = cov XL, one needs expansions for the quantities £ ? ^ ( m 1 / 2 X m μ ) and

Eφ{mιt2Xm\i),

where φ(x) = h({x,Qx);α), φ(x) =

2h'((x,Qx);α){XuQx).

If the function h is sufficiently smooth, the required expansions axe well known (see, e.g., Gδtze (1985)). If h(x] ά) = l{x < α}, then, evidently, P(Tm < α|Xi) = P(Tm{1

+ (XU Q X m | 1 ) < α - m ^ Q ^ ) ) ,

and the problem reduces to proving expansions for the distribution of the quadratic polynomial Γ m |i + (x, Q X m | i ) Such expansions have been proved up to order O(m~ι) in Bentkus and Gδtze (1999). Prom the proof of Theorem 2.1 it is clear that one needs to control the variance of E(h(Tm\α)\X\) and the correlation between Eh(Tm',α) and Eh(Tm/2;b) for α,6 G R. It is also clear that if αp — 0, one needs to assume that a second order approximation for Eh(Tm\α) holds. Consider, for example,

295

Bootstrap Sample Sizes Assumption (I\). There exist two functions φuφ2 dependent on P, such that for all α,b E R

:

x

R

R ->• -R? possibly

(^(6)

Assumption (I2).

EPh(Tm;α)

= Eh^T^α)

(l))

+ m~ιβp{α)

and

+ ίp, m (α), where

1

/?P G 1^2 (A*) and 5p > m is ^ ( m " ) in L 2 ( μ ) .

fR{θmn{Pn; α) - θmn{P; α))2μ(dα) = oP(m/n + m~ 2 ).

Assumption (J'J.

Let Ci?C2 denote mean zero Gaussian processes with covariances equal to φι and correlation Eζι(α)ζ2(b) = 2~l/2ip2(a,b), a,b £ R. Theorem 2.3 Suppose that the assumptions (/i),(/2), and {J1) axe satisfied and βp Φ 0. Choose sequence m G n(l/3,7). Depending on 7 = 00 and 0 < 7 < 00 we may choose norming sequences τmn = (n/m)1/2 and m respectively such that (2.1) holds where (writing CQ = 2~ιί2) (foojffco) = (Ci> Ci -C0C2), (ί 7 » ^7) = (7Ci + /?P? 7(Ci - co C2) -

/?P)

for

0 < 7 < 00.

Moreover, (2.2) holds where TQ = 1 aiid r 7 is a random variable when 0 < 7<1 Example 2.2 In this example, we assume that Xι,...,Xn is a random sample from the uniform distribution P on the interval (0, θ). The maximum likelihood estimator of θ is the extreme order statistic θ = maxi
τ

(x

x

Pλ

Let Ln(P\α) denote the distribution function of this statistic. bootstrap approximation L^(Pn;α) is defined by

Lm(Pn]α)=

(n) \m/

where

Set

/•

El = JR (L*m(Pn; α) - L*m/2(Pn; E2m=

ί JR

α))2μ(dα),

(L*m(Pn;α)-Ln(P;α))2μ(dα).

As a consequence of Theorem 2.3 we get in this example:

Its (^)-

296

Friedrich Gόtze and Alfredas Rackauskas

Corollary 2.4 Assume that m E n(l/3,7). We have with the notations and choices of Theorem 2.3 Em En

v

Proof We shall verify the assumptions of Theorem 2.3, for h(x; ά) = ϊ{x < α}. Obviously we have, for 0 < α < n,

(2.3)

2n

n 2 ( l — α/n)2'

where \θ\ < 1. Since Tn converges in distribution to an exponential random variable T ^ , the assumption (I'2) with βp{α) = —e~~αα2/2 follows by (2.3). By straightforward calculations we see that E\L*m(Pn,α)-L*m(P,α)\<-.

Tί

;

This, clearly, yields assumption (J ). To check assumption (I\) note that for Zm(α) := P(Tm < α\Xι) - P(Tm < o) = = (1 - α/m)m-1

[(1 - α/m) - 1{XX < 0(1 - α/m)}],

using (2.3) we have EZm(α)Zm(b) = exp{—(α + 6)}min{α; b}m~ι + o{m~ι) and EZm(α)Zm/2(b) = exp{—(α + 6)}min{α;2b}m~ 1 + o(m~ι). Hence assumption (Ji) follows, which completes the proof of the corollary. •

3

Moon bootstrap with replacements

In the cases, where the classical nonparametric bootstrap fails for the distribution Ln(P), it often turns out that the limit distribution of Ln(Pn) is random. In order to describe it, we introduce, following Beran (1997), a local weak convergence property of the statistic which we want to approximate in distribution. In order to introduce this notion, we need some preparation. Fix a set T C L<ι(β,P). Let ί^F) denote the Banach space of realvalued bounded functions on T equipped with the supremum norm | | z | | ^ = |^(/)|. Assume that the envelope function F(s) = sup{|/(β)| : / ε T} < oo

for all a E 5,

and the quantities sup{|Q/| : / G T] < oo for all Q E Vo, where Qf = / fdQ are bounded. Then the mappings δs : T —> R given by δsf = f(s) and Q : T -> R given by Qf = J fdQ, where Q E Vo, are in

297

Bootstrap Sample Sizes

) In order to avoid measurability problems connected with the nonseparability of too^T), we assume throughout, that T is either a countable or an image admissible Suslin set. For the relevant definitions concerning these matters, we refer to Dudley (1984). Let the empirical process up)Tl = (vp,n(f),f G T) be defined by up)Tl = ι 2 n l (Pn — P ) , n G N. Note that, for each Q G Vo, Z/Q,Π is a random element with values in ί Definition 3.1 The statistic Tn = Tn(Xι,...,Xn;P) G R is called locally T-weakly convergent at P eVo, if there exists a family of probability measures {L(P, z), z G ^ooC?7)} on R such that Ln{Qn)

—> L(P,z)

weakly

n—>oo

for every z G iooi.?) &nd every sequence {Qn} C VQ such that \\nl'\Qn-P)-z\W

^

0.

Remark 3.1 If the model is parametric, e.g., P = PQ, where θ = ( . . . , θd(P)) then the notion of locally .F-weakly convergent statistics reduces to the notion of a 'LAWC statistic introduced by Beran (1997) when taking

Remark 3.2 If T is P-Donsker and if the statistic Tn is locally .T7-weakly convergent at P, then there exists a random element Z with values in too{F) such that the random probability measure Ln(Pn), converges in distribution to a random probability measure L(P,Z). Indeed, since T is P-Donsker, the empirical processes vp^n converge in law in ^ooC?7) to a Gaussian process Gp — { G p ( / ) , / G F] with zero mean and covariance EGp(f)Gp(g) = Pfg-PfPg, /, g G T. By a result of Dudley (1985) there exist a probability space (Ω,Σ,P) and perfect functions gn : (Ω, Σ , P ) -> (Ω,Σ,P) such that p o gn = P and Hn^ίPnO^-PJ-Gpoflϋl^

^A

0.

Hence, there is a set Ωo C Ω such that P(Ωo) = 1 and \\n1'2(Pnogn{ω)-P)-Gpog0(ω)\\r

^

7

0

for each ω G Ωo By the definition of .T -weakly convergent statistics we have LniPn ° 9n(v))

L

—> (P, GP o gQ) weakly.

298

Friedrich Gδtze and Alfredas

Rackauskas

Example 3.3 The fact that a characteristic function is real if and only if the corresponding distribution function is symmetric at 0 suggests the statistic +oo / / +oo

/

for testing symmetry. Here

^

\ 22 \

2

^2Im(cQ^(t))

+ α{t)J g(t)dt,

/•-f /•-f-oo

,n(t) = / J —O

denotes the empirical characteristic function corresponding to the distribution Q, g is an integrable weight function, and α(t) satisfies / ^ α2(t)g(t)dt < oo. This statistic Tn is locally .T7-weakly convergent at any symmetric distribution P, when the class T is chosen as T = {x —> cosfcr, t G R}. A parametric version of the following proposition is given in Beran (1997). Proposition 2.2 Assume that T is P-Donsker and that the statistic Tn is locally f-weakly convergent at P. Let {m(n), n > 1} denote any sequence of positive integers such that m(n) —> oo and m{n)/n —» 0 as n —> oo. Then L m ( n )(P n ) is d-consistent in probability, where d is any metric metrizing the weak convergence. Proof We have \\mι/2(Pn°gn-P)-Q\\r

—t 0 a.s.

n—> o o

By Definition 3.1, Lm(Pn ogn(ω)) converges weakly to ί/(P,0) for almost all ω eΩ. This completes the proof. • Next we investigate the stochastic equivalence of the random distances d(Lm(Pn), Ln(P)) and rf(Lm(Pn), Lm/2(Pn))> where d is either the Kolmogorov or the bounded Lipschitz distance. A unified way to consider both distances is to consider the more general class of uniform distances over a class of measurable bounded functions Ή, say. Define for distributions F, Q, the uniform distance dn(F,Q) = sup \Fh-Qh\. hen Indeed, if Ή. is chosen as the class of indicator functions I{(—oo, α]}, α E R, dκ(F,Q) will coincide with the Kolmogorov distance. If % is the class of measurable functions h : R —• R such that supα|Λ(α)| + supα_^6 \h(α) —

299

Bootstrap Sample Sizes

h(b)\/\α—b\ < 1, then du{F, Q) corresponds to the bounded Lipschitz metric. Note that distances du , where % consists of higher order smooth functions, have been used as well for investigating the accuracy of the bootstrap. Write Anm

= dn(Lm(Pn),Ln(P)),

Anm

=

dn(Lm(Pn),Lm/2(Pn)).

Theorem 2.3 Suppose that the sequence of statistics satisfies assumptions (A), (£?), (C), and (£>), which are stated and discussed below. Assume furthermore that T is a P-Donsker class, and that \\upin\\jr is uniformly integrable. For sequences m G n(l/2,7) we may choose norming sequences τrnn = (n/m)1/2 and ra1/2 corresponding to 7 = 00 and 0 < 7 < 00 such that for random variables £,£1,^2 and a constant c\ > 0 which axe defined in the proof in Section 4

^

(£^)

where (writing c := 1-2" 1 / 2 , d = 2ι'2-l, ) (ξoc,r?oo) = (l,c)ξ, (&,%) = (1, d) cι and (f7, ηΊ) = (ξu ξ2) for 0 < 7 < 00. Thus Theorem 3.3 yields the stochastic equivalence of the random distances A-um and Δ ^ m , as n -» 00. In order to formulate the assumptions (A) - (D), fix a distance d on the set Vo and, for given constants Co > 0 and c\ > 0, consider the neighborhood V(P) C Vo of P defined by

V(P) = {Q e Vo : d(Q,P) < cu

\\nι'2{Q - P)\\ < CQ}.

The first assumption concerns the local ^*-weak convergence property of the sequence of statistics. Roughly speaking, we assume that parameterized expansions for Ln(Q)h hold uniformly in the neighborhood V(P). A parameι 2 terization will be given by the quantity n l (Q — P) considered as an element in ίooiJ7). In many cases, T will consist of a finite number of functions only. Assumption (A). For each Q G V(P), there exist a set {L(Q, z), z G ί of probability distributions on R and a set {ί(Q, z),z G 4o (F)} of real valued functions on Ή such that for every h G %

Ln{Q)h = L{Q,nιl2(Q - P))h + n-l'2i{Q,nll2{Q +

- P))h

ι 2

Rn{Q,n l {Q-P),h),

where

sup sup RniQ,^2^ Qev(P) hen

- P),h) = oin-1/2).

Furthermore, we assume first order smoothness for L(Q, z) and a Lipschitz condition for ^(Q, z) as a function of z G ^0

300

Friedrich Gδtze and Alfredas Rackauskas

Assumption (B). For each h E Ή and Q E V(P) we have L(Q, z)h = L(Q, 0)Λ + Li(Q, Λ)2r + Λ(Q, h, z\ where L\(Q,h) is a bounded linear functional on ί^F) and 2

sup sup\R(Q,h,z)\
and s u p Λ € W |ί(P,0)/ι| < oo.

Finally, we need the continuity of the limiting distribution L(P) = L(P, 0) as well as the continuity of the function £(P) = t(P, 0) at P. Assumption (D). d w (L(P n ,0),L(P)) = O p ^ " 1 / 2 ) and sup \l(Pn,Q)h - £{P)h\ = Op{n-1'2). hen Remark 2.4 If in assumption (B) the first order approximation vanishes, i.e. if Li(Q, h) = 0, we need again a second order expansion term for L(Q, z) defined on z E ^oo(^) to discriminate between the choice of m verses ra/2 although now at a lower level. Hence, assumption (B) should be replaced by the following: f

Assumption (B ). For each h E Ή. and Q E V(P) we have L(Q, z)h = L(Q, 0)h + L 2 (Q, h)z2 + R{Q, Λ, z), 7

where L2(Q,Λ) is a bounded bilinear functional on ^(J ) and sup QeV(P

Moreover, s u p ^ ^ |^2(P, Λ)| < oo. Example 2.5 Here we continue example 3.3. Consider the uniform distances (2.4)

Δ m = sup|L m (P n ;r) - L n (P;r)|, r . ^° . Δ m = sup|L m (P n ;r) - Lm/2(Pn;r)\ r>0

Bootstrap Sample Sizes

301

corresponding to the distribution function of the statistic Tn defined in Example 3.3. There we assume α φ 0. Write Tn(Q) = φα(Sn + zn), where 1 2

Sn(t)

=n- / Σk=ir)k(t),

ηk(t) = [cos Zkt - E cos Zkt], φα(x) = Πo(a(t) +

2

α(t)) g(t)dt and, finally, zn(t) = nιl2Ecos Z\t. We consider Sn as a sum of i.i.d. random elements in the Hubert space L2(R,g{t)dt). We obtain, by the general result given in Theorem 2.4 below, that the random distances Δ m and Δ m are stochastically equivalent as n —> oo. Namely, if m G n(l/2,7), then Δ m / Δ m -^» £ 7 , where £<» = 2 1 / 2 - 1 , £ 0 = 1-2" 1 / 2 , and £ 7 is a random variable for 0 < 7 < 00. Moreover, we get EpAm/EpAm —> c τ (P), where cΊ (P) is a constant depending on 7 and P. Example 2.6 Let H be a separable Hubert space with the norm || || and inner product ( , •). Consider random elements Λ Ί , . . . , X n ,.. in H that are independent and identically distributed with distribution P. Assume that EX\ = 0 and P is taken from a class Po, with the following properties: i) Q is non symmetric around zero, ii) JH \\x\\4:Q(dx) < 00, and iii) the covariance operator VQ of Q has at least 13 (counted with multiplicities) eigenvalues exceeding a given β > 0. The eigenvalues of a positive operator V : H —» H will be denoted by λi(V) > λ2(Vr) > . It is well known (see, e.g., Gohberg and Krein (1969)) that |λj(Vί) — λj(T^)| < ||Vi — V^|| for linear completely continuous positive operators Vi, V2 on H. Let || H2 denote the Hilbert-Schmidt operator norm. One easily checks that 1

en

f \\x\\4P(dx).

It follows that, with probability tending to one, at least d > 13 eigenvalues of the covariance operator Vp will exceed a number βo > 0. Hence, without loss of generality we can assume that VQ contains the empirical distribution

PnofPeVo

For Q € VQ and αE H, α φ 0, define n

Tn,α{Zu •• ,Zn;Q)

= n\\n-1 Σ,Zk + α|| 2 . fc=l

Let L n j O (Q, r) denote the distribution function of this statistic. Furthermore, let T denote the class of functions on H given by x\(z) = (x, z) together with 2^2(2) = (s, z)2 indexed by x £ H with ||x|| = 1. Note that the evaluation at 7 a point y E H defines an embedding H C ^ocC? ), via y{x\) := x\{y) — (z,y) and foryeH and 2/(2:2) := 2:2(1/) = (x,y)2 It is easy to verify that the statistic Tn,α is locally ^-weakly convergent at any P E VQ which has zero mean. We aim to prove the stochastic equivalence as n -¥ 00 of the uniform errors Δ m and Δ m defined by (3.1).

302

Friedrich Gδtze and Alfredas Rackauskas

Denote by μQ the mean zero Gaussian measure on H with covariance operator VQ, and let D^(x)μQ be the jth directional derivative of μ in the 2 direction of x. Set Vr(z) = {x E H : \\x + z\\ < r}. For P,QG VQ, consider the distance d(Q, P) = \JH \\x\HQ(x) - P(x)]\ + \\VQ - VP\\. ι 2

Define V(P) = {Q E V : d(P,Q) < cu n l \\Q - P ^ < c 2 }. An inspection of the proof of Theorem 2.1 in Bentkus (1984) gives the following uniform expansions (compare with assumption (A)). For any Q G V(P) we have 1 2

3

Ln(Q,r) = μQ(VΓ(α + zn))+ln- / ED (Z1)μQ(Vr(α 0

+ zn))

(2.5)

where zn = nll2EZ\ and, for any ε > 0 and Co > 0, there exists a constant c > 0 such that SUp SUp SUp Rn(Q, 2, 0 = C7l~ 1+ε . <3eV(P)||α||0

Furthermore, note that, for z E if, we have (2.6)

μg(Vr(α + z)) = μQ(VΓ(α)) + £>(2ί)μg(Vr(α)) + Λ(Q, z; r),

where sup

sup sup|,R(Q,z;r)| < c 2 (P)|k||3τ.

QeV(P)\\α\\0

Indeed, by standard arguments (inversion formula, Lebesgue lemma) it easily follows that (2.7)

Dt{z)μQ{Vτ{α)) = (2π)"1 £ j

~

[(it)~ιe~itrqt(α + τz)\ \τ=Qdt,

where qt(α) is the characteristic function corresponding to μQ(Vr(α)). By elementary calculations one proves that (2.8)

\D2(z)μQ(Vr(α))\ < C^Q)

• X,(Q))-1/2 max{l, ||α|| 2 }||z|| 2 .

Hence, (2.6) follows by Taylor's formula and (2.8). Note, that (2.6) implies assumption (B). By (2.7) (which is assumption (C)) we get (

' '

\ED\Z{)μQ{Vr{α + z)) - ED\Zx)μQ{α)\ < C(λ x (Q) • • . λ 9 ( Q ) ) - 1 / 2 m a x { i ) | | α | | 4 } | N | £ J | | Z | | 3 .

Finally, collecting the bounds (3.2), (3.3), and (3.6) we have proved the following result.

303

Bootstrap Sample Sizes

Theorem 2.4 Suppose that Q E V(P) and J xQ(dx) φ 0. Then, for any constants Co > 0 and ε > 0, there exists a constant c > 0 such that L n (Q,r) = μQ(Vr(α)) + 1

D(zn)μQ(Vr(α)) 3

where sup sup\Rm,n(r)\ < c ( n ~

sup

1+e

' " |H|O

3

1

2

+ | | 2 n | | + n- / ||z n ||).

Arguing as in Gδtze and Zitikis (1995) one can prove that sup sup I/is (VΓ(α)) - μp{Vτ{α))\ =

||α||0

n

^

and sup s u p K 1 Y^D\Xk)μp IMI<<* Γ>o fc=i

(Vr(α)) - EPD3(X^P(Vr(a))\

=

Combining this with Theorem 2.4, we get Corollary 2.5

If the sequence m € n(l/2,j),

then

where ξo = 1? Coo = 1 — 2" 1 / 2 , and ξ 7 is a random variable for 0 < 7 < 00. Remark 2.7 We excluded the case α = 0 since here Z ) ( ^ ) ^ Q ( K ( 0 ) ) = 0 and D3(z)μQ(Vr(0)) = 0. Therefore, here one needs to use asymptotic expansions for Ln(Q\r) up to order o(n~1.) This would require some additional use of Cramer's condition of smoothness for the measures Q E V(P). 3

Proofs

Proof of Example 1.1 Set ΐ_ = (ί - ~Xnmι/2)/σ, t+ = (ί + Xnmι/2)/σ, where, as usual, σ 2 denotes the sample variance. We also denote, for s > 0, κ s = EX*. By Edgeworth expansions we have P(m(X*m)2 < t2\Xu...,Xn)

= Φ(ί_) + Φ(ί+) - 1

304

Friedrich Gδtze and Alfredas Rackauskas

where

and Qi and Q2 denote the plug-in estimates of Q\ and Q2 respectively. Here Hs denote the standard Hermite polynomials. Straightforward calculations show that P(m(X*J2
(3>1)

- - ^

m where Yn = (nι/2Xn)/σ. (3.2)

Furthermore,

P (n(Xn)2

< t2) = 2Φ(ί/σ) - 1 + -

Set

and ζ m (ί) = ζm(t) - Cm/2(*) If n/m2 -> 0, then (4.1) and (4.2) together yield -sup|Cm(ί)| = yn2sup|Φ"(t)|+oF(l) TΠ t>0

ί>0

and - sup \ζm(t)\ = 2-^ ^

t>0

sup |Φ"(ί)| + op(l). t>0

Combining this with the central limit theorem we get (1.2). The proof of (1.3) immediately follows by (4.1), (4.2), and the law of large numbers. For the proof of (1.4), we have, by (4.1), (4.2) combined with the law of large numbers,

ί>0

= sup |(m/n 1 / 2 )Φ"(ί)y n 2 - 2Q[(t)Yn + 2{nιl2/m)Q2(t)\ t>0

and t>o

|

|

= sup |(m/2n 1 / 2 )Φ / / (t)y n 2 - 2(n 1 / 2 /m)Q 2 (t)| t>o

+ oP(l)

305

Bootstrap Sample Sizes It is not difficult to verify that (1.5) is valid with fι(Y) ι

2Q[(t)Y + 2Ί- Q2(t)\

and f2(Y) = sup t > 0

= sup t > 0

\ηΦ"(t)Y2-

\2'^

The proof of Theorem 2.1 is using two results about [/-statistics due to Vitale (1992) which we state below. For a sequence of i.i.d. random elements Xι,...,Xn taking values in a measurable space (S,A) and a sequence of functions (/ι m ), where hm : Sm -» R is a real-valued kernel of degree m < n, define the [/-statistic Un^mhm by /

Unrnhm=

I V'V

\-i

I

2, l < 2 < <

whereas the conditional kernel hm^ : Sk —> R is defined by = /

Js fs

Js

hm(xι,...,xm)P{dxk+ι)

'P(dxm)

Then the Hoeίfding decomposition of Un,mhm is given by

where

The degree of degeneracy of the [/-statistics Un,mhm is the largest integer r such that /ιm(fc) = 0 for k = 1,..., r. Lemma 3.1 Suppose that Uniπιhm is the U-statistic based on the symmetric kernel hm satisfying Eh^ < oo. If the degree of degeneracy of Un is equal to r — 1, then /m\2

/ m \

var(ϋn) < ^ - var(/> m | r ) + ^ ( V a r ( / ι

m

) - (7) v

\r+V

\r)

Lemma 3.2 if [/n?m/ιm is as in Lemma 3.1, the sequence where k = r , . . . , m is nondecreasing.

Proof of Theorem 2.1

Write

Cn,m(<0 = ^ m W + Km(α) + Λπm(α), where Unm(α) is the [/-statistic with kernel Λ m ( x i , . . . , x m ; α) = h{Tm(xu

. . . , xm; P ) ; α) - 0 m ( P ; α);

306

Friedrich Gόtze and Alfredas Rackauskas

Vnm{α) = θm(P] α) - θn{P; α), Rnm{α) = θmn{Pn; α) - 0 m n ( P ; α), α e R. By assumption (J), ||i2nm||2 = oP({m/n)1/2 + πΓιl2) whereas assumption (I) yields Km(α) = -7=-Bκ(-XΊ; α) + vnm(α), /m where ||v n m||2 = o(m~ιl2).

α e R,

Hoeffding's decomposition yields

where = £7(Λ(T m ;α)|Xi = x) - 0 m (

gm{x\α)

and, in view of Lemma 4.1 and Lemma 4.2, we obtain Er2nm(α)

< ^EihiTm

α) -

Tli

This gives | | r n m | | 2 = op^m/n)1/2

+m~1/2).

θm(P;α))2.

Finally, assumption (J) leads to

(3.3) for every α £ R, where ||κ m , n ||2 = op^m/n)1/2 + 1/ra 1 / 2 ). Now the result easily follows from the representation (3.3) and the central limit theorem in Hubert spaces. • Proof of Theorem 2.3 Under the assumption (J1) the L2-weak limiting behavior of the process r m n ζ ^ coincides with that of the process τ m n £ 7 n m , where Umn denotes the L2-valued [/-statistic, Umn(α) = θmn(P\ α) — θn(P\ α) (where α E R). Hoeffding's decomposition, Lemma 4.1, Lemma 4.2 and assumption (J2) reduce the proof to the weak convergence of the L2-valued random elements 1

Smn{α) = —^gm(Xk;α) + —βp{α), α e R. m n m ^ This can be easily shown using the results of Cremers and Kadelka (1986) on weak convergence in Lp spaces. • 1 2

ι 2

Proof of Theorem 2.3 Introduce τ m > n = (m/n) / + m~ l and fixing constants CQ > 0,c\ > 0, define the set Ωo = {ll^nllj 7 < coίn/m) 1 / 2 } Π {d(Pn,P) < ci}. On this set Ωo, we have by assumption (A) H{h) := Lm(Pn)h

- Ln(P)h = L(Pn,m^2(Pn n

- P))h -

- P))h ^

307

Bootstrap Sample Sizes where s u p Λ e 7 ί \Rm,n(h)\ = Write αn(h)

o

( m ^

P

= Lι{Pn,h)(vp,n),

bn{h)

Using assumptions

= £(Pn,0)h.

(5) and (C) we get H(h) = {m/n)ιl2αn{h) where s u p ^ e ^ |Γ m n (Λ)| = op(τmn). (3.4)

+πC1l*bn{h)

+Tmn(h),

In view of assumption (2?), we conclude

H(h) = (m/n)ι/2αn(h)

+ m~ιl2b{h)

+

Tmn(h),

where αn{h) = Xi(P,Λ)(i/p>n), b(h) = £(P,0)h. Now (3.4) yields (3.5) H*(h) := Lm(Pn)h-

Lm/2(Pn)h _

-Φb{h)

1)m

where sup / ι G ^ |T^ n (/ι)| = o p ( r m n ) . Since J 7 is P-Donsker and ||^p,n||.F is uniformly integrable, we obtain (3.6)

A ^oc

sup\αn(h)\ hen

n

sup|Li(P,Λ)G P |. hen

Using (3.4), (3.5), and (3.6) it is easy to complete the proof of the theorem. Moreover, we observe that ξ = sup / ι 6 ? ΐ \Lι(P, h)Gp\, c\ — sup / ι G ^ | whereas ξι = sup I7L1 (P, Λ)Gp + b(h) \ and 1

2

ξ 2 = sup | 7 ( 1 - 2" / )L 1 (P,/ i )Gp + (1 - y/2)b(h)\. hen This concludes the proof • Acknowledgements The first author would like to thank Peter Bickel and Willem van Zwet for a number of stimulating and very helpful discussions on the construction of subsampling procedures.

308

Friedrich Gόtze and Alfredas Rackauskas

REFERENCES Arcones, M.A. and Gine, E., (1989). The bootstrap of the mean with arbitrary bootstrap sample size. Ann. Inst. Henri Poincare 25, 457-481. Athreya, A.C., (1987). Bootstrap of the mean in the infinite variance case. Annals of Statistics 15, 724-731. Bentkus, V., (1984). Asymptotic expansions for distributions of sums of independent random elements of a Hubert space. Lithuan. Math. J. 24, 305-319. Bentkus, V. and Gotze, F., (1999). Optimal bounds in non-Gaussian limit theorems for [/-statistics. Ann. Probab. 27, 454-521. Beran, R., (1982). Estimated sampling distributions: the bootstrap and competitors. Ann. Statist. 10, 212-225. Beran, R., (1997). Diagnosing bootstrap success. Ann. Inst. Statist. Math. 49, 1-24. Bickel, P.J., Gotze, F. and Zwet, W.R., (1997). Resampling fewer than n observations: gains, losses, and remedies for losses. Statistica Sinica 7, 1-32. Bickel, P. J. and Freedman D. A., (1981). Some asymptotic theory for the bootstrap. Ann. Statist. 9, 1196-1217. Bretagnolle, J., (1983). Lois limites du bootstrap de certaines fonctionelles. Annales Inst. Henri Poincare 19, 281-296. Cremers, H. and Kadelka, D. (1986). On weak convergence of integral functionals of stochastic processes with applications to processes taking paths in Lξ. Stoch. Proc. Appl. 21, 305-317. Davison, A.C. and Hinkley, D.V., (1998). Bootstrap Methods and Their Applications. Cambridge University Press, United Kingdom, New York, Melbourne. Datta, S., (1996). On asymptotic properties of bootstrap for AR(1) processes. J. Stat. Plan. Infer. 53, 361-374. Datta, S. and McCormik, W.P., (1995). Bootstrap inference for a first order autoregression with positive innovations. JASA 90, 1289-1300. Deheuvels, P., Mason, D. and Shorack, G., (1993). Some results on the influence of extremes on the bootstrap. Ann. Inst. H. Poincare 29, 83-103. Dudley, R.M., (1984). A course on empirical processes. Lect. Notes in Math. Springer, Berlin 1097, 1-142. Dudley, R.M., (1985). An extended Wichura theorem, definition of Donsker class and weighted empirical distributions. Lect. Notes in Math. Springer, Berlin 1153, 141-178.

309

Bootstrap Sample Sizes

Gohberg, I. C. and Krein, M. G., (1969). Introduction to the Theory of Linear Nonselfadjoint Operators. American Mathematical Society, Providence. Gόtze, F., (1985). Asymptotic expansions in functional limit theorems. J. Multivariate Anal. 16, 1-20. Gotze, F., (1993). IMS Bulletin. Gotze, F. and Zitikis, R. (1995). Edgeworth expansions and bootstrap for degenerate von Mises statistics. Prob. Math. Stat. 15, 327-351. Hall, P., (1992). The Bootstrap and Edgeworth Expansion. Springer, New York. Heimann, G. and Kreiss, J.P., (1996). Bootstrapping general first order autoregression. Stat. Probab. Letters 30, 87-98. Politis, D. N. and Romano, J. P., (1994). A general theory for large sample confidence regions based on subsamples under minimal assumptions. Ann. Statist. 22, 2031-2050. Shao, J., (1994). Bootstrap sample size in nonregular cases. Proc. Amer. Math. Soc. 122, 1251-1262. Shao, J., (1996). Bootstrap model selection. JASA 91, 655-665. Swanepoel, J. W. H., (1986). A note on proving that the (modified) bootstrap works. Comm. Statist. Theory Methods 15, 3193-3203. Vitale, R. A., (1992). Covariances of symmetric statistics. J. Multiυar. Ann. 41, 14-26. DEPARTMENT OF MATHEMATICS BIELEFELD UNIVERSITY POSTFACH

100131

33501 BIELEFELD GERMANY

[email protected] DEPARTMENT OF MATHEMATICS VILNIUS UNIVERSITY NAUGARDUKO 24

2600 VILNIUS, LITHUANIA alfredas. rackauskas @maf. υu. It

CONFORMAL INVARIANCE, DROPLETS, AND ENTANGLEMENT

GEOFFREY GRIMMETT 1

University of Cambridge Very brief surveys axe presented of three topics of importance for interacting random systems, namely conformal invariance, droplets, and entanglement. For ease of description, the emphasis throughout is upon progress and open problems for the percolation model, rather than for the more general random-cluster model. Substantial recent progress has been made on each of these topics, as summarised here. Detailed bibliographies of recent work are included. AMS subject classifications: 60K35, 82B43.

Keywords and phrases: Conformal invariance, droplet, large deviations, entanglement, percolation, Ising model, Potts model, random-cluster model.

1

Introduction

Rather than attempt to summarise the 'state of the art' in percolation and disordered systems, a task for many volumes, we concentrate in this short article on three areas of recent progress, namely conformal invariance, droplets, and entanglement. In each case, the target is to stimulate via a brief survey, rather than to present the details. Much of the contents of this article may be expressed in terms of the random-cluster model, but for simplicity we consider here only the special case of percolation, defined as follows. Let £ be a lattice in Rd; that is, C is an infinite, connected, locally finite graph embedded in Rd which is invariant under translation by any basic unit vector. We write C = (V,£), and we choose a vertex of C which we call the origin, denoted 0. The cubic lattice, denoted Zd, is the lattice in Rd with integer vertices and with edges joining pairs of vertices which are Euclidean distance 1 apart. Let 0 < p < 1. In bond percolation on £, each edge is designated open with probability p, and closed otherwise, different edges receiving independent designations. In site percolation, it is the vertices of C rather than its edges which are designated open or closed. In either case, for A,B CV, we write A <-> B if there exists an open path joining some a G A to some b E B, 1

This work was aided by partial financial support from the Engineering and Physical Sciences Research Council under grant GR/L15425.

Conformal Invaxiance, Droplets, and Entanglement and we write A «•> oo if there exists an infinite open self-avoiding path from some vertex in A. Let Pp denote the appropriate product measure, and let θ(p) = Pp(0 <+ oo)

be the probability that the origin lies in some infinite open self-avoiding path. The critical probability of the process is defined by pc = sup{p : θ{p) = 0}. Note that the values of θ{p) and pc depend on the choice of C and on the type (bond or site) of the process; we shall suppress this information whenever it is clear from the context. For more information about the mathematics of percolation, see Grimmett (1997, 1999). Periodic reference will be made to a more general model called the random-cluster model. There is a sense in which the latter model includes percolation, Ising, and Potts models as special cases; the reader is referred to Grimmett (1995) for a recent account of random-cluster processes. The next three sections contain summaries of recent progress on conformal invariance, the droplet problem, and the question of entanglement, respectively. 2

Conformal invariance

Consider a random spatial process in R2, perhaps a percolation process on some two-dimensional lattice. Let us assume the existence, in an appropriate sense, of the limit process obtained by a spatial re-scaling of the original process by an increasing sequence of factors. Under certain assumptions, the law of the limit process is expected to be invariant under conformal maps of the underlying space R 2 . This remarkable speculation has emerged from conformal field theory, and is relevant to a variety of random processes including the percolation and Ising models. For simplicity, we consider here the case of percolation only. Let £ be a lattice in two dimensions, and let pc be its critical probability (we shall not at this stage be specific whether it is bond or site percolation under consideration). Consider a percolation process on £, with p = pc. The hypothesis of universality suggests that the chances of long-range connections should, to some degree, be independent of certain 'local fluctuations' in C. In particular, local deformations of space, within limits, are not expected to affect such probabilities. One family of local changes arises by local rotations 2 and dilations, and such mappings of R constitute the set of conformal maps.

311

Geoffrey Grimmett

312

We make this specific in the manner surveyed and pursued by Langlands et al. (1992, 1994). Take a simple closed curve 7 in the plane, and disjoint arcs α, β of 7. For a dilation factor r > 1, define 7iγ(7; α, β) = P(ra <-» rβ in rη) where P = PPc and the event in question is the event that there exists an open path of C whose intersection with the inside of 7 contains an open connection between ra and rβ, (See Figure 2.1.)

ra y

β

/ / (\

•N :

Ί J >

\ : \-

• •

\

\

:

L

r

f

Ψ

r

•β

Figure 2.1. An illustration of the event that ra is joined to rβ within r7, in the case of bond percolation on Z 2 .

The first conjecture is the existence of the limit (2.1)

τr(7;α,/?) = lim πr(η\a,β)

for all triples (75 a,/3). Some convention is needed in order to make sense of (2.1), arising from the fact that rη lives in the plane R2 rather than on the lattice £; this poses no major problem. Only in very special cases is (2.1) known to hold. For example, in the case of bond percolation on Z 2 , self-duality enables a proof of the existence of the limit when 7 is a square and α, β are opposite sides thereof; for this special case, the limit equals 1/2. Let φ : R2 —> R2 be conformal on the inside of 7 and bijective on the curve 7 itself. The hypothesis of conformal invariance states that (2.2)

π(7; α, β) = π(<^γ; φot, φβ)

for all such φ. This conjecture concerns only percolation of a specific type (bond or site) on a specific lattice. More general forms of the conjecture come readily to hand; see Langlands, Pouliot, and Saint-Aubin (1994). First, it is natural to extend the conjecture to include the existence (or not) of crossings between more than one pair of arcs of the curve 7. Secondly, the conjecture may be extended to include a hypothesis of universality.

Conformed Invariance, Droplets, and Entanglement

313

Lengthy computer simulations, reported by Langlands, Pouliot, and Saint-Aubin (1994), support these conjectures. Particularly stimulating evidence is provided by a formula known as Cardy's formula. By following a sequence of transformations of models, and applying ideas of conformal field theory, Cardy (1992) was led to the following explicit formula for crossing probabilities between two arcs of a simple closed curve 7. Let 7 be a simple closed curve, and let ^1,^2,^3,^4 be four points on 7 in clockwise order. There is a conformal map φ on the inside of 7 which maps to the unit disc, taking 7 to its circumference, and the points Z{ to the points W{. There are many such maps, but the cross-ratio of such maps, (2 3)

u=

is a constant satisfying 0 < u < 1 (we think of z% and W{ as points in the complex plane). We may parametrise the W{ as follows: we may assume that w\ = eιδ,

W2 = e~ιδ,

W3 = —eιδ,

W4 = —e~ιδ

for some δ satisfying 0 < δ < \Έ. Note that u = sin 2 θ. We take α to be the segment of 7 from z\ to ^2, and β the segment from z 3 to z±. Using the hypothesis (2.2) of conformal invariance, we deduce that π(7; α, β) may be expressed as some function /(u), where u is given in (2.3). Cardy (1992) has derived (non-rigorously) a differential equation for / , namely

subject to the boundary conditions /(0) = 0, /(I) = 1. The solution is (2-4)

/(«)

where 2^1 is a hypergeometric function. The derivation is somewhat speculative, but the predictions of the formula may be verified by Monte Carlo simulation (see Figure 3.2 of Langlands, Pouliot, and Saint-Aubin (1994)). The function in (2.4) appears complicated, and calls for an intuitive motivation. It turns out that there is a special choice for the triple (7; α, β) for which the formula in (2.4) takes an extremely simple form, as follows. Consider site percolation on the triangular lattice, illustrated in Figure 2.2; the critical probability of the process is pc = \ (see Grimmett (1999), Section 11.9). Let x satisfy 0 < x < 1. Take 7 to be an equilateral triangle of unit side-length; let α be one side of 7, and β be a sub-interval of another side, with length x and having the vertex opposite to α as an endpoint. (See Figure 2.3.) It may be conjectured that (2.5)

π{r,<*,β)=x

Geoffrey Grimmett

314

Subject to a suitable generalised form of the hypothesis of conformal invariance, (2.5) is equivalent to Cardy's formula. Conjecture (2.5) appears to be due to L. Carleson. It is supported by numerical simulations of Bain (1999) and probably others. Formula (2.5) may be justified by the self-matching property when x—\Note that (2.5) should be valid for other processes also, such as bond percolation on the triangular lattice when p equals its critical value.

Figure 2.2. Part of the triangular lattice.

4

4- - • —»

•--••--•

Figure 2.3. An illustration of the triple (7; α, β) and of the event relevant to the probability τrr(7;α,/3). The above 'calculations' are striking. As suggested by Aizenman (1995), similar calculations may well be possible for more complicated crossing probabilities than the cases discussed above. For example, Watts (1996) has performed numerical simulations which give support to a conjecture for the limiting probability that a large rectangle is crossed from left to right and

Conformed Invariance, Droplets, and Entanglement

315

simultaneously from top to bottom. In the above formulation, the principle of conformal invariance is expressed in terms of a collection {π(7;α,/?)} of limiting 'crossing probabilities'. It would be useful to have a representation of these π(j\ α, β) as probabilities associated with a specific random variable on a specific probability space. Aizenman (1995) has made certain proposals about how this might be possible. In his formulation, we observe a bounded region DR = [0, i?] 2 , and we shrink the lattice spacing α of bond percolation restricted to this domain. Let p = pc, and let Gα be the graph of open connections of bond percolation with lattice spacing α on DR. By describing Gα through the set of Jordan curves describing the realised paths, he has apparently obtained sufficient compactness to imply the existence of weak limits as α -> 0. Possibly there is a unique weak limit, and Aizenman has termed an object sampled according to this limit as the 'web'. The fundamental conjectures are therefore that there is a unique weak limit, and that this limit is conformally invariant. Further work in this direction may be found in Aizenman and Burchard (1999). The quantities π(7;α, β) should then arise as crossing probabilities in 'web-measure'. This geometrical vision may be useful to physicists and mathematicians in understanding conformal invariance. Mathematicians have long been interested in the existence of long open connections in critical percolation models in Md (see, for example, Kesten (1982), Kesten and Zhang (1993)). An overall description of such connections will depend greatly on whether d is small or large. When d = 2, a complex picture is expected, involving long but finite paths on all scales whose geometry may be described as 'fractal'. See Aizenman (1997, 1998) for accounts of the current state of knowledge. A particular question of interest is to ascertain the fractal dimension of the exterior boundary of a large droplet (see Section 3 of the current paper). Such questions are linked to similar problems for Brownian Motion in two dimensions. The (rigorous) conformal invariance of Brownian Motion has been used to derive certain exact calculations, some of which are rigorous, of various associated critical exponents (see Lawler and Werner (1998) and Duplantier (1999), for example). Such results support the belief that similar calculations are valid for percolation. The picture for large d is expected to be quite different. Indeed, Hara and Slade (1999a, 1999b) have recently proved that, for large d, the twoand three-point connectivity functions of critical percolation converge to appropriate correlation functions of the process known as Integrated SuperBrownian Excursion. In one interesting 'continuum' percolation model, conformal invariance may actually be proved rigorously. We drop points {Xi,X2,. } in the

316

Geoffrey Grimmett 2

plane R in the manner of a Poisson process with intensity λ. Now divide 2 R into tiles {T(X1),T(X2),...}, where T(X) is defined as the set of points 2 in R which are no further from X than they are from any other point of the Poisson process (this is the 'Dirichlet' or 'Voronoi' tesselation). We designate each tile to be open with probability ^ and closed otherwise. This continuum percolation model has a property of self-duality, and, using the conformal invariance and other properties of the Poisson point process, one may show in addition that it satisfies conformal invariance. See Aizenman (1998) and Benjamini and Schramm (1998). We note that Langlands et al. (1999) have reported a largely numerical study of conformal invariance for the two-dimensional Ising model. 3

Droplets and large deviations

Consider the Ising model on a finite box B of the square lattice Z 2 with + boundary conditions, and suppose that the temperature T is low. (We omit a formal definition of the Ising model, which is known to many, and which is not central to this short review.) The origin may lie within some region whose interior spins behave as in the — phase, but it is unlikely that such a region, or 'droplet', is large. What is the probability that this droplet is indeed large? Conditional on its being large, what is its approximate shape? For low T, such questions were answered by Dobrushin, Kotecky, and Shlosman (1992), who proved amongst other things that droplets have approximately the shape of what is termed a Wulff crystal (after Wulff (1901)). In later work, such results were placed in the context of the associated randomcluster model, and were proved for all subcritical T; see Ioffe (1994, 1995), Ioffe and Schonmann (1998), and the references therein. In a parallel development for percolation on Z 2 , Alexander, Chayes, and Chayes (1990) explored the likely shape of a large finite open cluster when p > Pc They established a Wulίf shape, and proved in addition the existence

of η(p) € (0, oo) such that (3.1)

—\= logP p (|C| = n) -> η(p) as n -> oo

where C denotes the set of vertices which are connected to the origin by open paths. The geometrical framework for such results begins with a definition of 'surface tension'. The details of this are beyond this article, but the very rough idea is as follows. Let k be a unit vector, and let σ(k,p) denote 'surface tension in direction k'. For the Ising model, σ(k,p) is defined in terms of the probability of the existence of a certain type of interface orthogonal to k; for percolation, one considers the probability of a certain type of dual path of closed edges which is, in a sense to be defined, orthogonal to k. When

Conformal Invariance, Droplets, and Entanglement suitably defined, these probabilities decay exponentially, and the relevant exponents allow a definition of 'surface tension' in each case. Given a closed curve 7, one may define its energy 7/(7) as the integral along 7 of σ(k,p), where k denotes the normal vector to 7. We say that 7 encloses a 'Wulff crystal' if 7/(7) < 7/(7') for all closed curves 7' enclosing the same area as 7. We make this discussion of surface tension more concrete in the case of two dimensions, following Alexander, Chayes, and Chayes (1990). For a unit vector k and an integer n, let [nk] be a vertex of Z 2 lying closest to nk. The existence of the limit

σ(k,p) = Jim^ | - 1 logPi_p(0 ** [nk]) J follows by subadditivity, and this may be used as a definition of surface tension. Consider now the percolation model on Ί? with p > pc. If \C\ < 00, the origin lies within some closed dual circuit 7. For a wide variety of possible 7, the circuit 7 contains with large probability a large open cluster of size approximately θ(p)\ ins(7)|, where ins(7) denotes the inside of 7. It turns out that, amongst all 7 with 0(p)|ins(7)| = n, say, the 7 having largest probability may be approximated by the Wulff crystal enclosing area n/θ(p). The length of such 7 has order >/n, and one is led towards (3.1). A substantial amount of work is required to make this argument rigorous. It is a great advantage to work in two dimensions, and until recently there has been only little progress towards understanding how to prove such results in three dimensions. Topological and probabilistic problems intervened. However, a recent paper of Cerf (1998) has answered such problems, and has shown the way to a Wulff construction in three dimensions. Cerf has proved a large deviation principle from which the Wulff construction emerges. A key probabilistic tool is the 'coarse graining' of Pisztora (1996), which is itself based on the results of Grimmett and Marstrand (1990); see also Grimmett (1999, Section 7.4). Cerf's paper has provoked a further look at the Ising model, this time in three dimensions. Bodineau (1999) has achieved a Wulff construction for low temperatures, and Cerf and Pisztora (1999) have proved such a result for all T smaller than a certain value Tsiab believed equal to the critical temperature T c . The latter paper used methods of Pisztora (1996) concerning 'coarse graining' for random-cluster models. 4

Entanglement

The theory of long-chain polymers has led to the study of entanglements in systems of random arcs of M3. Suppose that a set of arcs is chosen within R 3 according to some given probability measure μ. Under what conditions

317

318

Geoffrey Gήmmett

on μ does there exist with strictly positive probability one or more infinite entanglements? Such a question was posed implicitly by Kantor and Hassold (1988), and has been studied further for bond percolation on the cubic lattice 3 Z by Aizenman and Grimmett (1991), Holroyd (1998), and Grimmett and Holroyd (1999). It is first necessary to decide on a definition of an 'entanglement'. Let 3 E be the edge set of Z . We think of an edge e as being the closed line 3 segment of M joining its endpoints. For E C E, we let [E] be the union 3 of the edges in E. The term 'sphere' is used to mean a subset of M which is homeomorphic to the unit sphere. The complement of a sphere S has two connected components, an unbounded outside denoted out(5), and a bounded inside denoted ins(5). For E CE and a sphere 5, we say that S separates E if S Π [E] = 0 but [E] has non-empty intersection with both inside and outside of S. Let E be a non-empty finite subset of E. We call E entangled if it is separated by no sphere. See Figure 4.1.

Figure 4.1. The left graph is not entangled; the right graph is entangled.

There appears to be no unique way of defining an infinite entanglement, and the 'correct' way is likely to depend on the application in question. Two specific ways propose themselves, and it turns out that the corresponding definitions are 'extreme' in a manner to be explained soon. Let E be a (non-empty) finite or infinite subset of E. (a) We call E strongly entangled if, for every finite subset F of £7, there exists a finite entangled subset F' of E satisfying F CFf. (b) We call E weakly entangled if it is separated by no sphere. Note that all connected graphs are entangled in both manners, and that a finite subset of E is strongly entangled if and only if it is weakly entangled. Let £, (respectively E^) be the collection of all strongly entangled sets of edges (respectively weakly entangled sets). It is proved in Grimmett and Holroyd (1999) that 8, C E^, and that these sets are extreme in the sense that ί , C ί C EQQ for any collection E of non-empty subsets of E having the following three properties:

Conformed Invariance, Droplets, and Entanglement

319

(i) the intersection of £ with the set of finite graphs is exactly the set of finite entangled graphs; (ii) if E G E, then E is separated by no sphere; (iii) let JE?i, E<ι,... E S be a sequence such that, for every pair i and j , E{ and Ej have a common vertex; then \J{ E{ G £. Furthermore, £/ and ^ satisfy conditions (i)-(iii). The reason for the notation ε, and £<» is that the notions of weak and strong entanglement arise naturally through a consideration of finite entanglements within the box [—n, n]3 in the limit of large n, with 'free' and 'wired' boundary conditions respectively. Let Jo (respectively J\) be the event that the origin of Z 3 lies in an infinite, open, strongly (respectively weakly) entangled set E\ note that Jo and J\ are increasing events. We define the strong and weak entanglement probabilities by θO{p) = Pp(Jo),

0i (p) = Pp{Jl),

and the associated entanglement critical points

°=

p°c = sup{p : ft(p) = 0}, Since

it is immediate that 0Q(P) < #i(p)? whence p[? > p\.

Figure 4.2. The left graph is weakly but not strongly entangled; the right graph is both strongly and weakly entangled. It is proved in Grimmett and Holroyd (1999) that 0 o (p) = θ\{p) for all values of p sufficiently close to 1, and it may be conjectured that p°c=pιc=

pln\

θι(p) = θo(p)

for some p* n t E (0,1), i

320

Geoffrey Grimmett

Such mathematical questions were not treated in the initial paper of Kantor and Hassold (1988). Numerical work reported there suggested the nt nt existence of an 'entanglement critical point' pl satisfying pl ~ p c — 1-8 x 7 10~ . No formal definition of this critical point was presented, and indeed the discussion of this initial paper concerned only finite entanglements. The strict inequality p® < pc follows by the argument presented in Aizenman and Grimmett (1991). The complementary inequality pi > 0 has been proved by Holroyd (1998). The list of open problems concerning entanglement in percolation includes: proving the almost sure equivalence of the notions of strong and weak entanglement, establishing an exponential tail for the size of the maxnt imal finite entanglement containing the origin when p < pl , and deciding whether or not there exists an infinite entanglement of a given type when p equals the appropriate critical value. Uniqueness of the infinite entanglement when p exceeds the corresponding critical value has been proved recently by Haggstrόm (1999), but the critical process (whenp equals the critical value) is still not understood. The following combinatorial question may prove interesting. Let ηn be the number of finite entangled subsets of E which contain the origin and have exactly n edges. Does there exist a constant A such that ηn < An for alln?

REFERENCES Aizenman, M., (1995). The geometry of critical percolation and conformal invariance. Proceedings STATPHYS 1995 (Xianmen), 104-120, (eds: Hao Bai-lin). World Scientific. Aizenman, M., (1997). On the number of incipient spanning clusters. Nuclear Physics B 485, 551-582. Aizenman, M., (1998). Scaling limit for the incipient spanning clusters. Mathematics ofMultiscale Materials (eds: K. Golden, G. Grimmett, R. James, G. Milton, P. Sen). IMA Volumes in Mathematics and its Applications 99, 1-24. Springer, New York. Aizenman, M. and Burchard, A., (1999). Tortuosity bounds for random curves. Preprint. Aizenman, M. and Grimmett, G. R., (1991). Strict monotonicity for critical points in percolation and ferromagnetic models. Journal of Statistical Physics 63, 817-835.

Conformed In variance, Droplets, and Entanglement

321

Alexander, K. S., Chayes, J. T., and Chayes, L., (1990). The Wulff construction and asymptotics of the finite cluster distribution for two-dimensional Bernoulli percolation. Communications in Mathematical Physics 131, 1-50. Bain, A. F. R., (1999). Unpublished. Benjamini, L, Schramm, O., (1998). Conformal invariance of Voronoi percolation. Communications in Mathematical Physics 197, 75-107. Bodineau, T., (1999). The Wulff construction in three and more dimensions. Preprint. Cardy, J., (1992). Critical percolation in finite geometries. Journal of Physics A: Mathematical and General 25, L201-L201. Cerf, R., (1998). Large deviations for three dimensional supercritical percolation. Asterisque, to appear. Cerf, R. and Pisztora, A., (1999). On the Wulff crystal in the Ising model. Preprint. Dobrushin, R. L., Kotecky, R., and Shlosman, S. B., (1992). Wulff construction: a global shape from local interaction. AMS, Providence, Rhode Island. Duplantier, B., (1999). In preparation. Grimmett, G. R., (1995). The stochastic random-cluster process and the uniqueness of random-cluster measures. Annals of Probability 23, 1461-1510. Grimmett, G. R., (1997). Percolation and disordered systems. Ecole dΈte de Probabilites de Saint Flour XXVI-1996 (eds: P. Bernard). Lecture Notes in Mathematics 1665, 153-300. Springer, Berlin. Grimmett, G. R., (1999). Percolation (2nd edition). Springer, Berlin. Grimmett, G. R. and Holroyd, A. E., (1999). Entanglement in percolation. Proceedings of the London Mathematical Society, to appear. Grimmett, G. R. and Marstrand, J. M., (1990). The supercritical phase of percolation is well behaved. Proceedings of the Royal Society (London), Series A 430, 439-457. Haggstrόm, O., (1999). Uniqueness of the infinite entangled component in three-dimensional bond percolation. Preprint. Hara, T. and Slade, G., (1999a). The scaling limit of the incipient infinite cluster in high-dimensional percolation. I. Critical exponents. Preprint.

322

Geoffrey Grimmett

Hara, T. and Slade, G., (1999b). The scaling limit of the incipient infinite cluster in high-dimensional percolation. II. Integrated super-Brownian excursion. Preprint. Holroyd, A. E., (1998). Existence of a phase transition for entanglement percolation. Mathematical Proceedings of the Cambridge Philosophical Society, to appear. Ioffe, D., (1994). Large deviations for the 2D Ising model: a lower bound without cluster expansions. Journal of Statistical Physics 74, 411-432. Ioffe, D., (1995). Exact large deviations bounds up to Tc for the Ising model in two dimensions. Probability Theory and Related Fields 102, 313-330. Ioffe, D. and Schonmann, R. H., (1998). Dobroshin-Kotecky-Shlosman theorem up to the critical temperature. Communications in Mathematical Physics 199, 117-167. Kantor T. and Hassold, G. N., (1988). Topological entanglements in the percolation problem. Physical Review Letters 60, 1457-1460. Kesten, H., (1982). Percolation Theory for Mathematicians. Birkhauser, Boston. Kesten, H., Zhang, Y., (1993). The tortuosity of occupied crossings of a box in critical percolation. Journal of Statistical Physics 70, 599-611. Langlands, R., Lewis, M.-A., and Saint-Aubin, Y., (1999). Universality and conformal invariance for the Ising model in domains with boundary. Preprint. Langlands, R., Pichet, C, Pouliot, P., and Saint-Aubin, Y., (1992). On the universality of crossing probabilities in two-dimensional percolation. Journal of Statistical Physics 67, 553-574. Langlands, R., Pouliot, P., and Saint-Aubin, Y., (1994). Conformal invariance in two-dimensional percolation. Bulletin of the American Mathematical Society 30, 1-61. Lawler, G. and Werner, W., (1998). Intersection exponents for planar Brownian motion. Preprint. Pisztora, A., (1996). Surface order large deviations for Ising, Potts and percolation models. Probability Theory and Related Fields 104, 427-466. Watts, G. M. T., (1996). A crossing probability for critical percolation in two dimensions. Journal of Physics A: Mathematical and General 29, L363.

Conformal Invariance, Droplets, and Entanglement

323

Wulff, G., (1901). Zur Prage der Geschwindigkeit des Wachsturms und der Auflόsung der Kristallflachen. Zeitschrift fur Krystallographie und Mineralogie 34, 449-530. STATISTICAL LABORATORY, DPMMS UNIVERSITY OF CAMBRIDGE 16 MILL LANE CAMBRIDGE CB2

1SB

UNITED KINGDOM

g. r. grimmett@stαtslαb. cam. ac. uk

NONPARAMETRIC ANALYSIS OF EARTHQUAKE POINT-PROCESS DATA

EDWIN CHOI 1 AND PETER HALL 1

Centre for Mathematics and its Applications, ANU Motivated by multivariate data on epicentres of earthquakes, we suggest nonparametric methods for analysis of point-process data. Our methods are based partly on nonparametric intensity estimation, and involve techniques for dimension reduction and for mapping the trajectory of temporal evolution of high-intensity clusters. They include ways of improving statistical performance by data sharpening, i.e. data pre-processing before substitution into a conventional nonparametric estimator. We argue that the 'true' intensity function is often best modelled as a surface with infinite poles or pole lines, and so conventional methods for bandwidth choice can be inappropriate. The relative severity of a cluster of events may be characterised in terms of the rate of asymptotic approach to a pole. The rate is directly connected to the correlation dimension of the point process, and may be estimated nonparametrically or semiparametrically. AMS subject classifications: Primary 62G05, 62G07; Secondary 63M30. Keywords and phrases: Bandwidth, bias reduction, correlation dimension, data sharpening, density estimation, epicentre, geophysics, intensity, Japan, Kanto, kernel methods, magnitude, pole, regular variation, smoothing.

1

Introduction

An earthquake-process dataset may often be interpreted as a realisation of a 5-dimensional point process, where the first three, spatial components denote latitude, longitude and depth below the earth's surface, the fourth represents time, and the fifth is a measure of 'magnitude', for example on the Richter scale. Goals of analysis can be very wide-ranging. At one level they may be purely descriptive, perhaps summarising features of the dataset. In this regard, some form of dimension reduction is often critical, putting the information on five dimensions into a form that is more readily accessible and interpretable. At another level the goals may be exploratory, suggesting directions for future analysis, or they may be more explicit and detailed, perhaps with the aim of elucidating properties of subterranean features that played a role in generating the data. Supported by the Australian Research Council.

Point Process Analysis

325

In this paper we discuss nonparametric methods for summarising earthquake data, for exploring the main features of the data, and for addressing more structural problems such as the location of poles and pole lines, the way in which those poles migrate with time, and the value of the correlation dimension of clusters of epicentres. (Poles and pole lines are points and line segments, respectively, at which the intensity of the point process asymptotes to infinity.) Many of our arguments are based on kernel-type estimators of intensity, while others employ methods that are parametric in simple cases but are nevertheless valid in contexts which are quite distant from the parametric model. The aim is to develop analytical tools that offer greater diversity, and robustness against departures from structural models, than more traditional parametric approaches. The latter include the popular Epidemic Type Aftershock Sequence (ETAS) model (Ogata, 1988), which is used to describe temporal behaviour of an earthquake series; and refinements of Hawkes' (1971) self exciting point process model, which describe spatial-temporal patterns in a catalogue. The paper by Ogata (1998) gives detailed discussion of recent extensions of these models. Disadvantages of parametric models in this setting include their instability when even small amounts of new data are added, and their relative insensitivity to anomalous events, arising from the fact that models tend to be formulated through experience of relatively conventional earthquake activity. Indeed, anomalies are typically the root cause of the aforementioned parameter instability. Since anomalous events are often of at least as much interest as conventional ones (see Ogata, 1989), procedures that tend to conceal anomalies are not necessarily to be preferred. Figure 1 depicts spatial components of the type of data that motivate this paper. They are part of the 'Kanto earthquake catalogue', and were compiled by the Centre for Disaster Prevention at Tsukuba, Japan. The points are longitude-latitude pairs representing the locations of earthquakes that occurred in the region of Kanto, Japan, between 1980 and 1993. We have restricted attention here to events whose location was between 138.6° and 139.7° longitude and 34.6° and 35.7° latitude, whose depth was less than 36 km, and whose magnitude was at least 2.0 on the Richter scale. There are 8187 points in the dataset. The diagonal line on the figure is a linear approximation to the location of the volcanic front of the Izu-Bonin Arc (Koyama, 1993), which is a known source of earthquake activity. The region with a dotted boundary defines a smaller subset, near the island of 0-shima, which will also feature in our analysis. Section 2 describes methods for intensity estimation based on point process data, and outlines applications to which such estimates may be put. Techniques for enhancing multivariate intensity estimates, and for deducing structure from them, are outlined in Section 3. Section 4 introduces methods

326

Edwin Choi and Peter Hall

Figure 1. Spatial coordinates of Kanto earthquake data. Data in the smaller region, indicated by the dotted-line boundary, will be used for analysis described in section 2.3. The dashed line diagonally across the figure represents a linear approximation, A, to the volcanic front of the Izu-Bonin Arc.

for estimating the locations and strengths of poles in intensity functions. Some discussion of use of the term 'magnitude', and of the 'Richter scale', is in order. The many different measures of magnitude include those based respectively on energy and on different measures of the amplitudes of shock waves produced by an earthquake. Local Magnitude, more popularly referred to as Richter Magnitude, is of the latter type and is representable in terms of the logarithm of the maximum trace amplitude, measured in micrometers on a standardised seismometer. The magnitude to which we refer in this paper is Local Magnitude, although we shall henceforth call it, and the scale on which it is measured, by its popular name. 2

2.1

Data summarisation and exploration

Dimension reduction

Information about depth in a seismic data vector is often not particularly

Point Process Analysis

327

accurate, and for example is typically represented in bins up to 10 kilometres wide. Reflecting this difficulty, we suggest pooling bins. The longitude and latitude components too are recorded with varying degrees of error, which depend on, among other matters, the spatial distribution of recording stations around the location of the event, and event depth. We shall not attempt to employ such information in our analysis — it is sometimes explicitly available (see e.g. Jones and Stewart, 1997), or deducible from other measurements — but it can be incorporated. Even after removing the depth dimension, data vectors can have as many as four components. We suggest looking at the two remaining spatial components separately, by projecting longitude-latitude pairs onto first one axis and then another. An appropriate axis is often clear from physical considerations; see Figure 1. Neglecting the magnitude component for the time being, we now have two bivariate datasets where in each case one component represents time and the other is a spatial coordinate. Each may be used to produce nonparametric estimates of point-process intensity, enabling perspective plots (where the third dimension represents intensity) to be produced. Of course, contemporary dimension-reduction methods, such as projection pursuit, might also be used to determine projections in the continuum that maximise the 'interestingness' of the associated bivariate scatterplots. In their full generality, such approaches can be hard to justify in the present setting, since rotations of axes that are as distinct as time and space are difficult to interpret. Even if dimension reduction is contemplated only for the spatial coordinates, physical interpretation can sometimes be facilitated by using information from outside the dataset (for example, in the case of the Kanto data, the physically-meaningful Izu-Bonin Arc) to determine an appropriate axis. Magnitude may be depicted by adding colour or a grey shade to graphs of estimated intensity. Therefore, magnitude can be included on the plots described above, without increasing the complexity of the set of projections. It can be shown separately, however, in plots broadly similar to those for intensity. Since magnitude is recorded with error, and only at scattered points in space and time, it is generally necessary to smooth magnitude measurements.

2.2

Kernel estimation of intensity

If space-time data pairs (Xi,Ti) are available after projection of spatial coordinates onto an axis, then the space-time intensity per unit area at (x, t) is estimated by <2«

*<«

328

Edwin Choi and Peter Hall

where K\ is a univariate kernel, usually taken to be a symmetric probability density function, and /ii,/i2 are bandwidths for spatial and temporal components, respectively. A local method for bandwidth choice is critical when estimating the intensity of point processes related to earthquakes, since in many cases there is evidence that the intensity function is neither bounded nor square-integrable. Estimators of the type at (2.1) have been discussed by, for example, Wand and Jones (1995, p. 167f). They were introduced by Ramlau-Hansen (1983) and Diggle (1985). Related methods in a geophysical context have been discussed by Davis and Prohlich (1991). The panels in Figure 2 are intensity estimates computed using (2.1). We employed the biweight kernel. Bandwidth was chosen using the following 'coupled' near-neighbour method. We chose h\ and /12 to minimise hι+αli2, subject to the rectangle (x — Λi, x + hi) x (t — h<ι, t + /12) containing at least k space-time pairs (Xi,Ti). The value of k controls the overall level of smoothing, with larger fc's providing a greater amount of smoothing. The relative emphasis placed on spatial and temporal coordinates is governed by α. In particular, if α is chosen small then the resulting rectangle is elongated along the time axis, and so the intensity estimator is localised more in space than in time. We took α to equal the ratio of the ranges of the spatial and temporal data sets, although substantial changes to the values of A; and α (e.g. doubling or halving their values) do not alter the man features of Figure 2. Longitude and latitude components of the original dataset are depicted in Figure 1. The straight line there (denoted here by Λ) was the axis onto which spatial components were projected for the first panel. For the second panel, spatial projection was in the perpendicular direction. Even by itself, without reference to data on magnitude, Figure 2 is revealing. For example, patterns of spatial variation of events that occur close together in time are clearly evident. Similar plots may be derived when event magnitude, rather than intensity, is featured on the vertical scale. A comparison of intensity and magnitude plots can provide a particularly informative description of the way in which a cluster of earthquakes develops in time and space. The relationship between intensity, and magnitude, spacetime surfaces is complex, and far from being proportionate. It tends more towards being inversely proportionate, with a high frequency of relatively low-magnitude events being similar in some respects to a low frequency of high-magnitude ones. However, even this is a significant oversimplification. Further details are available from figures in Choi and Hall (1998). A reader familiar with parametric analysis of earthquake data might query our use of a symmetric kernel in at least the temporal component at (2.1). In particular, it is known that earthquake processes are quite asymmetric in time, with few foreshocks and many aftershocks; see for ex-

Point Process Analysis

329

(a)

Figure 2. Perspective plots of intensity estimates. Panels (a) and (b) show plots of space-time intensity when spatial components are projected onto and perpendicularly to, respectively, the diagonal line A in Figure 1.

ample Kagan (1994). A virtue of nonparametric analysis is that it usually adapts well to asymmetries of this type, however. In a related setting, one would seldom have qualms about using a symmetric kernel to estimate an asymmetric probability density, and similar arguments apply here. 2.3

Spatial migration of peak intensity

As Figure 2 suggests, intensity increases rapidly, in both temporal and spatial terms, to a peak that could be interpreted as a pole. The temporal trajectory of the spatial location of the pole is of particular interest. To estimate the trajectory we first estimate spatial intensity as a function of time, in the continuum. To this end we choose a time window of width 2/i2, and utilise those data with occurrence times in the interval T(t) = [t — h<χ,t + h<^. As before, we bin the depth component of data vectors, but we no longer

330

Edwin Choi and Peter Hall τ

project data onto the axis A Therefore, the data (Y?,Ti) that we employ are vectors of length 3, with Y{, a column vector of length d = 2, representing the spatial coordinates of the datum observed at time T;. The spatial intensity, μ(y\ £), of events that occur at time t is defined to equal the expected number of events that occur per unit area at point y in the plane, when the process is observed at time t. It is estimated by

where if2 is a bivariate kernel and I13 is a new bandwidth. Here, y is a column vector of length 2, and we have in effect employed a bandwidth matrix /13/2 for K<ι, where I<ι denotes the 2 x 2 identity matrix. When used in the present context, near-neighbour methods tend to produce closely neighbouring, multiple peaks in places where intensity is high, and so we do recommend them. We employ instead a global bandwidth. Once μ(-\t) has been computed we may readily calculate the spatial location of its maximum, being a bivariate function of t; and thence we may estimate the trajectory of the maximum, again as a function of t. Since earthquake activity is typically low between relatively short periods of high intensity, it is usually necessary to threshold the maximum intensity at some value r, say, in order to obtain a trajectory that is representative of the more interesting episodes. Panel (a) of Figure 3 depicts spatial migration of peak intensity within the smaller region shown in Figure 1, indicated by the dotted-line boundary there. The great majority of events in that smaller region are the result of high-magnitude but low-intensity volcanic activity near the island of Oshima, in the lower right of Figure 1. We make this specialisation so as to restrict attention to a region where earthquake activity has a relatively homogeneous cause. Without it, a plot of peak migration is an interwoven mixture of movements representing quite different subterranean activities in different regions; over time, the trajectory jumps from one region to another, often without there being any plausible relationship between adjacent locations. The total number of data points within the smaller region in Figure 1 is 5654. Intensity was computed using the estimator at (2.2), taking K\ to be the biweight kernel and K2 to be its multivariate version, with h<ι = 50 and hs = 0.1. Time was discretised on 301 points. Panels (b) and (c) of Figure 3 depict the longitude and latitude components, respectively, of the trajectory shown in panel (a). The crosses in both panels represent those among the 301 values of t where estimated peak intensity exceeded the threshold r = 50, and the general locations are numbered 1-10 in respective order. The points corresponding to greatest intensity in each of the 10 clusters of

Point Process Analysis

331

crosses are respectively 30 June 1980, 20 January 1983, 5 September 1984, 13 October 1986, 11 May 1987, 20 February 1988, 2 August 1988, 9 July 1989, 10 January 1993 and 31 May 1983.

0

1000

2000

3000

4000

5000

time (days from 1 January 1980)

139.10

139.15

139.20 longitude

139.25 0

Ί000

2000

3000

4000

5000

time (days from 1 January 1980)

Figure 3. Spatial migration of peak intensity. Panel (a) depicts spatial movement of peak intensity, in the direction indicated by increasing numbers from 1 to 10. Panels (b) and (c) give plots, against time, of longitude and latitude components, respectively, of peak intensity estimates. Attention is confined to data in the region represented by the dotted-line boundary in Figure 1.

3 3.1

Data sharpening for intensity estimation General principles

The term 'data sharpening' refers to methods for pre-processing data so that, when they are substituted into a conventional estimator, performance is improved relative to what would it be if the raw data were employed. The idea is to enhance performance without degrading the attractive properties that relatively simple estimators enjoy. For example, in the context of kernel density estimation one can achieve high orders of accuracy by replacing a nonnegative kernel by one with a large number of vanishing moments. But this typically destroys positivity of the estimator, and the high-order advantages are often only available for relatively large sample sizes. By appropriately preprocessing the data before substitution into an estimator with a conventional, positive kernel, it is possible to improve accuracy yet retain the property of positivity (and the property that the estimator integrates to 1). The methods of intensity estimation discussed in Section 2.2 have much

332

Edwin Choi and Peter Hall

in common with those for nonparametric density estimation, and in particular share bias properties with that approach. They tend to underestimate intensity at peaks, and overestimate it at troughs. This is a major source of bias — it means that intensity estimators are generally biased downwards at local maxima, and upwards at local minima. If we could move data values closer together near peaks and further apart near troughs, then in large part these problems would vanish, and bias would be reduced, without us having to modify the estimator itself. Possibilities along these lines have been discussed by, for example, M.C. Jones and Signorini (1997) and R.H. Jones and Stewart (1997). In the context of the method described in Section 2.2, this argument might be made in the following form. Consider the surface S defined by the true bivariate intensity λ, of which λ is an estimator. Then S is the locus of triples (z, £, y) defined by y = λ(rr, £). If we are at a point Pi = (X;, T») in the plane then, to a good approximation, we shall move across the plane in the direction of the projection of the line of steepest ascent up <S, if we map Pi to the value of the average of all data values in some small neighbourhood of Xi. Such a mapping will tend to move data in precisely the direction needed to overcome biasing problems associated with standard intensity estimators. Performance of the algorithm is of course influenced by the definition taken for 'neighbourhood'. This may be given very simply in terms of the neighbourhood used for the intensity estimator itself. For example, suppose we are using a standard kernel estimator such as that at (2.1), with bandwidth h = hi = h>2 (after rescaling) and a nonnegative, symmetric kernel ϋfi, to estimate intensity; and that we wish to sharpen our data in this context. Then a canonical version of our data-sharpening transformation amounts to mapping {X^Ti) to the value taken by the standard multivariate Nadaraya-Watson density estimator evaluated at (Xi,T;), when bandwidth equals 2~ιl2h and the data are used as both exogenous and endogenous variables in the regression problem. It may be shown that this procedure reduces bias by two orders of magnitude, from O(/ι2), expressed relative to true intensity, to O(Λ4). The key to validity of this assertion is use of the factor 2" 1 / 2 to adjust bandwidth. Without the factor, bias may be reduced but not by an order of magnitude. The method suggested by Jones and Stewart (1997) does not employ such a factor, although the context of Jones and Stewart's work is different from that discussed above. These authors are interested in identifying ridges in intensity estimates computed from geophysical datasets, not in intensity estimation per se. See Section 3.3 for further discussion. Derivation of the bandwidth-multiplication factor will be summarised in section 3.2. Further details are available from Choi and Hall (1999b). Suffice it to say here that the factor is unrelated to the bivariate nature of

Point Process Analysis

333

the problem, but instead depends on the order of the smoothing technique. In particular, if we were estimating an intensity in p > 1 dimensions, and employing an estimator based on nonnegative, symmetric kernels, then we would use the data-sharpening method suggested two paragraphs above, employing bandwidth 2~ιl2h in the sharpening step based on the NadarayaWatson estimator, regardless of the value of p. Bias would be reduced from O(h2) to O(hA). More generally, however, if we were using kernels of order r then we would replace 2" 1 / 2 by r~χlτ. The method has simple analogues for other linear estimators, for example those based on orthogonal series or singular integrals.

3.2

Implementation

Rather than reanalyse the data addressed in Section 1 we shall introduce a new data set, this time wholly spatial rather than involving both space and time. It has the advantage of requiring relatively little variation of bandwidth. Panel (a) of Figure 4 depicts epicentres of earthquakes with magnitude at least 2 on the Richter scale, occurring between 100° and 160° longitude and -30° and 30° latitude during the years 1984 to 1995. The number of events which satisfy these criteria is 24471. Data were compiled by NOAA and USGS (1996); see also the printed account in the PDE catalogue (1997). A kernel estimator of intensity at x — (x^,x^), based on data such as these, is

(compare (2.1)), where now Xi = (x\ \x\ ) represents a spatial data pair, K is a bivariate kernel and h a bandwidth. The sharpened data are

where K and h are as at (3.1). The 'diagonal' terms may be dropped from the numerator and denominator at (3.2), without affecting first-order asymptotic properties. We suggest substituting Xi for Xi at (3.1), to compute the sharpened form of v\

Panels (a) and (b) of Figure 5 show v and ί>s, respectively, computed from the data in panel (a) of Figure 4, using bandwidth h — 2.0 (chosen subjectively) and taking K to be the product of two univariate biweight

334

Edwin Choi and Peter Hall (a): Raw data

100

110

120

130 140 longitude

(b): One-fold sharpened data

150

160 ~Ϊ00 110

(c): Two-fold sharpened data

(d): Three-fold sharpened data

CO

30-

120 130 140 150 160 ' longitude -'

/.

•

T

\

\

CM

20-

.

10

•/:•

latitude o

10-

V 1

-10

-10-

-20<

-20-

-30

100

110

120

130 140 longitude

150

160

-30

100

110

120

130 140 150 160 longitude

Figure 4: Scαtterplots of sharpened data with decreasing bandwidth. Panel (a) shows the raw data, and panels (b)-(d) show the effects of applying the sharpening algorithm m = 1, 2 or 3 times, respectively. The bandwidth used to compute the sharpened scatterplots decreases in proportion to the inverse of the square root of m.

kernels. Visibly, the intensity peaks are depicted more sharply after sharpening. In particular, the estimated peak intensity at (146.33°,—6.50°) of the data-sharpening intensity estimate exceeds that of the standard kernel estimate by 23%. Finally we give a short technical argument to demonstrate the appropriateness of the bandwidth-multiplication factor, 2" 1 / 2 . Of course, we assume K is a symmetric bivariate probability density. Suppose the data X{ are p-variate and come from a distribution with density /, and consider the empirical transformation

where, for the present, h\ denotes an arbitrary constant multiple of h. If we

Point Process Analysis

335

(a)

(b)

(c)

(d)

Figure 5: Intensity estimates computed from sharpened data with decreasing bandwidth.

Estimates in panels (a)-(d) were computed from the scatterplots in panels (a)-(d), respectively, of Figure 4. The bandwidth used to compute each intensity estimate from the sharpened data was kept fixed at h = 2.0, although the bandwidths used to produce successive sharpened scatterplots were decreasing.

take that constant to be 2" 1 / 2 then X{ — ψ(Xi). More generally, by Taylor expansion, E{φ{x)} = x + —ΛΓ / u K{u) f(x + h\u) du + o(h2) j \ x)

j

where K = J{u^)2 K{u) du, V/ = (/W,..., ftψ, and fV\x) denotes the partial derivative of }{x) with respect to x^\ Put V 2 / = {f^)2 + ... + (y(p))2_ Then, using the above expansion of E(ψ), and the fact that ψ has variance of smaller order than the variance of a kernel estimator of / using

336

Edwin Choi and Peter Hall

bandwidth /i, we see that

= ft {/(*) + \ K ft2 V 2 /(i) i) - Kft?V2/(x) + o(ft2)} = ft{/(l) + o(ft2)} , where the last identity requires hi = 2~ι/2h. This confirms that using the bandwidth-multiplication factor 2" 1 / 2 reduces bias from O(h2) to o(/ι 2 ). The symmetry of K may be used to prove that bias is actually reduced to O(/ι 4 ).

3.3

Iteration

There is no difficulty iterating the sharpening step. To describe the method, let Xj * = Xj denote a 'raw' data value, and consider using bandwidth ho rather than 2~ιl2h at (3.2). We may generalise the notion of sharpened data by defining the fc-fold sharpened form of Xj by

for k > 1. In order to preserve the order of bias reduction, from O(h2) to O(/ι 4 ), it is necessary to change bandwidth from 2~ιl2h to ho = {2m)~ιl2h if a total of m iterations is contemplated (i.e. if we employ (3.3) for 1 < k < m, and then compute our final estimator using formula (3.1) with X^ replacing Xi). Note that the order of bias reduction does not change with increasing m. More generally, we could employ ho = hk, say, in (3.3), where 2 Λi,..., hm satisfy h\ + . . . + h^ = h /2. Panels (b)-(d) of Figure 4 are scatterplots of the points x\m\ for m = 1,2,3 respectively. They were computed using bandwidth h0 = (2m)~ 1 / 2 /ι, with h = 2.0, in formula (3.3). The increasing amount of structure observed as m increases is the result of points X{ moving to positions closer to the projections, into the x plane, of ridge lines on the surface S described by the equation y = v(x). As m increases the ridge projections become sharper, and the structure becomes more complex and 'crinkly', partly because of the movement of points and partly because bandwidth is decreasing. However, the structure starts to degenerate as we increase m further. This is starting to become apparent in panel (d) of Figure 4, representing

Point Process Analysis

337

m = 3. By the time m = 5 the ridge structure that is clearly visible for m = 2 has almost entirely disappeared. It is replaced by a new scatterplot of highly isolated clusters of points. On the scale of Figure 4 it looks similar to panel (a), except that only a few percent of the original data appear to remain. These 'points' are actually high-intensity data clusters around several hundred modes, or local maxima, the latter arising because the effective bandwidth has been substantially reduced. This tendency reverses for large m, and in fact the limit, as m -> oo, of the sharpened scatterplot is the original scatterplot in panel (a) of Figure 4. Panels (a)-(d) of Figure 5 show versions of the intensity estimate vs that correspond to the scatterplots in panels (a)-(d), respectively, of Figure 4. Note that we use the same bandwidth, h = 2.0, for all the functions in Figure 5, although the scatterplots in Figure 4 are calculated using successively smaller bandwidths. A key feature of Figure 5 is that intensity estimates are virtually identical in each of panels (b)-(d). While the number of iterations of data sharpening (with steadily reducing bandwidth) has a substantial impact on the point process pattern (see Figure 4), it has little effect on the intensity estimates produced from the patterns, at least for the numbers of iterations employed to generate the last three panels of Figure 5. In particular, the ratio of the height of the largest peak for a given value of ra, to that when m = 0 (i.e. for the standard kernel estimator), increases to 1.25 when m = 2, and decreases only slightly for m = 3 and 4, always remaining above 1.20. There is some evidence of increased variability of intensity estimates for larger values of ra, but the increase is substantially less than the changes in the point-process maps (see Figure 4) produced by increasing m. Simulation studies for simpler target intensities show that for fixed sample size, bias also tends to increase if ra is increased beyond a certain point. Therefore, employing a high order of iteration to compute the 'final' estimate is not recommended. Results are quite different if we keep bandwidth fixed while iterating the data-sharpening step. Now complexity and 'crinkliness' are actually reduced for relatively large numbers of iterations. See Figure 6. However, successively less information is available in parts of the plane where the 'true' intensity is not high. As a result, if we compute the intensity estimator v from datasets that have been successively sharpened in this way, its shape alters rapidly as the number of iterations increases, and overall performance deteriorates. This is clear from Figure 7. There, the local maxima of the intensity estimate become progressively more pronounced as the number of iterations is increased. Indeed, the growth is so rapid that we have had to truncate height at 300 in order to present panels (c) and (d) of Figure 7; the heights of the largest peaks for those datasets are actually 360 and 430, respectively. In

Edwin Choi and Peter Hall

338

this setting, where bandwidth is kept fixed in the data-sharpening step, the ratio of the height of the largest peak for a given value of m, to that when m = 0 (i.e. the standard kernel estimator), increases monotonically with ra, being 1.3, 1.6, 1.9 and 2.0 when m = 1,2,3,4 respectively. By way of comparison, when bandwidth is reduced as m increases, the corresponding ratio at first increases and then, for the last two panels, decreases only slightly (and monotonically); see Figure 5. (b): One-fold sharpened data

(a): Raw Data

100

110

120

130 140 longitude

150

160

100

(c): Two-fold sharpened data

100

110

120

130 140 longitude

150

110

120

130 140 longitude

150

160

(d): Three-fold sharpened data

160

100

110

120

130 140 longitude

150

160

Figure 6: Scatterplots of sharpened data with constant bandwidth. Panel (a) shows the

raw data, and panels (b)-(d) show the effects of applying the sharpening algorithm m = 1, 2 and 3 times, respectively, this time keeping bandwidth fixed at h = ho = 2.0 in all data sharpening steps.

An algorithm similar to that illustrated by Figure 6 was first suggested by Jones and Stewart (1997). It was designed to elucidate structure in geophysical data of the type depicted in panel (a), not as an aid to intensity estimation. It may be thought of as a device for forcing points to 'walk' up empirical approximations to lines of steepest ascent on the surface 5, rep-

Point Process Analysis

339

Figure 7: Intensity estimates computed from sharpened data with constant bandwidth.

Estimates in panels (a)-(d) were computed from the scatterplots in panels (a)-(d), respectively, of Figure 6.

resenting a plot of the intensity estimator against spatial location. Initially, in the first few steps, those lines are approximately perpendicular to ridge lines, but after a point has attained a reasonable height on S its path starts to turn and, as the number of iterations increases, it moves in a direction that is increasingly parallel to a ridge line. Increasing the number of iterations beyond this stage tends to reduce performance of the algorithm as a means for elucidating structure. In particular, detail about structure is lost at places where intensity is relatively low; this is already apparent in panel (d) of Figure 6. Jones and Stewart (1997) suggested a stopping rule to help overcome this problem. An alternative approach is to modify the algorithm by forcing the projection in the x-plane of the vector of motion up S to be similar to that in early steps. Constraints such as this can substantially improve performance of Jones and Stewart's

340

Edwin Choi and Peter Hall

algorithm.

4

Estimating the locations and strengths of poles

We have seen in Sections 2 and 3 that the intensity function associated with a point process of earthquake epicentres can rise very steeply from the plane, and give every appearance of having poles or pole lines in places of high intensity. Nonparametric methods may be employed to estimate both the location of a pole or pole line, and the rate at which intensity diverges in its vicinity. For the sake of simplicity we shall confine attention here to poles. We begin with an idealised model for both the location and 'strength' of a pole. Estimators suggested by the model are appropriate very generally, and so the model amounts only to a device for pointing the way to methodology, not to a specific structural assumption. To this end, we assume that in the vicinity of a point v in the plane, the intensity v(x) is asymptotic to a constant multiple of ||x — v\\~a, where α > 0 represents pole strength. In order for the expected number of points in each bounded, nondegenerate region to be finite, we need α < 2. Of course, if the point process were Poisson then the actual number of data in a region would be infinite, with probability 1, if the expected number there were infinite. The value of α is related to the correlation dimension, D, of the point process by the formula D = 2(2 - α). See Grassberger and Procaccia (1983). It may be shown that, even if the data are from a Poisson process with this intensity, maximum likelihood estimation of v is not feasible. Nevertheless, given an estimator v of υ, a form of maximum likelihood estimation of a is possible. Estimators of v may be based on maximising the number of points within a small region, and have at least two forms, as follows. Let T> = T>(w, r) denote the closed disc of radius r centred at w. Define v to be either that value of w which minimises the area of V(w,r) subject to this disc containing at least a given number, N say, of points; or a value of w which maximises the number of points contained in V(w, r) for a given value of r. If the points X{ are distributed in the continuum then the former v is uniquely defined with probability 1, while the latter is not unique, with the same probability. For this reason we favour the former estimator. Given v we may define an estimator ά of α to be the minimiser of

' log \\Xi - ϋ\\ + M{V2\Vλ)

log ( f \ JV2\Dl

\\x - v\\~a dx] , )

where V\ C T>2 are concentric discs centred at ΰ, £ / denotes summation over those points Xi that lie in V2\Vι, and M(V2\Di) equals the number of such points. The equation (d/dά) i(ά) = 0 has a unique solution ά. The radii of 2?, V\ and V2 play the roles of smoothing parameters.

Point Process Analysis

341

It may be proved that ά and v are consistent for α and v, under conditions that are much more general than those asserted by the motivating model v(x) = vo(x) = \\x — Ή | ~ α A theory describing rates of convergence may be developed (Choi and Hall, 1997), having points of contact with that for semiparametric estimation of parameters in distributions with regularly varying tails; see e.g. Embrechts, Klϋppelberg and Mikosch (1997, Chapter 6). It is straightforward to incorporate effects of noise into the intensity model, for example by assuming that an independent bivariate Gaussian vector with zero mean is added to each point in the plane. Provided the covariance matrices of these vectors are known they do not materially complicate estimation of α. However, it may be shown that there is negligible information in the data for estimating noise covariance, under intensity models such as v = z/o Fortunately, information about noise properties is often available from knowledge of the placement of recording stations around the epicentre of an event, and of the nature of the rocks through which shock waves passed on their way to those stations. Indeed, each measurement of longitude, latitude and depth is sometimes accompanied by its own error covariance matrix; see for example Jones and Stewart (1997). We may estimate υ and α for data from temporal clusters, and thereby compute estimators of the spatial trajectory, as a function of time, that are alternative to those considered in Section 2.3. Table 1 provides information about location and pole strength of nine of the ten shallow (i.e. no deeper than 36 km) Kanto event clusters that were the subject of Figure 3. (The cluster numbered 6 in Figure 3, occurring during February 1988 and having relatively low intensity, has been omitted from the present analysis.) Figure 8 depicts the corresponding trajectory, and should be compared with Figure 3. Smoothing parameters were chosen by performing a simulation study involving models that produced realisations approximating the data clusters. Details of the analysis are given by Choi and Hall (1999a). Methods for estimating α in related problems have been discussed by Theiler (1990), Smith (1992), Grassberger and Procaccia (1993), Mikosch and Wang (1995), Haxte (1996) and Vere-Jones (1999). Acknowledgements We are grateful to Dr. D. Harte and Professor D. Vere-Jones for providing the earthquake data analysed in this paper, and to Professor B.L.N. Kennett for helpful comments during the course of our work. The constructive comments of two referees have also been particularly helpful.

Edwin Choi and Peter Hall

342

Year 1980 1983 1984 1986 1987 1988 1989 1 1993 1993 2

Longitude 139.0,139.3 139.0,139.3 139.1,139.4 139.0,139.3 139.1,139.4 139.05,139.35 138.95,139.25 139.0,139.3 139.0,139.3

Latitude 34.8,35.1 34.8,35.1 34.8,35.1 34.8,35.1 34.8,35.1 34.8,35.1 34.8,35.1 34.8,35.1 34.8,35.1

N

vx

Vy

222 238 383 207 489 237 175 337 614

139.186 139.200 139.217 139.175 139.258 139.195 139.108 139.177 139.130

34.966 34.938 34.928 34.948 34.913 34.951 34.986 34.937 34.976

: includes only events in January : includes only events in May and June Table 1: Locations and pole strengths for 9 shallow Kanto event clusters.

8 CO

CO

CD CO •σ

to

s' Si

139.10

i

139.15

139.20

139.25

longitude Figure 8. Spatial migration of pole. Data are clusters within the dataset used to compute Figure 3, and represent shallow Kanto events.

Point Process Analysis

343

REFERENCES Choi, E. and Hall, P. (1998). A nonparametric approach to analysis of spacetime data on earthquake occurrences. J. Computαt Graph. Statist, under revision. Choi, E. and Hall, P. (1999a). On the estimation of poles in intensity functions. Biometrika, to appear. Choi, E. and Hall, P. (1999b). Data sharpening as a prelude to nonparametric density estimation. Biometrika, to appear. Davis, S.D. and Prohlich, C. (1991). Single-link cluster analysis of earthquake aftershocks — decay laws and regional variations. J. Geophys. Res. 96, 6335-6350. Diggle, P.J. (1985). A kernel method for smoothing point process data. Appl. Statist. 34, 138-147. Embrechts, P., Klύppelberg, C. and Mikosch, T. (1997). Modelling Extremal Events. Springer, Berlin. Grassberger, P. and Procaccia, I. (1983). Measuring the strangeness of strange attractors. Physica D 9, 189-208. Harte, D.S. (1996). Multifractals— Theory and Applications. PhD thesis, Victoria University of Wellington. Hawkes, A.G. (1971). Point spectra of some mutually exciting point processes. J. Roy. Statist. Soc. Ser. B. 33 438-443. Jones, M.C. and Signorini, D.F. (1997). A comparison of higher-order bias kernel density estimators. J. Amer. Statist Assoc. 92, 1063-1073. Jones, R.H. and Stewart, R.C. (1997). A method for determining significant structures in a cloud of earthquakes. J. Geophys. Res. 102, 82458254. Kagan, Y.Y. (1994). Observational evidence for earthquakes as a nonlinear dynamic process. Physica D 77, 160-197. Koyama, M. (1993). Volcanism and tectonics of the Izu Peninsula, Japan (in Japanese). Kagaku (Science) 63, 312-321. Mikosch, T. and Wang, Q. (1995). A Monte-Carlo method for estimating the correlation exponent. J. Statist. Phys. 78, 799-813. NOAA and USGS (1996). Seismicity Catalogs, Volume 2: Global and Regional, 2150 BC-1996 AD (CD rom). National Oceanic and Atmospheric Administration (NOAA), National Geophysical Data Center, Boulder, Colorado; and U.S. Geological Survey (USGS), National Earthquake Information Center, Denver, Colorado. Ogata, Y. (1988). Statistical models for earthquake occurrences and residual analysis for point processes. J. Amer. Statist. Assoc. 83, 9-27. Ogata, Y. (1989). Statistical model for standard seismicity and detection of anomalies by residual analysis. Tectonophysics 169, 159-174.

344

Edwin Choi and Peter Hall

Ogata, Y. (1998). Space-time point-process models for earthquake occurrences. Ann. Inst. Statist Math. 50, 379-402. PDE (Preliminary Determination of Epicenters) Catalogue, (1997). Monthly Listings. U.S. Department of the Interior/National Earthquake Information Center, Denver, Colorado. Ramlau-Hansen, H. (1983). Smoothing counting process intensities by means of kernel functions. Ann. Statist. 11,453-466. Smith, R.L. (1992). Estimating dimension in noisy chaotic time series. J. Roy. Statist. Soc. Ser. B. 54, 329-351. Theiler, J. (1990) Statistical precision of dimension estimators. Phys. Rev. A. 41, 3038-3051. Vere-Jones, D. (1999). On the fractal dimensions of point patterns. Advances in Applied Probability 31, to appear. Wand, M.P. and Jones, M.C. (1995). Kernel Smoothing. Chapman & Hall, London. CENTRE FOR MATHEMATICS AND ITS APPLICATIONS AUSTRALIAN NATIONAL UNIVERSITY CANBERRA, ACT 0200, AUSTRALIA

halpstat@fac. anu. edu. au

A NOTE ON ESTIMATORS OF GRADUAL CHANGES

M. HUSKOVA

1

Charles University, Praha In the present paper we focus on the estimators in location models with gradual changes described by power α that can be known or unknown. Least squares type estimators of the parameters are studied. It appears that the limit behavior (both the rate of consistency and limit distribution) of the estimators of the change point in location models depends on the type of gradual changes. AMS subject classiβcations: 62G20, 62E20, 60F17. Keywords and phrases: gradual changes in location model, estimators.

1

Introduction and main results

Consider the location model with a change after an unknown time point mn\ (1.1)

Yi = μ + J n ( ^ - ^ ) ° ° + ehi = 1,...,n,

where α+ = max(0, α), μ, δn φ 0 and mn(< n) and αo £ [0,1] are unknown parameters. We assume that ei,..., en are independent identically distributed random variables Eei = 0, 0 < σ 2 < oo, E\ei\2+A < oo with some Δ > 0 and (1.3)

mn = [771] with some 7 E (0,1),

where [α] denotes the integer part of α. Concerning the slope parameter £ n , we assume that, as n —> 00,

(1.4)

|ίnhθ,

.f^l

->oo,

Vloglogn which covers local alternatives (δn -> 0), and if αo Φ 0 also fixed alternatives (δn = δφ 0). 1

This research was supported by the grant GACR-201/97/1163 and by the grant CES:J13/98:113200008.

346

M. Huskova

We study the least squares type estimators μn,δn,&n, ™>n of the paramm eters μ,ί n ,αo ? ίi These are defined as the minimizers over μ G i£, δ G ϋ , α G [0,1] and k = 1,..., n of

Straightforward (but tedious) calculations give that ( L 5 )

^ =

n( while αn and m n are the maximizers over α G [0,1] and k = l,....,n — lof

(Σ=ife*W)2) ' where

Σ() If there are more solutions we take the pair fhn,αn with the smallest first component. The parameter mn is the change point and it is the parameter of main interest. The parameter αo characterizes the type of change (abrupt - c*o = 0 or gradual αo > 0) and it is usually also of interest, whereas μ, δn and σ2 are nuisance parameter. The case when αo G [0,1] is known has been studied in the past, e.g. Horvath and Csδrgό (1997), Antoch and Huskova (1998) for survey results in case αo = 0 and Huskova (1999) for αo G (0,1], However, in reality α is usually unknown. Related test procedures with αo = 0 or αo = 1 are studied in a number of papers, see e.g. Hinkley (1971), Lombard (1987), Jaruskova (1998a,b), Csδrgδ and Horvath (1997), Siegmund and Zhang (1994). We will study here the limit behavior of the estimators fhn and α n and compare it with the limit behavior of the corresponding estimators if only one of these parameters is unknown. We use the notation, for 7 G (0,1) and α G (1/2,1] (or for [1,1] in the case of an), (1.8)

347

Estimators of Changes αn(α, 7 ) = ( / {x - Ί)2_?\og2(x Wo (1-9)

) dx

Ί +

- Jo(y -

α22(α,7) =α\j\x-

)^~2dx - (j\(y -

Ί

2 M

\{j\

Oί

αi2(«,7) =α2i(α,7) = «( / (^ ~ T ) ^ " 1 log(x Wo

(1.11)

- (J\y - Ί)α+-ιdy){j\z - Ί)% \oφ - Ί)+dz)) - α(j\x - rff-'dx - (J\(y - Ί)%-ιdy){j\z - Ί)%dz))

y

ti(x - Ί)\« log(x - Ί)+dx - Sj(y - 7) W Q ( Z ~ Ί)% logjz - -γ)+dz Jo1 (x - j)2+α log(x - Ί)+dx - (Jo1 (y -

)«+dyγ

Ί

The integrals can be easily calculated, however, the resulting expressions are neither simpler nor more transparent, e.g. an(0,7) = 7(1 — 7)(log(l — 7) — In the following, a J l (a,7) denote the elements of the inverse matrix We formulate the main results in three theorems that cover the cases. In Theorem 1.1 we consider αo Ξ [0? 1/2)? i n Theorem 1.2 we consider αo = 1/2 and finally in Theorem 1.3 the case αo £ (1/2,1]. Theorem 1.1 (α 0 E [0,1/2)) Let (1Λ)-(1.4) be satisfied. Ifα0 E [0,1/2), then, as n —> 00,

(1.13) where (1.14)

2

Vα = argmax{WQ(ΐ) - Γ ° ((x +1)% - x%) dx/2; t G R}, J-OO

348

M. Huskova

with {Wα{t)', t G R} being a Gaussian process with zero mean and covariance structure, for s,t E R, (1.15)

cov(W β (t), Wα(s)) = Γ

((x + s)% - x%) ((x +1)% - x%)dx,

J—oo

Ifαo=O

then, as n -> oo,

δny/n(αn

- α 0 ) - ^ max(y,0),

where Y has distribution N(0, σ 2 αJ" 1 1 (0,7)). The estimators δ n and m^ are asymptotically independent. T h e o r e m 1.2 ( α 0 = 1/2) Let (L1>-(L4) be satisfied. ίfα 0 = 1/2, then, as n -> oo? (1.13) holds true and (1.16)

d

v

^

The estimators άn and fhn are asymtotically

> independent.

T h e o r e m 1.3 ( α 0 G (1/2,1]) Let (1.1)-(1A) be satisfied. Ifα0 G (1/2,1), then, as n -» oo,

<M7> where Λ~ x (αo,7) denotes the inverse matrix to A(αo?7) If αo = 1, then, as n —> oo, and

δny/n(fhn - mn)/n -A M2, where Mi has distribution iV(0, σ 2 α^2"(l,7)) and M2 has distribution

lN(0,σ2α^(l,Ί))

+ iiV(0,σ2α22(l)7)).

The proofs are postponed to the next section. Here we discuss various consequences of the stated results. Note that the rate of consistency of αn depends neither on c*o nor mn while the rate of consistency fhn depends on αo The best rate of consistency of αn is reached for αo = 0, and the worst one for αo E (1/2,1]. This is in accordance with the results of Ibragimov and Hasminski (1981) concerning consistency of estimators in regular, almost regular and singular cases.

Estimators of Changes

349

If αo is known then comparing the results of the present paper with Theorems 1.1-1.2 in Huskova (1999) we realize that in case α 0 G [0,1/2] the limit behavior of fhn is the same as if αo is unknown. In case αo G (1/2,1] by Theorem 1.3 in Huskova (1999), as n -» oo, δny/τι(fhn — Tfij^jJTi —y iV(0, <τ αj~^ (αo,^y)), which means that the rate of consistency and the type of limit distribution remains the same, however, the variance is smaller in case of αo known. If the change point mn is known then it can be proved along the line of the proofs of Theorems 1.1-1.3 that (1.13) remains true for αo G [0,1] (not only for αo G [0,1/2] when mn is unknown). It can be shown that if αo is unknown then the estimator ran(α) defined as the maximizer over k = 1,..., n — 1 of

\ΣLiχik(<*)Yί\

with α G [0,1] prechosen, does not even need to be consistent. It should be pointed out that if αo G [0,1/2] the estimator fhn is not generally asymptotically optimal; there exists a Bayesian like estimator that performs better. These results together with some related problems will be discussed in a separate paper. The present results can be extended to other types of estimators, e.g. to M- and i?-estimators. Other type of gradual changes can also be considered, e.g. Y{ = μ + δng((mn — h)+/n\θ) + e^i — 1,...,n, where g(t\θ) is a known nonincreasing smooth function in t G [0,1) and a smooth function in 0; and θ is a nuisance parameter. The type of smoothness then determines the limit behavior of the estimators. Concerning the limit behavior of the estimators μn and ί n , it can be shown, using standard techniques, that their limit behavior is not affected by the fact that αo and mn are known or unknown.

2

Proofs

We start with some lemmas. The proofs of Theorems 1.1-1.3 are at the end of the section. Some of the lemmas are quite technical and require tedious calculations. We try to shorten them. Lemma 2.1 as n -* oo,

Let assumptions (1Λ)-(1.4) be satisfied. Then, for α G [0,1],

(2.1)

, ^ max

350

M. Huskova

(2.2)

max

1ΣLI^(^|

( l )

for any e E (0,1). Proof This is a slight modification of Lemma 2.5 in Huskova (1999) and therefore the proof is omitted. • L e m m a 2.2 Let assumptions (1.1)-(1.3) with α G [0,1] and (1.4) be satisfied. Then, as n -> oo, αn - A α 0 . (ra n - mn)/n

- A 0.

Proof Clearly, for k = 1, ...,n — 1, and α G [0,1],

Σ?=i 4 The first term on the right hand side of 2.3 is stochastic one and by Lemma 2.1 the maximum of its absolute values over A; and α is of order (log log n ) 1 / 2 . The second term on the right hand side is nonstochastic and elementary calculations give

=

Jo(s - 7)?(s - t)%dx - ft(x - Ί)%°dx(x - t)%dx

= Q(a,αo,t,7), (say), for ί € [0,1), 7 € (0,1), and α, αo G [0,1]. By the Schwarz inequality, for t e (0,1), 7 € (0,1), α, α0 G [0,1],

where equality holds only with α = αo, t = 7. The function Q(.) is continuous in all variables. This implies for αo? £ [0? 1]> and 7 G (0,1),

ίG(o,7-e)u(7+6,i) α€(0,αo-e)U(α0+e,l) α€[0fl]

L

Q 1 / 2 ( α , α, t, t)

351

Estimators of Changes

for e > 0 small enough. This in combination with the property of the stochastic term on the right hand side of 2.3 implies the assertion of our lemma. • To obtain stronger properties than the above consistency we have to investigate both stochastic and nonstochastic terms in more details. The estimators fhn and αn can be equivalently defined as the maximizers over k e { 0 , 1 . . . , n - 1} and α E [0,1] of

It is useful to decompose the single terms as follows

(

( Σ?=i *

}

= Ak + 2δn(Bkι

+ Bk2 + Bk3 + BM) - δ2n{Ckι + 2Ck2 + Ck3 + CM),

for k = 1, ...,n, α G [0,1], where

_

2=1

n

Σ?=l Σ?=i

Ck2 = Σ{xik{α)

- Ximn{α)){ximn{α)

- Zimn(c*o))

x

i=\ ik\

352

M. Huskova

Σ?=i Notice that

fcl

+ ... + Bk4}

2

= σ {Ckι + ... + Ck4).

In the next few lemmas we investigate the single terms C'ks and B'ks for α close to αo and k close m n . We start with C'ks. L e m m a 2.3 Let assumptions (1.1)-(1.4) with α$ G [0,1] be satisfied. Then, as α —> αo and | m n — fc|/n ->- 0, n -> oo: = (α - α o ) 2 α n ( α o , 7 ) ( l + o(|α - αoΓ + 1))

(2.6)

-Ckl

(2.7)

Icfc2 = ( α -

- (Jo1 (z x (l + o ( | α - α 0 | κ + l)). uniformly for \k — mn\ = o(n) for some K > 0. Proof We derive the assertion for one term only, since others are treated in the same way. We have α

mn\

fi-

Γ(f

= Γ(f\x-

= (α - α 0 ) f\x Jo

)f ]og(* -Ί)+dx)dβ

Ί

- 7)l Ω 0 Iog2(x - Ί)+dx(l + o(\α - <x)\κ + 1)),

Estimators of Changes

353

where we used elements of classical calculus. • Lemma 2.4 Let assumptions (Ι.l)-(IΛ) and n —>• oo, (1) forαe (1/2,1]: (2.9)

be satisfied. Then, as α -> α 0

uniformly for |A — m n | = o(n) for some K > 0. (2) for α = 1/2: (2.10) uniformly for \k — mn\ = o(n) for some K > 0.

(3) for αe [0,1/2): (2.11) i c f e 3 = 77.

oo

uniformly for \k — mn\ = o{n) for some n > 0. Proof The lemma is a slight generalization of Lemmas 2.2-2.4 in Huskova (1999) and therefore the proof is omitted. • Lemma 2.5 Let assumptions (1.1)-(1.4) be satisfied. Then for α G [0,1], as α -> αo and n -> oo, Bkι + Bk3 = (α - α o ) y n i ( l + o P (|α - α 0 Γ + 1)) uniformly for \k — mn\/n = o(l) for some n > 0, where (2.12)

Ynl =

o 1 ^ - 7 ) + α o log(g ~ 7)+
Proof The crucial part of the proof is to show that

( ( Ϊ Γ ) ί—1

+

( i r ) +)

= (α

~α o )

354

M. Huskova

Clearly,

is continuous in α and we have for all 0 < αi,α2 < 1 and n large enough

._- - -

ιv

- i

-

n

/+

/

^

τι

/+

for some C > 0. Hence by Theorem 12.3 in Billingsley (1968) we have that the sequence Sn = {S n (α),α £ [0,1]} is tight and also

)> = Σ( ( ^ x(l + op(\α-α0\κ

+ l))

uniformly for α -» αo and n —)• oo. The lemma is proved. • The following lemmas are slight generalizations of Lemma 2.6-2.9 in Huskova (1999); their proofs are omitted. Lemma 2.6 Let assumptions (Ι.l)-(IΛ) as α ->> αo &nd n —>> oo,

be satisfied. Then for αo € [0? 1]?

= - I 7 ^ - ^ y 2 n (i + O p ( | α _

Sjb4

αo

| - + i))

uniformly for \k — mn\/n = o(l) for some K > 0, where

(2 13)

ΐ = 1

/o1^ - Ί?r-ldx - ^(y - T)?- 1 ^ fo(* ~ Lemma 2.7

fαo ^ [1/2,1]) Let assumptions (1.1)-(1A) be satisfied. Then,

as α —> αo a n d n —> oo,

(1) for αo€ (1/2,1]: 71

355

Estimators of Changes uniformly in \k - mn\/n = o(ί) for some K > 0. (2.14)

γM-α

(2) for α 0 = 1/2 mn-kl

A

/ΐ~

uniformly for \k - mn\/n = o(l) for some « > 0 . Next, we introduce the process VnOLp = {Ki,α(ί); |*| < Γ } with α E [0,1/2), T being a positive number and = δnBk2,

k = l,....,n,

and piecewise linear otherwise. Lemma 2.8 (α 0 G [0,1/2)) Let assumptions (1.1)-(1.4) be satisfied. Then for any e > 0 and 77 > 0 there exist Hjη > 0, j = 1,2, and n^ such that for n > nη P(

max

\\α-αo\
max

, | , ^ ' >η)

e>|fc-mn|/n>fl-2τ7((J2n)1/(2αo+i) | d n | C f c 3

<η

/

and, as α —>• αo and n -^ 00,

where Wαo,τ = {Wao; \t\ < T} is a Gaussian process with zero mean and the covariance function given by (1.15). Proof of Theorem 1.3 The proof is divided into two steps. First we show that, as n —> 00 and α —> o>o (2.15)

(fhn - mm)/n =

(2.16)

1

OP{{δnVn)- ),

δn-αo

and then the limit distribution of fhn and 3 will be derived. By Lemma 2.2 it suffices to investigate max |α-αo|<e,|fc-mn|/τi<£

{Ak + 2δn(Bkl

+... + BkA) - δ2n(Ckι + Ck2 + 2Ck3

356

M. Huskova

for e > 0 small enough. Denoting GD = {(α,fc);|α - αo\ < D{\δn\V^)~\

\k - mn\/n < D{\δn\

and HD = {(α,fc);|α - α 0 < e, \k - mn\/n

<e}-GD

we notice that for any D > 0 and e > 0 small enough max{δl(Ckl

+ 2Ck2 + Ck3 + CkA)} = 0,

and by Lemmas 2.3-2.4 2Ck2 + 2Cks + C fc4 )} = m a x ( - D 2 ( α n ( α 0 , 7 )

^

- £ ) 2 α n ( α o , 7 ) , -D2α22{α0,7)}

x (1 + ^(1))

where αij(αo^) are defined by (1.9)—(1.11). Clearly, αn(αo,7)+2αi2(αo,7)+ ^22(^057) > 0. By assumption (1.4) and by Lemma 2.1 we find that the terms A^'s are negligible and also that for D > 0 large enough the nonstochastic terms δ^(Ckι + Ck2 + 2Cks + Ck^) dominate the stochastic terms δn(Bkι + Bk2 + Bks + Bk4) for (α, k) G Hp with probability close to 1. Since max\δn\\Bkι

+ ... + BkA\ = Op{\δn\y/n)

and since D can be chosen arbitrarily large we find that (2.15)-(2.16) hold true. In order to obtain the limit behavior of our estimators we investigate the maximum of (2.17)

2δn(Bkι

+ ... + Bk4) - δ2n(Ckl + 2Ck2 + C f c 3 + CkA)

over the set G/> a n Writing α = c*o + hiδnVn)"1 d k = m + t2n(δny/n)~ι we get by Lemmas 2.3-2.7 that our problem reduces to investigating the maximum of

with respect to t\ and t2. Since A(αo5 7)? defined in (1.8) is a positive definite matrix and since by the CLT (Yί n /\/n, (Y2n + Y3n)/V™) has asymptotically ΛΓ((O,O)τ,σ2A(αo,7)) distribution, we find after some standard steps that the assertion of Theorem 1.1 holds true. • Proof of Theorem 1.2 We proceed similarly as in the proof of Theorem 1.3. Checking the behavior of B^'s and CVs for αo = 1/2 (Lemmas 2.32.8) we realize that, as n —> 00 and α —>• αo, fc3

( ( ^ ) g (

n

) ) ,

CM = o(Ck3),

357

Estimators of Changes

_L β f c 2 = O p ( ] * z p l ( l o g ( n _ m n ) ) i / 2 ) 5

Bk4 = Op{Bk2)

uniformly for k - mn = o(n). The terms Ck\,Ck2 and Bk\,Bk$ are not affected in this way, which leads to the conclusion that the rate of consistency of αn is the same as in case α0 G (1/2,1] while for fhn we have, as n -» oo and α -> αo ^ ^ V ^ ) " 1 log~ 1 / 2 (n -

)

Moreover, it is enough to study the maximum of 2δn(Bkι

+ Bk2 + Bk3) - δ2n(Ckl + Cks)

over a properly modified set Hp. The proof is now finished in the same way as of Theorem 1.3. • Proof of Theorem 1.1 This is omitted since it is in principle the same as that of Theorem 1.2. • Acknowledgements The author thanks two anonymous referees for their helpful comments.

REFERENCES Antoch, J., Huskova, M. and Veraverbeke, N., (1995). Change - point estimators and bootstrap. J. Nonpαrαm. Statist. 5, 123 - 144. Billingsley, P., (1968). Convergence of Probability Measures. Wiley, New York. Csorgό, M., and Horvath, L.,(1997). Limit theorems in change point analysis. Wiley, New York. Hinkley D., (1971). Inference in two-phase regression. J. Amer. Statist. Assoc. 66, 736 - 743. Huskova, M., (1999). Gradual changes versus abrupt changes. To appear in Journal Statististical Planning and Inference. Ibragimov, LA., and Hasminski, R.Z., (1981). Statistical Estimation: Asymptotic Theory. Springer Verlag, New York. Jaruskova, D., (1998a). Testing appearance of linear trend. Journal Statistical Planning and Inference 70, 263 - 276. Jaruskova, D.,(1998b). Change-point estimator in gradually changing sequences. CMUC39, 551 - 561.

358

M. Huskova

Lombard, F.,(1987). Rank tests for change point problem. Biometrikα 74, 615 - 624 Siegmund, D., and Zhang, H., (1994). Confidence regions in broken line regression. In Change-point problems 23. IMS Lecture Notes - Monograph Series, 292 - 316. DEPT. OF STATISTICS CHARLES UNIVERSITY SOKOLOVSKA 83

CZ-186 00 PRAHA CZECH REPUBLIC

huskova @karlin. mff. cuni. cz

ESTIMATION OF ANALYTIC FUNCTIONS

I. IBRAGIMOV

1

St.Petersburg Branch of Steklov Mathematical Institute Russian Ac.Sci. In this paper we present a review of some results on nonparametric estimation of analytic functions and in particular derive minimax bounds under different conditions on these functions. AMS subject classifications: 62G05, 62G20. Keywords and phrases: Nonparametric estimation, Minimax bounds.

1

Introduction

The aim of this paper is to present a review of some results about nonparametric estimation of analytic functions. A part of it is written in exposition style and summarizes some recent work of the author on the subject (detailed versions have already been published). The rest of the paper contains new results in the area. Sometimes the proofs are only outlined and will be published elsewhere. Generally the problem looks as follows. We are given a class F of functions defined on a region D C Rd and analytic in a vicinity of D. It means that all / G F admit analytic continuation into a domain G D D of the complex space Cd. To esimate an unknown function / G F one makes observations Xε. Consider as risk functions of estimators / for / the averaged Lp(D)-norms

J/(t)-/(t)|Λj

,

l
- /lloo = E{sup |/(a:) - /(x)| D

where in the case of noncontinuous / the supremum is understood as an essential supremum. Put

Δp(ε,F) = Δ(F) = inf supE/ll/ - /|| p . 1

This work is partly supported by the Russian Foundation for Basic Research, grants 99-01-00111, 96-15-96199, 00-15-96019 and INTAS, grant 95-0099, 99-1317.

360

I. Ibragimov

The last expression gives some understanding of our estimation problem and below we study the asymptotic (ε -» 0, say) behaviour of Δ p (ε, F ) . The rate of convergence of Δ depends on F and the understanding of this dependence is the main interest of the paper. Of course the problem is formulated too general and to have reasonable results we are to consider concrete variants of it. Below we are dealing with the following problems. Problem I An observed signal Xε on the interval [α, b] is of the form

dXε{t) = f(t)dt + εdw(t). Here w(t) is the Wiener process, ε is a known small parameter and the unknown function / belongs to a known class F of functions analytic in a vicinity of [α,&]. Problem II Assume that one observes iid random variables Xi, X<ι,... Xn taking values in ϋ 1 and having a density function / G F where again F consists of functions analytic in a vicinity of an interval D = [α, 6]. The problem is to estimate f on the interval [α, b]. Problem III Let {Xt} be a real-valued stationary Gaussian process with discrete or continuous time f, mean zero and spectral density /(λ). We assume that the spectral density / is unknown and should be estimated on the base of observations Xt,0 < t < T. Assume further that / G F where F is a given (known) class of spectral densities. Let [α, 6] be an interval and assume that we would like to estimate the restriction of / to [α, 6], {/(λ), α < λ < &}, when all / E F are analytic in a vicinity of [α, b]. Problem IV Let / be an unknown function belonging to a known class F of functions analytic in a vicinity of an interval [α, b]. To estimate / one makes n observations of the function / at the points -XΊ, X<ι, Xn and observes

where Έ(Gj{Xj^ω)\Xι^ -Xj-i) = 0 and the noise variables Gj are conditionally independent under a given observation plan (see in more detail

One may ask why such attention to the estimation of analytic functions, isn't the class of such functions too special and too narrow? I believe that apart of their mathematical beauty these problems are rather natural. I appeal to two arguments of authority. "When I asked Hubert what were the general considerations which brought him to the hypothesis (proved later by myself) that all solutions

361

Estimation of Analytic Functions

to regular problems of the calculus of variations are analytic, the celebrated geometer answered that by his opinion solutions to all naturally posed problems should be analytic." (S. Bernstein, Sur la deformation de surfaces, Ann.Math., 1905). "It may seem that analytic functions are a refined, a recherche object and that such a refined property as analyticity is not necessary in numerical analysis. In fact it is not true. Analytical functions constitute a very disseminated object and many natural scientific problems lead to analytic functions. Ignoring the analytic nature of solutions we narrow drastically our possibilities and operate squanderingly." (K. Babenko, Foundations on numerical analysis). Notice also that in the case of stationary processes a fast decay of the correlation function implies the analyticity of the spectral density. If the correlation radius is finite, the spectral density is even an integer function. Below we denote through c, C with or without indices various constants. In different places the same notation may stay for different constants. 2

Functions analytic on bounded regions

In this section we suppose that the class F consists of functions analytic on the bounded region G D [α, b] of the complex plane and bounded there by a constant M . The region G and the constant M are supposed to be known. Thus rb

Δf>(P)=infsup(/

\f{t)-f(t)\pdt)

l/p

T h e o r e m 2.1 Under the conditions of Problem I when ε -» 0 the minimax risk Δ P ( F , ε) satisfies the following asymptotic relations Δp(F))xε0n(l/ε),

1
Δ 4 (F) x εy^l/εXlnlnU/ε)) 1 / 4 ,

(2.1)

ι 2 p

Δ P (F) x ε(ln{l/ε)) - / ,

4 < p < oo.

The constants under the x sign depend on G, M, and p only. Recall that the sign " α n x 6 n " means that 0 < c = liminf(α n /6 n ) < limsup(α n /b n ) = C < oo. T h e o r e m 2.2

Under the conditions of Problem II when n —> oo the

362

I. Ibragimov

minimax risk Δ p ( F , n) satisήes the following asymptotic relations

Δ4(F) >c V. nf e Δ P (F)

4 < p < oo.

The constants depend on G, M, and p oniy. Theorem 2.3 Let Xt be a stationary gaussian process with discrete time. Under the conditions of Problem III when T —>• oo the minimax risk Δ P (F, T) satisβes the following asymptotic relations

Δ 4 (T) 4
p

<

1