Weighted Empirical Processes in Dynamic Linear Models, Second Edition (Lecture Notes in Statistics 166)

J

(1.2.1)


Hn(y, t)

:=

n- I

L I(Yni ~ y + x~it), i=1

Y E~, t E W.

1.2. M-, R- and scale estimators

3

A significant portion of non parametric inference in models (1.1.1) deals with M- and R- estimators of f3 (Adichie; 1967. Huber; 1973) and linear rank tests of hypotheses about f3 (Hajek - Sidak; 1967) . By viewing these procedures as functions of {V(y ,t) ,y E IR,t E W}, it is possible to give a unified treatment of their asymptotic distribution theory, as is done in Chapters 3 and 4 below . There is a vast literature in nonparametric inference that discusses inferential procedures based on functionals of empirical processes in the k-sample location model such as the books by Puri and Sen (1969), Serfli.ng (1980) and Huber (1981) . Yet their appropriate extensions to the linear regression model are not readily accessible. This monograph seeks to fill this void . The methodology and inference procedures studied here extend many known results in the k-sample location model to the model (1.1.1), thereby giving a unified treatment . An important result needed for study of the asymptotic behavior of R-estimators of f3 is the asymptotic uniform linearity of the linear rank statistics of (1.2.1) in the regression parameter vector. Jureckova (1969, 1971) obtained this result under (1.1.1) with i.i.d . errors. A similar result was proved in Koul (1969, 1971) and van Eeden (1972) for linear signed rank statistics under i.i.d. symmetric errors . Its extension to the case of non-identically distributed errors is not readily available. Theorems 3.2.4 and 3.3.3 prove the asymptotic uniform linearity of linear rank and linear signed rank statistics with bounded scores under the general independent errors model (1.1.1). In the case of i.i.d. errors, the conditions in these theorems on the error d .f. are more general than requiring finite Fisher information for location. The results are proved uniformly over all bounded score functions and are consequences solely of the asymptotic sample continuity of V - processes and some smoothness of {Fnd . The uniformity with respect to the score functions is useful when constructing adaptive rank tests that are asymptotically efficient against Pitman alternatives for a large class of error distributions. Chapter 3 also contains a proof of the asymptotic normality of linear rank and linear signed rank statistics under independent alter-

1. Introduction

4

natives and for indicator score functions. This proof proceeds via the weak convergence of certain basic W.E.P. 's and complements some of the results in Dupac and Hajek (1969). Section 4.2.1 discusses the asymptotic distribution of M- estimators under heteroscedastic errors using the asymptotic continuity of V- processes. Section 4.2.2 presents some second order results on bootstrap approximations to the distribution of a class of Mestimators. In order to make M-estimators scale invariant one often needs an appropriate robust scale estimator. One such scale estimator, as recommended by Huber (1981) and others, is 81

=

m ed{lYni - x~iJI, 1:S i :S n} ,

where i3 is an estimator of {3. The asymptotic distribution of 8 1 under heteroscedastic errors is given in Section 4.3. If the errors ar e i.i.d. , this asymptotic distribution does not dep end on i3 provided the common error distribution is symmetric around O. This observation naturally leads one to construct a scale estimator based on the symmetrized residuals , thereby giving another scale estimator 82

:=

m ed{lYni - x~ii3 - Yni + x~ii3l ; 1 :S i, j :S n} .

As expected , the asymptotic distribution of 82 is shown to be free from the estimator i3 in the case of i.i.d . errors , not necessarily symmetric. It also appears in Section 4.3. Section 4.4 discusses the asymptotic distribution of a class of R-estimators under heteroscedastic errors using the asymptot ic uniform linearity results of Chapter 3. The R-estimators considered ar e asymptotically equivalent to Jaeckel's estimators. The complete rank analysis of the linear regression model (1.1.1) requires an estimate of the scale parameter Q(J) :=

!

fdrp(F)

where f is density of the unknown common error d .f. F and sp is a nondecreasing function on (0,1) . This estimate is used to standardize the test statistic and estimate the standard error of the R-estimator

1.3. M.D . ESTIMATORS & GOODNESS-OF-FIT TESTS

5

corresponding to the score function cpo This parameter also appears in the efficiency comparisons of rank procedures and it is of interest to estimate it , after the fact, in an analysis. Lehmann (1963) , Sen (1966) , Koul (1971), among others, provide estimators of QU) in the one - and two - sample location models and in the linear regression model. These estimators are given in terms of the lengths of Lebesgue measures of certain confidence intervals or regions. They are usually not easy to compute when the dimension p of f3 is larger than 1. In Section 4.5, estimators of QU) , based on kernel type density estimators of f and the empirical d.f H n , are defined and their consistency under (1.1.1) with i.i.d. errors is proved. An estimator whose window width is based on the data and is of the order of square root n, is also considered. The consistency proof presented is a sole consequence of the asymptotic continuity of certain W.E .P.'s and some smoothness of the error d.f. 'so

1.3 1.3.1

M.D. Estimators & Goodness-of-fit tests Minimum distance estimators

The practice of obtaining estimators of parameters by minimizing a certain distance between some functions of observations and parameters has been present in statistics since its beginning. The classical examples of this method are the Least Square and the minimum Chi Square estimators. The minimum distance (m.d.) estimation method, where one obtains an estimator of a parameter by minimizing some distance between the empirical d.f. and the modeled d.f, was elevated to a general method of estimation by Wolfowitz (1953, 1954, 1957). In these papers he demonstrated that, compared to the maximum likelihood estimation method, the m.d. estimation method yielded consistent estimators rather cheaply in several problems of varied levels of difficulty. This methodology saw increasing research activity from the mid 1970's when many authors demonstrated various robustness proper-

1. Introduction

6

ties of certain m.d. estimators. See, e.g., Beran (1977, 1978), Parr and Schucany (1979), Millar (1981, 1982, 1984), Donoho and Liu (1988 a, b) , among others. All of these authors restrict their attention to the one sample setup or to the two sample location model. See Parr (1981) for additional bibliography on m.d. estimation till 1980. In spite of many advances made in the m.d. estimation methodology in one sample models, little was known till late 1970's as to how to extend this methodology to one of the most applied models, v.i.z., the multiple linear regression model (1.1.1). A significant advantage of viewing the model (1.1.1) through V is that one is naturally led to interesting m.d. estimators of f3 that are natural extensions of their one- and two- sample location model counterparts. To illustrate this, consider the m.d. estimator {j of the one sample location parameter 0, when errors are i.i.d. symmetric around 0, defined by the relation

{j:= argmin {Tn(t); t E JR}, with

Tn(t) = !{n-l/2tI(Yni::; y+t)-I(-Yni < y-t)]}2dG(y), t E JR, i=l

where G E 'DI(JR). Since (1.1.1) is an extension of the one sample location model, it is only natural to seek an extension of {j in this model. Assuming that {eni} are symmetrically distributed around 0, the first thing one is tempted to consider as an extension of (j is defined by the relation

e;

f3t := argmin{Kt(t) ; t

E

JRP},

with

Kt(t)

.-

! {n-

1

/

2t

[I(Yni ::; y +

x~it)

i=l

- I (- Yni < y -

x~i t)]

r

dG (y), t E JRP.

However, any extension of {j to the linear regression model should have the property that it reduce to {j when the model is reduced to

1.3.1. Minimum distance estimators

7

the one sa mple locati on model and , in addit ion, t hat it redu ce to an appropriate extens ion of {; to the k-sample locati on model when the model (1.1.1) is reduced to it. In t his sense f3t does not provide t he right exte nsion but f3 j( does, where

f3 j(: = arg min{K i(t) ; t E JRP },

(1.3.1)

wit h

Ki (t) y+'

.-

J

.-

(V1+ , ... , V/ ),

Y +'( y, t )(X'X)-ly+ (y, t )dG(y) , t E IRP,

n

Vj+(y, t ) .- Vj( y,t) - L Xnij

+ Vj( -y ,t ),

i= l

for 1 ::; j ::; p , y E ~, t E W. In the case errors are not symmet ric bu t i.i.d . according to a known d .f. F , so that EVj( y, (3 ) == L i xnij F (y), a suitable class of m.d. estimators of f3 is defined by t he relati on (1.3.2)

{3x

K x (t)

' -

.-

ar gmin {K x(t); t E JRP },

J

2

II W (y, t )11 dG(y ),

W (y, t ) .- (X' X) - 1/2{y(y , t ) - X'IF (y )}, I' .- (1" " , 1h xn .

y E ~, tE JRP ,

Chapter 5 discusses t he existe nce, the asymptot ic distribution, t he robustness and the asymptotic optimality of f3j( and {3x under (1.1.1) with het eroscedastic errors. For exa mple, if p = 1 in (1.1.1) and the design vari abl e is nonnegative then th e asy mptotic variance of f3 j( is smaller th an t hat of f3t for a large class of symmetric error d.I. 's F and integrating measures G. A similar resu lt holds abo ut {3x and for p ~ 1. Chapter 5 also discusses severa l ot her m.d. estimators of f3 and t heir asymptotic th eory under (1.1.1) wit h heteroscedastic errors . T hese include analogues of {3x when t he common error d.f. is unk nown and some m.d. est imators correspo nding to certain supremum distances based on Y.

1. Introduction

8

1.3.2

Goodness-of-fit testing

Closely related to the problem of m.d. estimation is the problem of testing the goodness-of-fit hypothesis H o : Fn i == Fo, Fo a known d.f.. One test statistic for this problem is

fh := supln 1 / 2{Hn (y,,8) - Fo(y)}l,

(1.3.3)

y

where ,8 is an estimator of f3 . This test statistic is suggested by looking at the estimated residuals and mimicking the one sample location model technique. In general, its large sample distribution depends on the design matrix. In addition, it does not reduce to the Kiefer (1959) tests of goodness-of-fit in the k-sample location problem when (1.1.1) is reduced to this model. Test statistics that overcome these deficiencies are

b,

:=

sup IWo(y, ,8)1,

fh

:=

y

sup IIWo(y,,8)11 , y

where WO is equal to the W of (1.3.2) with F = Fo. Another natural class of tests is based on K~,.(,8x), where K& is equals to the Kx of (1.3.2) with W replaced by WO in there. All of the above and several other goodness-of-fit tests are discussed at some length in Chapter 6. Section 6.2.1 discusses the asymptotic null distributions of the supremum distance statistics Dj ,j = 1,2 ,3. Also discussed in this section are asymptotically distribution free analogues of these tests, in a sense similar to that discussed by Durbin (1973, 1976) and Rao (1972) for the one-sample location model. Section 6.2.2 discusses smooth bootstrap approximations to the null distributions of tests based on W .E.Po's . Tests based on L 2 - distances are discussed in Section 6.3. Some modifications of goodness-of-fit tests when F o has a scale parameter appear in Section 6.4 while tests of the symmetry of the errors are discussed in Section 6.5.

1.3.3

Regression model fitting

Another problem of interest is that of fitting a regression model. More precisely, let X, Y be r .v .ts with X being p-dimensional. In

1.5. Randomly weighted etapixicels

9

much of the existing literature the regression function is defined to be the conditional mean function /-l(x) := E(YIX = x), xE JRP , assuming it exists, and then one proceeds to fit a parametric model to this function, i.e., one assumes the existence of a parametric family

M = {m(x, 0) : x E W , 0 E 8} of functions and then proceeds to tests the hypothesis

/-l(x) = m(x, ( 0 ) ,

for some 00 E 8 and V x EX,

based on n i.i.d. observations {(Xi ,Yi); 1 ::; i ::; n} on (X, Y), where X is a compact subset of JRP . We consider the problem of fitting the model M to the 'ljJ- regression function defined as follows. Let 'ljJ be a nondecreasing real valued function such that EI'ljJ(Y - r)1 < 00, for each rElit Define the 'ljJ-regression function m'lj; by the requirement that

E['ljJ(Y - m'lj;(X)) IX]

= 0, a.s.

Observe that, if 'ljJ(x) == 'ljJo.(x) := I(x > O)-(l-a) , for an 0 < a < 1, then m'lj;(x) == mo.(x), the a th quantile of the conditional distribution of Y, given X = x, and if 'ljJ(x) == x, then m'lj; = /-l. Tests for fitting a model to m'lj;, i.e., for testing the hypothesis

H o : m'lj;(x) = m(x, (0),

for some 00 E 8 and "Ix EX,

will be based on the weighted empirical process n

Sn,'Ij;(x,O)

1 2

:= n- /

L: 'ljJ(Yi - m(Xi, 0)) I(£(Xd ::; x), i=l

where x E [-00, (0), l is a known function from ~p to ~, and 0 is a n 1 / 2 _ consistent estimator of 0 0 . This process is an appropriate extension of the usual partial sum process useful in testing for the mean in the one sample problem to the current regression setup. Section 6.6 discusses the weak convergence of this process and gives a transform of this process whose asymptotic null distribution is known, so that tests based on this transformed process are asymptotically distribution free.

1. Introduction

10

1.4

R.W.E. Processes and Dynamic Models

A randomly weighted empirical process (RW.E .P.) corresponding to the r.v. 's (nl, ' " (nn , the random noise bnl , ' " , bnn and the random real weights h nl, .. . ,hnn is defined to be n

(1.4.1)

Vh(X)

:= n-

l

I: hniI((ni :s; x + bnd ,

x E JR, n ::::: 1.

i =l

Examples of RW.E.P.'s are provided by the W.E .P.'s {Vj; 1 :s; j :s; p} of (1.1.2) in the case the design variables are random. More importantly, RW .E .P.'s arise naturally in autoregressive models. To illustrate this let Yo = (X o, ' " , X l - p)1 be an observable random vector. In the pth order linear autoregressive (LAR(p)) model one observes {Xi} obeying the relation (1.4.2) for some P = (PI , '" , pp)' E JRP , where {Ei ,i::::: I} are i.i .d , r.v .'s, independent of Yo . Processes that play a fundamental role in the robust estimation of P in this model are randomly weighted residual empirical processes T = (T I , ' " ,Tp )' , where, for x E JR, t E W , n

(1.4.3)

Tj(x , t)

:=

n-

I

I: gj(Yi -dI(X

i

:s;

X

+ t'Y i - l ) ,

i =I

with Y~-l (Xi-I, ..., Xi-p) , i ::::: 1, and where the components gj , 1 :s; j :s; p , of the vector g' = (gl, ' " ,gp) , are p measurable functions, each from JRP to JR. Clearly, for each 1 :s; j :s; p, Tj (x, P + n- l / 2t ) is an example of Vh(X) with (ni == Ei , bni == n- l / 2 t'Y i _ l and hni == 9j(Yi-d · It is customary to expect that a method that works for linear regression models should have an analogue that will also work in autoregressive models. Indeed the above inferential procedures based on W.E.P.'s in linear regression have perfect analogues in LAR(p) models in terms ofT. The generalized M-estimators of P as proposed by Denby and Martin (1979) corresponding to the weight function 9

1.5. Randomly weigbted empiricals

11

and the score function 'ljJ are given as a solution t of the p equations

J

'ljJ(x)T(dx, t)

= 0,

assuming that E'ljJ(c) = O. Clearly, the classical least square estimator is obtained upon taking g(y) == y , 'ljJ(x) == x in these equations. A generalized R-estimator PR corresponding to a score function sp is defined by the relation

PR := argmin {IIS(t)ll ;t

(1.4.4)

E

W},

where

S(t)

J

.-

cp(Fn(x, t))T(dx, t), n

Fn(x, t)

n-

:=

1

L I(X

i ::; X

+ t'Yi -d ,

x E JR, t E W.

i=l

An analogue of an R-estimator of (1.2.1) is obtained by taking g(x) == x in (1.4.4). The m.d. estimators pi that are analogues of'si of (1.3.1) are defined as minimizers, w.r.t . t E W , of

L J[n- L: X i_ j { I (X i ::; X+t'Yi-l) n

p

K+(t)

:=

1 2 /

j=l

i =l

- I( <X, <

X -

t'Yi -

1 ) } ] 2 dG(x) .

Observe that K+ involves T corresponding to 9j(Y i - 1 ) == Xi-j . Chapter 7 discusses these and some other procedures in detail. Section 7.2 contains a result that establishes the asymptotic uniform linearity in t of the R.W.E.P.'s {T(x, p+n- 1 / 2 t ), x E JR, Iltll ::; b} and the residual empirical processes {Fn(x , p + n- 1 / 2 t ), x E JR, litH::; b}, for every 0 < b < 00 . These results are used to investigate the asymptotic behavior of G-M- and G-R- estimators in Sections 7.3.1 and 7.3.2 respectively. In order to carry out the rank analysis in LAR(p) models, one needs a consistent estimator of Q(J) where now f is the error density of {ci}. A class of such estimators is given

12

1. Introduction

in Section 7.3.3. A large class of m.d. estimators and their asymptoties appears in Section 7.4. Section 7.5 discusses an extension of an important methodology of the regression quantiles of KoenkerBassett (1978) and the related rank scores to linear autoregression. Section 7.6 briefly discusses some tests of goodness-of-fit hypotheses pertaining to the error d.f, in linear AR models. The problem of fitting a given parametric autoregressive model of order 1 to a real valued stationary ergodic Markovian time series Xi, i = 0, ±1, ±2, '" is discussed in Section 7.7. Much of the development here is parallel to that of Section 6.6 above . Define the 'l/J-autoregressive function m1/J by the requirement that

E['l/J(X1

-

m1/J(Xo))IXo]

= 0, a.s.

Tests of goodness-of-fit for fitting the model M to m'/J are to be based on the process ~

n -1 '2 '"""'

M n, 1/J(x, lJ) := n i L 'l/J(Xi

-

m(Xi -

1,

~ lJ)) I(Xi -

1 ~

x),

i=l

for x E [-00,00] . Section 7.7 discusses a martingale type transform of this process whose asymptotic null distribution is known . The decade of 1990's has seen an exponential growth in the applications of nonlinear AR models to economics, finance and other sciences. Tong (1990) illustrates the usefulness of homoscedastic dynamic AR models in a large class of applied examples from physical sciences while Gourieroux (1997) contains several examples from economics and finance where the ARCH (autoregressive conditional heteroscedastic) model of Engle (1982) and its various generalizations are found useful. Most of the existing literature has focused on developing classical inference procedures in these models. The theoretical development of the analogues of the estimators discussed above that are known to be robust against outliers in the innovations in linear AR models has relatively lagged behind. Chapter 8 discusses the asymptotic distributions of the analogues of M-, R-, and m .d.- estimators and sequential empirical processes of residuals for a class of nonlinear AR and ARCH time series models. The main

1.5. Randomly weighted empiricals

13

point of the chapter is to exhibit the significance of the weak convergence approach in providing a unified methodology for establishing these results for non-smooth underlying score functions 1/J and .p, The contents of Chapter 2 are basic to those of Chapters 3, 4, 7, 8 and parts of Chapter 6. Sections 2.2.1 , 2.2.2 and 2.2.3, contain, respectively, proofs of the weak convergence of suitably standardized W .E.P.'s, R.W.E.P.'s and the partial sum processes an to continuous Gaussian processes. Even though W.E.Po's are a special case of R.W.E.Po's, it is beneficial to investigate their weak convergence separately. For example, the weak convergence of Ud is obtained under a fairly general independence setup and minimal conditions on {dni} whereas that of Vh is obtained under some hierarchal dependence structure on {ryni' hni , 8ni} and the boundedness and/or the square integrability of the weights {hni}. The process an is different from the Vh process because in these processes the weights are the innovations, whereas in the weighted empiricals process of innovations, the jumps occur at the innovations. In Section 2.3, the asymptotic continuity of certain standardized W.E.P.'s is used to prove the asymptotic uniform linearity of V(., t) in t , for t in certain shrinking neighborhoods of 13, under fairly general independent , non-identically distributed errors. This result is found useful in Chapter 4 when discussing M- estimators and in Chapter 6 when discussing supremum distance test statistics for goodness-of-fit hypotheses. The asymptotic continuity is also found useful in Chapter 3 to prove various results about rank and signed rank statistics under heteroscedastic errors. The asymptotic continuity of Vh - processes is found useful in Chapters 7 and 8 when discussing the AR and ARCH models . Chapter 2 concludes with results on functional and bounded laws of the iterated logarithm pertaining to certain W.E.Po's. It also includes an inequality due to Marcus and Zinn (1984) that gives an exponential bound on the tail probabilities of W .E.Po's of independent r.v .ts. This inequality is an extension of the well celebrated Dvoretzky, Kiefer and Wolfowitz (1956) inequality for the ordinary empirical process. A result about the weak convergence of W .E.Po's when r .v.ts are p-dimensional is also stated. These results are in-

14

1. Introduction

eluded for completeness , without proofs. They are not used in the subsequent sections. A martingale property of a properly centered Ud process is proved in Section 2.4.

2 Asymptotic Properties of W.E.P. 's 2.1

Introduction

Let, for each n ~ 1,17nl , ' " -Tlnn. be independent LV .'S taking values in [0, 1] with respective d.f.'s G n1, ... , G nn and dn1, .. . , dnn be real numbers. Define n

(2.1.1)

Wd(t) =

L dni{I(17ni :::; t) -

Gni(t)} , 0:::; t

< 1.

i =l

Observe that Wd belongs to V[O, 1] for each n and any triangular array {dni, 1 :::; i :::; n}, while Vh of (1.4.1) belongs to V(IR) for each n and any triangular array {h ni, 1 :::; i :::; n}. In this chapter we first prove certain weak convergence results about suitably standardized Wd, Vh and an processes. This is done in Sections 2.2.1, 2.2.2, and 2.2.3 , respectively. Section 2.3.1 uses the asymptotic continuity of a certain W d - process to obtain the asymptotic uniform linearity result about V(- , u) of (1.1.2) in u. Analogous result for T(·, u) of (1.4.3) uses the asymptotic continuity of a certain Vh - process and is proved in Section 7.2. The weak convergence results about an is used in Section 7.6 to derive the limiting distribution of some tests for fitting an autoregressive model to the given time series . H. L. Koul, Weighted Empirical Processes in Dynamic Nonlinear Models © Springer-Verlag New York, Inc 2002

2. Asymptotic Properties of W .E.P. 's

16

A proof of an exponential inequality for a stopped martingale with bounded differences due to Johnson, Schechtman and Zinn (1985) and Levental (1989) is included in Section 2.2.2. This inequality is of a general interest . It is used to carry out a chaining argument pertaining to the weak convergence of Vh with bounded h. Section 2.4 treats laws of the iterated logarithm pertaining to W d , the weak convergence of Wd when hd are in [O,lJP, the weak convergence of Wd w.r.t . some other metrics when {1Jd are in [0,1], an embedding result for Wd when {1Jd are i.i.d . uniform [0, 1J r.v.'s and a proof of its martingale property. It also includes an exponential inequality for the tail probabilities of W.E.P.'s of independent r. v.'s. This inequality is an extension of the well celebrated Dvoretzky, Kiefer and Wolfowitz (1956) inequality for the ordinary empirical process. These results are stated for the sake of completeness, without proofs. They are not used in the subsequent sections.

2.2 2.2.1

Weak Convergence Wd

-

processes

In this section we give two proofs of the weak convergence of suitably standardized {W d } to a limit in e[O, 1J. Accordingly, let n

(2.2.1) Gd(t)

:=

I: d;iGni(t) , i=l n

Cd(S, t)

:=

I: d;i[Gni(S

1\

t) - Gni(S)Gni(t)], 0:::; s, t :::; l.

i=l

Let p denote the supremum metric . Theorem 2.2.1 Let {1Jnd , {dnd and {Gnd be as in Section 2.1 . In

addition assume that the following hold;

17

2.2.1. Weak convergence ofWd

Then, [or every (i)

E

> 0,

lim lim sup P (sup 0-+0

It-sl
n

(ii) Moreover, Wd

I Wd(t)

- Wd(S)

I> E)

= O.

some W on (1)[0,1], p) ij and only ij [or every 0 ::; s, t ::; 1, Cd(s,t) converges to some covariance junction C(s, t). In this case W is necessarily a continuous Gaussian process with covariance junction C and W(O) = 0 = W(l). =}

Remark 2.2.1 Perhaps a remark about the labeling of the conditions is in order. The letter N in (N1) and (N2) stands for Noether who was the first person to use these conditions to obtain the asymptotic normality of certain weighted sums ofr.v .'s . See Noether (1949) . The letter C in the condition (C) stands for the specified conti-

nuity of the sequence {G d}. Observe that the d.f.'s {Gd need not be continuous for each i and n; only {G d } needs to be equicontinuous in the sense of (C). Of course if {7Jd are i.i.d. G then, because of (N1), (C) is equivalent to the continuity of G. 0 The proof of the theorem will be given after the following two lemmas.

Lemma 2.2.1 For any 0 ::; s ::; t ::; u ::; 1 and each n

~

1

E!Wd(t) - Wd(s)12IWd(U) - W d(t)!2

(2.2.2)

< 3 [Gd(U) - Gd(t)][Gd(t) - Gd(S)]

(2.2.3)

::; 3 [Gd(U) - Gd(s)f

Proof. Fix 0 ::; s ::; t ::; u ::; 1 and let

Pi = Gi(t) - Gi(S) , qi = Gi(u) - Gi(t) , ai = I(s < 7Ji ::; t) - Pi, f3i = I(t < 7Ji ::; u) - % 1 ::; i ::; n. Observe that Ea, = 0 = Ef3j for all 1 ::; i, j ::; n, {ad are independent as are {f3i} and that o; is independent of f3j for i =/= j. Moreover,

Wd(t) - Wd(S) =

L diai, i

Wd(U) - Wd(t) =

L dif3i · i

2. Asymptotic Properties of W .E.P. 's

18

Now expand and multiply the quadratics and use the above facts to obtain

(2.2.4) EIWd(t) - Wd(s)12IWd(U) - Wd(t)\2

"D d~2 ECt~f3? 2 2

+" " d~d2J ECt2d~ECt~ Ef32J DD J 2

2

2

iii

+ 2 I:I: dTdjE( Ctif3d E(Ctjf3j) . i#j

But

ECtT

=

ECtTf31

=

Pi(1 - pd , Ef3] = qj(1 - qj) , 2piq; (1 - pd + (1 - qd 2qiPT + Piqi(1 - qi - Pi)

< {(I - Pi) + (1 - qi) + (1 - qi - Pi)}Piqi < 3piqi, -(1 - pi)Piqi - (1 - qdqiPi + Piqi(1 - qi - pi)

E(Ctif3i)

l:;i:;n,l:;j:;n .

Piqi, Therefore,

LH8(2.2.4)

< 3{

3

:~;>ttPiqi + ~:L d; d]Piqj }

[~dlPi] [yd]qj] .

This completes the proof of (2.2.2) , in view of the definition of {pi, qj} . That of (2.2.3) follows from (2.2.2) and the monotonicity of the Gi , 1 :; i :; n . 0

Lemma 2.2.2 For every

(2.2 .5)

E

> 0 and s :;

U,

P[ sup IWd(t) - Wd(s)1 2: E] s9'5:u :; f,;E -

4

[Gd(U) - Gd(s)f

where r: does not depend on

E,

+ P[IWd(U)

- Wd(s)1 2: E/2]

n or on any underlying quantity.

2.2.1. Weak convergence Of Wd Proof. Let 8 = u - s, m (2.2.6)

~j

~

1 be an integer , and for 1 :::; j :::; m , let

Wd((j /m)8

=

19

+ s) -

Wd(((j - 1)/m )8 + s),

k

2::= ~j, u.; =

s,

j=l

max

l ::;k ::;m

ISkl·

The right cont inuity of W d implies that for each n and each sample pa th , M m -+ sup{ IWd(t ) - Wd(s)l; s :::; t :::; u} as m -+ 00, w.p.L In view of Lemma 2.2.1, Lemma 9.1.1 in the Appendix is applicable to t he ab ove LV . ' S { ~j } with 2, a 1 and

,=

=

Hen ce (2.2.5) follows from t hat lemma and the right cont inuity of Wd. 0 Proof of Theorem 2.2.1. For a 8 > 0, let r = [8- 1 ], th e grea tes t integer less than or equa l to 1/8. Define tj = j8, 1 :::; j :::; rand to = 0. Let f j = Wd(tj ) - Wd(tj-d , 1 :::; j :::; r. Then

p( It-sl< sup IWd(s) o

Wd(s)1

~

E)

r

:::; 2::= p( j=l

su p

r

2 :::; KE- 2::= [Gd(t j ) - Gd(t j -dF j= l :::; KE -

r

+ 2::= P[ifj \ ~ E/ 6] j=l r

2

sup [Gd(t

+ 8) -

09 ::;1- 0

(2.2.7)

~ E/3)

IWd(S) - Wd(tj - d l

t j _ l ::;S::;t j

= I n( 8)

+ II (8) ,

Gd{t)]

+ 2::= P[lfjl j= l

~ E/ 6]

(say).

n

In t he above, the first inequ ality follows from Lemm a 9.1.2 of the Appendix, the second inequ ali ty follows from Lemma 2.2.2 above and t he last inequ ality follows becau se, by (N l) , r

(2.2.8)

2::= lGd(t j ) - Gd(tj - !lJ :::; Gd{l) j=l

= 1.

2. Asymptotic Prop erties of W.E.P.'s

20 Next, observe that a;

.- Var(fj)

L d;{G(t j) - Gi(tj _t}}{l - Gj(tj)

+ Gi(tj-l)} ,

and , by (2.2.8), th at T

(2.2.9)

L aj :s;

[Gd(t + 0) - Gd(t) ], all rand n .

sup O::; t:9 - <5

j=l

Furthermore, (Nl) and (N2) enable one to apply the Lindeberg Feller Central limit Theorem (L-F CLT) to conclude that atfj -+d Z , Z a N( o, 1) L V . Therefore, for every 0 > 0 (or r < (0) , T

II (0) - L P (IZ I ~ (E / 6)aj l)

(2.2.10)

-+ 0 as n -+

00 .

j=l

n

By the Markov inequality applied to the summa nds in the second term of (2.2.10) and by (2.2.9) , lim sup n

IIn (0)

T

< 3 lim sup L (6aj / E)4 (E Z 4 = 3) j=l

n

<

IiE-

4

lim sup n

sup [Gd(t + 0) - Gd(t )]. O
The resul t (i) now follows from t his, (2.2.7) and t he assumption (C ).

Proof of (ii) . Suppose Cd -+ C. Let m be a positive integer , o :s; iv . : . . .t-« :s; 1 and aI , . . . ,am be arbitrary but fixed numbers . Consid er m

t;

(2.2.11)

:=

L ajWd(tj ) = LdWi j=l

where

n

i= l

m

Vi := L aj{I(17i :s; tj ) - Gi(t j)} , j=l

1

:s; i :s;

n.

21

2.2.1. Weak convergence OfWd Note that m

IVil ::; L Jajl < 00,

(2.2.12)

1 ::; i ::; n.

j==l

Also, Var(Tn ) ---+ (]"2:= 'LJ==I'LJ==lajarC(tj,t r) . In view of (Nl) and (N2) , the L-F CLT yields that Tn ---+d N(O,l). Hence all finite dimensional distributions of Wd converge weakly to those of a Gaussian process W with the covariance function C and W(O) = 0 = W(l) . In view of (i) , this implies that Wd =?- W in (1)[0,1], d) with W denoting a continuous Gaussian process tied down at 0 and 1. Conversely, suppose Wd ==?- W. By (i), W is in C[0,1]. In particular the sequence Tn of (2.2.11) converges in distribution to T := 'LJ==1 aj W(tj) . Moreover, (2.2.12) and (Nl) imply that , for all n ~ 1,

E(f diVi)

4

t==l

< 3

=

f d;Ev; + f f; 3

t==l

t==l J :;<:t ,l

(~Iajr

Therefore {T;, n ~ I} is uniformly integrable and hence m

ET;

=L

m

m

m

L ajakCd(tj , tk) ---+ L L ajakCov[W(tj), W(tk)]

j==lk==l

j==lk==l

for any set of numbers 0 ::; {tj} ::; 1 and any finite real numbers al ,·· · , am' Hence Cd(S , t) ---+ Cov[W(s), W(t)] = C(s , t) for all 0 ::; s, t ::; 1.

Now repeat the above argument of the "only if" part to conclude that W must be a tied down Gaussian process C[O, 1]. 0 Another set of sufficient conditions for the weak convergence of {Wd} is given in the following

2. Asymptotic Prop erties of W .E.? 's

22

Theorem 2.2.2 Under the notation of Th eorem 2.2.1, suppose that

(N'l ) holds. In additio n, assume that the follo wing hold:

(B)

n max d; i = 0(1) . l
(D)

1

n- L

Gni(t) - t is nonincreasing in t , 0 ~ t ~ 1, n 2:: 1.

i =l

Th en also (i) and (ii) of Th eorem 2.2. 1 hold. Remark 2.2.2 Clearly (B) implies (N2). Moreover

[Gd(t

+ <5)

- Gd(t)] < n max dT[n- 1 L {Gi (t I

+ <5)

- Gi(t)}]

+ <5)

- (t

. I

nmrxd7[n-l L{Gi(t

+ <5)}

i

_n- 1 L {Gi (t ) - t}

+ <5]

i

< n max dT <5, I

0 ~ t ~ 1 - <5, by (D) .

Thus (B) and (D) together imply (N2) and (C) . Hence Theorem 2.2.2 follows from Theorem 2.2.1. However , we can also give a different pr oof of Theorem 2.2.2 which is direct and qu it e interest ing (see (2.2.15) below) . This pro of will be bas ed on the following two lemmas. Lemma 2.2 .3 Under (D) , for all n 2:: 1, 0

~

s, t

~

1,

(2.2.13)

Proof. Suppose 0 ~ s ~ t ~ 1. Let ai and Pi be as in the pr oof of Lemma 2.2.1. Usin g the indep endence of {ad and th e fact th at Eo; = 0 for all 1 ~ i ~ n, one obtains

E(Ldia i f

o~ d

4 Ea 1

I

I

+ 30 ~ ~ d~d~Ea2 Ea~ 0 J J

"2;- d; {Ea; I

I

I

ih

2( 3E aT)}

+ 3 ( L dT E aT) 2 I

2.2.1. Weak convergence

23

OfWd

+3 [

~ d;Pi(1- Pi)]

2

z

k~{ n-

< But s

:s: t

2

LPi

+3

z

(n-1~Pi) 2}. z

and (D) imply

a:s: n- 1 LPi =

n- 1

L[Gi(t) - Gi(S)J

i

:s: (t -

s).

i

Hence the

L .H.S.(2.2.13) :s: k~{n-l(t - s) + 3(t - sf} ,

o:s: s :s: t :s:

1.

The proof is completed by interchanging the role of sand t in the above argument in the case t :s: s. 0 Next , define, for (i - l)jn :s: t :s: ijn, 1:S: i :s: n,

(2.2.14) Zd(t)

=

Wd((i - l)jn)

+ {nt -

(i - l)}[Wd(ijn) - Wd((i - l)jn)].

Lemma 2.2.4 The assumption (D) implies that V a:s: s , t

:s: 1, n

~

1, (2.2.15) If, in addition, (N!) and (B) hold, then

(2.2.16)

sup IWd(t) - Zd(t)1 = op(l). t

Proof. Let n ~ 1 and a :s: s, t integers 1 :s: i , j :s: n such that

(i - l)jn

(2.2.17)

:s:

:s: s :s: ijn

1 be arbitrary but fixed. Choose

and (j - l)jn

:s: t:s:

jjn.

For the sake of convenience, let Ok,m

.-

4k~[(m - k)jnf ,

bk,m ~u,v

IZd(mjn) - Zd(kjn)1

.-

IZd(U) - Zd(V)! ,

= IWd(mjn) - Wd(kjn)l,

m , k integers ;

2. Asymptotic Properties of W .E.? 's

24 From (2.2.13), (2.2.18)

E8tm

< k~{3(m - k)2 jn 2 + n- 2lm - kl} < 4k~[(m - k)jn]2 =: bk,m'

The proof of (2.2.15) will be completed by considering the following three cases . Case 1. i < j - 1. Then because of (2.2.14) and (2.2.17), D.s,t ::; max{ 8i ,j -

l , 8i ,j , 8i - 1 ,j - l , 8i - 1 ,j }

which entails that (2.2.19)

< E{8 4 . 1 + 84 . + 84 1 ,J.- 1 + 84 1 ,J. } < bi,j-l + bi,j + bi-1 ,j-l + bi-l ,j < 4bi-l ,j = 16k~[(j - (i - l)) jnf ~,J -

~ ,J

~-

~-

where the second inequality follows from (2.2.18) and the last from o ::; j - i - I < j - i < j - (i - 1). Note that (2.2.17), i < j -1 and i ,j integers imply that

3(t - s) 2: [j - (i - l)] jn . From this and (2.2.19) one obtains (2.2.20) Case 2. i = j . In this case (i - l)jn ::; s , t ::; ijn. From (2.2.14) one has D.s,t =

nit - sI8i -

1 ,i

so that from (2.2.18) (2.2.21) The last inequality follows because n( t - s) ::; 1. Case 3. i = j - 1. By the triangle inequality

2.2.1. Weak convergence Of Wd

25

Thus by Case 2, applied once with sand ifn and once with i fn and t , one obtains

E,6.~,t

< 24 { E~~,i/n ,t } < 26 k~ {(i fn - s)2 + (t - ifn)2}

:s 27k~(t - sf .

In view of this, (2.2.21) and (2.2.20) , t he proof of (2.2.15) is comp lete . To prove (2.2.16), let di+ = max(O, dd , di.: = max( O, - di ) . Then one has d; = di+ - di _ . Decompose Wd and Zd accordingly. Note = dT, I :s i :s n. This and (Nl) t hat max (dT+ , dL ) = dT+ + imply that Td+ :s 1, Td- :s 1. It also implies that if (N2 ) is sat isfied by the {dd then it is also sa t isfied by {di+ , di - } . By the triangle inequality,

dL

Moreover , di+ 1\ di.: 2: 0, for all i . Therefore, it is enough to prove (2.2 .16) for d i 2: 0, I :s i :s n. Accordingly suppos e t hat is the case . T hen

(2.2.22) where m!tx

sup

\Wd (t ) - Wd((i - 1)/n)J,

m ax

sup

IWd(t ) - Wd(i f n )1

l ::;z::;n (i- l )/ n::; t::;i/n l ::;z::;n (i- 1)/n::;t::;i/ n

:s t :s ifn, and a;

For (i - l)fn

2: 0,1

:s i :s n ,

IWd(t) - Wd(if n)1

< I

n

n

j=l

j=l

I: dj I (t < rlj :s i f n )1+ I: dj i

i - I

n

[Gj (i f n ) - Gj (t )] i

i - I

< I Wd(~) - Wd(---;-) I + 2 I:dj [Gj ( ~ ) - Gj (---;- )] j=l

< Ji -

1i

,

+ 2 1<max d ', '
by (D)

2. Asymptotic Properties of W .E.P.'s

26

This inequality, (2.2.18) and the Markov inequality imply that for every to > a and for n sufficiently large such that 2 maxj dj < E, the existence of which is guaranteed by (B) ,

P(U2 2:: E)

2m~xdd

< P(max6i- 1,i 2:: El

l

n

< (E-2maxdd-4"I:-E6f-l ,i l

< (E -

i==1

2maxdi)-4.4k~n-l -+ O. z

Exactly similar calculations show that U1 = op(1) .

D

Proof of Theorem 2.2.2 . Observe that Zd(O) = a = Zd(1) and that Zd E C[0,1] for every n 2:: 1 and each sequ ence {dd · Hen ce by (2.2.15) and Theorem 9.1.2 of the Appendix, {Zd} is tight in C[O , 1]. Thus claim (i) follows from (2.2.16). To prove (ii) just argue as in 0 th e proof of (ii) of Theorem 2.2.1 above. The following corollary will be useful later on. To state it we need some mor e notation. Let Fn1, ..., Fnn denote d.f.'s 011 JR and Xn i be a LV . with dJ. F ni , 1 < i ~ n. Define, for x E JR, 1 ~ i ~ n, a ~ t S; 1,

(2.2.23) H( x)

n- 1

.-

n

"I:- Fni(x) ;

H- 1(t):= inf{ x ; H( x) 2:: t} ;

i== 1 n

Lni(t)

'-

Fni(H-

1(t)) ;

Ld(t) :=

"I:- d~iLni(t) , i==1

n

W;(t)

"I:- «; {I(Xni ~ H- 1(t)) -

:=

Lni(t)} .

i== 1

Corollary 2.2 .1 As sum e that (2.2.24)

X n1, . . . , X n n are independent r.v. 's with respective d.f. 's Fn1, . . . , Fnn on JR.

In addition, suppose that {dnd , {Fnd satisfy (N1) , (N2) and (C*)

lim lim sup n

o-tO

sup [Ld(t + 6) - Ld(t)] = O.

0::;1::;1-0

Then, for every E > 0, (2.2.25)

lim limsupP( sup IW;(t) - W;(s)1 2:: E) = O. o-tO

n

It-si d

27

2.2.2. Vh - Processes

Proof. Follows from Theor em 2.2.1 (i) applied to

1]i

=H (Xd , G = i

L i ,1 :'S i:'S n.

0

Remark 2.2.3 Not e that if H is continuous then n

n-

1

2:: Lni(t ) = t. i= l

Therefore, sup [L d(t +8) -Ld(t)] :'S nm~xd;i8.

O
~

Thus , if we strengthen (N2) to require (B) then (C*) is a priori sa tisfied. That is, the conditions of Theorem 2.2.2(i) are satisfied. If Fni = F, where F is a cont inuous d.f. , th en Lni(t) = t. Therefore, in view of (N l) , (C*) is a priori sa tisfied . Moreover C;[( s, t ) := Cov (Wd'(s) , W d' (t )) = s (1 - t) , a :'S s :'S t :'S 1. Therefore we obtain

Corollary 2.2.2 Suppose that X n 1 , . . . , X nn are i.i. d. F , F a contin-

uous d.f. . Suppose that {dnd satisfy (Nl) and (N2 ). Then W d' in ("P[O, 1], p) with B a Brownian bridge in C[a, 1].

~

B

Ob serve th at dni = n - 1 / 2 satisfy (N l) and (N 2) . In ot her word s t he above corollary includes t he well celebrated result , viz., the weak convergence of t he sequen ce of t he ordinary empirical processes. Note. A vari an t of Theorem 2.2.1 was first proved in Koul (1970). The above formul ation and proof is based on thi s work and that of Withers

(1975). Theorem 2.2.2 is motiv at ed by the work of Shorack (1973) which deals only with the weak converge nce of the W1-process, t he pr ocess Wd with d-« == n- 1!2. The sufficiency of condit ion (D) for (2.2.13) was observed by Eyst er (1977).

2.2.2

Vh

-

0

processes

In t his subsect ion we shall investi gat e the weak convergence of the R.W.E.P.'s {Vh(x ), x E IR} of (1.4.1). To state t he genera l result we need some more st ructure on t he un derlyin g r. v.'s.

2. Asymptotic Properties of W.E.P. 's

28

Accordingly, let (n, A , P ) be a pr obability space and G be a d.f on 1Ft For each integer n ~ 1, let (( ni , h ni , 8nd, 1 :::; i :::; n, be an array of trivariate LV. 'S defined on (n ,A) su ch that {(ni, 1 :::; i :::; n} ar e i.i.d. G LV . 'S and (ni is ind ependent of (hni' and for each 1 :::; i :::; n. Furthermore, let {And be an array of sub o-fields such that A ni c A n,i+l ,l :::; i:::; n,n ~ 1; (hnl,an d is A nI - measurable; the L V. ' S {( nl ,'" ,(n,j - l; (h ni, and , 1 :::; i :::; j } are A nj - measurabl e, 2 :::; j :::; n; and (nj is indep endent of A nj , 1 :::; j :::; n . Define, n

I: hniI (( ni :::; x + ani),

l

(2.2.26) Vh(X) := n -

i= l

n

Jh( X)

:= n-

l

I: E [hniI(( ni :::; x + 8ni )IA ni] i=l n

= n-

I: hniG(x + ani),

l

i= l

n

Vh*(x)

:= n -

l

I: hniI (( ni :::; x),

n

Jh (x )

i= l

Uh(X) := n l / 2 [Vh (x ) - Jh(X)], Uh(x ) := n l / 2 [Vh*(x ) - Jh(x)],

:= n -

l

I: hniG(x) , i= l

x E lIt

We shall give three results here. The first one st ates and pr oves t he weak converge nce of t hese processes for t he bounded weights while t he second states the same resul t under weaker assumptions for only square integrable weights. The proofs of t hese resul ts are similar in sp irit yet different. An addit iona l similar theorem along with it s proof per taining to ana logues of t he Vh process suitable for heteroscedastic autoregressive models is also given in this sect ion. To begin with we state th e following Theorem 2.2.3 In addition to the above, assume that the f ollowing

conditions hold: (2.2.27) (2.2.28)

sup max Ihnil :::; c, a.s. f or som e constant c

n>l

max I

I

lanil =

Op(l).

< 00 .

2.2.2. Vh

-

29

Processes

Then, for every x E lR at which G is continuous,

(2.2.29) In addition, if

(2.2.30) (2.2.31)

G has a uniformly continuous a. e. positive Lebesgue density g,

then

(2.2.32) If, in addition,

(2.2.33)

Ani C A n+l,i, 1 ~ i ~ n; n 2:: 1.

(2.2.34)

(n

-1","""

2 )1 /2

Z:: h ni

=

0:

+ op(l),

0:

a positive r.v.,

Uk

0:'

B(G)

i

then

(2.2.35)

Ui, =?

0: '

B(G) ,

=?

where B is a Brownian Bridge in e[O, 1], independent of 0:.

The proof of (2.2.29) is straightforward using Chebbychev's inequality, while that of (2.2.32) uses a restricted chaining argument and an exponential inequality for martingales with bounded differences. It will be a consequence of the following two lemmas.

Lemma 2.2.5 Under {2.2 .27} - {2.2.31}, \:IE>

li~n P (sx~; n-

1 2 /

t,

Ihnil'IG(y + Oni) - G(x

°

and for r = 1,2,

+ onill ,,; 2,'

where the supremum is taken over the set

{x ,y E lR;n 1/ 2 IG(x ) - G(y)1 ~

E} .

c) ~

1,

2. Asymptotic Properties of W .E.P. 's

30

Proof. Let E> 0, q(u) := g(G- 1 (u)) , 0 :::; u :::; 1; I n := maxi Ibil , sup{lq(u) - q(v) l; lu - vi :::; m - 1/ 2 }

.-

Wn

sup{ lg(x) - g(y) l; IG(x ) - G(y) 1:::; m- 1/ 2 } , sup{lg( x) - g(y)l ; Iy - z ] :::; I n }.

L\n .-

By (2.2.31), q is uniform ly continuous on [0,1] . Hence , by (2.2.28) , (2.2.36)

L\n = op(l ),

Wn

= 0(1 ).

But n

supn- 1 / 2 x ,y

L

Ihi n G(y + bi) - G(x

+ bd l

i= l

n

<

sup n- 1 / 2 x, y

L

n

Ihi II'IG(y) - G(x)1

+ n-

i= l

1

L

Ihin bil[w n + 2L\n]

i= l

< c1' E + Op(l ).op(l ),

by (2.2.30) and (2.2.36).

o

This completes the pr oof of the Lemma.

Lemma 2.2 .6 Let {Fi , i 2: O} be an in creasing sequence of a - fields , m be a positive integer,

T :::;

m be a stoppi ng tim e relative to

{Fi} and

{(i , 1 :::; i :::; m} be a sequen ce of real valued martingale differen ces ui.r.i . {Fd. I n add it ion , suppose that for so m e con stan t M , L

I(il

(2.2.37)

<M

< 00 ,

1 :::; i :::; m ,

T

L E((TIFi-d :::; L .

(2.2.38)

i= l

Th en, for every a

(2.2.39)

p

> 0,

(It \;1a) >

:::;

aT

2 exp { - (a /2M) arcsi nh(Ma/2L)} .

Proof. Write = E((fIFi-d , i 2: 1. F irst , consider th e case T = m : Recall t he following elementary facts : For all x E JR ,

< 00 ,

31

2.2.2. Vh - Processes exp (x ) - x - I::; 2(cosh x -1 ) ::; x sinhx, sinh(x)jx is increasin g in [z], x ::; exp (x - 1).

(a) (b) (c)

Because E(~i IFi- d 1 ::; i ::; m , E{[exp(o~d

-

== 0 and by (2.2.37 ), for a

°>

0 and for all

1J1Fi-d < E{o6 sinh(06)I F i- l } , < a; o sinh(oM)jM,

(2.2.40)

by (a), by (b).

Use a condit ioning ar gum ent to obtain

}

E ex+t,~i

~ E[exp (S~
<

E[exp (S~
<

E[exp (S~
<

E[exp (;~ <}xp {(L - ~ a; ).(S/M ) Sinh(SM)}] ,

where th e second inequ ali ty follows from (2.2.40) while the first follows from (c). Obs erve that L - ~{:; is F j - 2 measurabl e, for all j ~ 2. Hence, it erating the above argument m - 1 times yields

a;

E exp { S

t,
cxp {L.(S/M) . sinh(SM)} .

Now, by the Markov inequ ality, Va > 0,

< eXP { o[L j M . sinh(oM )- aJ} .

2. Asymptotic Properties of W .E.? 's

32 T he choice of equa lity p

a=

(l/M) arcsinh(Ma/2L) in this leads to the in-

(~ ~i ~ a)

"

exp{( - a/2M) arcs inh(Ma/2L)} .

An application of this inequality to { -~i} will yield the same bo und for P(Z= ~l ~i ::; - a), thereby completing the proof of (2.2.39) in the case T = m. Now consider the general case T ::; m : Let xs = ~j I(j ::; T). Becau se the event [j ::; T] E Fj -l , it follows t hat {Xj , Fj} satisfy th e conditions of t he pr evious case. Hence

P(l t~il ~ a)

p ( l f Xil ~ a)

i= l

i= l

< exp{( -a/2M)arcsinh(Ma/2L)} .

0

Proof of Theorem 2.2.3. For th e clarity of th e pr esent proof it is important to emphasize the dep end ence of various underlying processes on n . Accord ingly, we shall write Vn , U'; et c. for Vh, Uh etc. in the proof. On IR define th e matric d(x, y) := IG(x ) - G(y) 11 / 2 . T his metric makes IR totally bounded. T hus, to prove th e th eorem, it suffices to prov e

IUn(Y) - U~( y)1 = op(l ),

(a) (b)

VE (i)

> 0:3 a > a lim sup P( n

(ii)

3

IUn(Y) - Un(x) 1 > E) < E,

sup d(x ,y )):S.c5

I U~ ( y )

lim sup P ( sup n

V yE IR,

- U~(x) 1

> E) < E.

d(x ,y) :S.c5

Proof of (a). The fact that Un -U~ is a sum of condit ionally centered bounded r.v.'s yields that n

Var(Un(y) - U~(y)) ::; En - 1

L

h~IG (y

+ ad -

i= l

by (2.2.27), (2.2.28), (2.2.31) and t he D.C.T .

G(y)1 = 0(1),

33

2.2.2. Vh - Processes

Proof of (b)(i) . The following proof of (b)(i) uses a restricted chaining argument as discussed in Pollard (1984, p. 160-162), and the exponential inequality of Lemma 2.2.6 above . Fix an E > O. Let an := [n l / 2 IE], the greatest integer less than or equal to n 1/2 IE, and define the grid 1i n := {Yj ;G(Yj) = jm- l / 2 ,

1::; j

::; an},

n 2:: 1.

Also let

= hi+ -

Write hi

hi-

Uh( X)

= max(O, hd , so that

n- l / 2

n

L hi+Zi( x) -

n- l / 2

i= l

n

L hi-Zi(X) i= l

UJ~(x) - Ui:(x) , say .

Thus to prove (b)(i) , by the triangle inequality, it suffices to prove it processes. The details of the proof shall be given for the u;t for process only ; those for the U;; being similar. Next , let , for k 2:: 1, and x , Y E JR,

o;

k

Ddx ,y)

:=

L hr+E {[Zi( X) -

Zi(y)]2IAi} ,

i=l

and define th e sequence of stopping times

+ ._ { . Dk(x , y)) < 3EC2 n } . . - n 1\ max k 2:: 1, max d2 ( x, yE 1i n x ,Y

Tn

Observe that T;t ::; n. To adapt the present situation to that of the Pollard , we first prove that P(T;t < n) --7 0 (see (2.2.41) below). This allows one to work with n- l / 2(T;t)1 /2U\ -instead of u;t . By r« Lemma 2.2.6 and the fact that arcsinh(x) is increasing and concave in x, one obtains that if X,Y in 1i n are such that d2 (x , y ) 2:: tEn - 1/ 2 then P(n-l/2(T:)1/2IU~(x) - U~(Y)I 2:: t) 2

::; 2 exp {- 2Cd 2t(2 , y) EarCSinh(1 /(6E c))}, X

for all t > O.

2. Asymptotic Properties of W .E.? 's

34

This enables on e to carry out the chaining arg ume nt as in P ollard. What rem ains to b e done is t o connec t between the points in IR and a point in 7-l n whi ch will b e done in (2.2.42) b elow. We shall now prove

P (r;t < n) -+ O.

(2.2.4 1)

Proof of (2.2.41) . For Yj , Yk in 7-l n with Yj < Yk , d2 (Yj , Yk) (k - j) E n- 1/ 2 . Hence, us ing the fact t hat (hi+)2 :::; hI ,

>

n

h;[G(Yk + 6d

< L

- G(Yj + 6d]{ (k -

12 j )E }-l n /

i= l

n

< {(k - j)E} - ln 1/ 2 L i= l

k- l

h; L[G(Yr+l

+ 6i )

-

G(Yr + 6dJ

r=j

n

Now apply Lemma 2.2.5 with r = 2 t o obtain P ( max

ll eA~ ,J -::: a n

2 } Dd X,Y) { d 2 (x. , Y ) < 3EC n ----* l.

This completes t he proof of (2.2.41). Next , for each x E IR, let Yjx deno te t he p oin t in 7-l n t hat is the closest to x in d-met ric from t he po ints in 7-l n t hat satisfy Yjx :::; x . We sha ll now prove: V E > 0, P (su p lu;t(x ) - u;t (Yjx)1 > SCE) -+ O.

(2.2.42)

x

Proof of (2.2.42) . Now write Vn+, J;t for Vn , I n wh en {hi } in these qu anti ti es is repl aced by { hi+} ' The defin it ion of Yjx' G increasing, and t he fact tha t lu.; :::; Ihil for all i, imply t hat sup In 1 / 2 [J;t (x ) - J;t (Yjx)J! x

35

2.2.2. Vh - Processes An application of Lemma 2.2.5 with r = 1 now yields that P(sup In 1/ 2[J: (x) - J:(Yj x)]1

(2.2.43)

x

> 4CE) ----+ O.

But hi+ 2:: 0, 1 ~ i ~ n , implies that Vn+ is nondecreasing in x . Therefore, using the definition of Yjx'

n - 1/ 2[U: (Yjx_d - U:(Yjx)]

+ J:(Yjx-d -

J:(Yj x)

Vn+(Yjx-d - Vn+(Yjx)

< Vn+(x) - Vn+(Yjx) < Vn+ (Yjx+1) - Vn+(Yjx) =

n - 1/ 2 [U: (Yjx+l ) - U:(Yjx)]

+ J:(Yjx+d

- J:(Yjx)'

Hence, (2.2.44)

sup In 1/ 2[Vn+(x ) - Vn+(Yjx] I x

< 2

max

l :'SJ :San

1U:(Yj+d - U:(Yj)1

+2 max In 1/ 2[Jlt (Yj +l ) - J:(Yj)]1 l :SJ :Sa n

Thus, (2.2.42) will follow from (2.2.43) , (2.2.44) and

P( m ax 1U:(Yj+l) - U:(Yj)1 > CE) -+ O.

(2.2.45)

l :'SJ :'San

In view of (2.2.41), to prove (2.2.45) , it suffices to show that (2.2.46) P( max n- 1/ 2(T: )1/21U++(Yj+d - U\(Yj)1 > CE) -+ O. t«

l:S J:San

r«

But the L.H.S. of (2.2 .46) is bounded above by an

T;;

L p(1 L hi+[Zi(Yj+l j=l

Zi(Yj)] I > etn

1 2 / ) .

i= l

Now apply Lemma 2.2.6 with ~i == hi+[Zi(Yj+d - Zi(Yj)], Fi-l == A i, T == T;; , M = c, a = etn 1 / 2 , m = n . By the definition of T;;, L = 3c2E2n1 / 2 . Hen ce, by Lemma 2.2.6,

T;;

p(1 L

lu.; [z, (Yj+d - Zi(Yj)]! > cm

i= l

n 1/ 2E

< 2exp[--2-arcsinh(1/6E)] .

1 2 / )

2. Asymptotic Properties of W.E.P. 's

36

Since this bound does not depend on j , it follows that L.R.S . (2.2.46)

n 1/ 2 E

~ 2E- 1n1 / 2 exp[--2-arcsinh(1/6E)] ~ 0.

This completes the proof of (2.2.42) for u;t. As mentioned earlier the proof of (2.2.42) for U;; is exactly similar, thereby completing the proof of (b)(i). Adapt the above proof of (b)(i) with Oi == to conclude (b)(ii) . Note that (b )(ii) holds solely under (2.2.27) and the assumption that G is continuous and strictly increasing, the other assumptions are not required here. The proof of (2.2.32) is now complete. The claim (2.2.35) follows from (b)(ii) above, Lemma 9.1.3 of the Appendix and the Cramer - Wold device . 0 As noted in the proof of the above theorem, the weak convergence of Uh holds only under (2.2.27), (2.2.34) and the assumption that G is continuous and strictly increasing. For an easy reference later on we state this result as

°

Corollary 2.2.3 Let the setup of Theorem 2.2.3 hold. Assume that G is continuous and strictly increasing and that {2.2.27}, {2.2.34} hold. Then, Uh '* a · B (G), where B is a Brownian bridge in C[a, 1], independent of a. Consider the process Uh(t) := Uh(G- 1(t)),0 ~ t ~ 1. Now work with the metric It - 81 1/ 2 on [0,1]. Upon repeating the arguments in the proof of the above theorem, modified appropriately, one can readily conclude the following

Corollary 2.2.4 Let the setup of Theorem 2.2.3 hold. Assume that G is continuous and that {2.2.27}, {2.2.34} hold. Then {Uh} ==> a.B, where B is a Brownian Bridge in C[a, 1], independent of a . Remark 2.2.4 Suppose that in Theorem 2.2.2 the r.v.'s 1]nl,'" Tln» are i.i.d . Uniform [0,1] . Then, upon choosing h n i == n 1/ 2 dn i , (ni == G- 1 (1]n d , one sees that Ui, == Wd(G), provided G is continuous. Moreover the condition (D) is a priori satisfied , (B) is equivalent to (2.2.27) and (Nl) implies (2.2.34) trivially. Consequently, for this

37

2.2.2. Vh - Processes

sp ecial setup , Theor em 2.2.2 is a special cas e of Corollary 2.2.4 . But in general t hese two res ults serve differe nt purposes. T heore m 2.2.1 is the most general for the indep endent setup given there and ca n not b e deduced from Theorem 2.2 .3. The next t heo rem gives an analog of t he above theorem when t he weights are not necessarily bounded r. v. 's nor do they need to be square integrable.

Theorem 2.2.4 Suppos e th e setup in (2.2.26), and the assump tions

(2.2.28), {2.2. 30) , (2.2.31), and (2.2.34) hold. Th en (2.2.32) continues to hold. If, in addition, (2.2.33) holds, and

(2.2.47) For each n 2: 1, {h ni ; 1::; i ::; n} are square in tegrable, th en (2.2.35) cont in ues to hold. A proof of this theorem appears in Koul and Ossiander (1994) . This proof uses the ideas of bracket ing and chaining and is a bit involved , hence not included her e. We are rather interested in its numerous applications to time ser ies models as will be seen la ter in the Chapters 7 and 8 of this monograph. The next t heorem gives an analogu e of the above two res ults useful in the presence of het eroscedasticity. According ly, let Tni, 1 ::; i ::; n b e another array of r. v.'s indep en dent of (ni , for each 1 ::; i ::; n . Let {And b e an array of sub a -fields such that Ani C A n,i+I , 1 ::; i ::; n , n 2: 1, (h nl, Snl , Tnl) is AnI - meas urable; the r.v. 's {(n l ,· · · ,(n,j - I;(hni,Sni, Tni), 1 ::; i ::; j} are A nj - measurable, 2 ::; j ::; n ; and ( nj is indep endent of A nj , 1 ::; j ::; n . Define, n

(2.2.48)

Vh(x)

I 2 := n- /

L

hniI((ni ::; x

+ XTni + Sni) ,

i= l

n

Jh( x)

I 2

:= n- /

L

hniG( x

+ XTni + Sni ),

i= l

x ER

2. Asymptotic Properties of W.E.P. 's

38

Next , we introduce the addit ional needed assumptions. (2.2.49)

G has Lebesgue den sity 9 satisfying the following:

> 0 on the set { x : 0 < G(x) < I} , g(G- 1( u))

9

is uniformly cont inuous in 0 :::; u :::; 1, (2.2.50)

sup(1

+ Ixl)g(x) < 00 ,

x EIR

(2.2.51)

lim sup Ixg(x(1

u-.OxEIR

E

(2.2.52)

(n- t 1

+ u))

- xg(x )1=

o.

h;i) = 0(1) .

1=1

(2.2.53)

max ITnil = op(I). l 'S I'Sn

(2.2.54)

n'/' E[n-' t{h~;(IOn;1 + ITn;I)}]' ~ 0(1).

(2.2.55)

n-

n

1 2 /

L

IhniTnil

= Op(I) .

i= l

Remark 2.2.5 Not e that (2.2.50) impli es that for some const ant

0
00 ,

(2.2.56)

D( s) := sup IG(x

+ xs) -

G(x)1 :::; C [s],

vs E R

xEIR

Clearly, D( s) :::; 1 :::; 21 sl , for all lsi 2: 1/2 . Now consider -1 /2 O. In this case -log(1 + s ) :::; -2s = 21 sl , and D( s) =

l

x

X(!+ S)

1 yg(y)-dy :::; sup Iylg(y){ -log(1 y yE IR

+ s)}

<s<

:::; Cl sl·

A similar and somewhat simpler argument proves the claim in the case 0 < s < 1/2, using the fact log(1

+ s ) :::; S .

D

Theorem 2.2.5 Suppo se the setup in (2.2.26), and the assu mption s (2.2.28) , (2.2.30) and (2.2.4g)-{2.2.55) hold. Th en, (2.2.57)

sup IUh(X) - U':( x)1 xE IR

=

op(I).

39

2.2.2. Vh - Pro cesses

Suppose T n i == O. Then (2.2.57) is equivalent t o the conclus ion (2.2.32) of Theor em 2.2.4. But the conditio ns of Theor em 2.2.5 are clearly mu ch st ro nge r t han those needed by Theor em 2.2.4. Theor em 2.2.4 gives t he b est kn own result of t his ty pe useful in homosced ast ic autoreg ress ive (AR) t ime series models. Theorem 2.2.5 gives on the ot her hand a sim ila r result useful in autoreg ressive condit ionally het erosced astic (ARCH) time series mod els. As will be seen in the Chapters 7 and 8 of this mon ograph, these results are used t o ob tain t he asymptotic uniform lin earity of the analogues of M and R scores, and of the empirica l process of the residuals in standardized underlying paramet ers in a large class of AR and ARCH models . The latter result about the emp irical processes in turn is useful in variety of inference problem s in these models, including the investigation of the consiste ncy of t he er ror density estimates . Theorems 2.2.4 and 2.2.5 are also useful in obtaining the asympto t ic uniform qu adraticity result of a certain class of minimum dist ance scores in t hese mod els.

Proof of Theorem 2.2.5 . Ass u me withou t loss of generality t hat all b-« are non-n egati ve. Nex t , wri te Uh(X) = U:(x) + U;:(x) , wher e U:(x), U;:(x) correspond to t hat part of t he sum in Uh(X) whi ch has T n i 2: 0, Tn i < 0, respectively. Decompose Uh(x) sim ila rly. It t hus suffices to show tha t (2.2.58)

sup IU:(x ) - U~ +(x) 1

op( l) ,

x EIR

(2.2.59)

sup IU;:(x) - U~- (x ) 1

op( l ).

xE IR

Details will be given only for (2.2.58) , they b ein g similar for (2.2.59) . Now, fix a o > oa nd let - 00 = Xo < Xl < . .. < Xrn - l < Xrn = 00 I (j8/ n l / 2 ) , 0 ~ j ~ r - 1 and be a partition of ~ wh ere Xj = n l 2 T ri := [n / /8J+ 1. Note that

a-

(2.2.60) The dep enden ce of X j 's on n is supp ressed for the sake of convenience . Also , in t he proof be low C is a generic constant , possib ly differen t in different conte xt, but never dep ending on n or t he {x j }.

40

2. Asymptotic Prop erties of W .E.P. 's

Using the monotonicity of th e indicator fun cti on and the d.f. G, we obt ain that for Xj- I < x ~ Xj,

(2.2.61) IU: (x) - U~ + (x)1

< IU: (x j ) - U~+(xj- Il l + IU:(xj- Il - U~ +(xj)1 -+ *+ + 2 1J-+ h (Xj) - J h (xj- I)I + 2 1Jh (Xj ) Bnj,1 + Bnj ,2 + 2Bnj,3 + 2Bnj,4, say .

Jh*+ (xj- Ill

Note th at by (2.2.52), n

I~~~n Bn j,4 ~ n _J_

I

L Ih i= 1

n il8

= Op(8).

Next , conside r Bnj,l. For th e sake of br evity, let tni = Then , one can rewrite

Bnj ,1 =

n- I / 2

T

ni

+

1.

n

L hni { I ((ni ~ Xjtni + 8nil - I ((ni ~ .Tj - Il i=1

which is a sum of mar t ingale differences . We sha ll apply the inequality (9.1.4) given in t he Append ix below, with p = 4, V i = A ni' and

Di == n- I/ 2 h ni { uc; ~ Xjtni + 8nil

- I( ( ni ~ xj - Il

- G(Xjtni + 8nil + G(x j-Il} . Use

IDil ~ n - I / 2 hni , \:j i,

to obtain, for an

E

and th e fact

> 0,

P[jBnj,l ! > E] n

< CE- 4

(n- L Eh~i 2

i= 1

+E [n- I

t i= 1

h~i lG(Xjtni + 8ni) -

G(xj -Il!f ).

41

2.2.2. Vh - Processes

The first term in the above inequ ality is free from j . To deal with the second term, use the boundedness of g, (2.2.56), and a teles coping argument to obtain that for all 1 ::; j ::; r«. 1 ::; i ::; n,

G(Xjtni + ond - G(Xj-l) G(Xj) - G(x j-d

+ G(Xjtni + Oni) -

G(Xjtni)

+G( Xjtnd - G(Xj)

< C [0 n- 1!2 + IOnil +

ITnil].

Therefore,

Hence, using r n p (

= O(n 1!2),

~ax IBnj,l I > E)

l SJSrn

< C (n- 1! 2 (1 + 02) n-1

n

L Eh; i i=l

+ n 1!2E[n - 1 t h; i(IOnil + ITnil)f) , i=l which in turn, together with (2.2.52) and (2.2.54) , implies that maXlSj Srn IBnj,l I = op(l) . A similar state ment holds for Bnj,2. Next , consider n

1

Bnj ,3 = n - !2

L hni [G(Xjtni + ond -

G(Xj-ltni + Oni)]

i=l n

<

n-

1

n

L

hniO+ n - !2 1

i=l

L hniTni[Xjg(Xj) -

Xj- l g(xj- d ]

i=l n

+n- ! 2 L hni[G(x jtni + Oni) - G(Xjt nd - Onig(Xjtni)] 1

i=l n

+n- !2 L hni[G(Xj tnd - G(Xj) - TniXjg(Xj)] 1

i=l

2. A symptotic Prop erties of W .E.P. 's

42 n

1 2 - n- /

L hni[G( Xj-ltnd -

G(xj -d

i=l n

1 2 _n- /

L hni[G( Xj-ltni + Oni) -

G( Xj -ltni)

i=l n

+ n-

1 2 /

L hni oni [g(Xj tnd -

g(Xj- ltni )J.

i=l Now, let m.; := maxl ::;i::;n IOnil , /-Ln := maxl ::;i::;n ITnil· Not e that , uniformly in 1 :S j :S Tn , the sum of the absolute values of the t hird and sixt h term in th e above bound is bounded above by Cn - 1/ 2

n

L

Ihni ond

i=l

sup Ig(x) - g(y)1 = op ( l), I:r - yl::;mn

by (2.2.30) and the uniform continuity of g , implied by the the boundedness of 9 and th e uniform cont inuity of g(G- 1 ) on [0, 1J . Next , we bound the fourth term; the fifth term can be handled similarly. Clearly, the absolute valu e of the fourth term is bounded above by 1 2

n- /

n

~ IhniTniXjl

S; n -1/2

~ IkniTnil

[1

io

Ig(Xj

+ t XjTni)

l' ~~~{IxlIg(x

- g(Xj ) ldt

+ txli) -

g(x)11 dt = op(I) ,

by (2.2.51) and (2.2.55) . Finally, consider t he seventh term; the second term can be dealt with sim ilarl y. To begin with observe that by (2.2.56), _ max IG(xjtnd - G( xj)1 :S C m ax ITnil· l ::;t::;n,l ::;J::;r n l ::;t::;n Hen ce by the uniform cont inuity of g( G- 1 ) assure d by (2.2.49) , and by (2.2.53) , max Ig(x -t -) - g(x -)1 l ::;i::;n,l ::; j ::;rn J nz J =

_ max Ig(G-1(G(Xjtnd)) - g(G-1(G(xj)))1 = op (l ). l ::;t::;n,l ::;J::;r n

43

2.2.3. an - processes

Upon combining all these bounds and using E(n- 1 L:~l hnd = 0(1) , we obtains l~'~m IBn j ,31::; <5 Op(l) _ J_

+ op(l).

AIl of the above facts together with the arbitrariness of <5 thus imply 0 (2.2.57) , thereby completing the proof of Theorem 2.2.5.

Not e: The inequality (2.2.39) and its proof has its origins in th e papers of Johnson , Schechtman and Zinn (1985) and Levental (1989) . The proof of Theorem 2.2.3 has its roots in Levental and Koul (1989) and Koul (1991) . Theorem 2.2.5 was first proved in Koul and Mukherjee (2001). 0

2.2.3

(Xn-

processes

In this sub-section we sh all prove the weak convergence of a sequ ence of partial sum pro cesses useful in autoregressive mod el fitting. Accordingly, let X i , i = 0, ±1 ,··· be a strictly stationary ergodic Markov process with st ationary d.f. G, and let F, = 0"(Xi ,Xi - 1, ... ) be the o-field generated by the observations obtain ed up to time i. Furthermore, let , for each n 2: 1, {Zni , 1 ::; i ::; n } be an array of LV. ' S adapted to {Fd such that (Zni , X i) is strictly stationary in i , for each n 2: 1, and satisfying (2.2.62)

1 ::; i ::; n .

Our goal here is to establish the weak convergence of the pro cess n

(2.2.63)

a n( x) = n -

1 2 /

2: Z niI(Xi - 1 ::; x ),

x E R

i == l

The process an is also a weighted empirical process. It is also called a marked empirical process of the X i-1'S, the marks being given by the martingale difference array {Znil. An example of this process is the partial sum process of the innovations where Zni = Zi = (Xi - E(Xi/Xi-t}) , assuming the expectation exists. Further examples appear in Section 7.7 below. The process an takes its value in the Skorokhod sp ace D( - 00 , 00). Extend it continuously to ± oo by putting n

an(- oo) =0 and an (+oo) =n-1 /22:Zni . i == l

2. Asymptotic Properties of W.E.P. 's

44

Then an becomes a process in 1)[-00,00] . We now formulate the assumptions that guarantee the weak convergence of an to a continuous limit in 1)[-00,00] . To this end, let

Ly(x) := P(X I

-

ep(Xo) S; xlXo = y),

x, Y E

~,

where ip is a real valued measurable function . Because of the Markovian assumption, Lxo is the d.f. of Xl - ep(Xo), given Fo. For example, if {Xd is integrable, we may take ep(x) := IE[Xi+1 I Xi = x], so that epi+l := X i+! - ep(Xd are just the innovations generating the process {Xi} ' In the context of time series analysis the innovations are often i.i.d . in which case L y does not depend on y. However , for our general result of this section, we may let L y depend on y . This section contains two Theorems. Theorem 2.2.6 deals with the general weights Zni satisfying some moment conditions and Theorem 2.2.7 with the bounded Zni . The following assumptions will be needed in the first theorem: For some 7J > 0, 6 > 0, K < 00 and for all n sufficiently large: (2.2.64) (2.2.65) (2.2.66) There exists a function ip from ~ to ~ such that the corresponding family of functions {L y, y E ~} admit Lebesgue densities ly which are uniformly bounded:

suply(x) S; K < 00.

(2.2.67)

x ,y

There exists a continuous nondecreasing function [0, (0) such that \:j x E [-00,00], (2.2.68)

l n-

n

L E[Z;iIFi-l] I(X

i-

7

2

on ~ to

2(X) + op(l) . 1 S; x) = 7

i=l The assumptions (2.2.64) - (2.2.67) are needed to guarantee the continuity of the weak limit and the tightness of the process an , while (2.2.68) is needed to identify the weak limit . Here, B denotes the Brownian motion on [0, (0) .

45

2.2.3 . an - processes

Theorem 2.2.6 Under (2.2.64) - (2.2.68}, an ~ B

(2.2.69) where B time

0 7

2

0 7

2

in the space D[-oo, 00],

is a continuous Brownian motion on JR with respect to

2 7 .

Proof. For convenience, we shall now not exhibit the dependence of Zni on n . Apply the CLT for martingales given in Lemma 9.1.3 of the Appendix, to show that all finite dimensional distributions converge weakly to the right limit, under (2.2.64) and (2.2.68). As to tightness, fix -00 ~ tl < t2 < t3 ~ 00 and assume, without loss of generality, that the moment bounds of (2.2.64)- (2.2.66) hold for all n ~ 1 with K ~ 1. Then we have

[an (t3) - an (t2)J2[an (t2) -- an (td J2 n

2

n-2[l:ZiI(t2 < i =l

n-

2

X i-l

n

~ t3)] [l:ZiI(tl < Xi-l ~ t2)]

2

i=l

l: ~i~j(k(l, i,j ,k,l

with ~i = ZiI(t2 < X i-l ~ t3), ( i = ZiI(tl < X i-l ~ t2)' Now, if the largest index among i , j, k , l is not matched by any other, then JE{~i~j(k(l} = O. Also, since the two intervals (t2,t3J and (tl,t2J ar e disjoint , ~i(i == O. We thus obtain (2.2.70)

E{ n-

=

l: ~i~j(k(l} n - l: E{(i(j~n + n- 2 l: E{~i~j(n· 2

i,j,k,l 2

i,j< k

i,j
The mom ent assumption (2.2.64) guarantees that th e above expectations exist . In this proof, the constant K is a generic constant, which may vary from expression to expression but never dep ends on n or the chosen t 's . We shall only bound the first sum in (2.2.70), the second being dealt with similarly. To this end , fix a 2 ~ k ~ n for the time being

2. Asymptotic Prop erties of W .E.P. 's

46 and write

(2.2.71)

L

E{ (i(j~n

i,j
k- l

i=1

i=1

E{ (L (if~k} = E{ (L (iflE(~k IFk-d} k-2 < 2E{ (L (if E(dIFk-d}

+ 2E{ (LI E(~k IFk-d } .

i= 1

The first expectation equa ls

k- 2 E{ (L (ifI(t2 < X k- 1

(2.2.72)

::;

i=1

t3)IE(Z~IFk-d}.

Write for an appropriate fun cti on r . Not e that du e to stat ionar ity r is the sa me for each k . Condit ion on Fk- 2 and use t he Mark ov property and Fubini 's Theorem to show t hat (2.2.72) is th e sa me as

E{(?= Vi ) Jr( x , X k- 2, . . .) lX k- 2

t3

2

1= 1

k_ 2 ( X

t2

k-2

JIE{ (?= Vi ) t3

2 r( x , X k- 2, . ..

t2

te

<

- 1fJ(X k- 2))dx}

) lXk _ 2 ( X

-

IfJ (X k- 2))} dx

1=1

k- 2 I {E(?= (i)4V

J

t2

1= 1

2

1

{E( r( x , X k- 2,' " )lXk_2(X - IfJ (Xk - 2))) } 2 dx ,

X

where the last inequ ality follows from the Cauchy-Schwarz inequ ality. Since the (i's form a centered martingale difference array, Burkholder 's inequ ality (Chow and Teicher : 1978, p. 384) and the moment inequality yield

k-2

E( L

(ir : ; KE(L (If::; K

i=1

k-2

i=1

(k - 2)2 E(t.

47

2.2.3. an - processes

We also have

where L 1 (t) = EZt I(Xo :::; t) ,

- 00 :::;

t :::;

00 .

Let , for - 00 :::; t :::; 00,

! t

L 2(t ) =

{E(r( X,Xk_2, · · ·) lXk_2(X-
- 00

Not e that due to stationarity, L 2 is the same for each k . It thus follows that (2.2.72) is bounded from the above by (2.2.73) The assumption (2.2.64) implies th at L 1 is a cont inuous nondecreas ing bounded fun cti on on the real line. Clearly, L 2 is also nondecreasing and cont inuous. We shall now show that L2(00) is finit e. For this , let h be a st rictly posi tive continuous Lebesgue density on the real line such that h(x) '" lxi -I -I) as x -t ± oo, where fJ is as in (2.2.65) . By Holder 's inequ ality,

!

00

L 2(00) :::; [

E(r (x ,Xk_2, . .. ) lx k_2(x -
]!.

- 00

Use th e assumpt ion (2.2.67) to bound one power of lXk_2 from the above so that the last integral in turn is less than or equal to

The finit eness of th e last exp ect ation follows, however , from assumption (2.2.65) . We now bound th e second expectat ion in (2.2.70) . Since Zk - l is measurable w.r.t. Fk - l , t here exists some fun ct ion s such that

2. Asymptotic Properties of W .E.P. 's

48

Put u = r s with r as before. Then we have, with the 0 as in (2.2.66) ,

E{ d-I E(~~ IFk -t} } =

E{ J (t2 < X k- l ::; t3)J(tl < X k- 2 ::; t2) U( X k- l , X k- 2,··· )} t3

J

E{ tu, < X k- 2 ::; t2) u( x , Xk- 2, · · . )

t2

6

< [G(t2) - G(tt}] I H ts

X

J 1~6 E

{

ul+ 8(x , X k- 2, · . . ) lk~~2 (x -
t2

Put , for - 00 ::; t ::; 00, t

£ 3(t ) =

J 1~6 E

{ u l +8 (x , X k- 2, · · . )lk~~2 (x -
- 00

Arguing as above, now let q be a positive continuous Leb esgue density on the real line such th at q(x) rv !xl - l - } as x -+ ± oo. By Holder 's inequ ali ty, £ 3(00)

[J

00

<

1

E{ ul+8(x ,Xk_ 2, ... )

Ik~~2(X -
- 00 1

< K {E(Ul+ 8(Xk_ 1 , .. . ) q-8(Xk_1))}I H 1

< K {E(Z~(1 +8 )Z~(1 +8 )q - 8 (Xt})}I H , where the last inequ ali ty follows from Hold er's inequality applied to cond it iona l expec tat ions. The las t expect a tio n is, however , finite by ass umption (2.2.66). Thus £ 3 is also a nond ecreasing continuous bounded fun cti on on t he real line a nd we obtain

49

2.2.3. an - processes

Upon combining this with (2.2.70) and (2.2.73) and summing over k = 2 to k = n , we obtain

n- 2

L

E{ (i (j ~n

i,j
< K {[L1 (t2) -

+ [G (t2) -

Ll (td] ~

[L2(t3) - L 2(t2]

G(tl )]1~" IL3(t3) - L 3(t 2}]}.

One has a similar bound for the second sum in (2.2.70). Thus summarizing we see that the sums in (2.2.70) satisfies Chents ov's crite rion for ti ghtness. For relevant details see Billingsley (1968: Theorem 15.6) . This completes the pr oof of Theorem 2.2.6. 0 The next theorem covers t he case of uniformly bounded {Zni }· In this case we can avoid th e moment condit ions (2.2.65) and (2.2.66) and repl ace the conditio n (2.2.67) by a weaker cond itio n. Theorem 2.2.7 Suppose the r.v. 's {Znd are un iformly bounded and {2.2.68} holds. In addit ion, suppose there exists a m easurabl e function sp from IR to IR such that the corresponding f am ily of f unctions { L y , y E 1R} has Lebesgue densities {ly , y E 1R} satisfying

(2.2.74)

J[E

f or some 0

li~8 (x -

1


> O. Th en also the con clusion {2.2.69} holds.

Proof. Proceed as in the proof of the previous theorem up to (2.2.71) . Now use the boundedness of {Zd and argue as for (2.2.73) to conclude that (2.2.71) is bo unded from the above by

K

f E{ (~ (i)' lx._, f (EI~ ({" )'t.{Ell~'

<

( x - 'I' (Xk-2) ) } dx

K

s

(x - '1'( X o )) }

< K (k - 2) [G (t2) - G(tl )] 1+" [f(t3) - r(t2)],

.~.

dx

2. Asymptotic Properties of W .E.P. 's

50

with 8 as in (2.2.74) . Here t

J{E ll~8(x

f(t) =

-
-00 :::;

t :::;

00 ,

- 00

Note that (2.2.74) implies that I' is strictly incr easing, continuous and bounded on R Similarly

E{ eLl E(~~IFk-d} <

KE{ eLl I(t2 < Xk-l :::; ts) } E{I(tl < X k- 2:::; t 2)E[ZL I I(t2 < X k- l:::; ts)!Fk -2]}

<

KE{ tu, < X k- 2 :::; t2)

[1:

3

lXk_2(X -
r

E{ tu, < X k- 2 :::; t2) lXk_2(X -
lt2

Upon combining the above bounds we obtain that (2.2.70) is bounded from the above by

Summation from k = 2 to k = n thus yields

n- 2

L

lE{eiej~n

: :; K [G(t2) -

G(tl)]1~<5 [f(ts) - r( t2)]'

i ,j
The rest of the details are as in the proof of Theorem 2.2.4. 0 Note : Theorems 2.2.6 and 2.2.7 first appeared in Koul and Stute (1999).

2.3

AUL of Residual W.E.P.'s

In this section we shall obtain the AUL (asymptotic uniform linearity) of residual W.E .P.'s. It will be observed that the asymptotic continuity property of the type specified in Theorem 2.2.1(i) is the

51

2.3. AUL of residual W .E.P. 's

basic tool to obtain this result . Accordingly let {Xnil , {Fnil , {H} and {L nil be as in (2.2.2 3) and define

n

L dniI(Xni ~ H -

.-

(2.3.1) Sd(t , u)

1(t)

+ C~iU) ,

i=l n

J-ld(t,U) .-

i=l Sd(t , u) - J-ld(t , u) ,

.-

Yd(t, u)

L dniFni(H - l (t ) + C~iU) , o~

t ~ 1, uEW ,

where {Cni' 1 ~ i ~ n} are p x 1 vectors of real numbers . We also need n

(2.3.2)

S~( x , u)

:=

L dnJ(Xni ~ x + C~iU) , i =l n

J-l~( x , u)

:=

L dniF~!i(X + C~iU) , i= l

YJ( x , u)

:=

S~(x , u) - J-l~(x , u) ,

-

00

~ x ~

00,

u E W.

Clearly, if H is strictl y increasin g then S~( x , u) = Sd(H( x) , u) . Similar rem ark applies to other fun ctions. Throughout the text, any W.E.P. with weights dni n - 1 / 2 will be indicated by the subscript 1. Thus, e.g., \j - 00 ~ x ~ 00 , u E W ,

=

n

1 2 n- /

(2.3.3) S~(x , u)

L tix;

~x

+ C~iU) ,

i=l n

1 2 n- /

L {I (X ni ~ x + C~iU)

-

Fni(x

+ c~iU)}.

i= l

Theorem 2.3.1 In addition to (2.2.24), (N1) , (N2), and (C*) as-

sume that d.f. 's {Fni , 1 ~ i ~ n } have densities {fni, 1 w.r.t. A such that the following hold: (2.3.4) (2.3.5)

sup Ifni(X) - fni (y )1= 0,

lim lim sup max

o-t O m ax 1,n

n

lS;iS;n Ix- ylS; o

Ilfnilioo

~ k

~ i ~

<

00 .

n}

52

2. Asymptotic Properties of W .E.P. 's

In addition, assume that (2.3.6)

0(1),

lJ!fln II Cni II = n IIdnicnill =

L

(2.3.7)

0(1) .

i==l

Then , for every 0 < b <

00 ,

n

(2.3.8)

sup ISd(t, u) - Sd(t, 0) - u'

L dnicniqni(t) I = Op(l) , i==l

where qni := fn iH-1, 1 ::; i ::; n, and the supremum is taken over o ::; t ::; 1, lIull::; b. Consequently , if H is strictly increasing on JR, then n

(2.3.9)

sup IS~(x, u) - S~(x, 0) - u'

L dnicndni(x)1 = op(l) . i==l

where the supremum is taken over

-00 ::;

x ::;

00 ,

lI uli ::; b.

Theorem 2.3.1 is a consequence of the following four lemmas. In these lemmas the setup is as in the theorem. In what follows, SUPt ,u stands for the supremum over 0 ::; t ::; 1 and lIuli ::; b, unless mentioned otherwise. Let , for 0 ::; t ::; 1, u E W , n

(2.3.10)

Vd(t)

:=

L dnicniqni(t) i==l

Rd(t, u) := Sd(t, u) - Sd(t , 0) - U'Vd(t).

Lemma 2.3.1 Under (2.3.4) - (2.3.7) , sup IfLd(t , u) - fLd(t, 0) - u' v d(t )1 = 0(1).

(2.3.11)

t ,u

Proof. Let On = bmaxi [c.] . By (2.3.4) , {Fd are uniformly differentiable for sufficiently large n, uniformly in 1 ::; i ::; n . Hence , the L.H.S. of (2.3.11) is bounded from the above by n

L i ==l

IIdi ci li max z

sup

lli( x) -li(y)1 = 0(1),

Ix-yl :SJn

by (2.3.4) , (2.3.6) and (2.3.7).

o

2.3. AUL of residual W.E.P. 's

53

Lemma 2.3.2 Under (Nl) , (N2 ) , {C*}, {2.3.4} - {2.3.7}, Vllull ::;

b, (2.3.12)

Proof. F ix a llu ll ::; b. The lemma will follow if we show

(i)

Yd(t , u ) - Yd(t , 0 ) = op(l) for each 0 ::; t ::; 1,

(ii)

V E > 0, and for a = u or a = 0 , lim lim sup P ( sup IYd(t , a ) - Yd(s, a ) 12:

6--+0

It- sid

n

E) = O.

Since Yd(-, O) = W;(-) of (2.2.23), for a = 0, (ii) follows from (2.2.25) of Coro llary 2.2.1. To verify (ii) for a = u , take 'T]i = H (X i e~u ) , 1 ::; i ::; n , in (2.2.1) . Then Yd(- , u) == Wd(-) of (2.2.1) and G i( ·) = Fi(H- 1 (-) + e~u ) , 1 ::; i ::; n. Moreover , by (2.3.4)-(2 .3.6), n

(2.3.13)

sup

L d;[Fi (H -

1

(t + 8) + e~ u ) - Fi(H- 1 + e~u) ]

09 :::1 - 6 i= 1

::; 2bk max lIeill

+

[Ld(t

sup

+ 8) -

Ld(t)],

0
l

0(1 ), as n ----+ 00, and th en e ----+ 0, by (C *). Hen ce, (C) is sa tisfied by the above {Gd . The other condit ions being (N l ) and (N 2) which ar e also ass um ed here, it follows t hat th e above {'T]i} and {Wd } satisfy the conditions of Theorem 2.2.1(i) . Thus (ii) for a = u follows from this theor em . To obtain (i) , not e that again by (2.3.4)-(2 .3.6) , n

Var [Yd(t , u) - Yd(t , 0)] <

L d;IF (H i

1

(t)

+ e~u)

-

Fi (H - 1 (t))l

i=1

<

bkm~xll eill = 0(1). l

This and th e Chebych ev inequ ality imply (i) and hence (2.3.12). 0

2. Asymptotic Properties of W .E.P. 's

54

To state and pro ve t he next lemma we need some more not ation. Let lini = Ilcnil\' 1 ::; i ::; n , and define for 0 ::; t ::; 1, u E W , a E JR, n

(2.3.14) Sd(t , u , a)

L dniI(X ni ::; H - 1(t ) + C~iU + alind , i= l

J-Ld(t , u , a) =

ESd(t , u , a),

Yd*(t , u ,a)

Sd(t , u , a) - J-Ld(t , u , a).

Lemma 2.3.3 Und er (Nl) , (N2 ) , (C *) , (2.3.4) - (2.3. 7), V 0, 0 < b <

00 ,

lal < 00

and

n

>

Il ull ::; b,

(2.3.15) lim lim su pP ( sup IYd*(t , u , a) - Y;(s , u , a)1 8~ O

E

It- sid

~ E) = O.

Proof. In T heor em 2.2.1(i) , take 17i = H (X i - c~ u - alid , 1 ::; i ::; n. T hen W d(·) = Yd*(- , u ,a) and G i(-) = Fi( H - 1(-) + ci , u + alid, l ::; i ::; n . Again simi lar to (2.3.13), sup

[Gd(t

+ J) -

Gd(t)] < 2k(b + a) max IIcili ~

O::;t::;1- 8

+

sup [Ld( t + J) - Ld(t )] O
0(1),

by (2.3.6) and (C *) .

o

Hence (2.3.15) follows fro m Theorem 2.2.1(i).

Lemma 2.3.4 Und er (Nl) , (N2 ) , (C*), (2.3.4}-(2.3. 7), V E > 0

< b < 00 , IIvll ::;

th ere is a J

>0

(2.3. 16)

lim supP ( sup IRd(t , u ) - Rd(t , v )1 n t,llu- vll::;8

suc h that for eve ry 0

b,

~ E) = 0,

where Rd is defined at (2.3. 10) .

Proof. Assu me, wit hout loss of generality, that d; ~ 0, 1 ::; i ::; n . For, ot herwise write d; = di+ - di:. , 1 ::; i ::; n, where {d i+ , di -} ar e as in the proof of Lemma 2.2.4. T hen Sd = Td+Sd+ - Td-Sd- , Rd = Td+Rd+ - Td- Rd- , where TJ+ = ~i(di+)2 , T1- = ~i(di_)2 . In view of (N l ), Td+ ::; 1,Td- ::; 1. Moreover , if {dd satisfy (N 2) and (2.3.7)

2.3. AUL of residual W.E.P.'s

55

above , so do {di+ ' di - } because dy+ V dL = dy+ + dL = dy, 1 ::; i ::; n. Hence the triangle inequality will yield (2.3.16), if proved for R d + and Rd- . But note that a., 1\ a.: ~ 0 for all i . Now Ilu - vII ::; 8 implies

~

Therefore, because di

0 for all i ,

S;t(t, v , -8) ::; Sd(t, u) ::; S;t(t, v , 8) for all t , yielding (2.3.18)

L 1(t ,u,v)

.-

S;t(t, v , -8) - Sd(t ,v - (u - V)'Vd(t)

< Rd(t, u) - Rd(t ,v) < S;t(t,v , 8) - Sd(t,v) - (u - V)'Vd(t) -'

L 2 (t, u , v) .

We shall show that there is a 8 > 0 such that for every (2.3.19)

p (

sup \Lj(t , u , v)\ t,llu-vll:So

~ E)

= 0(1),

Ilvll ::; b,

j = 1,2.

We shall prove (2.3.19) for L 2 only as it is similar for L 1 . Obs erve that

(2.3.20)

L 2 (t , u , v) ::; IYd*(t , v , 8) - Yd'(t,v,O)1

+1J-l:J(t , v , 8) - J-l:J(t , v , 0)1

+ I(u -

The Mean Value Theorem, (2.3.4) , (2.3.5) and

V)'Vd(t)\

Ilu - vII ::; 8 imply n

(2.3.21)

sup 1J-l:J(t, v , 8) - J-l:J(t , v , 0)1

< 8k

L

IldiCill,

i == l

n

sup t

I(u -

v)'vd(t)1

< k8

L i == l

Iidicill-

2. Asymp tot ic Properties of W .E.P. 's

56

Let M(t) den ot e t he first t erm on the R. H.S. of (2.3.20) . I.e .,

M(t) = Yd*(t , v , 0) - Yd*(t , v , 0) ,

0 ~ t ~ 1.

We shall first prove that

(2.3.22)

sup IM(t) 1 = op( l) . t

To begin wit h , n

Va r(M(t))

<

L d;[Fi(H -

<

s» max Iii ,

by (2.3.4), (2.3.5) ,

0(1),

by (2.3.7) .

1

(t ) + c~v + <5kd - Fi(H - 1 (t ) + c~v)]

i=l 2

Hence

(2.3.23)

M(t) = op(l) ,

VO ~t ~1.

> 0,

Next, note t hat , for a , sup IM (t ) - M( s)1 It-s l<,

<

sup IYd*(t , v , 0) - Yd*( s, v , <5)1 It- sl< , + sup IYd*(t , v , 0) - Yd*( s, v , 0) 1· It-sl <,

Apply Lemma 2.3.3 twic e, on ce with a = <5 and on ce with a = 0, to obtain that V E > 0, lim lim sup P ( sup IM (t ) - M( s)1 n It-s l<,

, -t o

~ E) = O.

T his and (2.3.23) prove (2.3 .22) . Now cho ose <5 > 0 so t hat n

(2.3.24)

lim sup <5k n

L

II di cdl ~ E/3.

(Use (2.3.7) here).

i =l

From t his , (2.3.20) , (2.3.21) and (2.3.22) one readily obtains limsup P ( n

sup t,lIu- vll:<:;J

IL 2 (t , u , v )1 ~ E )

~ limnsup P (S~PIM(t) 1 ~ E/3)

= O.

2.3. AUL of residual W .E.P. 's

57

This prove (2.3.19) for L 2 . A similar argument proves (2.3.19) for L 1 with the same 8 as in (2.3.24), thereby completing the proof of the Lemma. 0 Proof of Theorem 2.3.1. Fix an E > 0 and choose a 8 > 0 satisfying (2.3.24) . By the compactness of the ball N(b) := {u E W ; Ilull :::; b} there exist points VI ,' " vr in N(b) such that for any u E N(b) , lIu - vjll :::; 8 for some j = 1,2"" , r . Thus

lim sup P(sup IRd(t , u)1 2:: E) n t ,U r

:::; L

limsupP( n

j=l

sup IRd(t , u - Rd(t , vj)1 2:: E/2) t,II U - V jll ::;J

r

+ LlimsupP(suPIRd(t ,vj)l2:: j =l

n

E/2) = 0

t

o

by Lemmas 2.3.2 and 2.3.4.

Remark 2.3.1 A reexamination of the above proof reveals that

Theorem 2.3.1 is a sole consequence of the continuity of certain W .E. P.'s and the smoothness of {Fnd . It do es not use the full force 0 of the weak converg ence of these W .E.P.'so Remark 2.3.2 By the relationship

and by Lemma 2.3.1, (2.3.8) of Theorem 2.3.1 is equivalent to sup IYd(t, u) - Yd(t, 0)1 = op(l). o::;t::;l,llull::;b

(2.3.25)

This will be useful when dealing with W .E .P.'s based on ranks in Chapter 3.

0

The above theorem needs to be ext ended and reformulated when dealing with a linear regression model with an unknown scale parameter or with M - estimators in the presence of a preliminary scale estimator. To that end , define , for x , S E~, 0:::; t :::; 1, u E ~p , n

Sd( S, t,

u)

:=

L dniI(Xni < (1 + sni=l

1j 2

)H - 1 (t ) + C~iU) ,

2. Asymptotic Properties of W .E.? 's

58 n

S~(s ,x , u)

L dniI (X ni :::; (1 + sn -1 /2 ) x +C~iU ) ,

:=

i= l

and define Yd(s , t, u ), f-Ld( S, t, u ) simi larly . We are now ready to prove

Theorem 2.3.2 I n addition to th e assumptions of Th eorem 2.3.1, assume that (2.3.26)

max sup Ixfni(x) 1:::; k l ,n

x

< 00 .

T hen (2.3.27)

su p ISd(S, i, u ) - Sd(O, t , 0) n

I

- L dnd sn - 1/ 2H - 1(t ) + C~iU} qni (t) = op(1). i= l

where the supremum is tak en over

lsi :::; b, Ilull :::; b, 0 :::; t

:::; 1.

Consequently, if H is stricilsi increasing for all n 2: 1, th en (2.3.28) sup

IS~(s , x , u ) - S~(O, x , 0) n

- L dnd sn- 1 /2x +c~iu } fni (x) 1 = op(1 ). i =l

where th e supremum is taken over

lsi:::; b, lI uli :::; b and

x E IR.

Sketch of Proof. T he argument is qu ite similar to t hat of T heorem 2.3. 1. We briefly indica te the modifica t ions of t he prev ious proof. An analogue of Lemma 2.3. 1 will now assert sup If-Ld( S, t, u ) - f-Ld(1, i, 0)

I

- {(n - 1/ 2"L,diqi(t )H- 1(t)) s + U' Vd(t )} = 0(1). T his uses (2.3.4), (2.3.5), (2.3 .6) , (2.3.7) , (2.3 .26) and (N l) . An analogue of Lemma 2.3.2 is obtained by applying T heor em 2.2.1 (i) to 7] := H (X i - C~u) O"; l ) , 1 :::; i :::; n , O"n := (1 + sn- 1/ 2). T his states t hat for every lsi :::; b and every Il ull :::; b, sup IYd (s, t , u ) - Yd(s , t, 0)1= op(1).

O
59

2.3. AUL of residual W.E.P. 's In verifying (C) for these {TJi} , one has an analogue of (2.3.13):

+ 0) -

sup [Gd(t 099- 8

Gd(t)]

:S 2k{B max Ilcill + bn- 1/ 2 } + sup [Ld(t + 0) - Ld(t)]. 099-8

z

Note that here Gd(t) = L:~=l d; Fi((]"nH-1(t) + c~u) . One similarly has an analogue of Lemma 2.3.3. Consequently, from Theorem 2.3.1 one can conclude that for each fixed s E [-b ,b], (2.3 .29)

sup IRd(S , t , uj] o::;t::;l,llull ::;b

= op(l) ,

where Rd(S , t , u) equals the L.R .S. of (2.3 .27) without the supremum. To complet e the proof, once again exploit the compactness of [-b ,b] and the monotonic structure that is present in 3d and fJd. Details 0 are left for int erest ed read ers. Consider now the specialization of Theor ems 2.3.1 and 2.3.2 to the case when Fni F , F a d .f. Not e that in this case (N1) implies that Ld(t) = t so that (C *) is a priori satisfied. To state these spe cializations we need the following assumptions:

=

(F1)

F has uniformly con ti n uous density f w.r.t. A.

(F2)

f > 0 , a .e.

(F3)

sUPx EIR IXf( x)1

A.

Note that (F1) implies that F is strictly increasing.

:S k < 00.

f is bounded and that (F2) impli es that

Corollary 2.3.1 LetXn1,' " , X nn be i. i.d. F. In addition, suppose that (N1) , (N2) , {2.3.6}, {2.3.1} and (1'1) hold. Th en {2.3.8} holds with qni = f(F- 1) . If, in addition, (1'2) holds, th en {2.3.9} holds with f ni

=f .

0

Corollary 2.3.2 Let X n1,' " , X nn be i. i.d. F . In addition, suppose that (N1), (N2) , {2.3.6}, {2.3.1}, (1'1) and (1'3) hold. Th en

{2.3.21} holds with H

=F and qni =f(F -

1

).

If, in addition, (1'2) holds, th en {2.3.28} holds with fn i

= f.

0

60

2. Asymptotic Prop erties of W .E.P. 's

We sha ll now ap ply the above results to the mod el (1.1.1) and the {Vj} - pro cesses of (1.1.2). The resul ts thus ob tained are useful in studying the asymptoti c distributions of certain goodness-of-fit t ests and a class of M-estimators of {3 of (1.1.1) when t here is an unknown scale par am et er also. We need the followin g ass umpt ion abo ut the design matrix X.

This is Noether 's cond it ion for t he des ign matrix X . Now, let (2.3.30) A

.-

(X ' X) - 1/2,

D := XA ,

q' (t ) .-

(qn 1(t), · · · ,qnn(t )),

A (t ): = diag(q (t )) ,

r 1 (t) .r 2 (t ) .-

AX' A (t )XA , n - 1/ 2 H - 1(t )D'q (t ),

0 ~ t ~ 1.

Wri te D = ((d i j ) ) , 1 ~ i ~ n , 1 ~ j ~ p , and let d U) denote t he j -th column of D . Note t hat D'D = I p x p . This in t urn implies t hat (2.3.31)

(N l) is sa tisfied by d (j) for all 1

~

j

~ p.

Mor eover , with a U) denoting the j -t h column of A , (2.3.32)

max dI2) . i p

m?-x(x' a U))2 ~ max I)x~aU ) ) 2 I

I

mfxx;

(t

j=l

a (j)a(j))

Xi

) =1

m?-x x;( X ' X)- l X i = 0(1),

by (N X) .

I

Let n

(2.3.33)

Lj (t ) :=

L dTj Fi( H - 1(t)) , i= l

We are now read y to state

0 ~ t ~ 1, 1 ~ j ~ p.

61

2.3. AUL of residual W.E.P. 's

Theorem 2.3.3 Let {(x~i,Yni),1 SiS n} ,,B ,{Fni,1 SiS n} be as in the model (1.1 .1) . In addition, assume that {Fnd satisfy (C*) is satisfied by each L j of (2.3.33) ,

(2.3.4), (2.3.5) and that 1

S

j

S p.

Then , [or every 0 (2.3.34)

< b < 00 ,

sup IIA{V(H- 1 (t) ,,B + Au) =

op(I).

where the supremum is over 0 S t S 1,

Ilull S

b.

If, in addition, H is strictly increasing for all n every 0 (2.3.35)

fdt)ull

V(H- 1(t),,Bn -

< b<

~

1, then, for

00 ,

sup IIA{V(x ,,B + Au) -

where the supremum is over

V(x,,Bn - f 1 (H(x))ull = op(I).

-00 :::;

Theorem 2.3.4 Suppose that obey the model

x S

00 ,

Ilull S

{(x~i ' Y ni) , 1

b.

SiS n} and,B E

ffi.P

(2.3.36) with {Eni} independent r.v. 's having d.f. 's {Fnd . Assume that (NX) holds. In addition, assume that (2.3.4), (2.3.5), (2.3.26) are satisfied by {Fnd and that (C*) is satisfied by each L j of (2.3.33}, 1 S j S p . Then for every 0

< b < 00 ,

(2.3.37)

A [V(aH- 1(t) ,,B + Au-r) - V(rH- 1(t) ,,B)] =

T 1 ( t) u - I' 2 ( t) v

where v := n 1 / 2(a - ,),-1, a

+ up ( 1) ,

> o.

If, in addition, H is strictly increasing for every n (2.3.38)

A [V(ax ,,B =

+ Au-y] -

~

1, then

V(rx , ,B)]

f 1 (H(x)) u - f 2 (H (x )) v

+ u p (I ).

In (2.3.37) and (2.3.38}, u p (l ) are a sequences of stochastic processes in (t , x , u , v) tending to zero uniformly in probability over 0 S t S

1,

-00

Sx S

00,

Ilull S band Ivl S

b.

2. Asymptotic Properties of W.E.P. 's

62

Proof of Theorem 2.3.3. Apply Theorem 2.3.1 to Xi = Yi x~{3 , c, = x~A , 1 ::; i ::; n . Then F; is the d.f. of Xi and the j -th components of AV(H- 1(t) ,{3 + Au) and AV(H- 1(t) ,{3) are Sd(t, u) , Sd(t, 0) of (2.3.1), respectively, with d i = d i j of (2.3.30) , 1 ::; i ::; n, 1 ::; j ::; p . Therefore (2.3.34) will follows by p applications of (2.3.8), one for each dUl' provided the assumptions of Theorem 2.3.1 are satisfied. But in view of (2.3.31) and (2.3.32), the assumption (NX) implies (Nl), (N2) for dUl' 1 ::; j ::; p. Also, (2.3.6) for the specified {cd is equivalent to (NX). Finally, the C-S inequality and (2.3.31) verifies (2.3.7) in the present case. This makes Theorem 2.3.1 applicable and hence (2.3.34) follows. 0 Proof of Theorem 2.3.4. Follows from Theorem 2.3.2 when applied to Xi = (Yi -x~{3h -1, c~ = x~A, 1 ::; i ::; n , in a fashion similar 0 to the proof of Theorem 2.3.3 above. The following corollaries follow from Corollaries 2.3.1 and 2.3.2 in the same way as the above Theorems 2.3.3 and 2.3.4 follow from Theorems 2.3.1 and 2.3.2. These are stated for an easy reference later on . Let C(b) := {s E W , IIA -l(S - {3)II ::; b} . Corollary 2.3.3 Suppose that the model (1.1.1) with F n i == F holds. Assume that the design matrix X and the d.f. F satisfying (NX) and (Fl) . Then , V 0

(2.3.39)

< b < 00,

A [V(F- 1(t) ,s) - V(F- 1(t),{3)] =

f(F- 1(t))A -l(s - {3) + up(1) ,

where u p(1) is a sequ enc e of stochastic processes in (t , s) tending to zero uniformly in probability over 0 ::; t ::; 1; sEC (b) . If, in addition, F satisfies (F2), then

(2.3.40)

IIA{V(x,s) - V(x ,{3)} - f(x)A -l(s - {3)II = up(1) ,

where u p(1) is a sequence of stochastic processes in (x, s) tending to zero uniformly in probability over

-00 ::;

x ::; 00 ; s E C(b).

0

Corollary 2.3.4 Suppose that the model (2.3. 36) with F n i F holds and that the design matrix X and th e d.f. F satisfy (NX),

2.4. Some additional results for W .E.P. 's

63

(Fl) and (F3). Then {2.3.37} holds with f(F- 1(t))Ip x p ,

(2.3.41) f 1 (t)

o:::; t :::; 1.

F- 1 (t)f(F- 1 (t) )AX'l,

f 2 (t )

If, in addition, F satisfies (F2), then {2.3.38} holds with rj(H)

= rj(F) ,j = 1,2,

i.e.,

A [V(ax ,,8 + Au-y) - V(--yx ,,8)] = f(x)u - xf(x)v

+ u p (l ), o

where v is as in {2.3.37} and u p (l ) is as in {2.3.38}.

We end this section by stating an AUL result about the ordinary residual empirical processes H n of (1.2.1) for an easy reference later on. Corollary 2.3.5 Suppose that the model {1 .1.1} with Fn i

=F holds .

Assume that the design matrix X and the d.f. F satisfying (NX) and

(Fl). Th en, V 0 < b <

00,

(2.3.42) n

=

1

f(F - (t ))

· n- 1 / 2

LX~iA . A -1(8

-

,8)1

+ u p (l ),

i=1

where u p (l ) is as in Corollary 2.3.3. If, in addition, F satisfies (F2), then, V 0

< b < 00 ,

n

=

f(x)· n- 1 / 2 LX~iA . A -1(8 -,8)

+ up(l),

i=1

where u p (l ) is as in {2.3 .40}.

Proof. The proof follows from Theorem 2.3.1 by specializing it to the case where d n i n -1 /2 and the rest of the entities as in the proof of Theorem 2.3.3. 0

=

Note: Ghosh and Sen (1971) and Koul and Zhu (1991) prove an almost sure version of (2.3.40) in the case p = 1 and p

> 1, respectively.

0

2. Asymptotic Properties of W.E.P. 's

64

2.4

Some Additional Results for W.E.P.'S.

For t he sake of general interest , here we state some further results abo ut W .E.P. 's. To begin with, we have

2.4.1

Laws of the iterated logarithm

In this subsec tio n, we ass ume that (2.4.1)

Define n n _ L d2 L di{ I (7]i ::::; t ) - Gi(t)} , (In2.'i, i=l i=l .- u; (t )/ {2(J; £n£n(J;} 1/2, » 1, t ::::; 1.

.-

(2.4.2) Un(t ) ~n (t)

>

°:: ;

°:: ;

Let r (s, t) := s /\ t - st , s, t ::::; 1, and H( r) be the reproducing kern el Hilb ert space gene rate d by the kernel r with II . li T denoting t he associated norm on H (r). Let

K = {f

E H( r) ; IlfilT:: :; I} .

Let p denote the uniform metric. Theorem 2.4.1 If7]1 ,7]2, '" are i.i .d. uniform on [0,1] r.v.'s an d d 1, d 2, . . . are any real numbers sa tisfying 2 · 1Im(Jn = n

.

00 ,

2

hm( max d i n

1:S1:Sn

)

£n£n(J; 2 (In

= 0,

th en P(P (~n , K) ~

°

an d the se t of lim it points of {~n } is K) = 1.0

This theorem was proved by Vanderzanden (1980 , 1984) usin g some of the results of Ku elbs (1976) and certain martingale properties of ~n'

Theorem 2.4.2 Let 7]1, fJ2, . . . be in dependen t no n n egat ive ran dom variables. Let {d i } be any real numbers. T hen lim sup sUP (J~ l IUn (L ) 1 n

t >O

< 00 a.s ..

o

2.4. Some additional results for W.E.P. 's

65

A proof of this appears in Marcus and Zinn (1984). Actually they prove some other interesting results about w.e.p.'s with weights which are r .v.'s and functions oft. Most of their results, however, are concerned with the bounded law of the iterated logarithm. They also proved the following inequality that is similar to, yet a generalization of, the classical Dvoretzky - Kiefer - Wolfowitz exponential inequality for the ordinary empirical process. Their proof is valid for triangular arrays and real r.v .'s. Exponential inequality. Let X n1, X n2 , ' " ,Xnn be independent r.v. 's with respective d.f. 's Fn1, ... , Fnn and {dnd be any real numbers satisfying (Nl) . Then, V .\ > 0, V n ~ 1, p (sup xE IR

t

i=1

dni{I(Xni ::; x) - Fni(X)}

~.\)

< [1 + (81f)1 /2.\] exp( _.\2 /8) .

o

The above two theorems immediately suggest some interesting probabilistic questions. For example, is Vanderzanden's result valid for nonidentical r. v.'s {'l7d? Or can one remove the assumption of nonnegative {'l7d in Theorem 2.4.1?

2.4.2

Weak convergence of W.E.P.'s in 1)[0, I]", metric and an embedding result.

III

pq -

Next, we state a weak convergence result for multivariate r.v.'s. For this we revert back to triangular arrays. Now suppose that 'l7ni E [0 , 1]P , 1 ::; i ::; n , are independent r.v.'s of dimension p. Define n

(2.4.3)

Wd(t)

:=

L dni{I('l7ni ::; t) -

Gni(t)}, t E [0, 1]P .

i=1 Let c.; be the j-th marginal of Gni , 1 ::; i ::; n, 1 ::; j ::; p. Theorem 2.4.3 Let {'l7ni, 1 ::; i ::; n} be independent P: variate r.v. 's and {dnd satisfy (Nl) and (N2) . Moreover suppose that for each 1 ::; j ::; p , n

L

lim lim sup sup d;i{Gnij(t 0-+0 n 09::;1-0 i=1

+ 8) -

Gnij(t)} = 0.

66

2. Asymptotic Properties of W.E.P.'s

Th en, f or every to >

°

2. Moreover, W d ==;> som e W on (V[O , l]P,p) if, and only if,

fo r each s, t E [O ,l]P, Co v( Wd( S), W d(t )) -+ Co v( W( s) , W (t) ) =: C(s , t ). In this case W is necess arily a Gaussian process, P (W E C[O, I]" ) = 1, W (O ) = = W(l ).

°

o

Theor em 2.4.3 is essent ially proved in Vanderzanden (1980) , using results of Bickel a nd Wi chura (1971) . Mehra and Rao (1975) , Withers (1975), and Koul (1977), among ot hers, obtain t he weak convergence resul ts for { Wd } - processes when {17nd are weakl y dep endent. See Dehling and Taqqu (1989) and Koul and Mukherj ee (1992) for similar resul ts when {1Jnd are lon g range dep endent . Shor ack (1971) pr oved the wea k convergence of W d/q - process in the p-met ric, wher e q E Q, with

Q :=

{q ,q a cont inuous fun ction on [0, l],q q(t) t , C 1/ 2q(t ) .j,. for 0

~ O,q(t)

= q(l -

< t < 1/2, fal q-2(t )dt

<

t) ,

oo}.

Theorem 2.4.4 Suppo se that 1Jnl , . .. , 1Jnn are in dependent random variables in [0,1] with respective d.f. 's G n1, ' " G nn such that n

n-

1

L Gni (t ) = t ,

0 ~ t ~ 1.

i= l

In addition, suppose that {dnd sati sf y (Nl ) and (B) of Theor em 2.2 .2. Th en, (i) \/ to > 0, \/ q E Q , lim lim sup P ( sup 8--+ 0

n

It- sid

IWqd(t()t)

I

- W d((s)) > to) = O. q s

(ii) q- l W d ==;> «:' W , W a continuous Gaussian process with 0 covariance function C if, and only if Cd -+ C.

2.4. Some additional results for W .E.P. 's

67

Shorack (1991) and Einmahl and Mason (1991) proved t he following embedding result.

Theorem 2.4.5. Suppose that rJnl , . . . Tln n. are i. i. d. Uniform [0 , 1] r.v. 'so In addition, suppose that {dni} satisf y (Nl) and that n

n

Ldni = 0, i=l

n

L d; i = 0(1) . i=l

Th en on a rich enough probability space there exist a sequence of versions Wd of the processes W d and a fixed Brownian bridge B on [0, 1] such that sup n V IWd(t) - B(t)1 l/n:::;t:::;l -l /n {t(l - t) P /2-v

= Op(l) ,

for all 0 ::; v < 1.

Th e closed interval 1/ n ::; t ::; 1 - 1/ n may be replaced by the open 0 interval min{rJnj ; 1 ::; j ::; n } < t < max{rJnj ; 1 < j ::; n }.

2.4.3

A martingale property

In this subsection we shall prov e a mar tingale pr op erty of W. E.P.'s. Let X n1, X n2, · · · , X nn be indep end ent real r.v .'s with res pect ive d .f.'s Fn1,· · · , Fnn ; dn1,· · · , dnn be real numbers . Let a < b be fixed real numbers. Define n

Mn(t) := L dni{I(Xni E (a, t ]}{ 1 - Pni(a, t]}i=l

1,

n

Rn(t)

:=

Ldni{I(Xni E (t ,b] - Pni(t ,b]}{1- Pni (t, b]}- l , t E JR, i=l

where

Pni(S, t] := Fni(t) - Fni( s) , 0 ::; s ::; t ::; 1, 1 ::; i ::; n . Let T 1 C [a , oo),T2 C (- oo,b] be such that M n (t)[R n (t)] is welldefined for t E T1[t E T 2] . Let

F1n(t) := a - field{I(Xni E (a, sD, a::; s ::; t , i = 1, · · · ,n}, t E T 1, F2n(t) :=a-field{I(XniE( s ,bD, t ::; s ::;b, i = 1, · · · ,n }, tET2.

2. Asymptotic Properties of W.E.P. 's

68

Martingale Lemma. Under the above setup, 'in 2: 1, {Mn(t) , Fln(t) , t E Tt} is a martingale and {Rn(t) , F2n(t), t E T 2} is a reverse martingale. Proof. Write qi(a, s] = 1 - Pi(a, s]. Because {Xd are independent , for a ::; s ::; t ,

E{ Mn(t)IFln(S)} n

Ldi1(Xi E (a,s]) i= l

x

[E{ [I (Xi EE (a, t]) - Pi(a, t]] IXi E (a, s]} qi(a, t]

+f(Xi ~ (a, sl )E{ [f(Xi E (a, t) -

p,(a, tl]

[x, ~ (a, sl] }]

n

Ldi {1(Xi E (a,s]) i=l

+

1(X.I 5'=, d (a s])

i=l

d,

' ,

- Pi(a, t l }

}

qi(a, t] 1(Xi

n

L

{Pi(S q-(a ,ts

E

(a,s]) -qi(a ,s] ()

qi a,s

A similar argument yields the result about Rn .

= M n (s).

o

Note; The above martingale lemma is well known when the r.v.'s {X ni} are i.i.d. and dn i == n- 1 / 2 . In the case {X n ;} are i.i.d. and {d n i } are arbitrary, the observation about {M n} being a martingale first appeared in Sinha and Sen (1979). The above martingale Lemma appears in Vanderzanden (1980, 1984). Theorem 2.4.1 above generalizes a result of Finkelstein (1971) for the ordinary empirical process to W .E.P.'s of i.i.d. r.v.'s .. In fact , the set K is the same as the set K of Finkelstein . DO

3 Linear Rank and Signed Rank Statistics 3.1

Introduction

Let {X ni , Fnd b e as in (2.2.23) and {cnd be p x 1 real vectors. The rank and the ab solu te rank of the i th residual for 1 ::; i ::; n , U E W , ar e defined , resp ectively, as n

(3.1.1)

Riu

=

L

I (X nj - U'Cnj ::; X ni - U'Cni),

j= l n

Rtu

= L I (IX nj -

u ' cnj l ::; IXn i - u 'cni l)'

j= l

Let sp be a nondecreas ing real valued fun cti on on [0, IJ and define

for

U

E W , where

cp+(s ) = cp((s + 1)/2 ), 0 ::; s ::; 1;

s(x) = I (x > 0) - I (x < 0).

H. L. Koul, Weighted Empirical Processes in Dynamic Nonlinear Models © Springer-Verlag New York, Inc 2002

70

3. Linear Rank and Signed Rank Statistics

The processes {Td(cp, u), u E Il~.P} and {Tt(cp , u) , u E ffi.p} are used to define rank (R) estimators of f3 in the linear regression model (1.1.1). See, e.g., Adichie (1967), Koul (1971) , Jureckova (1971) and Jaeckel (1972). One key property used in studying these R-estimators is the asymptotic uniform linearity (AUL) of Td(CP , u) and Tt(cp , u) in u over bounded sets. Such results have been proved by Jureckova (1969) for Td(cp, u) for general but fixed functions ip ; by Koul (1969) for Tt(I, u) (where I is the identity function) and by van Eeden (1971) for Tt (cp , u) for general but fixed cp functions. In all of these papers {X n i } are assumed to be i.i.d . In Sections 3.2 and 3.3 below we prove the AUL of Td(cp , .), Tt(cp , .), uniformly in those cp which have Ilcplltv < 00 , and under fairly general independent setting. These proofs reveal that this AUL property is also a consequence of the asymptotic continuity of certain W .E.P.'s and the smoothness of {Fnd . Besides being useful in studying the asymptotic distributions of R-estimators of f3 these results are also useful in studying some rank based minimum distance estimators, some goodness-of-fit tests for the error distributions of (1.1.1) and the robustness of R-estimators against certain heteroscedastic errors.

3.2

AUL of Linear Rank Statistics

At the outset we shall assume

(3.2.1) cp E C

:=

{cp : [0, 1J -+ JR, cp E VI[O , 1], with

Ilcplltv :=

cp(1) - cp(O) = 1}.

Define the W.E.P. based on ranks, with weights {dnd , n

(3.2.2) Zd(t , u)

:=

L dnJ(R

i u ::;

nt) , 0::; t ::; 1, u E JRP .

i=1

Note that with ndn

(3.2.3) Td(.p; u)

= 2::7=1 c:

J -J

cp(nt j(n + 1))Zd(dt, u) Zd((n + 1)tjn, u)dcp(t) + ndncp(1) .

3.2. AUL of linear rank statistics

71

This representation shows t hat in order to prove the AUL of Td(cp , ') , it suffices to prove it for Zd(t ,' ), uniformly in 0 ::; t ::; 1. Thus, we shall first prove t he AUL prop erty for the Zd-process. Define, for x EIR, O ::;t ::;1 , u E W , n

(3.2.4)

Hnu(x )

:=

n -

1

L tix;

-

C~iU

::;

x ),

i= l

n

.-

n-

1

L

Fni( x

+ C~iu) ,

i= l

inf{ x ; Hnu( x) 2: t} , inf{ x; Hu(x) 2: t} . Not e that H o is the H of (2.2.23) . We shall write t hat for any d.f. G ,

tt;

for H no. Recall

G( G- 1(t)) 2: t , 0 ::; t ::; 1 and G-1 (G(x )) ::; x , x E lR. This fact a nd t he relati on n Hnu(X i t ::; 1, 1 ::; i < n ,

-

c~u )

== Mu yield that V 0 ::;

For t echnica l convenience , it is desirable to cente r t he weigh ts of lin ear rank statistics appropriately. Accordingly, let (3.2.6) Then , with Zw den oting the Zd wh en weights are {Wni} ,

Hen ce, for all 0 ::; t ::; 1, u E IRP , (3.2 .7) Nex t define, for ar bit rary real weights {dnd , and for 0 ::; t 1, U E IRP, n

(3.2.8)

Vd(t , u )

:=

L dni I (Xni i= l

C~iU ~ H;~ ( t )) .

<

3. Linear Rank and Signed Rank Statistics

72

By (3.2.5) and direct algebra, for any weights {dnd ,

(3.2.9)

sup IZd(t, u ) - Vd(t , ul] ::; 2 max Idnil · f ,u

t

Cons ider t he condit ion

(N 3) In view of (3.2.7) and (3.2.9), (N 3) implies t hat t he problem of proving t he AUL for t he Zd-process is reduced to provin g it for t he Vw process. Next , recall the definition (2.3.1) and define

(3.2.10) Note t he basic decomposition : for any real numbers {d nd and for all 0 ::; l ,u E IRP,

i -:

provided H is strict ly increas ing for a ll n 2 1. This decomposit ion is basic to t he following proof of t he AUL prop erty of Zd. Theorem 3.2.1 Suppose that {Xni , Fnd satisfy (2.2.24) , (N 3) holds, and {end satisfy (2.3.6) and (2.3.7) with d ni = Wni ' I n addition, assume tha t (C*) holds with d ni = Wni , H is strictly in creasing, the densities {J nd of { Fnd satisfy (2.3.5) an d that

(3.2.12)

lim lim sup max

sup

n

IH( x )- H(y)l <J

o-tO

Th en , f or eve ry 0

(3.2.13)

t

Ifni (X) - f ni (y )1=

o.

< b < 00 ,

sup ITw(t , u ) - Yw(t , 0 ) - J1w(H H;;J(t ), 0 ) + J1w(t, 0 ) I = op(I) ,

whe re . the supremum is being taken ov er 0 ::; t ::; 1,

Hull::; b.

Befor e proceeding to p rove t he t heo re m , we prove t he followin g lemma which is of indep endent int erest . In t his resul t , no ass umptio ns ot her t han independence of {Xnd are be ing used .

3.2. AUL of linear rank statistics

73

Lemma 3.2.1 Let H , H n, H u and H nu be as in (3.2.4) above. Assuming only (2.2.24), we have

IIHn

(3.2.14)

-

Hlloo -+ 0,

a.s ..

If, in addition, (2.3.6) holds and if, for any (3.2.15)

°< b <

IH(x) - H(y)1 -+ 0,

sup

(m n

Ix-yl:S2m n b

00 ,

= max Ileill), ~

then, (3.2.16)

sup IHnu(x) - Hu(x)1 -+ 0, a.s .. Ixl
Proof. Note that Hn(x) - H( x) is a sum of centered independent Bernoulli r.v .'s . Thus E[Hn(x) - H(x)J4 = O(n- 2 ) . Apply the Markov inequality with the 4t h moment and the Borel - Cantelli lemma to obtain

IHn(x) - H(x)1 -+ 0, a.s., for every x E lit Now proceed as in the proof of the Glivenko - Cantelli Lemma (Loeve (1963), p. 21) to conclude (3.2.14). To prove (3.2.16), note that Ilull ::; b implies that

The monotonicity of H nu and H u yields that for

lIull ::; b,

Hn(x - bm n) - H(x - bm n) + H(x - bm n) - H(x

x E JR,

+ bmn)

::; Hnu(x) - Hu(x) ::; Hn( x

+ bmn) -

+ bmn) +H(x + bmn) H(x

H( x - bmn) .

Hence (3.2.16) follows from (3.2.15) and the following inequality: L.H.S. (3.2.16) ::; 2 sup IHn(x) - H(x)1 Ixl
+

sup

IH(x) - H(y)l .

Ix-yl:S2mn b

o

74

3. Linear Rank and Signed Rank Statistics

Proof of Theorem 3.2.1. From (3.2.11), for all 0 :; t :; 1, u E JRP,

Tw(t , u)

=

[Yw(HH;;~(t) , u) - Yw(H H;;~(t), OJ +[Yw(H H;;~ (t) , 0 - Yw(t, O)J +Yw(t , O) - [J.L w(t, u) - J.L w(t, 0) - u'vw(t)J +[J.Lw(H H;;~ (t), u) - J.L w(H H;;~ (t), 0) -u'vw(HH;;~(t))J

+J.Lw(H H;;~ (t) , 0) - J.Lw(t , 0)

+ u'vw(t).

Therefore L.H.S. (3.2.13)

< sup IYw(t, u) - Yw(t, 0) - Y(t , 0)1

+ sup IYw(HH;;~ (t) , 0) - Y(t , 0)1 + 2 sup lJ.Lw(t , u) - J.L w(t , 0) - u'vw(t)1 + sup lu'[vw(HH;;~(t)) - v w(t)J1 = Al + A 2 + A 3 + A 4 , say ,

(3.2.17)

where, as usu al, the supremum is being taken over 0 :; t :; 1, lIull :; b. In what follows, the range of x and y over which the supremum is being taken is JR, unless specifi ed otherwise. Now, (2.3.5) implies that IH(x) - H(y)1 :; Ix - ylk . This and (2.3.6) together imply (3.2.15). It also implies that sup Ifni(Y) - fni( X)1 :; Ix - y l<8

Ifni(Y) - fni( X)I ·

sup IH(x) -H (y) l
for all 1 :; i :; n and all fJ > O. Hence, by (3.2.12) , it follows that {fnd satisfy (2.3.4) . Now apply Lemma 2.3.1 and (2.3.25) , with dni = Wni, 1 :; i :; n , to conclude that

(3.2.18)

j = 1,3.

Next , observe that x ,u

+supIHu(x) - H( x)l , x ,u

sup IHu( x) - H( x)1 :; sup IH(x x ,u

x

+ mnb) - H( x - mnb)l.

3.2. AUL of linear rank statistics

75

Hence, in view of Lemma 3.2.1, we obtain

sUPIHH;J(t) - tl-7 0, a.s.

(3.2.19)

t ,u

(We need to use the convergence in probability only) . Now, fix a 0 > 0 and let B~ = [SUPt,u jHH;J(t) - tl < oj. By (3.2.19) ,

((B~) C)

limnsup P

(3.2.20)

Next, observe that Yd( ' ,O) = as in (3.2.17), for every fJ > 0,

Wd'O

= O.

of (2.2.23) . Hence , with A 2

limsupP (IA 2 1 ~ fJ) n

S limsupP ( sup n

It-sl <8

jW~(t) - W~(s)1 ~ n, B~) .

Upon letting 0 -7 0 in this inequality, (2.2.25) implies (3.2.21) Next , we have (3.2.22)

lim lim sup sup Ilvw(t) - vw(s)11 0-+0

<

It- sl
n

lim lim sup max 0-+0

n

~

Ifni(Y) - f ni(X)1

sup IH(x) -H(y)l<8

n

xL II wi

Cil1

i=l

=0

,

by (3.2.12) and (2.3.7).

From (3.2.22) and (3.2.20) one obtains, similar to (3.2.21) , that A 4 = op(l) . This completes the proof of the theorem. 0 From a practical point of view, it is worthwhile to state the AUL result in the i.i.d. case separately. Accordingly, we have Theorem 3.2.2 Suppos e that X n1, ' " , X nn are i. i.d. F . In addition, assum e that (Fl) , (F2) , (N3) , (2.3.6) and (2.3.7) with dni == Wni hold. Then, V 0 < b < 00 , n

L

I

(3.2.23) sup IZd(t, u) - Zd(t, 0) - u' wnicniq(t) = op(l), o:::;t:Sl,llull:Sb i= l

3. Linear Rank and Signed Rank Statistics

76

ITd(
(3.2.24) sup

+ u'

'PEc,lIull:Sb

t

WniCni

i=1

J

qd
= op(1) .

where q = !(F- 1 ) . Proof. Take Fni == F in Theorem 3.2.1. Then (F1) and (F2) imply that q is uniformly continuous on [0, 1] and ensure the satisfaction of all assumptions pertaining to F in Theorem 3.2.1. In addition, /-lw(t , 0) = 0, t ::; 1. Thus, Theorem 3.2.1 is applicable and one obtains sup ITw(t, u) - Yw(t, 01 = op(1)

°::;

t ,u

which in turn yields (3.2.25)

sup ITw(t, u) - Tw(t, 0)1 = op(1). t,u

Next , let p = 2:7=1 WniCni· From (3.2.7), L.H.S. (3.2.23)

= sup IZw(t, u)

- Zw(t, 0) - u' pq(t)l .

t ,u

Hence, L.H.S.(3.2.23)

< sup {IZw(t, u) - Vw(t, ul] + IZw(t, 0) - Vw(t, 0)1 t ,u

+ITw(t, u) - Tw(t, 0)1 =

+ l/-lw(t , u)

- u' pq(t)I}

op(1) ,

by (3.2.9), (3.2.10), (N3), (3.2.25) and Lemma 2.3.1 applied to Fn i == F, d n i == Wni . To conclude (3.2.24), observe that by (3.2.23), the uniform continuity of q and (2.3.7) with dn i == Wni , L.H.S.(3.2.24)

::;

sup {IZd(t, u) - Zd(t, 0) - u' pq(t)1 t ,u

-l-ju'pllq((n + 1)t/n) - q(t)l} op(l) .

0

Remark 3.2.1 Theorem 3.2.2 continues to hold if F depends on n, provided now that the {q} are uniformly equicontinuous on [0,1] . 0

3.2. A UL of linear rank statistics

77

Remark 3.2.2 An analogue of Theorem 3.2.2 was first proved in Koul (1970) under somewhat stronger conditions on various underlying entities. In Jureckova (1969) one finds yet another variant of (3.2.24) for a fixed but a fairly general fun ction

(a)

F has an absoluiels) continuous density f whose a.e. derivative f satisfies

o < I(f) < 00 ,

I(f) :=

(b)

{wnd satisfy (N3).

(c)

1.

I: 7=:1 (Cni -

2.

maxi(cni - cn )2 = 0(1),

(d)

(e)

Then,

cn )2 ::;

M

< 00 ,

J(j I

2

f) dF.

(recall here Cni is 1 xl) ,

cn = n - 1 I:~=:l Cni ·

Either (dni - dnj) (Cni - Cnj) ;::: 0, \:j 1 ::; i ,j ::; n , or (d ni - dnj)(Cni - Cnj) ::; 0, \:j 1 ::; i,j ::; n . \:j

0

< b < 00 , n

sup Td(

where b(

:= -

+uL

Wni Cnib(

i=:l

f~
0

The strongest point of Theorem 3.2.3 is that it allows for unbounded score functions , such as the "Normal scores " that corresponds to

being the d.f. of a N(O , 1) r.v . However, this

3. Lin ear Rank and Signed Rank Statistics

78

is balan ced by requiring (a), (cl ) and (e) . Not e that (b) and (cl ) together imply (2.3.7) with dn i = Wn i , 1 ~ i ~ n . Moreover , Theorem 3.2.2 does not require anyt hing like (e) . Claim 3.2.1 (a) implies that

f is Lip(I/2) .

First , from Haj ek - Sidak (1967), pp . 19-20, we recall that (a) implies th at f (x ) --t 0 as Ixl --t 00. Now, absolute continuity and nonnegati vity of f implies t hat

~

If (x ) - f (y )1

i

Y

l(j/ J)ldF, x < y. < y,

Therefore, by the Cauchy-Schwarz inequality, for x

{i (j / J) 2dF . [F (y ) _ F (X)]} 1/2 Y

(3.2.26) If( x) - f (y )1

<

(3.2.27)

< /1 / 2(1 ).

Letting y

--t 00

in (3.2.27) yields

(3.2.28) Now (3.2.26) and (3.2.28) together imply

A similar inequ ality holds for x > y , thereby giving

If( x) - f (y )! ~ J3/4(1)l y - x \1/2, V x, Y E JR, and pr oving the claim . Consequ ently, (a) implies (F1). Not e t hat f can be uniformly cont inuous, bounded , positive a .e. , yet need not sa tisfy / (1 ) < 00. For exa mple, consider

f( x)

.-

f( x)

o~ x

~ 1, 2 (x - 2j + 1)/ 2 + , 2j - 1

(1 - x )/ 2,

j

+1-

.-

(2j

.-

f (- x) ,

x)/2j+2 , 2j x

~

o.

~

x

~

x

~

~

2j

2j

+ 1,

j 2: 1;

3.2. AUL of linear rank statistics

79

The above discussion shows that both Theorems 3.2.2 and 3.2.3 are needed. Neither displaces the other. If one is interested in the AUL property of, say, Normal scores type rank statistics, then Theorem 3.2.3 gives an answer. On the other hand if one is interested in the AUL property of, say, the Wilcoxon type rank statistics, then Theorem 3.2.2 provides a better result . The proof of Theorem 3.2.3 uses contiguity and projection technique a La Hajek (1962) to approximate T d (

I2 n- /

(3.2.29)

n

L

Ilcnill

= 0(1) .

i= 1

Then , V 0

(3.2.30)

< b < 00,

sup o::;tSI,lluI ISb

In

l 2 / (H H;

where YI , VI etc . are Yd , Cons equently,

Vd

J (t) - t)

+ YI(t ,O) + u'VI(t)1

of (2.3.1) , (2.3.10) with dn i

= op(1),

== n- I / 2 .

(3.2.31)

sup In l / 2 (H H ; J (t ) - t)1 = Op(1) . OSt::;1,lluII Sb

Proof.

Write Yd ') ,I-lI(-) for YI(',O),I-lI(-, 0) , resp ectively. Let I

80

3. Linear Rank and Signed Rank Statistics

denote the identity fun ction and set ~nu := n l / 2 (H nuH ;;J -1) . Then n l / 2 (H H;;~ - 1)

nl /2(HH;;~ - HuH;;~ - HnuH;;~) =

-[J-LI (HH;;J , u ) - J-LI(HH;;~) - u'vI(HH;;~)] -u'[vI(HH;;~) -

(3.2.32)

+ ~nu

VI -

+ ~nu

Y1

-[YI(HH;;~ ,u) - YI(HH;;~)] - [YI(HH;;~) - Y1 ] .

Now, not e that

SUPt,u

l~nu(t)1 ::; n- I / 2 . Hence

sup Inl / 2(H H;;~(t) - t) t ,u

+ YI(t) + U'VI (t)1

< sup 1J-LI(t, u) - J-LI(t) - u'vI(t)1 t ,u

+B sup IlvI(H H;;~ (t)) t ,u

(3.2.33)

VI (t)

II

+ sup IY1 (t, u) t ,u

- Y1 (t) I

+suplwt(HH;;~(t)) - wt(t) l, t ,u

where we have used th e fact that YI(t) = Wt (t ) of (2.2.23). The first term on th e r.h.s. of (3.2.33) tends to zero by Lemma 2.3.1 when applied with d ni = n- I / 2 . The third term tend s to zero in probability by (2.3.25) applied with d ni = n - I / 2 . To show th at th e other two terms go to zero in pr obabili ty, use Lemma 3.2.1, (2.2.25) and an ana logue of (3.2.22) for VI and an ar gument similar to th e one th at yielded (3.2.21) and (3.2.23) above. Thus we have (3.2.30). Since SUP t,u IYI(t, 0 ) + u'vI(t)1 = Op(l ), (3.2.31) follows. 0 Lemma 3.2.3 In addition to the assumptions of Th eorem 3.2.1 and (3.2.29), suppose that for every 0 < k < 00, (3.2.34)

max l

n l / 2ILni(t) - Lni( s) - (t - s )£ni(s )1

sup It-s l:S kn-

1/ 2

= op(l)

where Lni := FniH- 1, £ni := f ni(H -1) /h(H - 1), 1 < h := n- l I:~l f ni. Moreover, suppose that, with n

w(t)

:= n -

l

2:= Wni£ni (t ), i= l

0 ::; t ::; 1,

2

< n , with

81

3.2. AUL of linear rank statistics sup n 1 / 2 !w(t)! = 0(1).

(3.2.35)

°9 :'S1

Th en, V 0 < b <

00 ,

(3.2.36) sup IlLw( H H;J(t)) - lLw(t ) + {Ydt)

+ U'Vl (t)}n 1/ 2w(t)I

= op( l) ,

where ILw(t) , Y1 (t) stand for ILw(t , 0), Y1 (t , 0), respectively, and where the supremum is being tak en over 0 :S t :S 1, lI ull :S b. Proof. Let M u := ILw(H H;;J) - ILw . From (3.2.31) it follows that V E > 0, 3 K € and N It such t hat (3.2.37)

where

A~ = sUPIHH;J(t) - t l:S K€n - 1 / 2 ]. t ,u

By assumption (3.2.34) , there exists N 2€ such that n 2: N 2€ implies (3.2.38)

m9X l

n 1/ 2I L i(t) - Li(S) - (t - s )fi(s )1 <

sup It- sl:'SJ(,n-

1/ 2

Define

In view of (3.2.38) and (3.2.37) , (3.2.39)

P

(m9X sup nl/2 IZ~i (t)1 > E) t,u

< E,

l

for all n 2: N € := N 1€ V N 2€ . Moreover ,

«:

Mu I(A~) n

+ Mu I((A~) C)

L Wi Z~i + Zuo + n i= 1

where

1

/

2[ HH;J - I ]n 1/ 2w,

E.

82

3. Linear Rank and Signed Rank St atisti cs

Note that P (sup IZ~ol t,u

f= 0) ::; P ((A~) C) < E,

\:j

n

> N c.

By t he C-S inequality, (N 3) and (3.2 .39) , p

(St~.f ~ WiZ~i(t) > E ) "E, V n > N, .

These derivations together with Lemma 3.2.2 and (3.2.35) complete 0 t he pr oof of (3.2.3 6) . We combine Theorem 3.2.1, Lemmas 3.2.2 and 3.2.3 to obtain th e following Theorem 3.2.4 Under th e no tation and assum ptions of Th eorem 3.2.1, Lem m as 3.2.2 and 3.2.3, \:j 0 < b < 00 , n

(3.2.40)

sup IZd(t , u ) - Zd(t , 0) -

u'2~)dni - dn(t)) Cniqni(t) I i= l

= op( l) ,

(3.2.41)

su p !Td(
+u'

Ji»:-

dn(t)) Cniqni(t)d
I

i =l

= op( l),

where th e supremum in (3.2.40) is over 0 ::; t ::; 1, IIull ::; b, in (3.2.4 1) over sp E C, IIull ::; b, and where dn(t) := n - 1 I:~l dni€ni(t), qni := f ni(H- 1 (t)) , 0 ::; t ::; 1, 1 ::; i ::; n .

Proof. Let p(t) := I:~l(di - d(t)) Ci qi(t ). Not e th at th e fact that n - 1 I:~=l €i(t) == 1 implies t hat p (t ) = I:~=l (Wi - W(t))Ci qi(t) , where {wd are as in (3.2.6) . From (3.2.7)-( 3.2.9) ,

(3.2.42) L.H .S.(3.2.40)

<

sup IZw(t , u ) - Zw(t , 0) - u' p (t )1 o9 :S 1 ,llulI :Sb 4 max IWil + sup !Vw(t , u ) - Vw{t, 0) - u' p(t )l. ~ o:St:Sl,lI ull:Sb

83

3.3. AUL of linear signed rank statistics

Now, from Theorem 3.2.1 and Lemma 3.2.3, uniformly in 0 ::; t ::; 1, Ilull ::; b, (3.2.43)

sup ITw(t, u) - Yw(t)

+ {Yt{t) + Vl(t)u}n 1/ 2w(t)!

= op(l) , where Yd(t) stands for Yd(t, 0) for arbit rary weights {dnd . Therefore, sup IVw(t, 0) - Vw(t, 0) - u' p(t)1

= sup ITw(t, u) - Tw(t, 0) + fLw(t, u) - fLw(t, 0) ::; sup ITw(t , u) - Tw(t, 0)

+ sup IfLw(t, u)

+

- u' p(t)1

1 2w(t)1 U'VI (t)n /

- fLw(t , 0) - u'vw(t)1

= op(l) ,

by (3.2.43) and Lemma 2.3.1 and the fact that p(t) = v w(t) vt{t)n 1 / 2w(t) . This completes the proof (3.2.40). The proof (3.2.41) follows from (3.2.40) in the same fashion as does that (3.2.24) from (3.2.23) .

of of 0

Remark 3.2.3 As in Remark 2.2.3, suppose we strengthen (N3) to require

(BI) Then (C*) and (3.2.35) ar e a priori satisfied by L w .

0

Remark 3.2.4 If one is interested in the i.i .d . case only, then The0 orem 3.2.2 gives a better result than Theorem 3.2.4.

3.3

AUL of Linear Signed Rank Statistics

In this section our aim is to prove analogs of Theorems 3.2.2 and 3.2.4 for the signed rank processes {Td' (.p; u) , u E W} , using as many results from the previous sections as possible. Many details are quite similar. Define , for u E W , 0 ::; t ::; 1, x 2: 0, n

(3.3.1)

zt(t , u) :=

L dniI(R~ ::; nt) S(X ni i=l

n

Jnu(x) := n-

1

L I(IXni -

c~iul ::; x)

i= l

= Hnu(x) - H nu( - x) ,

C~iU) ,

3. Lin ear Rank and Signed Rank Statisti cs

84 n

Ju (x ) .- n- 1 L[Fni( X + C~iU) - Fni (- x i=1 = Hu( x) - H u( - x) ,

+ C~iU)]

n

.-

L dniI(IXni - c~iul :S J;~ (t )) S(X ni - C~iU) , i=1

s t(t , u ) .-

L dniI (! Xn i - c~iul :S J - 1(t )) S(X ni - C~iU) , i=1

J.L d (t , u ) .-

L dni J.L~i (t , U) = ESt (t , u }, i=1 Fni(J - 1(t) + CniU) + Fni( -J- 1(t) + C~iU)

v t(t , u)

n

n

J.L~i( t , u)

.-

-2Fni(C~iU) ,

1 :S i :S n.

In the above and sequel, J and I n stand for Jo and J nO' resp ectiv ely. We also need

(3.3.2)

Yd+(t , u )

:=

st(t , u ) - J.L d (t , u ),

f ;j (t , u )

:=

v t (t , u ) - J.Ld(t , u ),

O:S t :S 1, u E IW .

An alogous to (3.2.11), we have the basic decomp osition : For 0 :S t :S 1, u E ffiP ,

Now, not e that , w.p . 1, for all 0 :S t :S 1, u E IRP ,

(3.3.4) Y/ (t , u ) =

Yd(H J - 1(t ), u ) +Yd(H (_J- 1(t)), u ) - 2Yd(H( O) , u ),

where Yd is as in (2.3.1). Therefore, by Theorem 2.3.1 (see (2.3.25)) , under the assumptions of that theorem and strictly increasing nature of J and H ,

(3.3.5)

sup IY/ (t , u ) - Y/ (t , 0)1= op( l). t, u

On e also has , in view of the cont inuity of {Fnd , a relation like (3.3.4) between J.L d a nd J.L d. Thus by Lemma 2.3.1, un der the ass ump ti ons t here, (3.3.6)

sup lJ.Ld (t , u) - J.L d(t , 0) - u'v+(t)1 = op(l ). t, u

3.3. AUL of linear signed rank statistics

85

where, for 0::; t ::; 1, n

vd(t)

:=

Ldni ni [Jni(J C

1

(t )) + fni(-J- 1(t)) - 2fni(0)].

i=l

We also have an analogue of Lemma 3.2.1: Lemma 3.3.1 Without any assumption except (2.2.24), sup IJn(x) - J(x)1 -+ 0 a.s . OS;xS;oo

If, in addition, (2.3.6) and (3.2.15) hold, then IJnu(x) - Ju(x)j -+ 0 a.s ..

sup oS;xS;oo,llullS;b

Using this lemma, arguments like those in Theorem 3.2.1 and the above discussion, one obtains Theorem 3 .3.1 Suppose that {X ni , Fnd satisfy (2.2.24) , (2.3.5) and that {dni , cnd satisfy (N1), (N2), (2.3.6) and (2.3.7). In addition, assume that (3.3.7)

lim lim sup max n

0-+0

l

sup

Ifni(x) - fni(y)1 = 0

lJ(x)-J(y)l
and that H is strictly increasing for every n. Then, for every 0 < b < 00 , (3.3.8) sup ITt (t, u) - Y/ (t , 0) - fhd(J J;~ (t) , 0) e
+ fhd (t, 0)1 0

We remark here that (3.3.7) implies (3.2.12) . Next , note that if {Fi } are symmetric about 0, then

fhd(t,O) = 0, 0::; t ::; 1, n ~ 1.

(3.3.9)

Upon combining (3.3.9) , (3.3.8) with (3.3.6) one obtains Theorem 3.3.2 In addition to the assumptions of Theorem 3.2.1, suppose that {Fni, 1 ::; i ::; n} are symmetric about O. Then , for every 0 < b < 00, n

(3.3.10)

sup 099,lIull:Sb

= op(I) ,

Izt(t , u) - zt(t , 0) - u'

L dnicnw;;i(t) I i=l

3. Linear Rank and Signed Rank Statistics

86

n

(3.3.11) sup Ir/(
+ u' L

1 1

dniCni

i=l

v;t(t)d
I

0

= op(l),

where v;t(t)

:=

2[jni(J- 1(t)) - fni(O)], 1:::; i :::; n ,

0:::; t :::; 1,

and where the supremum in (3.3.11) is taken over

= 0(1),

!

Thus (3.3.9) follows from this, (3.3.8), (3.3.7) and (3.3.6). The conclusion (3.3.11) follows from (3.3.9) in the same way as (3.2.24) follows from (3.2.23) . 0 Because of the importance of the i.i .d. symmetric case, we specialize the above theorem to yield Corollary 3.3.1 Let F be a d.j. symmetric around zero, satisfying (F1), (F2) and let X n 1 , ' " ,Xnn be i.i.d. F. In addition, assume that {d ni, cnd satisfy (N1), (N2) , (2.3.6) and (2.3.7) . Then, for every 0 < b < 00 ,

(3.3.12)

sup Izt(t , u) - zt(t, 0) - U'L;idniCniq+(t) I o:St:Sl,llull:Sb = op(l),

(3.3.13) sup Ir/(
where q+(t)

:=

+ L;idnic~iu

e

q+(t)d
Jo

2[j(F- 1((t + 1)/2)) - f(O)], 0 :::; t :::; 1.

0

Remark 3.3.1 van Eeden (1972) proved an analogue of (3.3.13) without the supremum over ip ; but for square integrable
3.3. AUL of linear signed rank statistics

87

Now, we return to Theorem 3.3.1 and expand the J.l!-terms further so as to recover an extra linearity term. Define , for 0 ~ t ~ l ,u E W ,

y; (t , u)

n

L dni[I(IX ni - cniul ~ J-1(t)) - Fit(J-l(t))) ,

:=

i= l

n

v;t(t)

:=

LdniCnilfni(J-1(t)) - fni(-J-1(t))] , i= l

where

Not e the relation: For arbitrary {dnd , (3.3.14)

From (3.3.14) and (2.3.25) applied with dni = n (3.3.15)

1/ 2 ,

we obtain

sup IYt(t , u) - Yt(t , 0)1 = op(I) . t ,u

Note that in the case dni == n- 1/ 2 , (2.3.7) reduces to (3.2.29) . Next , under (3.3 .7) and (2.3.7), just as (3.2.22) , (3.3 .16)

limlimsup sup IIv;t(t) - v;t(s)11 = 0, n It-sl
0---+0

for the given {dnd and for dni == n- 1 / 2 . Using (3.3.15) , (3.3.16) and calculations similar to those don e in the proof of Lemma 3.2.2, we obtain Lemma 3.3.2 Und er th e condit ion s of Theorem (3.2.) and (3 .2.29) (3.3.17)

sup Inl/2(JJ;~(t) - t) t ,u

+ Yt(t , 0) + u'vi(t)1

= op(I) .

Cons equently,

(3.3 .18)

o

Similarly arguing as in Lemma 3.2.3, we obtain the following Lemma 3.3.3. In it J.l!(t) , J.lt(t) etc. stand for J.l!(t ,O),J.lt(t ,O) etc . of (3.3.1) .

3. Linear Rank and Signed Rank Statistics

88

In addition to the assumptions of Theorem 3.2.1, (3.2.29), assume that for every 0 < k < 00 ,

Lemma 3.3.3

(3.3.19)

sup nl/2ItL~i(t) - tL~Js) - (t - s)e~i(s)1 It-sl:Skn- 1j 2 = 0(1)

max t

where {tL~i} are as in (3.3.1), e~i(S) := [fni(J- 1(S)) - fni(-J- 1(s))Jlh+(J -l(s)) , 0

S s S 1,

n

h+(x) := n- 1 L[fni(X) - fn i(-x)],

x 2: O.

i=1

Moreover, with dt(t) := n- 1 I:~=1 dnie~i(t) , 0 S t S 1, assum e that sup Inl/2d~(t)1 = 0(1) .

(3.3.20)

°9:S 1

Then, for every 0 < b <

00,

(3.3.21) sup ItLt(JJ;~(t)) - tLt(t)

+ {yt(t) + u'v;:(t)}nl /2d~(t)1

= op(l) ,

where the supremum is taken over the set 0 S t S 1, Ilull s b. Finally, an analogue of Theorem 3.2.3 is Theorem 3.3.3 Under the assumptions of Theorem 3.3.1, (3.2.29) , (3.3.19) and (3.3.20), for every 0 < B < 00 , (3.3.22)

sup

09:S 1 ,llull:Sb = op(l),

(3.3.23)

0

Izt(t , u) - zt(t , 0) - u'[vt(t) - v;:(t)nl/2d~(t)]1

sup

IT/(

+u' 10 [vt(t) = op(l) .

v;:(t)nl/2d~(t)]d
Remark 3.3.2 Unlike the case in Theorem 3.2.3, there does not appear to be a nice simplification of the term vt - vin 1/ 2dt. However,

3.4. Weak convergence of rank and signed rank W.E.P. 's

89

it can be rewritten as follows:

n

=

L diCi[Ji(J-1(t)) + Ji( -J-1(t)) -

2Ji(O)]

i=l

This representation is somewhat revealing in the following sense. The first term is due to the shift U'C i in the LV. Xi and the second term is due to the nonidentical and asymmetric nature of the distribution of X i, 1 ::; i ::; n . 0 Remark 3.3.3 If one is interested in the symmetric case or in the i.i.d, symmetric case then Theorem 3.3.2 and Corollary 3.3.1, re0 spectively, give better results than Theorem 3.3.3.

3.4

Weak Convergence of Rank and Signed Rank W.E.P. 'so

Throughout this section we shall use the notation of Sections 3.2 - 3.3 with u = O. Thus, e.g., Zd(t) , zt(t), etc . will represent Zd(t ,O), zt(t , 0), etc. of (3.2.2) and (3.3.1) , i.e., for 0 ::; t ::; 1, n

(3.4.1)

Zd(t) =

L dniI(Rni ::; nt) , i= l

n

zt(t) =

L

dniI(R~i ::; nt)s(Xni) ,

i =l

n

Vd(t) =

L i=l

n

dniI(Xni < H;;l(t)) ,

f1d(t) =

L dniLni(t) , i=l

where Rni (R~i) is the rank of X ni (IXnil) among X n1, . . , , X nn ( IXn11 , ... , IXnnl ). We shall first prove the asymptotic normality of Zd and zt for a fixed t, say t = v, 0 < v < 1. Note that this corresponds to proving the asymptotic normality of simple linear rank and signed rank statistics corresponding to the score function r.p that is degenerate at v.

90

3. Linear Rank and Signed Rank Statistics

To begin with consider Zd(V). In the following theorem v is a fixed number in (0,1) . Theorem 3.4.1 Suppose that {Xnd, {Fnd, {Lnd, L d are as in (2.2.23) and (2.2.24) . Assume that {dnd satisfy (N1), (N2) and that H is strictly increasing for each n. Also assume that lim limsup[Ld(V

(3.4.2)

0--+0

n

+ <5) -

Ld(v - <5)] = 0,

and that there are nonnegative numbers £ni(v), 1 ~ i ~ n , such that for every 0 < k < 00 ,

(3.4.3)

max I

sup n 1 / 2IL ni(t) - Ln i(v) - (t - v)£n i(v)1 It-v l::;kn- 1 / 2

= o(1). Denoting n

dn(v)

.-

n- 1

L

dni£ni(v),

i=l

n i=l

assume that

(3.4.4) liminf a~(v) >

(3.4.5)

n

o.

Then,

The proof of Theorem 3.4.1 is a consequence of the following three lemmas. In these lemmas the setup is the same as in Theorem 3.4.1. Lemma 3.4.1 Under the sol e assumption of (2.2 .24), sup IHH;;l(t) -

tl

= op(l).

09::;1

Proof. Upon taking u = 0 in (3.2.19) , one obtains sup IHH;;l(t) O
tl

~

sup IHn(x) - H(x)1 - oo::;x::;+oo

+ n- 1 =

op(l) ,

3.4. Weak convergence of rank and signed rank W.E.? 's

91

by (3.2.14) of Lemma 3.2.1. 0 Lemma 3.4.2 Let Yd(t) denote the Yd(t , 0) of (2.3.1). Then, under (3.4.3), for every E > 0, limlim supP( sup IYd(t ) - Yd(v )! > n Jt- vl
0-+ 0

E) =

0.

Proof. Apply Lemma 2.2.2 to Ilni = H (Xnd , G ni = L ni , to obtain t hat Yd == Wd of t hat lemma and th at

p( It-vl sup IYd(t )
Yd(v )1 >

E)

< IiE-2 [Ld(v + 0) - Ld(V - - : + P(lYd(V - 0) - Yd(v)1 > E/2) +P(lYd(V + 0) - Yd(v - 0)1>

E/4)

< (K+ 20)E- 2 [L d(v +0) - Ld(v -o) ] , (by Chebyshev) . The Lemm a now follows from t he assumption (3.4.3) . Lemma 3.4.3 Under (3.4.3), for every E > 0,

limsu pP(!Yd(HH;l(V)) - Yd(v)1 > E)

= O.

n

Proof. Follows from Lemmas 3.4.1 and 3.4.2.

o

Remark 3.4.1 Lemmas 3.4.2 could be deduced from Coro llary 3.3.1 wh ich gives t he tight ness of t he process Yd un der stronger cond it ion (C*). Bu t here we are interested in the behavior of Yd only in the neighb orh ood of one point v and th e above lemma proves t he continuity of Yd at th e point v at which (3.4.3) holds. Simil arly, many of the approxima tions that follow could of course be dedu ced from pr oofs of Theorem 3.2.1 and 3.2.2. Bu t these th eorems obtain results uniformly in 0 :S t :S 1 un der rather st ronger conditions t ha n would be needed in the pr esent case. Of course vari ous decompositi ons used 0 in t heir proofs will be useful here also. Proof of Theorem 3.4.1. In view of (3.2.9) and (N 2) , it suffices to prove that {ad(v )}-lTd(v ) --+ d N(O, 1), where

3. Linear Rank and Signed Rank Statistics

92

But, from (3.2.11) applied with u = 0,

Td(V)

+ ILd(HH;;l(v)) -lLd(V), Yd(V) + op(1) + ILd(HH;;I(v)) -lLd(V) ,

Yd(HH;;I(v)) =

Apply the identity (3.2.32) with u = n- 1j 2 to obtain ,

w.p.1 by (3.4.5) .

°and Lemma 3.4.3 with di ==

- YI (H H;;I (v)) - YI ( v)

+ op(1)

+ "»(1).

Since Yr(v) --'td N(O ,v(1 - v)) , WI (v)1 = Op(1) . Again, argue as for (3.2.36) with u == 0, t == v (i.e., without the supremum on the L.H.S. and with u == 0 , t == v), to conclude that

Combine this with (3.4.6) to obtain

(3.4.7) Td(V) Yd(V) -

n l / 2 d(v )Y1(v ) + op(1)

n

2:)dni -

d(v)){I(Xni :::; H -1(v)) - Lni(v)}

+ op(1).

i=1

The theorem now follows from (3.4.5) and the fact that {ad(v)}-Ix { leading term in the RHS of (3.4.7) } - t d N(o, 1) by the L-F CLT , 0 in view of (Nl) and (N2) . Remark 3.4.2 If {Fnd have densities {fnd then eni (v ) can be taken to be f ni(H-1(v)) jh(H -1(v)) , just as in (3.2.34) . However , if one is interested in th e asymptotic normality of linear rank statistic corresponding to the jump score function, with jump at v , then we need {L nd to be smooth only at that jump point. The above Theorem 3.4.1 bears strong resemblance to Theorem 1 of Dupac-Hajek (1969). The assumptions (Nl) , (N2) , (3.4.3), and (3.4.5) correspond to (2.2) , (2.13) and (2.22) of Dupac - Haj ek. Condition (3.4.2) above is not quite comparable to condition (2.12) of Dupac-Hajek but it appears to be less restrictive. In any case, (2.12) and (2.13) together imply th e boundedness of {ei(v)} and

3.4. Weak convergence of rank and signed rank W .E.P. 's

93

hence the condition (3.4.4) above. Taken together, then, the assumptions of the above theorem are somewhat weaker than those of Dupac - Hajek . On the other hand, the conclusions of the Dupac Hajek Theorem 1 are stronger than those of the above theorem in that it asserts not only {Zd(V) - J.Ld(v)}ai1(v) -+d N(O, 1) but also that E [ai1(v)(Zd(V) - J.Ld(V)W -+ 0, for r = 1,2 , as n -+ 00 . However , if one is only interested in the asymptotic normality of {Zd(V)} then the above theorem appears to be more desirable. Moreover , in view of the decomposition (3.2.11) , the proof presented below makes the role played by conditions (3.4.2) and (3.4.3) transparent. The assumption about H being strictly increasing is not really an assumption because, without loss of generality, one may assume that {Fi } are not flat on a common interval. For, if all {Fi } were flat on a common interval, then deletion of this interval would not change the distribution of R n1,' . . , R nn and hence of {Zd} . 0 Next, we turn to the asymptotic normality of zt(v) . Again, put u = 0 in the definition (3.3.1) to obtain, n

(3.4.8)

:L dniI( IXnil S J;l(t))s(Xnd ,

vt(t)

i= l n

st(t) =

:L dniI(IXni

!

S J -1(t))s(Xnd ,

i=l

J.L~i(t) =

Fni(J-l(t))

+ Fni( -J-1(t)) -

2Fni(0) ,

n

J.Lt(t)

=

:L dniJ.L~i(t) ,

Os t

S 1,

i=l

y+ = d

s d+ - J.Ld+'

Like (3.2.9), we have (3.4.9)

sup Iz t (t ) - vt(t) 1 OStSl

s 2 max Idil· t

Because of (N2) , it suffices to consider vt only. Observe that

3. Linear Rank and Signed Rank Statistics

94

where Yd is as in (2.3.1). Rewrite

Y/(t)

(3.4.10)

=

{Yd(H J-1(t)) - Yd(H(O))} -{Yd(H(O)) - Yd(-J-1(t))}

Yd1(t) - Yd2(t ), say . This representation motivates the following notation as it is required in the subsequent lemma. Let Pi := Fi(O) , qi := 1 - Pi and define for o:::; t :::; 1, 1 :::; i :::; n, (3.4.11)

L~(t)

.-

L1;(t) .-

{Fi(J-l(t)) - pd/qi,

qi > 0,

0,

qi = 0;

{Pi - Fi(-J -1(t))}/pi ,

Pi > 0,

0,

Pi = 0;

Observe that f-Lt(v) = qiL~(v) - PiL1;(v), 1 :::;

n

Ldl(t)

.-

i:::; n . Also define ,

n

Ld;lqiL~(t) ,

L~ (t)

:=

i=l

L d;PiL~(t) , 0 ~ t ~ l. i=l

Argue as for the proof of Lemma 2.2.2 and use the triangle and the Chebbychev inequalities to conclude Lemma 3.4 .4 For every

p( It -sup IYdj(t) vl
E

> 0 and 0 < v < 1 fixed,

Ydj(v)1 >

E)

~ ('" + 20)E - 2 [Lt (v + 6) - Lt(v - 6)], j = 1,2,

uihere r: does not depend on E,6 or any other underlying entities. 0

Theorem 3.4.2 Let X n1, '" ,Xnn be independent r.v. 's with respective continuous d.j. 's Fn1, . .. ,Fnn and d n1, .. . , d nn be real numbers. Assume that {dnd satisfy (Nl) , (N2) . In addition, assume the following.

3.4. Weak convergence of rank and signed rank W .E.? 's

95

For v fixed in (0,1 ), (3.4.13) lim lim sup ILJ.(v 8-+ 0

n

!1

+ 8) -

LJ.( v - 8)1 'J

= 0,

j

= 1,2.

(3.4.14) There exist num bers {ft(v ), 1 ::; i ::; n ; j = 1, 2} such such that max 1

< k < 00 , j

0

\j

sup It- vi:Sk n -

= 1,2 ,

n 1/2ILt (t) - Lt (v) - (t - v)ft(v)1 = 0(1). 1/2

With n

dt(v)

.-

n-

1

Ldni{qi f~( v) -Pi f 7;(v)} , i= l

2( T V) .-

n

L

{d~i [L~i(V)

-

{1t~i(v)} 2]

i= l

(3.4.15)

(a)

liminfT 2 (v) > O.

(b)

lim sup n 1 / 2 Idt

n

(v)1< 00.

n

Th en, (3.4.16)

where ItJ is as in (3.4.8). Proof. The proof of thi s th eorem is similar to th at of Theorem 3.4.1 so we shall be bri ef. To begin with, by (3.4.9) and (N2) it suffices to prove that {T(V)}-lrt(v) --'td N(0 ,1) , where r t(v) := Vd( v) - ItJ(v) . Apply Lemma 3.4.1 above to the LV. 'S jXn11,... ,IXnnl , to conclude that sup IJ(J;l(t )) - tl = op(1). O
From t his, (3.4.10), (3.4.12) and (3.4.13),

rt(v)

Yd+(JJ; l (v )) + ItJ (JJ;l (v )) - ItJ( v). Yd+(v ) + [lt J (JJ;l (V)) - ItJ (v )] + op(1).

3. Linear Rank and Signed Rank Statistics

96

Again, apply arguments like those that yielded (3.4.6) to {IXnil} to obtain n 1/2[JJ;1(V) = v] = -yt(V) + op(l), where yt(v) is as in (3.3.19) with t

r;j(v)

= Yd+(v)

= v and u = O.

- nl/2d~(v)yt(v)

Consequently,

+ op(l) = Kt(v) + op(l)

where

Kt(v) n

L

{dndI(J(IXnil) 'S v)s(Xnd -

/-l~i(v)]

i=l

-d~(v)[I(J(IXnil) 'S v) - L~i(V)]}. Note that Var(Kt(v)) = r 2(v) . The proof of the theorem is now completed by using the L-F CLT which is justified, in view of (Nl) , (N2), and (3.4.15a) . 0 Remark 3.4.3 Observe that if {Fi } are symmetric about 1/+ :::::: 0:::::: d+n and r 2(v ) = 'f:, I·d2L+(v) t"l nz nl .

a then 0

Remark 3.4.4 An alternative proof of (3.4.16), using the techniques of Dupac and Hajek (op .cit.}, appears in Koul and Staudte, Jr. (1972a) . Thus comments like those in Remark 3.4.1 are appropriate here also . 0 Next , we turn to the weak convergence of {Zd} and {zt}. These results will be stated without proofs as their proofs are consequences of the results of the previous sections in this chapter. Theorem 3.4.3 (Weak convergence of Zd). Let X n1, ... , X nn be independent r, v. 's with respective continuous d.j. 's Fn1, ' " ,Fnn . With notation as in (2.2.23), assume that (Nl), (N2) , (C*) hold. In ad-

dition , assume the following: There are measurable functions {£ni, 1 'S i 'S n} on [0,1], such that V a < k < 00 , (3.4.17) max I

sup n 1/ 2ILni(t) - Lni(s) - (t - s)£ni(s)1 = O. It- sl::;kn- 1 / 2

Moreover, assume that (3.4.18)

lim sup sup n 1 / 2 Idn (t )1 < n

O::;t::;l

00 ,

~,4 .

Weak convergence of rank and signed rank W.E.P. 's

(3.4.19)

lim lim sup sup n 1/ 2Idn( t ) - dn(s )1= 0, n

0-.0

(3.4.20)

97

it-s id

liminf0"2(t) > 0,

0 < t < 1.

n

Finally, with Kd(t) assume that

:= 2:~=1 (dni -

dn(t)){I(Xni ~ H-1( t )) - Lni(t)} ,

(3.4.21) C(t , s) lim COV(Kd(t), K d(s )) n

n

li~

'L

(dni - dn(t ))(dni - dn(s)) L ni(S)( 1 - Lni(t )),

i= l

exists for all a ~ s ~ t ~ 1. Then, Zd - fL d converges weakly to a mean zero, covariance C continuous Gaussian process on [0,1 ], tied down at a and 1. 0 Remark 3.4.5 In (3.4.17), withou t loss of generality it may b e assumed t hat n - 1L;i£ni( S) = 1,0 ~ s ~ 1. For , if (3.4.17) holds for some {£ni, 1 ~ i ~ n} , then it also hold s for £ni (S) == £~i ( s ) := n 1f2[Lni(S + n - 1/ 2) - Lni( s)], 1 ~ i ~ n , a ~ s ~ 1. Because

n- 12:;=1 Lni(s) == s, n - 1 2:~=1 £~i (s ) == 1. 0 Remark 3.4.6 Condit ions (C *), (N l ) and (3.4.18) may b e repl aced by t he condit ion (B) , b ecause, in view of t he previous rem ark, n

n 1/ 2Idn (t )1= In-1/2 'Ldni£ni(t )1 ~ n 1 / 2m ?-x jdn il, 1

i= l

a ~ t ~ 1.

0

Remark 3.4.7 In t he case Fni have density f ni, one ca n choose

£ni = f ni(H- 1)/n- 1

n

'L f nj(H- 1),

1 ~ i ~ n.

j= l

Remark 3.4.8 In t he cas e Fni == F, F a cont inuous and st rict ly increasi ng d.f., Lni(t ) == t , £ni (t ) == 1, so t hat (C *) and (3.4.17) (3.4.20) are t rivia lly satisfied . Moreover , C(s, t) = s(1 - t) , a ~ s ~ t ~ 1, so t hat (3.4.21) is satisfied . Thus Theor em 3.4.3 includes Theor em V.3.5.1 of Haj ek and SId ak (1967). 0 Theorem 3.4.4 ( Weak convergence of zt ). Let X n 1 , ' " , X nn be independent r.v. 's with respective d.f. 's F n 1 , " ' , Fnn and let dn 1 ,

98

3. Lin ear Rank and Signed R ank Statistics

... , dnn be real num bers. A ssume that (N 1) and (N 2) hold and that the following hold. (3.4.22) lim lim sup n

8-t0

sup

0
[Ld·(t 'J

+ 0) -

(3.4.23) There are measurable functions

1

sup It-sl:Skn -

(3.4.25)

= 0,

j

= 1,2.

i t , 1 ~ i ~ n , j = 1,2 , 00 ,

n 1/2I L ;j(t) - L;j(s ) - (t - s)it(s) 1= 0(1). 1/ 2

(3.4.24) lim sup sup n 1 /2I d~(t) 1 n

'J

such that for any 0 < k <

on [0, 1] max

Ld·(t)]

< 00 ,

0
lim lim sup sup n1 /2I d~(t) - d~ ( s)1 = n It - sl <6

o.

8-t0

(3.4.26) lim infT 2 (t) > 0, n

0

< t < 1.

(3.4.27) lim Cov(Kd (s ), K d (t )) = C+(s , t) exists, 0 n

< s ~ t ~ l.

Then, Zd - f.l d converges weakly to a continuous mean zero covariance C+ Gaussian process, tied down at O. 0 Remark 3.4.9 Rem arks 3.4.5 through 3.4.7 are applicable here also, with ap pro priate modifi cations. 0 Remark 3.4.10 Su pp ose t hat Fni n - 1 / 2 . T hen

== F, F cont inuous, and dni

sup IZd(t) - f.l d (t )1 0
sup n 1 / 2 1{H n (x ) - Hn(O)} - {Hn( O) - Hn(- x )}

o<x
-{F( x) - F(O)} - {F(O) - F (-x )} /, pr ecisely the st atisti c T~ prop osed by Smirnov (1947) to tes t t he hypothesis of symme try about F. Smirno v considered only t he null distribu tion. Theorem 3.4.4 allows one to st udy its asy mptotic distribution un der fairly general indep endent alternatives. For arbit rary {dnd , subject to (N 1) and (N 2) , t he stat ist ic sup{ IZd(t) - f.l d (t )I;O :S t :S I } may be considered a generalized Smirnov stat istic for testing symmet ry.

4 M, R and Some Scale Estimators 4 .1

Introduction

In th e las t four decad es stat ist ics has seen t he emerge nce and consolid ation of many compe titors of t he Least Squ ar e estimator of {3 of (1.1.1) . The most prominent are th e so-called M- and R- estimators. The class of M-estimators was introduced by Huber (1973) and its computational asp ects and some robustness properties are available in Huber (1981). The class of R-est imators is bas ed on th e ideas of Hod ges and Lehmann (1963) and has been develop ed by Adi chie (1967), Jureckova (1971) and J aeckel (1972). One of the attractive features of these est imators is th at they are robust aga inst certain outliers in erro rs. All of these estimators ar e translation invariant , whereas only R-estimators are sca le invariant. Our purpose here is to illustrate the usefulness of the results of Chapter 2 in der ivin g the asymptotic distributions of these estimators under a fairly general class of heterosced astic errors . The Section 4.2.1 gives the asymptot ic distributions of M-estimators while those of R-estimators are given in Section 4.4. Among ot her things, th e resul ts obtained ena ble one to study their qualitative robustness against an array of non-identical error d.f.'s converging to a fixed error d .f. T he sufficient cond iti ons given here are fairly general for H. L. Koul, Weighted Empirical Processes in Dynamic Nonlinear Models © Springer-Verlag New York, Inc 2002

100

4. M, R and Some Scale Estimators

the underlying score functions and the design variables.

Efron (1979) introduced a general resampling procedure, called the bootstrap, for estimating the distribution of a pivotal statistic. Singh (1981) showed that the bootstrap estimate B n is second order accurate, i.e. provides mor e accurate approximation to th e sampling distribution Gn of the standardized sample mean than the usual normal approximation in the sense that sup{IGn(x) - Bn(x)l ;x E 1R} tends to zero at a fast er rat e than that of th e square-root of n. This kind of result holds mor e generally as noted by Babu and Singh (1983, 1984).

Section 4.2.2 dis cusses similar results pertaining to a class of Mest imators of {3 when th e errors in (1.1.1) ar e i.i.d. It is not ed that th e th e distribution of the bootstrap est imators obtained by resampling the residuals according to a W .E.P. is second order acc urate . A similar result is shown to hold for Shorack's (1982) modified bootstrap est imators .

In an attempt to make M-estimators scale invariant one often needs a pr eliminary robust sca le est imator . Tw o such esti mators are th e MAD (median of absolute deviations of residuals) and the MASD (median of absolute symmetrized deviations of residuals) . The asymptotic distributions of th ese estimators under heteroscedasti c errors appear in Section 4.3.

In carrying out th e analysis of variance of an experimental design or a linear model based on ranks one needs an estimator of the asymptotic variance of certain rank statistics, see, e.g., Hettmansperger (1984). These variances involve the functional Q(J) = J fd rp(F) where sp is a known functi on , F a common error d.f. having a density f · Some est imators of Q(J ) under (1.1.1) are pr esented in Section 4.5. Again, the results of Cha pter 2 are found useful in proving th eir consiste ncy.

4.2. M-ESTIMATORS

4.2 4.2.1

101

M-Estimators First order approximations: Asymptotic normality

This subsection contains th e asymptot ic distributions of M - est imators of (3 when the errors in (1.1.1) are het eroscedastic. The following subsect ion 4.2.2 gives some results on t he boot st rap approximat ions to t hese distributions . Let the mod el (1.1.1) hold . Let 'ljJ be a nondecreasing functio n from JR to JR. The correspo nding M- est imato r .6. of (3 is defined to be a zero of the M - score J 'ljJ (y )V( dy , t) , where V is as in (1.1.2) . Our objec t ive is to invest igate t he asym ptot ic be havior of A - 1(.6. - (3) when the errors in (1.1.1) ar e het erosced astic. Our meth od is st ill the usu al one , v.i.z., to obtain the expansion of the M - score uniformly in t E {t ; IIA - 1(t - (3)11 ~ b}, 0 < b < 00, to observe t ha t there is a zero of the M - score , .6. , in this set and then to apply this expansion to obtain the approxima t ion for A -1(.6. - (3) in te rms of the given M - score at the true (3 . To make all this pr ecise, we need to standardize the M - score . For t hat reason we need some more not ation. Let (4.2.1)

.- diag (fn 1(y) , ' . . , fnn(Y) ), Y E JR, C .- AX' A * (y)d'ljJ (y )XA,

A *(y)

J A!

T( 'ljJ , t)

.- _C - 1

T( 'ljJ , t)

.-

'ljJ (y )V (dy , t) ,

A ~1(t - (3 ) - T( 'ljJ ,{3), tEJRP .

An approximat ion to .6. is given by the zero A of T('ljJ , t ), v.i .z., A - 1(A - (3)

(4.2.2)

= T('ljJ ,{3).

A basic resu lt needed to make this pr ecise is t he AUL of T( 'ljJ , t) in A -1(t - (3) , given in Theorem 4.2 .1 below. Let (4.2.3)

W :=

{'ljJ: JR I--7JR, 'ljJ E'DI(JR) , !'ljJ! tv bounded }.

Theorem 4.2.1 Let {(x~i ' Ynd , 1 ~ i < n}, {3 , {Fni , 1 ~ i ~ n } be as in the model (1.1.1) satisfying all conditions of Th eorem 2.3.3. In

4. M, R and Some Scale Estimators

102 addit ion, assume the follo wing:

lim supllC- 1 l1 oo <

(4.2.4)

00 .

n

Th en, V 0

(4.2.5)

< b < 00 ,

sup IIT(7P,,8

+ Au) - T( 7P,,8 + Au)11

= op(l) ,

l/J ,u

where the supremum is tak en over all 7P E w and Ilull ::s: b.

Proof. Rewrite, afte r int egration by parts ,

T( 7P, t) - T( 7P, t )

=

J

C - 1 A [V(y, t)

-V(y ,,8) - fdH( y))A - l (t - ,8)Jd7P(Y). Now (4.2.5) rea dily follows from this and (2.3.35). 0 In ord er to use this theorem , we must be able to argue th at IIA - l (Li - ,8)11 = Op(l) . To t hat effect , define J-li

.-

E7P (ed,

T? = V ar{ 7P (ei )},

1 ::S: i ::s: n ,

n

b.,

.-

ET(7P ,,8) = _C- 1 A L

XiJ-li ,

i=l

and observe that n

EllA - l (a -,8) - b n l12 = C- 1 LX~(X'X) -lxiTlc-1 = 0(1) , i=l

by (4.2.3), (4.2.4) and t he fact that 2::7=1 x~(X'X)-lX i == P < Hence by th e Markov inequ ality, V f. > 0, 3 0 < K € < 00 , 3 ,

00 .

Thus, assuming that n

(4.2.6)

LXiJ-li = 0, i=l

and arguing via Bro uwer 's fixed point theorem as in Huber (1981, p. 169), one concludes , in view (4.2.5), that IIA-l (A - ,8)11 = Op(l ).

4.2.1. Asymptotic normality of M-estim ators

103

A rout ine app lication of (4.2.5) enables one to conclude t hat

A -1(.6. - (3) = T ('IjJ ,{3 ) + op(l) .

(4.2.7)

Not e th at , un der (4.2.6), wit h T o('IjJ , (3 )

= CT ('IjJ , (3 ),

n

ETo('IjJ , {3) T~ ('IjJ , {3) = A LX~Xi TlA = AX'TXA i=l where T = diag (Tf , " . , T~) . Moreover , for any A E IRP , P

n

n

A'To ('IjJ ,{3) = L L Aj dij'IjJ (ed = LA'Ax~ 'IjJ ( ei ) i=l j=l i=l where {dij} are as in (2.3.30). In view of (2.3.31) and (2.3.32), (NX) and (4.2.6) imply t hat A'T o('IjJ ,{3) is asymptoti cally nor ma lly distrib ut ed wit h mean 0 and t he asymptot ic vari an ce A' AX'TXAA. T hus by the Cram er - Wold device [Theorem 7.7, p. 49, Billingsley (1968)], (4.2.4) and (4.2.7), :E- 1/ 2A - 1(.6. - (3 ) ---7d N( O, I p x p ) ,

(4.2.8) with :E as a

:=

C - 1 AX'TXAC - 1 . We summarize the above discussion

Proposition 4.2.1 Suppose tha t th e d.f. 's {Fnd of th e errors an d th e design m at rix X of (1.1.1) satisfy {4.2.4}, {4.2.6} an d the assumptions of Th eorem 2.3.3 including that H is stric tly increasing f or each n 2: 1. Th en {4.2.8} holds.

Now, consider t he case of the i.i .d. errors in (1.1.1) wit h Fn i Then ,

(4.2.9) Tl C

J (J

'ljJ 2 dF -

(J

jd'IjJ) I p x p ,

'ljJdF )2 = :E =

2 T ,

(J

j d'IjJ) - 2 T2 I p x p .

t XiJ 'ljJdF = O. i= l

F.

(say), 1 ::; i ::; n ,

Con sequent ly (4.2.4 ) is equivalent to requiring observe t hat (4.2.6) becomes (4.2.10)

==

J j d'IjJ > O.

Next ,

4. M, R and Some Scale Estimators

104

Obviously, this is satisfied if either I:~=l Xi = 0, i.e. if X is a centered design matrix or if f'lj;dF = 0, the often assumed condition. The former excludes the possibility of the presence of the location parameter in (1.1.1) . Thus to summarize, we have Proposition 4.2.2 Suppose that in (1.1.1), F ni

==

F. In addition,

assume that X and F satisfy (NX), (Fl), (4·2.10) and that f fd'lj;

>

O. Then ,

o Condition (4.2.10) suggests another way of defining M- estimators of (3 in (1.1.1) in the case of the i.i. d. errors. Let n

(4.2.11 )

Xnj

.-

l

n- L

Xnij,

1::; j

::; p;

i=l

x~

.-

X'

.- [x n ,"' , xn]nxp , X c := X - X .

Assume (NXl)

(Xnl,"', xnp ),

(X~XC)-I exists for all

n 2: p , maxi X~i(X~Xc)-IXni = 0(1).

Let n

(4.2.12) T*('lj;,t)

.-

Al L(Xni - xn)'lj;(Yi - x~it) , t E W , i=l

Al

.-

(X~Xc)-I/2 .

Define an M-estimator

a * as a solution t

of

T*('lj;, t) = O.

(4.2.13)

Apply Corollary 2.3.1 p times, lh time with dni = i th element of the i" column of X cA I , 1 ::; i ::; n,1 ::; j ::; p , to conclude an analogue of (4.2.5) above , v.i .z., sup ljI,IIA j l (t-{3)II :Sb

IIT*('lj;, t) - T*('lj; , t)11 = op(l)

105

4.2.1. Asymptotic normality of M-estimators where

T*(1/J , t) := A 1I (t - (3) - ( / f d1/J) -1 T*(1/J,{3) . The proof of this claim is exactly similar to that of (4.2.5) with appropriate modifications of replacing X by X , and A by Al and using Fni == F in the discussion there. Now, clearly, Fni == F implies that ET*(1/J , (3) = 0 , n

EIIT*(1/J,{3)11 2= 2:)Xi -x)/A~Adxi -X)T 2 =

0(1).

i= I

Hence , IIT*(1/J,{3)11 = Op(l). If A" is the zero ofT*(1/J, ·), then

Argue, as for (4.2.7) and (4.2.8) to conclude the following Proposition 4.2.3 Suppose that in (1.1.1), Fni == F . In addition, assume that X and F satisfy (NX1) , (F1) and (F2) . Then,

Remark 4.2.1 Note that the Proposition 4.2.2 does not require the condit ion J 1/JdF = O. An advantage of this is that d * can be used as a pr eliminary est imator when constructing adaptive estimators of

{3 . An adaptive estimator is one that achieves the Hajek - Le Cam (Hajek 1972, Le Cam 1972) lower bound over a large class of error distributions. Often a minimal condition required to const ru ct an adaptive estimator of {3 is that F have finite Fisher information, i.e., that F satisfy (3.2.a) of Theorem 3.2.3. See, e.g., Bickel (1982), Fabian and Hannan (1982) and Koul and Susarla (1983). Recall, from Remark 3.2.2, that this implies (F1) . On the other hand, the condition (NX1) does not allow for any location term in the linear regression model.

0

4. M, R and Som e Scale Estimators

106

So far we have been dealing with the linear regression model with kn own scale. Now consider the model (2.3.36) where"( is an unknown sca le param et er . Let s be an n l / 2 - consiste nt estimator of "(, i.e. (4.2.14) Define an M-estimat or (4.2.15)

t

x i7j!((Yi -

.6.. 1 of f3

as a solut ion t of

x~t)S-l) =

0 or

J

7j! (y )V (sdy , t) = O.

i= l

To keep exposition simple, now we shall not exhibit 7j! in some of the fun cti ons defined below. With C is as in (4.2.1) above, define, for an a > 0, t E W , (4.2.16)

f ' (y ) .-

(!I( y) , '" , f n(Y)),

Cl

.-

« :"? AX'

S(a , t)

.-

A

S( a , t)

.-

A - l (t -

J

y f (y )d7j! (y ),

J

7j! (y )V (a dy, t ),

f3h- l + C - lC l n l / 2( a

-

"(h- l

-C-lSh, (3) , Not e t hat by (NX) , (Fl) , (F3) , and (4.2.3), (4.2.17)

IICll1 = 0(1 ).

The following theorem is a direct consequence of Theorem 2.3.4. In it N; := {( a , t) : a > 0, t E W , IIA -l(t - (3)11 :::; lYy , Inl / 2( a- "()1 :::; lYy }, O < b < 00 . Theorem 4.2.2 Let {(x~i ' Y n i ) , 1 :::; i :::; n }, f3,"( , {Fn i , 1 :::; i :::; n} be as in {2.3 .36} satisfying all the condi tions of Th eorem 2.3.4. Moreover, assum e {4.2.3} and {4.2.4} hold. Th en, fo r ever y 0 < b < 00 ,

(4.2.18)

sup IIS(a , t) - S( a , t)11 = op (l ).

where the supremum is taken over all 7j! E W, and (a , t')' E Ni .

0

4.2.1. Asymptotic normality of M-estimators

107

Now argue as in the proof of the Proposition 4.2.1 to conclude Proposition 4.2.4 Suppose that the design matrix X and d.f. 's

{Fnd of {Eni} in {2.3.36} satisfy {4.2.5}, {4.2.6} and the assumptions of Theorem 2.3.4 including that H is strictly increasing for each n

:2:

1. In addition assume that there exists an estimate s of s satisfying

{4.2.14}. Then (4.2.19)

A -1(A 1 - 13),-1

= C- 1 Sb ,13) - C- 1C 1n 1/ 2 (s where

.6. 1

')'),-1

+ op(l), o

now is a solution of (is).

Remark 4.2.2 In (4.2.6), F; is now the d.f. of Ei, and not of ')'Ei, 1 i

~

n.

~

0

Remark 4.2.3 Effect of symmetry on

.6. 1 .

As is clear from

(4.2.19) , in general the asymptotic distribution of:0 1 depends on s . However, suppose that for all 1

d'ljJ(y)

=

~

i

~

n, y E JR,

-d'ljJ(-y) , /i(y) == /i( -y).

Then J y/i (y)d'ljJ (y) = 0, 1 ~ i ~ n, and, from (4.2.16), C 1 Consequently, in this case,

Hence, with

~

= o.

as in (4.2.8) , we obtain

Note that this asymptotic distribution differs from that of (4.2.8) only by the presence of ')'-1. In other words, in the case of symmetric errors

{Ei}

and the skew symmetric score functions {'ljJ} , the asymp-

totic distribution of M-estimator of 13 of (2.3.38) with a preliminary n 1/2 - consistent estimator of the scale parameter is the same as that of ')'-1 x M - estimator of 13 of (1.1.1). 0

108

4.2.2

4. M, R and Some Scale Estimators

Bootstrap approximations

Befor e discussin g t he specific b oot strap approximations we shall describe the conce pt of Efron 's b oot strap a bit more gen erall y in the one sample set up . Let 6 ,6, ' .. , ~n be n i.i .d . G LV . 's, Gn be t heir emp irical d.f. and Tn = Tn(en, G) be a fun cti on of e' := (6 ,6, ' " , ~n) and G such tha t Tn(e , G) is a LV. for every G. Let ( 1 , (2,' " , ( n den ot e i.i.d. Gn LV .'S and ( ~ := ((1, (2 , ' " ,(n). The bootstrap d.f. B n of Tn(e , G) is the d.f of Tn((n, G n) under G n. Efron (1979) showed , via numeri cal studies, that in several exam ples B n provides better approximat ion t o the d.f. I'n of Tn(en' G) under G than the normal approximation. Singh (1981) substantiated this obse rvat ion by provin g that in the case of the standardized sample mean the b ootstrap est imate B n is secon d order accurate , i.e.

(4.2.20)

supj lf' n(x) - Bn(x) l; x E ~} = o(n - l / 2 ) , a.s..

Recall that t he Edgeworth expansi on or t he Berry-Esseen b ound gives that sup [ I' n(x) - cI> (x )l; x E ~} = O(n- l / 2 ) , wh ere cI> is the d .f. of a N(O , 1) r .v. See, e.g., Feller (1966) , Ch. XVI ). Babu and Singh (1983, 1984), among ot hers , po inted out that this phen omen on is shared by a large class of statistics. For fur ther reading on boot st rapping we refer the reader to Efron (1982), Hall (1992). We now turn to the probl em of bootstrapping M-estimators in a lin ear regr ession model. For the sake of clarity we shall restrict our attention to a si m ple lin ear regr ession model only. Our main purpose is to show how a certain weighted emp irica l sampling distribution naturally help s t o overco me some inherent difficulties in defining boot strap M-estimators. What follows is based on the work of Lahiri (1989). No proofs will be given as they involve intricate te chnicalities of the Edgeworth expans ion for indep endent non-identicall y dist ributed LV .'s. Accordingly, ass ume that {ei , i ~ I } are i.i.d . F r. v.'s, {Xni' 1 ~ i ~ n} are t he known design points, {Yni , 1 ~ i ~ n } are obse rvable

4.2.2. Bootstrap approximations

109

r.v.'s such that for a (3 E JR, (4.2.21) The score function 'ljJ is assumed to satisfy (4.2.22)

J'ljJdF=O.

Let An be an M-estimator obtained as a solution t of n

(4.2.23)

L xni'ljJ{Yni -

Xni t ) = 0,

i=1

and Fn be an estimator of F based on the residuals eni := Yni XniAn , 1 ::; i::; n. Let {e~i' 1::; i::; n} be i.i.d. Fn r.v.'s and define (4.2.24)

1 ::; i ::; n.

The bootstrap M-estimator

~~

is defined to be a solution t of

n

(4.2.25)

L xni'ljJ{Y;i - Xni t ) =-= O. i=1

Recall, from the previous section, that in general (4.2.22) ensures the absence of the asymptotic bias in An . Analogously, to ensure the absence of the asymptotic bias in ~~ , we need to have Fn such that (4.2.26) where En is the expectation under Fn . In general, the choice of Fn that will satisfy (4.2.26) and at the same time be a reasonable estimator of F depends heavily on the forms of 'ljJ and F . When bootstrapping the least square estimator of (3, i.e., when 'ljJ{x) = x, Freedman (1981) ensure (4.2.26) by choosing Fn to be the empirical d.f of the centered residuals {eni - en., 1 ::; i ::; n} , where en. := n- 1 '2:/]=1 enj' In fact , he shows that if one does not center the residuals, the bootstrap distribution of the least squares estimator does not approximate the corresponding original distribution. Clearly, the ordinary empirical d.f. tt; of the residuals {eni; 1 ::; i ::; n} does not ensure the validity of {4.2.26} for general designs

110

4. M, R and Some Scale Estimators

and a general '1/;. We are thus forced to look at appropriate modifications of the usual bootstrap. Here we describe two modifications. One chooses the resampling distribution appropriately and the other modifies the defining equation (4.2.6) a La Shorack (1982) . Both provide the second order correct approximations to the distribution of standardized An .

Weighted Empirical Bootstrap Assume that the design points {xnd are either all non-negative or all non-positive. Let W x = L~=I IXnil be positive and define n

(4.2.27)

F1n(y) := w;1

L

jXnilI(eni :S y) , Y

E R

i= l

Take the resampling distribution Fn to be FIn. Then, clearly, TI

EIn'l/;(e~l) = w;1

L

I Xnil'l/;U~nd

i=1

n

sign(xt}w;1

L Xni'I/; (Yni -

Xni A) = 0,

i= 1

by the definition of An . That is, FIn satisfies (4.2.26) for any '1/; .

Modified Scores Bootstrap Let Fn be any resampling distribution based on the residuals . Define the bootstrap estimator ~ns to be a solution t of the equation n

(4.2 .28)

L Xni['I/; (Y; i -

Xni t ) - En'l/;(e~i)J = O.

i=1

In other words the score function is now a priori centered under Fn and hence (4.2.26) holds for any Fn and any '1/; . We now describe the second order correctness of these procedures. To that effect we need some more notation and assumptions. To begin with let r; := L~=I x~i and define n

TI

bi« >

L x~dr~, i= l

bx

:=

L i=l

IX~il/r~ .

111

4.2.2. Bootstrap approximati ons For a d .f. F and any sampling d .f. F n , define, for an x E JR,

I(X)

.- E 'ljJ (el - x) ,

w(x) = (J2(x ) := E { 'ljJ(el - x) - I(x)}2, In(X) := En 'ljJ ( e~l - x) ,

Wl(X) .-

E{'ljJ (el - x) - I(x)}3 ,

wn(x)

.-

(J~ (x) = En{'ljJ(e~ l - x) - In(x)}2 ,

Wln(X)

.-

En{ 'ljJ ( e~l - x) - ,n(x)}3 ,

{i: 1 ~ i ~ n , IXni l > cTx bx }, /1;n(c):= #An(c),

An( c) .-

C> 0.

For any real valued fun cti on g on JR, let g, 9 denot e its first and second derivat ives at wh enever t hey exist, resp ectively. Also, write In, Wn et c. for IN (O),WN(O), etc. Finally, let a := - -y/ (J, an := --Yn/(In and, define for x E JR, H 2(x) := x 2 - 1, and

°

Pn(x )

:=

In InWn .. " } X2 (x) - bix [{ - - - 3- -2 2 (In (In an

+

WIn ] -6 3 H 2(x ) ep (x ). (In

In t he following t heore ms , a .s, means for almost a ll sequences {ei; i 2: I} of i.i.d. F r. v.'s. Theorem 4.2.3 Let the model (4.2.21) hold. In addition, assum e

that 'ljJ has uniformly continuous bounded second derivative and that the following hold: (a) ---t 00. (b) a > O. (c) There exists a constant 0 < C < 1, such that livr; = O(K;n(c)) . (d) m xlnTx = O(Tx) . (e) There exist constants 0 > 0, 8 > 0, and q < 1, such that

T;

sup[IEex p{it'ljJ( el - x)}1 : [z ] < (f)

2:~=1 exp ( -

Th en, with

D.~

8, ltl > OJ < q.

AW;/ T;) < 00, V A > O. defined as a solution of (4.2.25) with Fn = FIn,

sup IPIn( anTx (D.~ - Lin) ~ y ) - 'Pn( y)1 = o(mx/Tx ), y

sup IPIn(a Tx(Lin - {3) ~ y) - PIn (Tx (D.~ - Lin) ~ y)1 y

= o(mx /T x)'

a.s .,

where PIn denot es the bootstrap probability under FIn, and where the supremum is over y E JR. 0

4. M, R and Some Scale Estimators

112

Next we state the analogous result for b. n s .

Theorem 4.2.4 Suppose that all of the hypotheses of Th eorem 4.2.3 except (J) hold and that b. n s is defined as a solution of {4.2.28} with Fn =

ti;

the ordinary em pirical of th e residuals . Th en,

sup IFn (anTx (b. ns - A n) ~ y) - Pn(y)1 = o(mx/Tx)' y

sup IPn (a Tx (A n,B) ~ y) - Pn(Tx(b. ns - A n) ~ y)1 y

=

where

i;

o(mx/Tx)'

a.s .,

denot es th e bootstrap probability under

it.;

o

The proofs of t hese t heorems appear in Lahiri (1989) where he also discusses analogous results for a non -smooth 'lj;. In th is case he chooses th e sampling distribution to be a smooth est imator obtained from the kernel type density est imator. Lahiri (1992) gives exte ns ions of th e above theorems to multiple linear regression models. Here we br iefly comment about the assumptions (a) - (f) . As is seen from th e pr evious section , (a) and (b) ar e minimally required for the asymptotic normality of M-estimators. Assumptions (c), (e) and (f) ar e required to carry out the Edgewort h expansions while (d) is slightly stronger than Noether's cond ition (N X) applied to (4.2.21). In particular, X i == 1 and X i == i sa tisfy (a) , (c) , (d) and (f). A sufficient cond ition for (e) to hold is that F have a positive density and 'lj; have a cont inuous positive derivative on an open int erval in JR.

4.3

Distributions of Some Scale Estimators

Here we shall now discuss some robust sca le estimators. Definitions. An est imator !1(X, Y ) based on th e design matrix X and the observat ion vector Y of f3 is said to be locatio n in vari ant if (4.3.1)

!1(X, Y

+ Xb) = !1(X , Y ) + b , V b E JRP.

It is said to be scale in variant if

(4.3.2)

!1(X ,aY) =a!1( X, Y), VaE JR, a#O .

113

4.3. Distributions of some scale estimators

A scale estimator 8{X , Y) of a scale parameter , is said to be location invariant if

8{X, Y

(4.3.3 )

+ Xb) = 8{X , Y) ,

V bE 1W.

It is sa id to be scale invariant if

8{X ,aY) = a8 {X , Y ), V a > O.

(4.3.4 )

Now observe that M-estimators Li and a * of f3 of Section 4.2.1 are location invariant but not scale invariant. The est imators Lil defined at (4.2.15) , ar e location and scale invariant whenever 8 satisfies (4.3.3) and (4.3.4) . Not e that if 8 does not satisfy (4.3.3) then Lil need not be location invari ant. Some of the candidat es for 8 ar e (4.3.5)

8

.-

{{n -

n

p )-1

2)Yi - x~/3)2}

1/ 2

,

i= 1

81

.-

med { IYi - x~/3I ; 1 < i

82

·-

med{ IYi -Yj -{Xi - Xj )'/3 1;1 ::; i < j ::; n } ,

::; n} ,

where /3 is a pr eliminary est imato r sa t isfying (4.3.1) and (4.3.2). Estimator 8 2 , wit h /3 as the least square est imator , is the usual est imator of the error var iance, ass uming it exists. It is known to be non-robust against outl iers in the errors . In robustness studies one needs scale estimators th at are not sensit ive to outliers in t he err ors . Estimator 8 1 has bee n mentioned by Huber (1981, p. 175) as one such candidate . The asymptotic properties of 8 1, 8 2 will be discussed shortl y. Here we just mention that each of these est imators est imat es a different scale paramet er but that is not a point of concern if our goal is only to have location and scale invariant M-estimators of f3 . An alte rn ative way of having location and scale invariant Mest imators of f3 is to use simultaneous M-estimation method for est imating f3 and , of (2.3.36) as dis cussed in Huber (1981). We ment ion here, without giving det ails, that it is possible to st udy the asymptotic joint distribution of these est imators under heteroscedastic errors by using th e results of Ch apter 2.

4. M, R and Some Scale Estimators

114

We shall now study the asymptotic distributions of 81 and 82 under th e model (1.1.1) . With F, denoting the d.f, of ei , H = n - 1 L:Z:::1 r; let (4.3.6)

pdy ) .- H(y ) - H( -y) ,

(4.3.7)

P2( y)

J[H(Y + x) - H( - y + x )JdH (x ), y

.-

~ O.

Define 1'1 and 1'2 by th e relations

P1hd =

(4.3.8)

1/2,

Not e that in th e case F, == F , 1'1 is median of the distribution of 1e1 1and 1'2 is median of the distribution of lei - e21· In general, 1'j , Pj , etc . depend on n , but we suppress this for th e sake of convenience. The asymptotic d istribution of S j is obtained by th e usual method of connecting the event { 8 j :S a} with certain events based on certain empirical processes, as is done when studying th e asy mptot ic distribution of th e sample medi an , j = 1,2. Accordingly, let , for y ~ 0, n

(4.3.9)

S( y)

'-

L I (IYi - x~,B1 :S y) , L L I (IYi - Yj i= l

T(y )

:=

(Xi -

xj)',B1:S y )

.

l :Si :Sj :Sn

Then , for an a > 0,

+ 1)2- 1},

(4.3.10) {Sl :S a}

=

{S( a) ~ (n

{S(a) ~n2-1}

C

{81 :Sa} ~ {S(a) ~nr1 -1} , neven.

n odd,

Simi larly, wit h N := n(n - 1)/2, for an a > 0, {T(a) ~ (N

(4.3.11)

{S2:S a} {T(a) ~ N2- 1}

C

+ 1)r 1},

N odd {82 :S a} ~ {T(a) ~ N2- 1 - I} , N even

Thus, to study the asymptotic distributions of 8 i - j = 1,2 , it suffices to st udy thos e of S(y) and T (y ), y ~ O. In what follows we shall be using the notation of Ch apter 2 with t he following modifi cations. As before, we shall write S~ , J-L~ etc . for

115

4.3. Distri bu tions of some scale estimators s~ , IL~ et c. of (2.3.2) whe never dni we shall take

== n- 1/ 2 . Moreover, in (2.3 .2),

(4.3.12) With these modifications, for all n

~

1, Y

~

0,

Sr(y , v ) - Sr( - y, v ) = n- 1/ 2

S (y)

n

L I(l ei -

c~v l :S y) ,

i= l

J

[S r(y + x , v ) - Sr( - y + x, v )JSr (dx , v) - 1,

with probability 1, where v = A - 1 (fJ - (3). Let (4.3.13) IL~(Y , u )

1L1(H(y) , u ),

y10 (y , u )

Y1(H(y ), u ),

J

K(y , u )

[y10(y

W(y , u ) 9i(X)

'-

hi( x)

.-

y

E IH;;

+ X, u ) - y 10( -y + X , u )JdH (x ),

y10(y , u ) - y 10( - y, u ), Y ~ 0, u E {fih'2 + x ) - Ji( + x)},

-,2 {fih'2 + x ) + Ji( -,2+ x )} ,

IH;P,

for 1 :S i :S n , x E R We shall write W(y) , K (y) etc . for W(y , O) , K (y,O) et c. Theorem 4 .3. 1 Assume that (1.1.1) holds with X and {Fnd satis-

fying (NX ) , (2.3.4) and (2.3.5) . Moreover, assume that H is strictly in creasing for each n and that lim lim sup

o~O

n

sup

O ~ s ~l - o

(4.3.14)

About {fJ} assume that (4.3.15)

[H(H - 1(s + 15) ± , 2) - H (H - 1(s) ± ,2)J = O.

4. M, R and Some Scale Estimators

116 Then, V a E JR, (4 .3.16)

P(n 1 / 2 ( s l

-

"n) ::; a"/'l)

= P ( W("/'l) + n- 1 / 2 ~ x~A{fk/'l) - Ji( -')'t}} . v

~ -a· ')'In-1 t[Ji(')'t} + Ji( -')'t}]) + 0(1), i= 1

(4.3.17) P(n / 2(s2 1

=

P (2K(')'2)

')'2) ::;

a')'2)

+ n- 3 / 2 L ~ Cij

9i(X)dFj(x) . v

J

1

~ -')'2 an - 1

J

tJ

hi( X)dH(X))

+ 0(1) .

1=1

where

Cij

=

(Xi -

Xj)' A,

1 S i , j ::; n .

Proof. We shall give the proof of (4.3.17) only ; that of (4.3.16) being similar and less involved . Fix an a E JR and let Qn(a) denote the left hand side of (4.3.17) . Assume that n is large enough so that an := (an- 1 / 2 + 1)')'2 > 0. Then , by (4.3.11), N odd

It thus suffices to study P(T(a n ) ~ N2- 1 + b), bE JR. Now, let

.- n- 1 / 2[2n- 1T(y) + 1] - n 1 / 2p2(Y) , y ~ 0, kn .- (N + 2b)n- 3 / 2 + n - 1 / 2 - n 1 / 2p2(an) .

Tt{y)

Then , direct calculations show that (4.3.18) We now analyze k n : By (4.3.8) ,

4.3. Distributions of some scale estimators

117

But

n 1/ 2(p2(an) - P2(,2)] =

n

1 2 /

J [{ H(a n + x) - H ('Y2

+ x)} -{H (-an + x) - H (

-,2 + x)} ]dH(x).

By (2.3.4) and (2.3.5), t he sequence of distributi ons {P2} is tight on (IR, B), implying that ,2 = 0( 1), n- 1/ 2 , 2 = 0(1). Consequently, 1 2

n /

J {H(±an + x ) - H(±' 2 + x )} dH (x) a,2 n - 1

t

J h(±, 2 + x )dH (x ) + 0(1),

i= l

- a, 2n - 1

(4.3.19 ) kn

t

J[h (,2

+ x) + fi(

-,2+ x) ]dH(x)

i =l

+ 0(1). Next , we approximate T1 (an ) by a sum of independ ent LV . ' S . The pr oof is similar to t he one used in approximating linear rank statistics of Secti on 3.4. From t he definition of T 1 , T 1 (y)

n - 1 / 2 J [Sr(y

+ x,v) -

Sr( - y + x ,v )]Sr (dx , v )

n - 1 / 2 J[YIO(y + x, v ) - y 10 ( - y + x ,v )]Sr (dx , v )

+ n- 1 / 2 J[f-L~ (Y + x, v ) - f-L~( - Y + x, v)]y1o(dx , v) + n- 1 / 2 J[f-L~(Y+ x,v)

-

f-L~( -y+ x,v)]f-L~(dx,v) - n 1/ 2p 2 (y)

Bu t E 3 (y)

.-

n- 1 / 2 J

[f-L~ (Y + x, v ) - f-L~( -Y + x , v)]f-L~ (dx, v ) -

n

1/ 2p2(Y)

4. M, R and Some Scale Estimators

118

n- 3/2

t 2; !

{Fl (y + X +

C~jv) -

-Fi(y + x) =

n- 3/2

Fi(-y + x

+ C~jv)

J

2=1

t L C~jV ! i =1

+ F1(-y + x) }dFj(X)

[Ji(y + x) - Ji( -y + x)]dFj(x)

+ u p (1),

j

by (2.3.3) , (NX) and (4.3.15). In this proof, up(1) means op(1) uniformly in Iyl :::; k , for every 0 < k < 00. Integration by parts, (4.3.15) , (2.3.25) , H increasing and the fact that J n -1 / 2f-l~ (dx , v) = 1 yield that

E2(Y) .-

n- 1/2 n - 1/2

!

! {JJ,~(Y + ! + {y10(y

x , v) -

f-l~( -y + x , v)}YI0(dx , v)

x , v) - y 10(-y + x, v)}f-l~(dx, v)

{y10(y + x ) - y 10(-y + x)}dH(x)

+ u p (1).

Similarly,

(4.3.20)

Et{y) =

n - 1/2

!

{YI0(y

+ x)

- y 10(-y + x)}S~(dx)

+ up (1).

Now observe that n-1/2S~ = H n , the ordinary empirical d.f of the errors {ei} ' Let

Eu(Y)

.- !

{y10(y + x) - Y?( -y + x)}d(Hn(x) - H(x))

Z(y) - Z( -y), where

Z(±y) := We shall show that (4.3.21)

!

y 10(± y + x)d[Hn(x) - H(x)], y 2: O.

4.3. Distributions of some scale estim ators

119

But

IZ(±an )

(4.3 .22)

-

Z(±'Y2)1

I![Yt{H(±an + x)) -

Yl(H(±,2 + x ))]

xd(Hn(x) - H(X)}I

S 2

sup 1Y1(H(y)) - Yl(H(z))1 Iy-s!:::; lain -1 / 2,

= op(l) ,

bec ause of (2.3.4}-(2.3.5) and Corollary 2.3.1 applied with dni n- l / 2 . Thus, to prove (4.3.21) , it suffices to show that

(4.3.23) But

IZ(±,2}!

111 <

[Yl (H(±'2 + H;;l(t)}} - Yl (H(±,2 + H-l(t)})]dtl

sup O
IY (H (± , 2 + H-l(H H;;l(t)}} - Yt{H(±'2 + H -l(t})}1 l

op(l }. by the assumption (4.3.14) , Lemma 3.4.1 and Corollary 2.3.1 applied with dni == n- l / 2 . This proves (4.3.21) . Cons equently, from (4.3.20) and an argument like (4.3.22) , it follows that

Et{a n} =

! !

{ylo(a n + x } - YlO( -an

+ x )}dH (x ) + op(l}

{Yl° (,2 + x } - YlO(-,2

+ x )}dH (x } + op(l} .

From these derivations and the definition (4.3.13), we obtain

Tt{a n} =

2K(,2}

+n- 3 / 2 ~ c~jA

!

{fi(,2

+ x} -

f i(

-,2 + x)}dFj(x} . v

2,J

(4.3.24)

+ op(l} .

4. M, R and Some Scale Estimators

120

Now, from the definition of k n and (4.3.19), it follows that the lim c., does not depend on b. Thus the limit of the l.h .s. of (4.3.18) is the same for b = -1 ,0,1/2, and, in view of (4.3.18), (4.3.19) and (4.3.24) , it is given by the first term on the r.h .s. of (4.3.17). 0 Remark 4.3.1 Observe that , in view of (4.3.8), n

Whd

n- l / 2

2:)I(l eil ::; gd

!

i=l

=

K(2)

{Hh2

+ x)

- 1/2} ,

- H(-"(2

+ x)}dYlO(x)

n

n- l / 2

L {Hh2 + ed -

H( -"(2

+ e.) -

1/2}.

i=l

Thus, Whl) and K(2) are the sums of bounded independent centered LV .'S and by the L-F CLT one obtains

where

ai .=

Var(Whd)

l n-

n

L {Fihd -

Fi( -"(1)}{1 - Fihl)

+ Fl (-"(I)},

i =l

a~

.-

Var(Kh2)) n-

l

t

![Hh2

+ x)

- H( -"(2

+ x )fdFl (x ) -

(1/4) .

0

i=l

Remark 4.3.2 If {F i } are all symmetric about zero, then from (4.3.25), (4.3.16) and (4.3.17) , it follows that the asymptotic distribution of

51

and

52

does not depend on the initial estimator

{3. In fact , in this case we can deduce that

(4.3.26)

/3 of

121

4.4. R-ESTIMATORS where

n

h(x)

:=

n- 1

L h(x) .

o

i= 1

Remark 4.3.3 i.i.d. case. In t he case F,

== F , t he asy mptotic

dist ribut ion of 8 1 depe nds on 13 unl ess F is symmetric around zero. However , t he asy mptotic dist ributi on of 82 does not depend on 13. This is so becau se in t his case t he coefficient of v in (4.3.17) is

n - 3 / 2 ~ ~(Xi ~

X j )' A

J[j b 2 + x ) - f( -,2

+ x )]dF (x ) =

0.0

J

T hat t he asy mptotic dist ribution of 82 is independent of 13 is not surpris ing because 82 is essent ially a symmetrized variant of 8 1. We summarize t his property of 8 2 as Corollary 4.3.1 If in the model (1.1.1), Fni == F, F satisfies (Fl) , (F2 ) and X satisfies (NX ), then T 2- 1 n 1/ 2(82- ' 2) -7d N(O , 1), where

now J[F(g2 + x ) - F(

-,2 + x )j2dF (x ) - 1/ 4

(J fb2 + x)dF(x))2 Not e t hat ,2 is now the med ian of t he distribution of leI Also, observe t hat t he cond it ion (4.3.14) now is equivalent to

[P (F (e1

sup

-

y)

0::;8::;1-8

e21 ·

~ 8 + 8) - P (F (el - y) ~ 8)] -7 0,

as 8 -7 0, VY E JR, which is implied by t he ass umptions on F .

4.4

R-Estimators

Consider t he model (1.1.1) and t he vecto r of linear rank statistics n

(4.4.1)

T (t ) := A L (Xni - xn ) cp ( ~tI (n + 1)), t E W , i= 1

o

4. M, R and Some Scale Estimators

122

where Al is as in (4.2.12) and Rit is the rank of Yni - x~it among {Ynj - Xnjt , 1 ~ j ~ n} . One of the classes of R-estimators of f3 is defined by the relation p

(4.4.2)

i~f IT(t)ll = IT(.8l)ll =

L

ITj(t)1 = 0,

j=l

T j being the lh component of T of (4.4 .1).

The estimators .81 were initially studied by Adichie (1967) for the case p = 1 and by J ureckova (1971) for p 2': 1. Another class of R-estimators can be defined by the relation (4.4 .3)

inf IIT(t)1I = IIT(.8)II · t

Yet another class of estimators , introduced by Jaeckel (1972) , is defined by the relation (4.4.4) n

J(t)

:=

L(Yni - x~ it)
Jaeckel (op .cit .) showed that for every observation vector (Yl , . . . , Yn ) and for every n 2': p, L~=l
°

°

4.4. R-Estim ators

123

where Tj (t ) is a Td(cp , u ) - statistic of (3. 1.2) wit h

x;

(4.4.5)

Yni - x~d3 ,

u = A

1

1

(t - (3 ),

a(j)(Xni - x n ), Cni = A l (Xni i" column of A I , 1 ~ j ~ p.

X n ),

1 ~ i ~ n;

T hus sp ecializing T heorem 3.2.4 to t his case readily yields

Lemma 4.4.1 Suppos e that (1.1.1) holds with Fni as a d.f. of eni , 1

~

i

~

n. In addition, assume that (X~Xc ) -1

(N X c)

exists for all

n 2: p ,

maxI::;i::;n(Xni - xn )'X~Xc ) - I (Xni - x n ) = 0(1).

A bout {Fni} assume that H is stri ctly increasing for each nand that (2.3.5) , (3.2.12) , (3.2.34), (3.2.35) hold and that for all j = 1""

, p,

(4.4.6)

lim lim sup

8 ~0

where f or

0

~

n

S

~

sup

[Lj( s

0::; s9 - 8

1, 1

~

+ 8) -

j ~ p,

n

L j( s)

2

L

:=

L j( s)] = 0,

( a(j)( Xni - X n )) Fni(H-I( s)) .

i= l

Th en, for every 0 < b < (4.4.7)

00 ,

IIT(t ) - T ((3 )

sup

+ K nA1 I (t -

(3 ) 11 = op( l)


K n := Al

n

rL

Jo i = 1

(Xni - xn( S))( Xni - x n)' qni (s )dcp(s )AI n

x n(s ) := n- I L

Xnif ni(S),

i= 1

o

4. M, R and Some Scale Estimators

124

In order to prove the asymptotic normality of 132, we need to show that IIA1 I(132 - ,8)11 = Op(l). To this effect let J-L

.-

S .-

Al t(Xni i== 1 T(,8) - J-L.

xn )

J

Fni(H-I)drp ,

Observe that the distribution of (132 -,8) does not depend on,8 , even wh en {end are not identically distributed.

Lemma 4.4.2 In addition to the assumptions of Lemma

4.4.1

sup-

pose that

liS + J-LII = Op(l) ,

(4.4 .8) (4.4 .9)

lim inf inf le'KneJ > ex for an ex n liell==1 -

(4.4 .10)

K~I ex is ts for all n 2': p ,

Th en , for eve ry

E

> 0,0 < z < 00 ,

> 0,

IIK~111 = 0(1).

there ex is t a 0

< b < 00

it and N,

such tha t

(4.4 .11)

P( inf IIT(AIu + ,8)112': z ) 2': 1 Ilull >b

E,

n 2': Ns,

Proof. Fix an E > 0,0 < z < 00 . Without loss of generality assume ,8 = O. Observe that by the C-S inequality inf IIT(A Iu)11 22': inf (e'T(rA Ie))2. lI ull>b Ilell==I,lr l>b thus it suffices to prove that there exist a 0 that

(4.4 .12)

P (

inf (e'T(rA Ie))2 2': Ilell==I ,lrl>b


z) 2':

1-

00

E,

and N, such

n>n..

Let , for t E II~.P , T(t) := T(O) - KnA1It , so that, by (4.4.7) for every 0 < b < 00 ,

(4.4 .13)

sup le'T(rAIe) - e'T(rAIe)1 = op(l) . Ilell==I ,lrISb

125

4.4. R-Estimators But e'T (rAIe)

= e' (S + p,) -

e'Kner.

By (4.4.8), t here exist a K E and an N I < such that

Choose b to sat isfy I, b ~ (K•, + z l/2)",u.

(4.4.14)

a

. (4.4.9) . as In

Then (4.4.15)

P (

inf (e'T(rA Ie))2 Ilell=I ,lr l>b

~ z)

> P (liS + p,11 :S _ zl /2 + b inf le'Kn el) lI ell=1

> P (liS+ ILII :S K t )

~ 1-

E/ 2, \fn

~

NI, ·

Therefore by (4.4.13) ad (4.4.15) t here exist N, and b as in (4.4.14) such t hat (4.4.16) But

where di = e'Al (Xi-X) , Rir is the rank of rj - r (xi-x)AIe. But such a linear rank statistic is nondecreasing in r , for every e . See, e.g., Hajek (1969; Theorem 7E , Chapter II) . This together with (4.4.16) 0 enables one to conclude (4.4.12) and hence (4.4.11). Theorem 4 .4.1 Supp ose tha t (1.1 .1) holds an d th at th e design matrix X an d th e error d.j. 's { Fn i } satisfy th e assumptions of Lem mas 4·4 · 1 and

4·4·2 above.

Th en

(4.4.17) Proof. Follows from Lemmas 4.4.1 and 4.4.2.

o

4. M, R and Some Scale Estimators

126

Remark 4.4.1 Arguing as in J aeckel combined with an argument of Lemma 4.4 .2, one can show t hat II A l (.82 - .83)1/ = Op(l) . Con-

1

sequently, under t he condit ions of Lemmas 4.4 .1 an d 4.4 .2, .82 and th e Jaeckel estimator .83 also satisfy (4.4.1 7). Remark 4.4.2 Cons ider the case when Fn i

0

== F, F a d.f. satisfying

(F l) , (F 2) . Then J.L = 0 and S = T ({3 ). Moreover, under (N X c ) all ot her ass umptions of Lem mas 4.4 .1 and 4.4.2 ar e a priori satisfied. Note t hat here

Moreover , from Theorem 3.4 .3 above, it follows that S converges in distribut ion to a

N( o, a~ Ip xp) ,

a~

=

LV . ,

where

t' r.p (u )du ) 2 . Jt'o r.p2(u )du - (Jo

We summar ize t he above d iscussion as Corollary 4.4.1 Suppo se that (i .i .i ) with Fn i

== F holds. Suppos e

that F and X satisfy (Fl ) , (F2 ), and (NX c ) . In addit ion, suppose that r.p

is nondecreasing bound ed on [0, 1] and

J f dr.p (F ) > O.

Then

A 1l (.82 - (3 ) =

(J

f dr.p (F )) - IT({3)

+ op(l) .

Hence,

T his resul t is qui te general as far as t he conditio ns on the design mat rix X and F are concerned but not that general as far as t he score function ip is concerne d . 0 Remark 4.4.3 Robustness against heteroscedastic gross errors . F irst we give a working definit ion of qualitative rob ustness .

Estim ation of QU )

127

Consider the model (1.1.1). Suppose that we have modeled the errors {eni' 1

~

i ~ n } to be i.i.d. F whereas their actua l d.f. 's ar e

{Fni , 1 ~ i ~ n }. Let p n := TI ~=l F, Qn := TI ~=l Fni denote the cor respond ing product probability measure. Definition 4.4.1. A sequence of est imators 13 is sa id to be qualit atively robust for f3 at F against Qn if it is consistent for f3 under P " and un der those Qn that sat isfy 'Dn

:=

max i SUPy IFni (Y) - F( y)1 -t O.

The above definition is a variant of that of Hampel (1971). On e could use the notions of weak convergence on product probability spaces to give a bit more general definition. For example we could insist that the Prohorov dist an ce between Qn and P " should tend to zero inst ead of requiring 'Dn -t O. We do not pursue this any further here. The result (4.4.17) can be used to st udy t he qu ali tativ e robust-

132

ness of

against certain heteroscedasti c errors. Consider, for ex-

amp le, t he gross errors model where , for some 0

~

6n i

~

1, wit h

maxi bni -t 0,

and , where G is d .f. having a uniformly cont inuous a.e. positive density. If, in addit ion, {6n d satisfy n

(4.4.18)

II A1 I)Xni - Xn)6nill =

0 (1 ),

i=l

t hen one can readily see that IIK;l ll = 0(1 ) and IIILII = 0(1). It follows from (4.4.17) t hat 132is qualitatively robust against the above het eroscedastic gross errors at every F that has uniformly continuous a.e. positiv e density. Examples of bni satisfyin g (4.4.18) would be

6n i == n- 1/ 2 or 6n i = p- l /2I1 At{x n i

-

xn )lI ,

1 ~ i ~ n.

It may be arg ued t hat the lat ter choice of contaminating prop ortions

{bni } is more natural to linear regression t ha n t he former. A similar remark is applicable to 131and 133' 0

4. M, R and Som e Scale Estimators

128

4.5

Estimation of Q(f)

Consid er the mod el (1.1.1) with Fni de nsity J on JR. Define (4.5.1)

QU ) =

JJ

F , where F is a d .f. with

drp(F ),

where sp E C of (3.2.1). As is seen from Corollary 4.4.1, the paramet er Q appears in the asy mptotic vari ance of R - est imato rs. The comp lete rank an alysis of the mod el (1.1.1) requires an est imate of Q. This est imate is used to standardize rank test stat ist ics when carrying out the ANOVA of linear models using J aeckel' s dispersion .J of (4.4 .4). See, for example, Hettmansp erger (1984) and refer ences t herein for the rank based ANOVA . Lehmann (1963) and Sen (1966) give est imators of Q in t he one and two sa mple location mod els. These est imators are given in te rms of lengths of confide nce interva ls based on linear rank statistics. Cheng and Serft ing (1981) discuss several est imators of Q when obs ervations are i.i.d . F , i.e., when t here are no nuisance pa rameters . Some of t hese est imators ar e obtained by replacin g J by a kernel ty pe dens ity estimator and F by an emp irical d. f. in Q . Schewede r (1975) disc usses similar estimates of Q in t he one sa mple location mod el. In this sect ion we discuss two ty pes of est imators of Q . Both use a kernel typ e density est imator of J based on t he residuals and the ordinary residual em pirical d. f. to estimate F. The d iffer ence is in the way the window widt h and the kernel are chosen. In one the window width is parti ally based on the data and is of the orde r of square root of n and the kernel is the histogram typ e whereas in the ot her the kernel and the window width are arbit rary. It will be observed t hat the AUL result abo ut the residual empirical process of Corollary 2.3.5 is t he basic too l needed to prove t he consistency of t hese est imat ors. We begin with th e class of est ima tors wher e the wind ow wid th is partly based on the data. Define (4.5.2)

p(y)

:=

J

[F(y

+x) -

F (- y + x )]drp (F (x )),

y

~ O.

4.5. Estimation of Q(J )

129

Since cp is a d.f., p(y) = P (le- e*1 ::; y), where e, e* ar e indep endent r.v .'s with respective d.f.'s F and cp(F) . Con sequ ently, under (Fl ), t he densi ty of pat 0 is 2Q . This suggests that an estimate of Q can be obtained by est imating t he slope of p at O. Recall the definiti on of t he residual empirical pr ocess H n (y, t ) from (1.2.1). Let fJ be an est imato r of f3 and define (4.5.3) A natural est imator of p is obtained by substituting

it;

for F in p ,

v.i.z. ,

Let - 00 = e(O) < e( l ) ::; e(2) ::; . . . ::; ern) < e (n + l ) = 00 denote t he ordere d residuals {ei := Yi - x~fJ , 1 ::; i ::; n}. Since cp(H n ) assigns mass {cpUI n ) - cp(j - 1) In))} to each e(j ) and zero mass to each of the intervals (e (j - l ) ' e(j) ), 1 ::; j ::; n + 1; it follows that 'Ii y E JR, (4.5.4)

Pn (y) n

L {cpU In) -

cpU - 1)l n ) }[Hn(y + e(j) ) - H n( - y + e(j))]

i= l

From t his one sees t hat Pn (y) has the following int erpret ation. For each j, one first computes the proportion of {e(i) } falling in th e interval [-y + e(j), y + e(j) ] and then Pn(y) gives the weight ed average of such proportions. Formula (4.5.4) is clearly suitable for computations . Now, if {h n } is a sequence of positive numbers te nding to zero , an est imator of Q is given by

T his estimator can be viewed from t he density est ima tio n po int of view also. Consider a kern el-type density est imator In of I base d on

4. M, R and Some Scale Estimators

130 the residuals {ei}:

n

fn(x)

:= (2nh n)-1

L 1(lx - eil :S hn ), i=l

which uses the window wn(x) = (1/2) . 1(lxl :S h n) . Then a natural estimator of Q is

Scheweder (1975) studied the asymptotic properties of this estimator in the one sample location model. Observe that in this case the estimator of Q does not depend on the estimator of the location parameter which makes it relatively easier to derive its asymptotic properties. In Qn, there is an arbitrariness due to the choice of the window width h n . Here we recommend that h n be determined from the spread of the data as follows . Let 0 < 0: < 1, t~ be o-th quantile of Pn and define the estimator Q~ of Q as (4.5.5) The quantile t~ is an estimator of the o-th quantile to: of p . Note that if cp(s) := s, then to: is the o:-th quantile of the distribution of leI - e21 and t~ is the o-th quantile of the empirical d .f. Pn of the LV . 'S {lei - ejl , 1 :S i,j :S n}. Thus, e.g., t~ = 82 of (4.3.5) . Similarly, if cp(8) = 1(8 ~ 0.5) then to: (t~) is o-th quantile of the d.f. of lei I (empirical d.f. of leil, 1 :S i :S n). Again, here t:/! would correspond to 81 of (4.3.5) . In any case , in general, to: is a scale parameter in the sense of Bickel and Lehmann (1975) . Note that the estimator Q~ is location and scale invariant in the sense of (4.3.3) and (4.3.4) , as long as is location and scale invariant in the sense of (4.3.1) and (4.3.2). This follows from the fact that under (4.3.1) and (4.3.2), Pn(Y, aY + Xb) = Pn(y/a , Y), t~(aY + Xb) = at~(Y) , for all y E~, a > 0, bE W. The consistency of Q~ is asserted in the following

/3

4.5. Estimation of QU)

131

Theorem 4.5.1 Let (1.1.1) hold with Fni (NX), (Fl) and (F2), assume that

/3

==

F.

In addition to

is an estimator of (3 satis-

fying (4.3.15). Then, (4.5.6)

sup IQ~

- QU)I =

op(1) .


The proof of (4.5.6) will be a consequence of the following three lemmas.

Lemma 4.5.1 Under the assumptions of Theorem 4.5.1, V 0 ::; a < 00

(4.5.7) Consequently, V 0 ::; a

< 00 ,

(4.5.8)

Proof. We shall apply Corollary 2.3.5 . Let

V= A -1(/3 - (3),

n

b~ = n- 1 / 2 LX~iA . i=1

Then, from (2.3.43) , (4.5.3) and (4.3.15) , we obtain (4.5.9) where n

Hn(y) == Hn(y , (3) == n - 1

L I(eni ::; y) ,

y E lit

i=1

Also , we will use the notation (2.3.1) with u = (4.5.10)

1 2 d n~. = - n- / ,

X' n~ -

"\7 . J. n~

X'ni(3'

°

and

Fn~· = - F.

Then Yl(t ,O) == n 1 / 2 [H n (F - 1 (t )) - t], 0 ::; t ::; 1. Write Y1 (-) for Y1 (-,0). Now, (4.5.9) and cp bounded imply that,

4. M, R and Some Scale Estimators

132

n 1 / 2 {Pn(Y) - p(y)} 1 2

n / /

{Hn(y

+ x)

- H n( -Y + x)}dip(Hn(x))

+ b~v /[J(Y+X)

- f(-y+x)]dip(Hn(x))

- n 1 / 2 p(y) + up (l )

+ R n2(y) + R n3(Y) + up ( l ),

Rnl(Y)

(4.5.11)

where u p (l ) stands for a sequence ofrandom processes that converge to zero, uniformly in Y E JR, ip E C, in probability, and where

R n1(y)

/{Y1(F(Y+X)) - Y1(F(-y+x))}dip(Hn(x))

b~v /[J(Y+x)-f(-y+X)]dip(Hn(X)) 1 2

n / { / [F(y

- /[F(Y

+ x)

- F( -y + x)]dip(Hn (x))

+ x)

- F( -y + x)]dip(F(x))},

for Y E R From (Fl) , (F2), the boundedness of ip ; and the asymptotic continuity of Y1 , which follows from Corollary 2.2.1, applied to the quantities given in (4.5 .10) , we obtain, with k = 2allflloo , (4.5.12)

IRn t{ n - 1 / 2 z)1 <

sup 0:Sz:Sa,
sup

1Y1(t)-Y1(s)1

It-sl :Skn - 1 / 2

op(l) . Again, (Fl) and the boundedness of ip imply, in a routine fashion, that (4.5 .13)

sup

IRn l ( n - 1/ 2 z)1=

op(l) .

0:Sz:Sa,
Now consider R n3. By the MVT, (Fl) and the boundedness of .p, the first term of R n 3(n - 1/ 2z ) can be written as

133

4.5. Estimation of Q(J)

where {~xzn} ar e real numbers su ch that I~xzn - z ] ::; an - 1 / 2 . Do the sa me with the second integral and put two to gether to obtain

Rn 3(n - 1/ 2z ) =

2z { / fd
fd
- /

+ u p (l )

{1 [q (FH;l (t )) - q(t )]d
2z

u p( l) .

But, (4.5.14) sup IFH;l(t ) °9 :::;1

tl <

n- 1

+ sup IHn(y) -

F( y)1

y

=

op(l) ,

by (4.5.9) and the Glivenko - Cantelli Lemma. Hen ce, q being uniformly continuous , we obtain

This together wit h (4.5.11) - (4.5.14) completes t he proo f of (4.5.7) whereas t hat of (4.5.8) follows from (4.5. 7) and the fact In 1/ 2p(n- 1/ 2 z ) - 2z Q (J )1 -+ 0,

sup 0:::; z :::; a ,ep EC

which in t urn follows from t he uniform cont inuity of

f.

o

Lemma 4.5.2 Und er the assump tions of Th eorem 4.5.1, Vy ~ 0,

sup IPn(Y) - p(y)1 = op(l ). ep EC

Proof. Proceed as in the proof of the pr evious lemma to rewrite

where Anj = n - 1 / 2 Rnj , j = 1,2 ,3 , with Rnj defined at (4.5.11). By Corollary 2.2.2 applied to the quantities given at (4.5.9), IJYliioo = Op(1) and hen ce f , sp bounded t rivially imply that sup ep EC,y?, O

IAnj(y)1 = op( l) , j = 1, 2.

4. M, R and Some Scale Estimators

134 Now, rewrite A n 3 (y) =

[ / F(y

+ x )dep (Hn(x )) -

/ F( y + x )dep (F (x ))]

- [ / F (- y + x )dep (Hn(x )) - / F (- y+ x )dep(F (x ))] An(y)

+ A n( - V),

say

Bu t , V Y E IR,

1 1

An(y)

= =

{ F (Y + F - 1 (FH;;I (t ))) - F(y

+ F- 1 (t ))} dep(t)

op(l ),

because of (4.5.14) and because, by (F l) and (F 2) , V y 2: 0, F (y + F - 1 (t )) is uniformly continuous function of t E [0, 1]. 0 Lemma 4.5.3 Und er the conditio ns of Th eorem 4.5 .1, "Y'

> 0,

E

Proof. Ob serve t hat t he event

[Pn(1 - E)t

Q )

< 0: :S Pn(l + E)t ~

[(1 - E)t

Q

Q ) ]

:s t~ :s (1 + E)t

Q ] •

Hence, by two applications of Lemma 4.5.2, once wit h y = (1 + E)t and once wit h y = (1 - E)t we obtain that

Q ,

Q

,

lim inf P (l t ~ - t Q I :S Et Q , V sp E C) n

> P(p( l - E)t =

Q )

< 0: :S p(( l + E) t

Q ) ,

Vep E C)

o

1.

Proof of Theorem 4.5.1. Clearly, V ep E C,

I Q~ - Q(f)1

=

(2t~)- 1 I nl /2Pn (n- l/2 t~ ) - 2t~Q(f)I ·

By Lemma 4.5.3, V E > 0,

P (o < t~ :S (1 + E)t

Q ,

Vep E C)

----1

1.

Hence (4.5.6) follows from (4.5.8) applied wit h a = (1 + E)t 4.5.3 and t he Slutsky Theorem .

Q ,

Lemma 0

4.5. Estimation of Q(J)

135

Remark 4.5.1 The estimator

Q~

shifts the burden of choosing the

window width to the choice of a . There does not seem to be an easy way to recommend a universal a . In an empirical study done in Koul, Sievers and McKean (1987) that investigated level and power of some rank tests in the linear regression setting, a to be most desirable.

= 0.8 was found 0

Remark 4.5.2 It is an interesting theoretical exercise to see if, for

some 0 < <5 < 1, the processes {n1 /2(Q~ - Q(J)), <5 ::; a ::; 1 <5} converge weakly to a Gaussian process . In the case cp(t)

== t ,

Thewarapperuma (1987) has proved under (F1) , (F2) , (NX) , and (4.3.15), that V fixed 0 < a < 1, n1 /2(Q~ - Q(J)) -td N(0 ,a 2), where a 2 = 16{J f3(x)dx - (J f2(x)dx)2}.

0

{t~ ,

(p E C} provides a class

Remark 4.5.3 As mentioned earlier,

of scale estimators for the class of scale parameters {to: , cp E C}. Recall that

81

and

82

of (4.3.5) are special cases of these estimators.

The former is obtained by taking cp(u) == I(u :::: 0.5) and the latter by taking cp(u) == u . For general interest we state a theorem below , giving asymptotic normality of these estimators. The details of proof are similar to those of Theorem 4.3 .1. To state this theorem we need to introduce appropriately modified analogues of the entities defined at (4.3.13) : K 1(y)

.-

K 2(y ) .-

K(y) where

.-

J J

[Y10(y + x) - Y10( -y + x)]dcp(F(x)), Y10(X)[J(y

+ x)

- f( -y + x)]{f(x)} - l dcp (F (x )),

K 1 (y) - K 2(y) ,

Y :::: 0,

yt is as (4.3 .13) adapted to the i.i.d . errors setup. It is easy to

check that K(tO:) is n- 1 / 2x { a sum of i.i.d. LV.'S} with EK(tO:) and 0 < (aO:)2:= Var(K(tO:)) < 00, not depending on n.

== 0

0

Theorem 4.5.2 In addition to the conditions of Theorem 4.5.1, assume that either cp( t) = I (t :::: u), 0

< u < 1,

fixed or sp is uniformly

136

4. M, R and Some Scale Estimators

differentiable on [0, 1] . Then ,

\:j

°< a < 1,

n1 /2(t~ - t a ) -+d

(v a )2 :=

((ja)2

J

{t a

[J(t a

N(o , (zp)2) ,

+ x) + f( _t a + X)]dCP(F(X))} -2

0

We now turn to the arbitrary window width and kerneltype estimators of Q. Accordingly, let K be a probability density and {ed be as on JR., h n be a sequ ence of positive numbers and before. Define, for x E JR.,

i3

in(x)

Qn

.:=

_1

nhn

J

'tK(-e i ) , .

l=l

hn

n

x - ei fn( x):= nh LK(-h-) ' 1 '""'" n i=l

n

in (x)dcp(iI n(x)).

Theorem 4.5 .3 Assum e that the model {i .l.l} with Fni == F holds.

In addition, assume that (FI), (F2) , (NX) and (4.3.15) and the following hold: (i) h n > 0, h n -+ 0, n 1/2 h n -+ 00. (ii) K is absolutely continuous with its a.e. derivativ e K satisfy ing

J KI

< 00 .

Then, (4.5.15)

sup IQn - Q(/) I = Op(l).
Proof. First we show in approximates f. This is done in several steps. To begin with, summation by parts shows that

so that ~ - f nlloo ~ (n 1/2 h -1 . lin 1/2 (H~ - H Ilfn n) n n)lIoo .

Henc e, by (4.5.9) and the fact that (4.3.15) , it readily follows that

lIin - f nll oo

Ib~

vi

JIKI. ·

= Op(l) guaranteed by

= Op(n1/ 2hn )- 1) = op(1) .

137

4.5. Estimation of QU) Now, let

Yn(x)

:=

h;;l

J

K((x - y)jhn)J(y)dy.

Note that integration by parts shows that

so that

Ilfn -Ynlloo

(4.5.16)

:S (nl /2hn)-11Inl/2[Hn - F]lIoo op(1),

=

by (i) and by the fact that Iln 1 / 2(Hn

llYn -

JIKI

flloo:S

-

F)lloo =

Op(1). Moreover,

sup If(y) - f(x)1 = 0(1), Iy- xl::;h n

by (Fl).

Now, consider the difference

a. - QU) Let q(t)

=

J

=

Dn1 + D n2, say .

= f(F -l(t)) .

(in - J)dcp(Hn) +

J

fd[cp(Hn) - cp(F)]

Then

sup IDn21:S sup Iq(F(H;l(t))) - q(t)1 = op(1) O::;t::;l

~EC

by the uniform continuity of q and (4.5.14) . Also, from the above bounds we obtain sup IDn11 :S ~EC

thereby proving (4.5.15).

Ilin - flloo

= op(1) ,

5 Minimum Distance Estimators 5.1

Introduction

The practice of obtaining estimators of parameters by minimizing a certain distance between some functions of observations and parameters has long been present in statistics. The classical examples of this method are the Least Square and the minimum Chi Square estimators. The minimum distance (m .d.) estimation method , where one obtains an estimator of a parameter by minimizing some distance between the empirical d.f. and the modeled d.f., was elevated to a general method of estimation by Wolfowitz (1953, 1954, 1957). In these papers he demonstrated that compared to the maximum likelihood estimation method, the m.d. estimation method yielded consistent estimators rather cheaply in several problems of varied levels of difficulty. This methodology saw increasing research activity from the mid 1970's when many authors demonstrated various robustness properties of certain m.d. estimators. Beran (1977) showed that in the i.i.d. setup the minimum Hellinger distance estimators, obtained by minimizing the Hellinger distance between the modeled parametric density and an empirical density estimate, are asymptotically efficient at the true model and robust against small departures from the model, where the smallness is being measured in terms of the H. L. Koul, Weighted Empirical Processes in Dynamic Nonlinear Models © Springer-Verlag New York, Inc 2002

5.1. INTRODUCTION

139

Hellinger metric. Beran (1978) demonstrated the powerfulness of minimum Hellinger distance estimators in the one sample location model by showing that the estimators obtained by minimizing the Hellinger distance between an estimator of the density of the residual and an estimator of the density of the negative residual are qualitatively robust and adaptive for all those symmetric error distributions that have finite Fisher information. Parr and Schucany (1979) empirically demonstrated that in certain location models several minimum distance estimators (where several comes from the type of distances chosen) are robust. Millar (1981, 1982, 1984) proved local asymptot ic minimaxity of a fairly large class of m.d. estimators , using Cramer - von Mises type distance, in the i.i.d. setup . Donoho and Liu (1988 a , b) demonstrated certain further finite sample robustness properties of a large class of m.d. estimators and certain additional advantages of using Cramer - von Mises and Hellinger distances. All of these authors restrict their attention to the one sample models or to the two sample location model. See Parr (1981) for additional bibliography on m.d.e. through 1980. Little was known till the late 1970's about how to ext end the above methodology to one of the most applied models , viz., the multiple linear regression model (1.1.1) . Given the above optimality properties in the one and two-sample location models, it became even more desirable to extend this methodology to this model. Only after realizing that one should use the weighted, rather than the ordinary, empiricals of the residuals to define m.d . estimators was it possible to ext end this methodology satisfactorily to the model (1.1.1). The main focus of this chapter is the m.d. estimators of (3 by minimizing the Cramer - von Mises type distances involving various W.E.P. 's. Some m.d. est imators involving the supremum distance are also discussed. Most of the estimators provide appropriate extension of their counterparts in the one - and two - sample location models. Section 5.2 contains definitions of several m.d. estimators. Their finit e sample properties and asymptotic distributions are discussed in Sections 5.3, 5.5, respectively. Section 5.4 discusses an asymptotic

5. Minimum Distance Estimators

140

theory about general minimum dispersion estimators that is of broad and independent interest. It is a self contained section. Asymptotic relative efficiency and qualitative robustness of some of the m.d. estimators of Section 5.2 are discussed in Section 5.6. Some of the proposed m.d. functionals are Hellinger differentiable in the sense of Beran (1982) as is shown in Section 5.6. Consequently they are locally asymptotically minimax (LAM) in the sense of Hajek - Le Cam.

5.2

Definitions of M.D. Estimators

To motivate the following definitions of m.d. estimators of (:J of (1.1.1) first consider the one sample location model where Y1 e,' " ,Yn - e are i.i.d. F , F a known d.f. Let n

(5.2.1)

Fn{y)

:=

n- 1 LI{Yi::; y),

YER

i=l

If e is true then EFn{y + e) = F{y), Vy E lIt This motivates one to define m.d. est imator iJ of () by the relation

iJ = argmin{T{t) ; t E IR}

(5.2.2)

where, for aGE lDI{IR), (5.2.3)

T{t)

:=

n ![Fn{y + t) - F{y)J2dG{y) ,

tER

Obs erve that (5.2.2) and (5.2.3) actually define a class of estimators

iJ, one corresponding to each G. Now suppose that in (1.1.1) we model the d.f. of eni to be a known d.f. H ni' which may be different from the actual d.f. Fni ,l ::; i ::; n. How should one define a m.d. estimator of {:J? Any definition should reduce to iJ when (1.1.1) is reduced to the one sample location model. One possible extension is to define (5.2.4)

/31 = argmin{K 1(t ); t

where , for atE JRP ,

E IRP } ,

141

5.2. D efinitions

If in (1.1.1) we take p = 1, Xni1 == 1 and Hni == F then clearly it reduces to the one sample location mod el and /31 coincides with of (5.2.2). Bu t this is also t rue for the estimato r /3x defined as follows. Recall t he definition of {ltj } from (1.1.2). Let , for y E JR, t E lW,

e

n

(5.2.5)

Zj (Y, t ) .- ltj(y , t ) Kx (t )

where Z'

:=

:=

J

L Xnij Hni(Y),

1 ~ j ~ p,

i= 1

Z' (y, t )(X'X)-1 Z (Y, t )dG(y),

(Z1 ,'" , Zp ) and define

(5.2.6)

/3x = argmin{K x (t ), t E JRP }.

Which of the two est imato rs is the right ext ens ion of e? Since {ltj , 1 ~ j ~ p} summarize t he data in (1.1.1) wit h pr obability one un der t he cont inuity ass umption of {eni' 1 ~ i ~ n} , /3x should In Sect ion 5.6 we shall see be considered t he right extension of t hat /3x is asymptotically efficient amo ng a class of est imators {/3D} defined as follows.

e.

Let D = ((d nij)) , 1 ~ i ~ n , 1 and define, for y E JR, t E JRP,

~ j ~

p , be an n x p real matrix

n

(5.2.7) Vjd(y, t ) .-

L dnij I(Yni ~ y + x~it ),

1~ j


i= 1

Define a class of estima tors, one for each D , (5.2.8)

/3D := argmin{K D(t) , t E JRP}.

If D = n- 1 / 2 [1, 0,' " ,O]nxp t hen /3D = /31 and if D = XA then /3D = /3x , where A is as in (2.3.30) . The above menti oned optimality of /3x is stated and proved in Theorem 5.6a.1 below.

Anoth er way to define m.d. est imators in t he case t he modeled

5. Minimum Distance Estimators

142

error d.j.

'8

are known is as follows. Let , for s E 0,1], Y E

~,

t E ~P ,

ns

(5.2.9)

M( s, y , t} .- n- 1 / 2 L {I (Yni ~ y} - Hni(Y - x~it)}, i=1

1J 1

Q(t)

{M(s , y ,t)}2dG(y}dL( s},

:=

where L is a d.f. on [O,lJ . Define

13 =

(5.2 .10)

argmin{Q(t} , t E W}.

The estimator 13 with L(s} == s is essentially Millar's (1982) proposal. Now suppose {Hnd ar e unknown. How should one define m.d. of /3 in this case.? Again, let us examine the one sample location model. In this case () can not be identified unless the errors ar e symmetric about O. Suppose that is the case. Then the r. v.'s Pi _.(), 1 ~ i ~ n} have the sam e distribution as {- Yi +(), 1 ~ i ~ n} . A m.d. estimator ()+ of () is thus defined by the relation ()+ = argmin{T+(t} , t E ~}

(5.2.11)

where

An extension of ()+ to the model (1.1.1) is Let , for y E ~, t E W , 1 ~ j ~ p, n

(5.2.12) Y j(y ,t)

:=

L

Xnij{I(Yni

/3*

defined as follows:

~ y+X~it}

i= 1

-I(-Yni

y+

K~(t)

.-

.-

< y - X~it)} ,

(V/, ... ,V/ )',

J

Y +(y , t}(X'X} -ly+(y, t} dG(y} ,

and define the est imator (5.2.13)

/3*

= argmin{K:i(t} , t E W} .

5.2. Definitions

143

More generally, a class of m.d. estimators of 13 can be defined as follows. Let D be as before. Define, for y E JR, t E W , 1 ::; j ::; p, n

(5.2.14) Y/(y,t)

2:dnij{I{Yni::;y+X~it)

:=

i=1

- I (- Yni < y - X~i t) },

Yi)

(YI+, . . . , y/ )/,

J

Y D+' {y, t)Y D+' (y, t) dG{y) .

+ KD(t)·and

f3fj by the relation f3fj

(5.2.15)

= argmin{K6{t), t E

JRP} .

Note that 131 is f3fj with D = XA . Next, suppose that the errors in (1.1.1) are modeled to be i.i.d. , i.e. H ni == F and F is unknown and not necessarily symmetric. Here , of course the location parameter can not be estimated. However, the regression parameter vector 13 can be estimated provided the rank of X, is p, where X , is defined at (4.2 .11) . In this case a class of m.d. of (5.2.8) provided we assume that estimators of 13 is defined by

i3D

n

(5.2.16)

2:dnij = 0, i= 1

An interesting member of this class is obtained upon taking D XcA I , Al as in (4.2 .12) . Another way to define m.d. est imators here is via the ranks of the residuals. With R it as in (3.1.1) , let n

(5.2.17)

Tj(s , t)

.-

2: dnijI{Rit ::; ns) ,

s E [0,1], 1::; j ::; p,

i= 1

Ki>{t)

:=

J

Tb(s, t) TD{S, t) dL(s) ,

t E JR,

where Tb = (TI, '" ,Tp ) and L is a d.f. on [0,1]. Assume that D satisfies (5.2.16). Define (5.2.18)

f3b

= argmin{Ki> (t), t E W}.

5. Minimum Distance Estimators

144

Observe that {,Bn}, {,Bi;} and {,B} are not scale invariant in the sense of (4.3.2). One way to make them so is to modify their definitions as follows. Define , for t E lRP , a 2: 0,

J

P

(5.2.19) Kn(a , t)

.- L

[Vjd(ay, t) -

j=l

Ki)(a , t)

n

L dnijHni(y)fdG(y) , i=l

J

+ Y n+' (ay, t)Yn(ay, t)dG(y).

:=

Now, scale invariant analogues of ,Bn and ,Bti are defined as , 0

(5.2.20)

.- argmin{Kn(s , t) , t E lRP } ,

,Bn

,Bti° .-

argmin{Kn(s , t), t E lRP } ,

where s is a scale estimator satisfying (4.3.3) and (4.3.4) . One can modify {,B} in a similar fashion to make it scale invariant. The class of est imators {,Bn} is scale invariant because the ranks ar e. Now we define a m.d. estimator based on the supremum distance in the case the errors are correctly modeled to be i.i.d. F, F an arbitrary d.f. Here we shall restrict ourselves only to the case of p = 1. Define n .(5.2.21) Vc(y , t) t, Y E lR, I:( Xi - x)I(Y; ::; y + txd , i= l

V~(t)

.-

sup{Vc(Y, t) ; Y E lR} ,

V;;(t)

.-

- inf{Vc(Y, t) ; y E lR} ,

V n (t)

.-

max{V~(t), V;;(t) ,V;;(t)}

sup{lVc(Y, t)l ; y E lR} ,

t

E lR.

Finally, define the m.d. estimator

/38

(5.2.22)

:=

argmin{Vn(t); t E lR} .

Section 5.3 discusses some computational asp ects including the existence and some finite sample properties of the above estimators . Section 5.5 proves th e uniform asymptotic qu adraticity of K n, Ki) , K and Q as processes in t. These results are used in Section 5.6 to study the asymptotic distributions and robustness of the above defined estimators.

n

5.3. FINITE SAMPLE PROPERTIES

5.3

145

Finite Sample Properties

The purpose here is to discuss some computational aspects, the existence and the finite sample properties of the four classes of estimators introduced in the previous section. To facilitate this the dependence of these estimators and their defining statistics on the weight matrix D will not be exhibited in this section. We first turn to some computational aspects of these estimators. To begin with, suppose that p = 1 and G(y) = y in (5.2.7) and (5.2.8). Write /3, Xi , di for 13, XiI, dil, respectively, 1 ~ i ~ n. Then

(5.3.1)

K(t)

~

J[t

d;{I(Y;

r

~ Y + Xit) - Hi(yll

I=I=didj! {I(Yi i=l j=l

~ Y+Xi t) -

x {I(Yj

dy

Hi(Y)}

~ Y + Xjt) - Hj(Y)} dy.

No further simplification of this occurs except for some special cases. One of them is the case of the one sample location model where Xi == 1 and Hi == F, in which case

K(t)

~ J[tdi{I(Y; ~ y) - F(y - Illr dy.

Differentiating under the integral sign w.r.t. t (which can be justified under the sole assumption: F has a density f w.r.t . ,X) one obtains

K(t)

=

2! I=

di{I(Yi

~ Y + t) -

F(y)}dF(y)

i=l

n

=

-2

L di{F(Yi -

t) - 1/2}.

i=l

Upon taking d; == n- 1! 2 one sees that in the one sample location model Bof (5.2.2) corresponding to G(y) = y is given as a solution of n

(5.3.2)

L i=l

F(Yi - B) = n/2.

146

5. Minimum Distance Estimators

Not e that this {) is pr ecisely the m.l. e. of () when F (x ) = {I exp( - x )} - 1, i.e., when the errors have logistic distribution! Another simplification of (5.3.1) occurs when we assume

+

n

and ~

Fix atE JR and let c := max{ Yi - x it; 1

f [;

(5.3.3) K (t )

i ~ n }. Then

diI (Y; :S Y + Xi tl ]' dy

tt J didj

I [ max(Yj

-

Xjt , Yi - Xit )

i=1 j=1

~ Y < c] dy n

=

-

n

L L di dj max (Y

j -

Xjt, Yi - Xit ).

bl,

a, s « JR,

i=1 j=1

Using t he relati onship (5.3.4)

2 ma x(a , b) = a + b+

la -

and t he assumption L ~=1 d; = 0, one ob tains (5.3.5)

K (t )

=- 2 L L 1 ~i

If di

= Xi -

did jlYj - Yi - (Xj - xi )t l·


X in (5.3.5) , th en the correspo nding ~ is asy mptoti-

cally equivalent to the Wilc oxon typ e R-estimat or of f3 as was shown by Willi am son (1979). This result will also follow from the general asymptotic theory of Secti ons 5.5 and 5.6 below. If d; = Xi - X , 1 ~ i ~ n, and Xi = 0, 1 ~ i ~ r; Xi = 1, r + 1 ~ i ~ n t hen (1.1.1) becomes the two sample location mod el and r

K (t ) = - 2

n

L L

IYj - Yi - tl +

a

LV.

constant in t .

i=1 j=r+1

Consequent ly here ~ = m ed{I Yj - Yi l, r + 1 ~ j ~ n , 1 ~ i ~ r} , t he usual Hodges - Lehmann estimator. The fact t hat in t he two

5.3. Finite sampl e properties

147

sample location model the Cr amer - von Mises type m.d. est imator of the location paramet er is the Hodges - Lehmann est imator was first noted by Fine (1966) . Note that a relation like (5.3.5) is true for general p and G. That is, suppose that p ~ 1, G E VI { ~) and (5.2.16) holds, then \j t E JRP, (5.3.6) K {t ) p

=

-2:L:L:L dij dkjIG {(Yk - x~t) _ ) - G( (Yi - x~t )_)I· j=1 19 < k~ n

To prove this pro ceed as in (5.3.3) to conclude first that p

K(t) = -2:L:L :L dijdk jG(max(Yk - x~t , Yi - x~t)_) j=1 1 ~ i < k~ n Now use the fact t ha t G(aVb)_ ) = G(a_ )VC{L) , (5.2.16) and (5.3.4) to obtain {5.3.6}. Clearly, this formula can be used to compute /3 in genera l. Next , consider K +. To simplify the expos it ion , fix a tE ~p and let r i := Yi - xit , 1 :::; i :::; n j b := max j r, - r i j 1 :::; i :::; n} . Then from {5.2.14} we obtain

Ob serve that the int egrand is zero for y > b. Now expand the qu adratic and integrate term by term, noting that G may have jumps, to obtain p

n

n

:L:L:L dijd kj{ 2G(r i V - rk)} - 2J (ri ) j=1 i=1 k=1

where J (y ) := G{y } - G{y_ ), t he jump in G at y E JR. On ce aga in use t he fact that G(aVb ) = G{a)VG {b ), {5.3.4} , t he invari an ce of the double sum under permutati on and t he definiti on of {rd to conclude

148

5. Minim um Distance Estim ators

that

(5.3.7) K +(t) n

p

=

n

L L L dij dkj [!G( Yi -

- G( -Yk

+ x~t)1

-~{ I G(Yi - x~t) _ ) -

G(Yk -

j= l i= l k=l

x~t)

+ IG(-

Yi

+ x~ t ) -

x~t)_ ) 1

G(- Yk + x~ t )

- J (Yi -

I}

x~t )] .

Before proceeding further it is convenient to recall at t his time the definition of symmet ry for a GE 1JI( ~) . Definition 5.3.1. An ar bitrary G E 1JI ( ~) , including a o - finite measur e on t he Borel line (~, B), is sa id to be sym m etric aro und 0 if

(5.3.8)

IG(y) - G (x )1 =

vx, Y E R

IG(-x - ) - G(- y_ )1,

Or (5.3.9)

dG (y ) = - dG (-y) ,

Vy E R

If G is cont inuous then (5.3.8) is equivalent to

(5.3.10)

IG(y ) - G(x )1 = IG( - x ) - G( -y) 1,

v x,y E R

Conversely, if (5.3.10) holds th en G is symmetric around 0 and continuous. Now suppose t hat G satisfies (5.3.8). Then (5.3.7) simplifies to

(5.3.11)

K+ (t ) p

n

n

L L L dijdkj [IG(Yi - x~t) - G(-Yk + x~t) 1 j =l i=l k=l

- IG(- Yi

+x~t ) -

G (-Yk +xkt )1

-J(Yi -

x~t)] .

5.3. Finite sample properties

149

And if G satisfies (5.3.10) then we obtain the relatively simpler expression (5.3.12)

K +(t) p

n

n

j==l

i == l

k==l

L L L dijdkj [IG(Yi -

x~t)

-IG(Yi -

- G(-Yk + x~t)1

x~t)

- G(Yk - xkt)IJ·

Upon sp ecializing (5.3.12) to the case G(y) and Xi == 1 we obtain

=

y, p

= 1, di

n- 1 / 2

K+(t) = n -

1

n

n

LL i== l

{IYi

+ Yk -

2tl-IYi - Ykl}

k== l

and the correspond ing minimizer is the well celebrated medi an of the pairwise means {(Yi + Yj )/2 ; 1 ::; i ::; j ::; n}. Suppose we spec ialize (1.1.1) to a complete ly randomized design with p treatments, i.e., take Xij

=

+ 1 ::; i

1,

m j -1

0,

ot herwise,

::; mj ,

where 1 ::; nj ::; n is the l h sa mple size, ma = 0, m j = n1 + ... + nj, l ::; j ::; p , m p = n. Then , upon taking G(y) == y, dij == Xij in (5.3.12) , we obtain K+(t) =

p

nj

nj

j == l

i ==l

k== l

LLL {IYij + Ykj -

2t jl-IYij - Ykjl} , t E W,

where Yij = the i t h observation from t he i" treatment , 1 ::; j ::; p . Consequently, f3+ = (13t ,· ·· ,13: ), where 13t = med{ (Yij + Ykj)2-1,1 ::; i::; k ::; nj} ,l ::; i : p . That is, in a complete ly randomized design with p treatments, f3 + correspo nding to the weights d , = X i and G( y) == y is the vector of Hodges - Lehmann est ima tors. Simil ar remark applies to the randomiz ed blo ck, factorial and other similar designs . The class of est imators also includes t he well celebrate d least absolut e deviat ion (LAD) est imator. To see this, assume that the

e:

5. Minimum Distance Estimators

150

errors are con tinuous. Choose G = 00 - the measure degenerate at 0 in K + , to obtain

K +(t)

t [t, t (t,

d;jsgn( Y; -

r

:s 0) -

l (Y; - x;t

> O)}

><:t))',

w. p .l, V

t E IR".

dij{I (Y; - x;t

Upon choosing d , == Xi , one sees that this expression is precisely the square of the norm of a .e. differential of the sum of absolute devi ations V(t) := L ~=::l lY'; - x~tl , t ], t E JRP . Clearly the minimizer of V( t) is also a minimizer of the ab ove K + (t) . Anyone of the express ions amo ng (5.3.7), (5.3.11) or (5.3.12) may b e used to com pute {3 + for a general G. It is also ap pare nt from the above dis cussion that both class es {,B} and {{3+} include rather int er esting est imators . On the on e hand we have a smooth unbounded G, v.i.z. , G(y ) == y, giving ris e to Hod ges - Lehmann ty pe estimators a nd on t he ot her hand a highly discrete G, v.i.z., G = 00 giving rise to t he LAD estimator. Any distributi on t heo ry should b e gene ral eno ug h to cover both of these cases. We now add ress the qu esti on of the exis tence of these est imat ors in the case p = 1. As befor e wh en p = 1, we write unb old letters for scala rs and di , Xi for di1, X il , 1 ::; i ::; n . Befor e stating the result we need t o define n

L

r(y) :=

I (x i = 0) d, {I(Y'; ::; y) - I( -Y'; < y)} ,

Y E JR.

i=:: l Ar guing as for (5.3.7) we obtain, with b = max{Y';, -Y'; ; 1 ::; i ::; n} , (5.3.13)

J

IfldG n

<

L i= l

<

00 .

I (xi

= 0) Idil [G(L)

- G(Y'; - ) + G(b_ ) - G( - Y';)]

151

5.3. Fini te sample properties Mor eover , directly from (5.3.7) we can concl ude that

Jr

(5.3.14)

2

dG <

00 .

Both (5.3.13) and (5.3.14) hold for all n :2: 1, for every sample {Yi} and for all real numbers {di } . Lemma 5.3.1. A ssum e that (1.1.1) with p = 1 holds. In addit ion, assume that either V 1 :s: i

(5.3.15)

:s:

n,

or (5.3.16)

V1

:s: i :s: n .

Th en a m in imizer of K + exis ts if either Case 1: G (JR) = 00, or Cas e 2: G (JR) < 00 and d; = 0 when ever Xi = 0,1 :s: i :s: n . If G is con tinuous then a minimizer is m easurable. Proof. The pr oof uses Fatou 's Lemm a and t he D.C.T. Sp ecialize (5.2.14) to t he case p = 1 to obtain

K +(t)

r

~ J[t, d;{ I(Y; $ y + x;l) - I( - Y; < y - x;t))

dG( y) .

Let K,+(y , t ) denote t he integrand wit ho ut t he squa re. Then

K,+(y , t ) = r( y ) + K,*(y , t ), where

K,*(y , t) n

=

L I( Xi > O)di{I (Yi :s: y + Xi t ) -

I( - Yi < y - Xi t )}

i= l

n

+ L I (xi < O)di{I (Y; :s: y + Xi t ) - I (-

Y; < y - Xit )}.

i= l

Clearly , V y , t E JR, n

1K,*(y , t)j

:s:

L I (xi -1= O)ldil i= l

=: a, say.

5. Minimum Distance Estimators

152 Hence (5.3.17)

f{y) - a ::; I(+{y , t) ::; f{y)

+a,

V y, t E llt

Suppose that (5.3.15) holds. Then, it follows that Vy E 1(* (y, t)

so that V y E (5.3.18)

~,

-+ ±a as t -+ ±oo,

~,

I(+{y, t) -+ f(y) ± a , as t -+ ±oo.

Now consider Case 1. If a = 0 th en either all X i == 0 or do = 0 for those i for which X i i= O. In either case one obtains from (5.3.14) and (5.3.17) that Vt E ~,K+{t) = Jf 2dG < 00 and hen ce a minimizer trivially exists. If a > 0 then, from (5.3.13) and (5.3.14) it follows that

J

(f(y) ± a)2dG{y) =

00 ,

and by (5.3.17) and the Fatou Lemma, lim inf K+{t) = t---7±OO

00 .

On the other hand by (5.3.7), K +(t) is a finite number for every real t , and hence a minimizer exists. Next , consider Case 2. Here, clearly I' == O. From (5.3.17), we obtain

and hence

K+{t) ::; a2G{~) , Vt E llt By (5.3.18) , I(+(y ,t) -+ ±a, as t -+ ±oo. By the D.C .T. we obtain

thereby proving the existence of a minimizer of K+ in Case 2. The continuity of G together with (5.3.12) shows that K+ is a continuous fun ction on ~ thereby ensuring the measurability of

5.3. Finite sample properties

153

a minimizer, by Corollary 2.1 of Brown and Purves (1973). This completes the proof in the case of (5.3.15). It is exactly similar when 0 (5.3.16) holds, hence no details will be given for that case. Remark 5.3.1. Observe that in some cases minimizers of K+ could be measurable even if G is not continuous. For example, in the case of LAD estimator, G is degenerate at 0 yet a measurable minimizer exists. Also, note that (5.3.15) is a priori satisfied by the weights d i = X i. The above proof is essentially due to Dhar (1991a) . Dhar (1991b) gives proofs of the existence of classes of estimators {,B} and {,8+} of (5.2.8) and (5.2.15) for p 2: 1, among other results. These proofs are somewhat complicated and will not be reproduced here. In both of these papers Dhar carries out some finite sample simulation studies and concludes that both, ,B and d " corresponding to G(y) = Y, show 0 some superiority over some of the well known estimators. Now we discuss ~ of (5.2.10). Rewrite

Q(t) =

n-

1

tt J

x~t)}

{I(Yi < y) - Hi(Y -

Lij

i==l j=l

X {I(Yj

::; y) - Hj(Y - xjt) } dG(y)

where L ij = 1- L(i V j)n- 1 ) , 1 ::; i ,j ::; n. Differentiating Q w.r.t. t under the integral sign (which can be justified assuming {Hd have Lebesgue densities {h i} and some other mild conditions) we obtain n

= 2n-

Q(t)

1

n

LL

Lij

J

{I(Yi ::; y) - (Hi(y -

x~t) }

i=l j=l

xhi(y - xjt) dG(y) Specialize this to the case G(y) integrate by parts , to obtain n

Q(t) =

-2n- 2

= y, L( s) = s, p

=

1,Xi

Xj.

= 1 and

n

L L min(n -

i , n - j){Hi(Yi - t) - 1/2}

i=l j=l

n

=

_n- 2 L(n - i)(n + i - l){Hi (Yi - t) - 1/2}. i==l

5. Minimum Distance Estimators

154

Now suppose fur ther t hat H i == F. Then

fJ

is a solution t of

n

(5.3.19)

2:: (n - i )(n + i - l ){F(Yi - t) - 1/ 2} =

o.

i=l

Compare th is fJ wit h iJ of (5.3.2). Clearly fJ given by (5.3.19) is a weight ed M-estimator of t he location parameter whereas iJ given by (5.3 .2) is an ordi nary M-estimator. Of course , if we choose L (s ) = 1(s 2: l) ,p = 1, Xi == 1, G(y) = y t hen iJ = (3. In genera l {3 may be obtained as a solut ion of Q (t ) = o. Next , consider f3* of (5.2.18 ). For the time be ing focus on the case p = 1 and di == X i - ii , Assume, without loss of genera lity, that t he data is so arranged t hat Xl ~ X2 ~ . . . ~ Xn . Let y := {(Yj - Yi)/(Xj - Xi); i < i , Xi < Xj }, to := min {t ; t E Y} and t l := max{t ;t E Y} . Then for Xi < Xj, t < to implies t < (Yj -Yd/ (x j -xd so t hat ~t < R jt . In ot her words the resid uals {Yj - tx j ; 1 ~ j ~ n} are naturally ord ered for all t < to, w.p .1., assum ing t he continuity of th e errors. Hence, with T( s , t ) denoting the Tl d(S, t) of (5.2.17) , we obtain for t < to,

T( s , t) =

'\"' L.., ik=l di ,

kin

0,

~

s < (k

+ l )/ n , 1 ~ k ~ n o~ s < lin, s =

1, 1.

Hence, K*(t) =

n~Wk -l {~k di }2

Similarly usin g the fact I.: ~=l di

= 0, one obtains

As t crosses over to only one pair of adjacent residuals change their ranks. Let Xj < Xj+! denot e t heir resp ective regression con-

155

5.3. Finite sample properties stants. Then

W(to_) -

r k~~#j {t,d;r I: {t r-{ ~ n

~Wk {t,d

W(to+)

i

-Wj {dj+l

Wk

+

di

}2

~=1

~

Wj [

di

dj+l

j

j- l

i= 1

i= 1

+

s.

L di < dj+l + L di < O. Hence K *(to_) > K *(to+) . Similarl y it follows that K *(t l+) K *(tl _) ' Con sequ ently, (31 and (32 are finit e, where

=

K *(~)} ,

(31 .-

min{t E YIK *(t+)

(32

max{t E Y ,K*(L) = inf K *(~)},

.-

inf

.6.EYC

>

.6.EY c

yc

and where denotes the complement of y . Then (3* can be uniquely defined by the relation (3* = ((31 + (32 ) / 2. This (3* corresponding to L(a) = s was studied by Willi amson (1979, 1982). In genera l t his estimator is asymptotically relatively mor e efficient than Wilcoxon type R-estimators as will be seen lat er on in Sect ion 5.6. There does not seem to be such a nice charac terizat ion for p 2:: 1 and general D sat isfying (5.2.16). However , pro ceeding as in the derivation of (5.3.6), a computat iona l formula for K * of (5.2.17) can be obtained to be

This formula is valid for a general used to compute {3 *.

(J" -

finit e measure L and can be

5. Minimum Distance Estimators

156

We now turn to t he m.d. est imator defined at (5.2.21) and (5.2.22). Let di == Xi - ii , The first obse rvat ion one ma kes is t ha t for t E JR, n

V n(t ) := sup yE IR

L di I(~ ~ y + tdd i=l

n

=

L

sup dJ( Rit ~ ns) 0'S s'S l i=l

Proceedings as in th e above discussion per tainin g to 13* , assume , without loss of generality, that the dat a is so ar ra nged that Xl ~ X2 ~ . . . ~ Xn so t hat d1 ~ d2 ~ . . . ~ dn· Let Y1 := {(Yj - ~ )/ ( dj - di);d i < O,dj 2 0, 1 ~ i < j ~ n }. It can be proved that V;i(D~) is a left cont inuous non-decreasing (right continuous non-increasing) st ep fun cti on on JR whos e points of discontinuity are a subse t of Y1. Moreover , if - 00 = to < ii ~ t2·· . ~ t m < t m+ 1 = 00 denote t he ordered members of Y1 then V ;i(t1 - ) = = V ;;(t m+ ) and v ;i(tm+) = 2:7=1 d; = V~ ( t1- ) , where dt == max (di , 0) . Consequent ly, t he following ent it ies are finite:

°

13s1 .-

inf{t E JR; V;i(t) 2 V ; (t )},

13s2 .-

sup] t E JR; V;i (t) ~ V ; (t )}.

Note t hat 13s2 2 13s1 w.p. l.. One can now take f3s = (131 + 13s2)/ 2. Willi am son (1979) pr ovides t he pr oofs of t he abov e claims and obtains t he asy mptotic dist ribut ion of 13s . This est imator is the pr ecise generalization of th e m.d. est imator of th e two sam ple location par ameter of Rao, Schuster and Lit tell (1975). Its asymptotic distribut ion is t he sa me as that of t heir estimator. We shall now discuss some add it ional dist ri bu tional prop erties of th e above m.d. estima tors . To facilit ate this discussion let /3 denote anyone of the est imators defined at (5.2.8), (5.2.15), (5.2.18) and (5.2.22). As in Section 4.3, we shall write /3(X , Y ) to emphasize the depe nde nce on the data { (x~ , Y i ) ; 1 ~ i ~ n} . It also helps to t hink of th e definin g dist an ces K , K + , etc . as fun cti ons of residu als. Thus we shall some times write K (Y - Xt) etc. for K (t ). Let K stand for eit her K or K + or K * of (5.2.7), (5.2.14) and (5.2.17). To begin with , obse rve t hat (5.3.21)

K (t - b ) = K(Y

+ Xb -

Xt ), \I t , b E W,

5.3. Finite sample prop erties

157

so that (5.3.22)

!3(X, Y

+ Xb) =

!3(X, Y)

+ b,

Vb E W .

Consequentl y, the dist ri bu ti on of !3 - {3 does not depend on {3. The dist an ce measure Q of (5.2.9) do es not satisfy (5.3.21) and hen ce the distribution of (:J - {3 will generally depend on {3. In general, the classes of est imators {,8} and {{3 +} are not scale invariant . However , as can be readily seen from (5.3.6) and (5.3.7) , the class {,8} corr esp onding to G (y) = y, H i = F and those {D} that satisfy (5.2.16) and the class {{3+} corres ponding to G(y) = y and general {D} ar e scale invariant in the sense of (4.3.2). An interesting property of all of the above m.d. estima tors is that th ey are invari ant under nonsin gular tran sformat ion of the design matrix X . That is, !3(XB , Y) = B - 1!3 (X, Y) V p x p nonsin gular matrix B. A similar st at ement holds for (:J. We sha ll end t his sect ion by discussin g t he symmetry pr operty of th ese est ima tors . In the following lemma it is implicitly ass umed t hat all integrals involved are finite. Some sufficient condit ions for t ha t to happen will unfold as we proceed in this chapter. Lemma 5.3.2. Let (1.1.1) hold wi th th e actual and the modeled d.f. of e; equal to H i , 1 ::; i ::; n . (i) If either (ia) {Hi ,l ::; i ::; n } an d G are sym me tri c aroun d 0 an d {Hi , 1 ::; i ::; n } are con tinuous, or (ib) dij = -dn- H1 ,j , Xij = - Xn- i+l ,j and H i = F V I::; i ::; n, 1 ::; j ::; p , th en

,8

an d {3* are symmetrically distributed aroun d {3 , whenever th ey exist uniquely.

(ii) If {Hi , 1 ::; i ::; n } and G are symmetric around 0 an d eit her { H i , 1 ::; i ::; n} are cont inuous or G is con tinuous, th en

5. Minimum Distance Estimators

158

13+

is symmetrically distributed around 13, whenever it exists uniquely. Proof. In view of (5.3.22) there is no loss of generality in assuming that the true 13 is O. Suppose that (ia) holds. Then /:J(X, Y) =d /:J(X, - Y). But, by definition (5.2.8) , /:J(X, - Y) is the minimizer of K( - Y - Xt) w.r.t. t . Observe that V t E W , K(-Y - Xt)

=

t J[t

t J[t

t J[t

r r r

dii{ I( -Yi :<: y + x:t) - Hi(y)}

dG(y)

dii{ 1 - I(Y; < -y - x:t) - Hi(y)} d;i{ I(Y; < Y - x:t) - Hi(Y-)}

dG(y)

dG(y)

by the symmetry of {Hd and G. Now use the continuity of {Hd to conclude that , w.p.L, K( - Y - Xt)

= K(Y + Xt) ,

V t E

~P ,

so that /:J(X , - Y) = -/:J(X, Y) , w.p.l , and the claim follows because -/:J(X, Y) = argmin{K(Y + Xt); t E ~P} . Now suppose that (ib) holds. Then

t J[t t J[t

r

K(Y +Xt)

~d

dn-i+U { I (Y; :<: y + x:.-iH t) - F(y) } dn- iH ,i{I(Yn - i+1 :<: y +

x~-Hlt) -

dG(y)

F(y) }] 2

xdG(y) K(Y - Xt),

V t E W.

This shows that -/3(X, Y) 13* is similar.

=d

/:J(X, Y) as required. The proof for

159

5.4. A general case Proof of (ii) . Again, ,8+ (X, Y) symmetry of {Hd . But,

=d

,8+(X, - Y), because of the

K+(-Y - Xt)

tf

[~dij{ I(-Y; S Y +x;t)-1 +I(-Yi:S

t f [~dij{

I(Y; < y +X;t)-1 + I(Yi

=

K+ (Y + Xt) ,

-Y+X~t)}rdG(Y)

Vt EW,

w.p .l , if either {Hd or G are continuous.

5.4

r

< -Y + x~t) } dG(y) o

A General M. D. Estimator

This section gives a general overview of an asymptotic theory useful in infer ence based on minimizing an objective function of the data and parameter in general models. It is a self contained section of broad interest. In an inferential problem consisting of a vector of n observations Cn = ((nl, ' " , (nn)', not necessarily independent, and a p - dimensional parameter 0 E W, an estimator of 0 is often based on an objective function Mn(Cn , 0) , herein called dispersion. In this section an estimator of 0 obtained by minimizing Mn (Cn, .) will be called minimum dispersion estimator. Typically the sequence of dispersion M n admits the following approximate quadratic structure. Writing Mn(O) for Mn(Cn 'O) , often it turns out that Mn(O) - Mn(Oo) , under 0 0 , is asymptotically like a quadratic form in (0 - ( 0 ), for 0 close to 0 0 in a certain sense, with the coefficient of the linear term equal to a random vector which is typically asymptotically normally distributed . This approximation in turn is used to obtain the asymptotic distribution of the corresponding minimum dispersion estimators.

160

5. Minimum Distance Estimators

The two class ical examples of the above typ e are Gauss's least sq uare and Fisher 's maximum likelihood estimators . In t he former th e disp ersion M n is t he error sum of squ ar es whil e in th e lat ter M n equals -log L n , L n denoting t he likelihood function of () based on CnIn the least squares method, M n( (}) - Mn((}o) is exac tly quadratic in (() - (}o), uniformly in () and (}o. The ra ndom vector appearing in t he linear te rm is typi cally asymptotically normally distributed. In the likelih ood method , the well celebrated locally asy mp totically normal (LAN) models of Le Cam (1960, 1986) ob ey t he above typ e of approxima te qu adrati c structure. Other well kn own examples include the least absolute deviation and th e minimum chi-squa re estimators. The main purpose of this section is to unify the basic structure of asy mptot ics underlyin g th e minimum dispersion est imators by exploiting t he above type of common asymptotic qua dratic struct ure inh erent in most of t he dispersions. The formulati on given below is general enough to cover some irr egular cases where t he coefficient of linear term may not be asy mptotically nor mally distribut ed . We now formulate genera l conditions for a given dispers ion to be uniformly locally asy mptotically quadratic (ULAQ). Accordi ngly, let 0 be an ope n subs et of IRP and M n , n ~ 1, b e a sequ ence of rea l valued functions defined on IRn x 0 such t hat M n ( · , (}) is meas urable for each (). We shall often suppress t he Cn coord inate in M n and wri te M n( (}) for Mn( Cn, (}). In orde r to state general cond it ions we need to define a sequence of neighb orh oods Nn( (}o) := { () E 0 , Idn((}o )((} - (}o)1 ~ b}, where (}o is a fixed par am eter valu e in 0 , b is a finit e number and {d n ((}o)} is a sequence p x p symmet ric positive definite matrices with norms Ildn((}o)11 tending to infinity. Since (}o is fixed , write dn, N n for dn((}o ), N n((}o), resp ectively. Similarly, let Pn denote the pr ob ability distribu tion of Cn when () = (}o ·

Definition 5.4.1. A sequence of disp ersions {M n ( (} ) , () E N n }, n ~ 1, satisfying condit ion (Ai) - (A3) given below is sa id to be uniformly locally asymptotically quadratic (ULAQ) . (Ai) There exist a sequence of p x 1 random vector S n((}O) and a seque nce of p x p , poss ib ly random , matrices W n((} o), such that ,

161

5.4. A general case for every 0

< b < 00,

Mn( O) Mn( Oo)

+ (0 -

OO)'S n(OO) + ~ (O - Oo)'Wn(OO)( O -

( 0)

+up (I ), where "u p (I )" is a seq uence of stochastic processes in 0 converging to zero, uniformly in 0 E N n , in Pn - pr obabili ty. (A2) There exists a p x p non-singular , poss ibly random, matrix W (O o) such that

(A3) There exists a p x 1

LV.

Y (O o) such t hat

where L n , L denote jo int probability dist ributions under Pn and in t he limit , respectively. Denote the conditions (Ai) , (A2) by (AI) and (A2) , respectively, whenever W is non-random in t hese conditions. A sequence of dispe rsions {Mn } is called uniformly locally asymptotically normal quadratic (ULANQ) if (AI) , (A2) hold and if (A3) , instead of (A3) , holds, whe re (A3) is as follows: (A3) T here exists a positive definite p x p matrix ~ (O o ) such t hat

If (Ai) holds without t he uniformity requirement and (A2), (A3) hold t hen we call t he given sequence M n LAQ (locally asymp totically qua dratic). If (A1) holds without the uniformity requirement and (A2), (A3) hold then t he given sequence M n is called locally asymptotically normal quadratic (LANQ). In t he case Mn( O) = - fn Ln(O ), t he cond it ion non-uniform (Al) , (A2) , (A3) wit h lI15n ll = O(n 1 / 2 ), determine t he well celebrated LAN models of Le Cam (1960, 1986). For t his particular case, W (O o), ~ (Oo ) an d t he limit ing Fisher information matrix, F (Oo), whenever it exists, are t he same.

5. Minimum Dist ance Estimators

162

In the above general formulation, M n is an arbitra ry dispersion satisfying (Ai) - (A3) or (AI) - (A3) . In the lat ter the three matrices W (9 0), ~ (9 0 ) and F (9 0) are not necessarily identical. The LANQ dispersions can thus be viewed as a generalizat ion of the LAN models. Typically in the classic al i.i.d. setup the normalizing matrix ~n is of the order square root of n whereas in t he linear regression model (1.1.1) it is of the ord er (X'X)1/2. In general ~n will depend on 9 0 and is determined by the ord er of the asymptotic magnitude of Sn(9 0). We now t urn to th e asymptot ic distribution of th e minimum dispersion estimators. Let {Mn } be a seq uenc e of ULAQ disp ersions. Define (5.4.1)

On = ar gmin{Mn(t), t E D}.

Our aim is to investigate th e asymptotic behavior of On and Mn( On)' Akin to th e study of th e asy mptotic distribution of m.l.e.'s, we must first ensure th at th ere is a On satisfying (5.4.1) such th at (5.4.2) Unfort unate ly th e ULAQ assumptions ar e not enough to guarantee (5.4.2). One set of additional assumptions th at ensures this is th e followin g. (A4) V E > 0 :3 a 0 < Zf < 00 and N 1f such th at

Pn(IMn( 9 0)1 :S Zf) ?: 1 (A5) V E > 0 and 0 and 0:) such t hat

E,

V n?: Nl{-

< 0: < 00, :3 an N 2f and a b (depending on

E

It is conveni ent to let , for a 9 E W ,

Qn(9 , 9 0)

:=

(9 - 90)'S n(90) + ~(9 - 9 0)'W n (9 0)(9 - 9 0),

and On := argmin{Qn(9 , 9 0) , 9 E W }. Clearly, relation (5.4.3)

On must

satisfy the

5.4. A general case

163

where B n := 8~ lwn8~1, where W n = Wn(l~O) ' Some generality is achieved by making the following assumption. (A6) 118 n(On - ( 0)11 = Op(I). Note that (A2) and (A3) imply (A6). We now state and prove Theorem 5.4.1 Let th e dispe rs ions M n satisfy (Ai) , (A4) - (A6). Th en , under ti;

(5.4.4)

I(On - On)'8 nB ndn(On - On)1 = op(I) , inf Mn(O) - Mn(Oo)

(5.4.5)

O ED.

= -(1 /2)(On -

Oo)'Wn(On - ( 0) + op(I) .

Con sequently, if (A6) is repla ced by (A2) and (A3), th en

8n(O n - ( 0) ~d {W(OO)} -ly(OO),

(5.4.6) (5.4.7)

inf Mn(O ) - Mn(Oo) O ED.

= -(1 /2)S~(00)8~113~18~ISn(00)

+ op(I) .

If, ins tead of (Ai) - (A3), M n satisfi es (AI) - (A3), and if (A4) and (A5) hold th en also (5.4.4) - (5.4.7) hold and

(5.4.8) whe re

Proof. Let

Zt

be as in (A4). Choose an a >

[ IMn(Oo)1 ~

Zt,

inf Mn(Oo Ilhll>b

C [ inf Mn(Oo +

Ilhll:Sb

C [ inf

IIhll>b

Mn(Oo

Zt

in (A5). Then

+ 8~lh) 2:: a]

8~lh) ~ Z t , Ilhll>b inf Mn(Oo + d~lh)

+ 8~lh) >

inf Mn(Oo Ilhll :Sb

+ 8~lh)]

2:: a ]

.

Hence by (A4) and (A5), for any E > 0 there exists a 0 (now dep ending only on E) su ch that V n 2:: NIt V N2t ,


00

5. Minimum Distan ce Estimators

164

This in turn ensures the validity of (5.4.2) , which in turn together with (Ai) yields (5.4.9) From (A6), the inequality inf [Mn(Oo)

OEN n

<

+ Qn(O, 00)] I

sup IMn(O) - [Mn(Oo)

+ Qn(O, 0 0 ) ] I

OEN n

and (Ai) , we obtain (5.4.10) Now, (5.4.9) and (5.4.10) readily yield

Qn(On , OO) =

a.»: 0

0)

+ op(l) , (P'l) ,

whi ch is pr ecisely equivalent to the statement (5.4.4). The claim (5.4.5) follows from (5.4.3) and (5.4.10). The rest is obvious. 0 R em a rk 5.4.1. Rou ghly speaking, the assumption (A5) assures t ha t th e sma llest value of Mn(O) for 0 outside of N n can be made asy mptot ically arbitrarily large with arbitrarily large probability. The ass umpt ion (A4) means that th e sequ ence of LV .'S {Mn(Oo)} is bounded in probability. This assumption is usually verified by an applicat ion of the Markov inequality in th e case EnIMn(Oo)1 = 0(1) , where En denotes the expec tat ion under Pn. In some applications Mn(Oo) converges weakly to a LV. which also implies (A4) . Oft en the verification of (A5) is rendered easy by an applicat ion of a vari ant of the C-S inequality. Examples of this appear in the next section when deal ing with m.d. estimators of t he pr evious sect ion . We now discuss the minimum dispersion tests of simple hypothesis , bri efly without many details. Consider the simple hypothesis H o : 0 = 0 0 . In the sp ecial case when M n is - £n L n , the likelihood ratio stat istic for testing H o is given by -2 inf{Mn(O)-Mn(Oo); 0 En}. Thus, given a general disp ersion fun ction M n , we ar e motivated to base a test of H o on th e st atisti c

5.4. A general case

165

wit h lar ge values of Tn being sign ificant . To study t he asymptotic null distribution of Tn, note t hat by (5.4.7), Tn = S~~;;- IB;;- I~;;- ISn + op(I) , (Pn) . Let Y , W etc. stand for Y (8 0), W (8 0), etc. Proposition 5.4.1. Under (Ai) - (A3) , (A4) , (A5) , the asymptotic null distribut ion of Tn is the same as that of Y'W -l y. Under (AI) - (A5) , the asymptotic null dist ribution of Tn is the sam e as that of Z'B Z where Z is a N( O, I pxp) r.u. and B = ~1 / 2W - I~ I /2 . 0 Remark 5.4.2. Clearly ifW (8 0) = ~ (8 0 ), then the asymptotic null distribution of Tn is X~ . However, if W(8 0) i= ~ (8 0 ), t he limit distribution of Tn is not a chi-square. We shall not discuss t he dist rib ut ion of Tn under alte rnatives. 0 Here we sha ll now discuss a few examples illust rati ng the above th eory. EXAMPLE 1: THE ONE SAMPLE L l. D . SETUP. Let e be an open interval in IR and {Fo; 0 E e} be a family of distribution functions on R Let 9 be real valued fun ction on IR x e such that g(·,O) is measurable for each 0 E e. Let Xl , ' " , X n be i.i.d. from an Foo ' where 00 is fixed value of 0 in e. Define n

Mn(O)

:=

L g(Xi, 0) ,

oE e.

i= l

We make t he following additional assumptions: Let A be an open neighborhood of 00 . a. g(x , 0) is absolutely continuous in 0 E A and for almost all x (Fo o)' wit h g(x,') denoting t he a .e. der ivative of g(x , ') satisfying (5.4 .11)

J

g2(x,0) dFoo(x) <

00,

b. T he fun ction ..\(0) := J g(x ,O) dFoo(x) is differentiable on A and the der ivative ..\ satisfies (5.4 .12)

c. The function s t--+ J[g( x , s) - g(x ,OoW dFoo(x) is cont inuous at 00 .

166

5. Minimum Distance Estimators

°

Under the above assumptions, for every < b < 00 and for every sequence of real numbers On E N n := {t: tEe, n 1 / 2 1t - 00 1~ b}, n

(5.4.13)

Mn(On) = Mn(Oo) - (On - ( 0 ) Lg(Xi,Oo) i=l

Proof. Let D n denote the difference between Mn(On) and the leading r.v .'s on the right hand side of (5.4.13), F = F oo' E, V denote the expectation and variance with respect to F , Un := On - 00 , and let t n := n 1 / 2 Un ' To prove (5.4.13), it suffices to prove (5.4.14)

ED n

-t

0,

First , suppose Un > 0. Consider n

E[Mn(On) - Mn(Oo) - (On - ( 0 ) Lg(Xi,Oo)] n n

J Jl

i=l

[g(x, On) - g(x . ( 0 )

(On - ( 0 ) g(x , ( 0 ) ] dF(x)

un

[g(x,00

l J l x l un

n

-

[g(x. eo

+ s) - g(x ,Oo)] ds dF( x) , + s) - g(x , ( 0 ) ] dF(x) ds ,

un

n

n 1/ 2

[

(00

+ s) - >.(( 0 ) ] ds

tn

[>'(0 0

+ sn - 1/ 2 ) -

>.(00 ) ] ds .

In the above , the second equality follows from a and (5.4.11) while the third uses Fubini. Note that t;.j2 = J~n s ds. Hence

167

5.4. A general case

<

b2(bn- 1/ 2) -

-+

0,

1

bn- 1 / 2

1

1~(Oo + s) - ~(Oo)1

ds

by (5.4.12), thereby proving the first part of (5.4.14) when To prove the second part in this case , similarly we have

V(D n)

nV [g(X , en) - g(X, ( 0 )

-

Un

> 0.

(On - Oo)g(X,( 0 ) ]

< nE[g(X, On) - g(X,Oo) - (On - Oo)g(X,Oo)f

J[l Jl

un

n

< n

{g(x , s + ( 0 )

un

u -,

Tnnl/21un

-

g(x , Oo)} ds

f

dF( x)

{g(x , s + eo) - g(x , Oo)} 2 dsdF(x)

J

{g(x , s + ( 0 )

-

g(x , ( 0 ) } 2 dF( x)ds

< b2(bn- 1/ 2) - 1

xl! s+ ( bn -

1/ 2

{g(x ,

-+

0,

0) -

g(x , Oo)}2dF(x)

by the assumption c .

This completes the proof of (5.4.13) in the case On > 00. The proof is similar in the opposite case , thereby completing the proof of (5.4.13) . Actually we can prove mor e under the same assumptions . Let

We sh all now show , under the same assumptions as above , that for every 0 < b < 00 , (5.4.15)

E(SUItl:SbP ID

n(t)I)2

-+ o.

To prove this , because of a , we can rewrite with probability 1 for all t 2': 0 and for all n 2': 1,

5. Minimum Distance Estimators

168

n

Dn(t)

=

L

[g(Xi,(JO+ tn- l / 2 )

-

g(Xi , ( 0)

i::::l

2

. ] t A(Oo) 2n

- tn" 1/2 g(Xi , ( 0) -

=

=

rt

Jo

n- l / 2

t

[g(Xi , 00 + sn- l / 2 )

-

g(Xi , ( 0)

i :::: l

- sn- l / 2 ~(OO) ] ds

it t

[g(Xi , 00 + sn- l / 2 )

n- l / 2

o

-

g(Xi , ( 0)

i::::l

- A(Oo + sn- l / 2 )

+ A(Oo) ] ds

t

+ 1

[

n l / 2[A(00

+ sn- l / 2 )

-

A(Oo)

- sn- l / 2 ~(Oo) ] ds say . Hence ,

E( sup ID

(t)12 )

nl

O
b

<

E(1

In- l / 2 t . [ g ( Xi,00+ sn- 1/ 2)-g(Xi ,00) _ A(Oo

<

+ sn- l / 2 ) + A(Oo) ] I dS) 2

blob E(n- 1!2t. [!i(Xi ,OO+sn-1!')-!i(Xi ,OO)

_ A(Oo

+ sn- l / 2 ) + A(Oo) ]) 2 ds

b

=

b1

V (g(X , 00 + sn - l / 2 ) - A(Oo

2(bn- l

< b

2)-11 /

bn -

-

g(Xi , ( 0)

+ sn- l / 2 ) + A(Oo) ) ds

1/ 2

E[g(X,00+U)-g(X,00)]2du

5.4. A general case

169

--+

by c.

0,

Similarly,

1

bn- 1 / 2

sup IDn2(t)1 099

b2(bn- 1 / 2 ) - 1

<

I~(eo + s)

-

~(eo)1 ds

by b.

0(1),

This completes the proof of (5.4 .15) for t ~ O. The details are exactly similar for the case t ::; O. Hence (5.4.15) is proved, which in turn together with the Markov inequality implies that for every 0 < b < 00, SUPltl::;b IDn(t)1 = op(I) . 2. Now consider a more general setup where the parameter (J is p-dimensional real vector, the observations X ni , 1 ::; i ::; n, are independent r.v.'s, each having possibly a different distribution depending on the true parameter value (Jo under a sequence of probability measures Pn := TI~=l Fni . Here , Fni denotes the d.f of X ni, 1 ::; i ::; n, that will typically depend on (Jo. Also, let Cni, 1 ::; i ::; n, be arrays of p-vectors in W, e s;;; W, and EXAMPLE

n

Mn((J) :=

L g(Xni , C~i(J),

(J E

e.

i=l Let C denote the n x p matrix whose i t k row is c~i' 1 ::; i ::; n , and Ani(t) := Eng(Xni , t), t E JR, where En is the expectation under Pn · About the constants Cni, assume the following: d. The vectors Cni are such that maxl::;i::;n Ilcnill = 0(1), and C'C = I pxp. e. The function 9 is as in a satisfying (5.4.16)

:t J IIc nil1

i=l

(5.4.17)

2

g2(x,

C~i(JO) dFni(x) =

lim lim sup max n-+oo l::;t::;n

s-e O

J

{g(x , C~i(JO

0(1),

+ s)

-g(x, C~i(JO) } 2 dFni(x) = O. f. The functions {And are absolutely continuous with their a.e.

170

5. Minimum Distance Estimators

derivatives {~ni} satisfy the following: For any sequence ani of positive numbers with maxl :::;i:::;n ani = 0(1) , (5.4.18) Using the ar guments of the typ e used in Example 1 above, one can prove that under d - f the followin g holds: For every 0 < b < 00 , n

(5.4.19) Mn( O) = Mn(Oo) - (0 - ( 0

)'2: cnig(Xni , C~i OO ) i=1

-

~(O -

n

(0

)'2: CniC~i ~ni (C~iOO)(O -

( 0)

i=1

where u p ( l ) is a sequence of pr ocesses in () that converges to zero uniformly over the set 110 - 0 0 II ~ b, in probability. Consider as a sp ecial case the M-dispersion used by Huber (1981) to construct M-estimators in the linear regr ession model (1.1.1) . Here one takes Cni == X ni (X ' X )- 1/ 2, and n

M n({3) =

2: p(Yni i=1

n

x~d3) =

2: p(eni -

Cni(X' X )I/2(,8 - ,80)) ,

i=1

where p is measurable fun ction from JR to JR. Upon identifying X ni == eni, 0 = (X'X)I / 2(,8 _,80) ' 0 0 = 0, one readily obtains the following: Suppose (1.1.1) holds with the errors eni i.i.d. F; p is ab solutely continuous wit h a .e. der ivative 'ljJ su ch t hat

!

1'ljJ( x

+ y)ldF( x) < 00 , V Y E JR,

!

'ljJdF = 0,

!

'ljJ2dF

< 00.

Let IT(t ) := f['ljJ( x - t) - 'ljJ (xW dF( x) , t E JR, r = 1, 2. Not e that upon ident ifying g(y , t ) of the EXAMPLE 1 with p(y - t) , one sees that g(y , t) == - 'ljJ(y - t) and that 1 1 = >'(0) - >'(t) . Suppose, additionally, that 1 1 is continuously differ entiable at 0 and that 1 2 is cont inuous at O. Then , under (NX) , it follows that Huber 's dispersion M n is

171

5.4. A general case ULANQ with n

=

00

~n

0,

= (X'X)I/2,

Sn(j3)

= - LXi1/J(J!i -

x~{3),

i=1

W

= -\(O)X'X, W = -\(O)Ipxp,

n({3)

:E = J'ljJ2dFlpxp .

If p is additionally assumed to be convex , then using a result in Rockafeller (1970), to prove the ULANQ, it suffices to prove that the score statistic Sn({3) is asymptotically linear in (X'X)I/2({3 - {30) , for {3 in the set II (X'X)I/2({3 - {30)11 :::; b, in probability. See Heiler and Weiler (1988) and Pollard (1991) for details. For p(x) = Ixl and F continuous, 'ljJ(x) = sgn(x) and Ir(t) = 2r IF (t ) - F(O)I. The condition on II now translates to the usual condition on F in terms of the density f at O. For p(x) = x 2, 'ljJ(x) == 2x , 11(t) == 2t , so that II is trivially continuously differentiable with ')'1 (0) = 2. Note that in general W =I :E unless -\(0) = J 'ljJ2dF which 0 is the case when 'ljJ is related to the likelihood scores.

3. An example where the full strength of (Ai) - (A3) is realized is obtained by considering the least square dispersion in an explosive AR(I) process where X o = 0 and for some 101 > 1, EXAMPLE

i = 1,2, ' " ,

where it is assumed that e, are mean zero finite variance i.i.d. r.v .'s . Let 1 n Mn(t) := 2 (Xi - tXi _ 1)2, ItI > 1.

L

i=1

Then the least square estimator of 0 is en := argmintMn(t). We now verify (Ai) - (A3) for this dispersion. Let 00 denote the true value. Rewrite, for a 101 > 1, 1 n

Mn(O)

= 2 L (s, -

(0 - 00)X i _ 1)2

i=1

1

n

n

2L i=1

(0 - 00 )

L i=l

ciXi - l

+

1

2(0 - 00 ) 2

n

L i=l

xLI

5. Minimum Distance Estimators

172

So one read ily sees that what Sn and W n should be. Now t he question is what should be On , Wand Y . F irst note that one can rewrite j

x, =

L O~ - i ei, i= l

so t ha t j

ooj x, =

L «' e, i= l

is a mean zero square int egrabl e martingale. Not e also that , with a 2 = E eI , and because 05 > 1, as j -7 00,

sup)· E ( 4)2 =

The above derivati ons also show t hat

T

2

°0 exists theorem , t here

Hence, by t he martingale convergence with mean zero and vari an ce T 2 such th at a.s. and in

£2 ,

as j

< a

00 .

LV.

Z

-7 00 .

This implies that IXjl explodes to infinity as j -+ 00. Also not e that becaus e ~!=1 00iei is a sum of i.i.d . mean zero finit e vari an ce r.v.'s , by the CLT , Z is a N(O , (05 _1) -1) LV. Now let Zi := 00 iXi . Then

Oo 2nW n =

n

L 002i Z~ _ i

-7

Z 2/(05 - 1),

a.s .

i=1

This suggests that th e On

= O~

and W

= Z 2/ (05 -

n

00 n s;

= L Oo iZn-i i= 1

en-HI ,

1). Now consider

173

5.5. Asymptotic uniform quadraticity

Because of the indep end ence of Ei from X i-I , Sn can be verified to be a means zero squa re int egrable martingale. By the martingale cent ral limit theorem it follows that

where ZI is a N (O , (115 - 1)-1 ) LV. , indep endent of Z . So here Y = !ZIZI. 0 The next secti on is devoted to verifyin g (A1) - (A5) for vari ous disp ersion introduce in Sect ion 5.2.

5.5

Asymptotic Uniform Quadraticity

In this section we sh all give sufficient conditions under which K D , KiJ of Sect ion 5.2 will satisfy (5.4.Al) , (5.4.A4) , (5.4.A5) and K and Q of Section 5.2 will satisfy (5.4.AI). As is seen from t he pr evious sect ion this will bring us a step closer to obtaining the asy mptot ic distributi ons of various m.d. est imators introduced in Sect ion 5.2. To begin wit h we shall focus on (5.4.A l) for KD ,KiJ and K Our bas ic objective is to study t he asy mptotic dist ribu tion of when t he actual d.f.'s of { e n i ' 1 ~ i ~ n} are {Fni , I ~ i ~ n} bu t we mod el them to be {Hn i , 1 ~ i ~ n} . Similarly, we wish to st udy t he asy mptotic distribution of 13i) when actually t he errors may not be symmetric but we model t hem to be so. To achieve these object ives it is necessary to obtain t he asy mptot ic resul ts un der as genera l a setting as possible. This of course makes the expos it ion t hat follows look somewha t complicated. The results thus obtained will ena ble us to study not onl y th e asymptotic distributions of th ese estimators at the true model but also some of their robustness pr op erties. With this in mind we pr oceed to state our assumptions.

n

n·

/3D

(a) X

satisfies

(NX).

(b) With d(j) denoting t he j -t h column of D , IId(j) 11 2 > 0 for at least one j ; Ild(j)1 12 = I for all t hose j for which Ild(j) 1I 2 > 0, 1 < j

<».

(c) {Fn i , 1

~ i ~

n} admit densiti es

{fni ,

1 ~ i ~ n} W.Lt. A.

5. Minimum Dist ance Estimators

174

(d) {G n } is a sequence in 'DI(IR). (e) With d~i

= (dni1,' " , dnip), the i-th

/t

Iidni l12 Fni (1 -

row of D , 1 ::; i ::; n ,

FnddGn

= 0(1) .

i= l

(f) W ith "in

:=

~7=1

Iidn i l12 L« ,

lim sup n

ibn/

"in (y + X )dG n (y )dx = 0

Jan

for any real sequences {an} ,{bn} ,an

< bn, bn - an -+ O.

(g) With dnij = d~ij-d;;ij ' 1 ::; j ::; p; Cni = AXni , /'\,ni i ::; n, V 6

lim sup n

:=

Ilcnill, 1 ::;

> 0, V IIvll ::; b, p

n

j= l

i= l

L / [L d;ij {Fni (y + v' Cni + 6/'\,nd - Fni (y + v ' Cn i

6/'\,ni)}

-

r

dGn (y)

where k is a constant not dep ending on v and 6.

(h). With

n., :=

t j=l

~7=1 dnij xndni' Vnj := ARnj , 1 ::; j ::; p ,

J[JL~j(Y, u ) - JL~j(Y' 0) -

U 'V n j

2

(y)J dG n (y) = 0(1).

(j) Wi th mnj := ~ ~=1 dnij[Fni - H ni ], 1 ::; j ::; p; m np),

mi:>

= (m n1 , ' . . ,

5.5. Asymptotic uniform quadraticity

175

(k) With r~(y) := (Vnl(Y), '" ,vnp(Y)) = D'A*(y)XA, where A* is defined at (4.2.1) , and with I'n := JrngndG n , where gn E Lr(Gn), r = 1,2, n ~ 1, is such that gn > 0,

°< li~inf

/ g;dGn

~ limnsup /

and such that there exists an a

g;dGn <

00,

°

> satisfying

liminfinf{e'I'ne; e E 1W ,lIell = n

1} ~ a .

(l) Either (1) e'dniX~iAe ~ 0, V 1 ~ i ~ n and VeE W , lIell = 1.

Or

(2) e'dniX~iAe ~ 0, V 1 ~ i ~ n and VeE W , Ilell = 1.

In most of the subsequent applications of the results obtained in this section, the sequence of integrating measures {Gn } will be a fixed G. However , we formulate the results of this section in terms of sequences {G n} to allow ext ra generality. Note that if Gn == G, G E VI(JR) , then th ere always exist a agE Lr(G) , r = 1,2, such that g > 0, < J ldG < 00 . Define , for y E JR, u E W , 1 ~ j ~ p ,

°

(5.5.1)

sJ(y , u)

.-

Yjd(y , Au) ,

yjO(y, u)

.- SJ(y , u) - f-L1(y , u).

Note that for each j , SJ , f-L1 , ~o are the same as in (2.3.2) applied to X ni = Yni , Cni = AXn i and dni = dnij , 1 ~ i ~ n , 1 ~ j ~ p . Notation. For any functions g , h : W+l -+ JR,

Igu -

hvl~ := /

{g(y , u) - h(y, v)}2dG n(y) .

Occasionally we shall write Igl~ for Igo I~ · Lemma 5.5.1. Let Yn 1 , ' " , Y n n be independ ent r.v. '8 with respective d.f.'s F n 1 , ' " , Fn n . Then (e) implies p

(5.5.2)

E

I: Ilj~l~ = 0(1) . j=l

5. Minimum Distance Estimators

176

Proof. By Fubini's Theorem, n

P

E

L IYJol; = / L j=l

2

IIdi ll Fi (l - Fi)dGn

i=l

and hence (e) implies the Lemma. 0 Lemma 5.5.2. Let {Ynd be as in Lemma 5.5.1. Then assumptions (a) - (d) , (f) - (j) imply that, for every 0 < b < 00, P

L

E sup IYj~ Ilull:Sb j=l

(5.5.3)

Proof. Let N(b) .- {u E VUE N(b),

jRP;

-

Yj~l; = 0(1).

IIuli :S b}. By Fubini 's Theorem,

P

L

E

I}~?u - Yj~I;

j=l

:S /

t

IIdi ll

r:(/

2lFi(Y

+ c~u) - Fi(y)ldGn

i =l

:S where b.,

= b max, K,i , In P

(5.5.4)

In(Y + x)dGn(y))dx

as in (f). Therefore, by the assumption (J),

ELI Yjou -

Yj~ I~ = o(1), VuE

jRP.

j =l

To complet e the proof of (5.5.3), because of the compactness of N(b) , it suffices to show that V E > 0, :3 a 8 > 0 such that V v E N(b) , P

(5.5.5)

lim sup E n

L

sup IL j u - Ljvl :S lIu- vll :So j =l

E,

where Lj u

:=

IYj~

-

Yj~I; ,

u E

JRP , 1:S j :S p .

Expand th e quadratic, apply the C-S inequality to the cross product terms to obtain , for all 1:S j :S p ,

177

5.5. Asymptotic uniform quadraticity ~

Moreover, for all 1 (5.5.6)

IYj~

j ~ p,

< 2{ISJu - SJvl~ + IltJu - ItJvl~}, ISJu - SJvl~ < 2{lstu - stl~ + ISju - Sjvl~}' IltJu - ItJvl~ < 2{llttu - Ittvl~ + Iltju - ItJvI~} ,

-

Yj~I~

are the SJ, ItJ with dij replaced by dtJ := max(O, dij), dij := dt; - dij , 1 ~ i ~ n , 1 ~ j ~ p . Now, Ilu - vii ~ 6, nonnegativity of {dtJ}, and the monotonicity of {Fd yields (use (2.3.17) here) , that for all 1 ~ j ~ p , where SJ'

It]=

r

± - Itjv ± 1n2 Itju

I

s;

J[t, d~

{F,(y + c;v +OK,) - F;(y + c;v - OK,)}

dGn(y).

Therefore, by assumption (g), p

(5.5 .7)

lim sup

sup

L IltJu - ItJvl~ ~ 4k6

2

.

Ilu-vll~o j=l

n

By the monotonicity of SJ and (2.3.17) , for all 1 ~ j ~ p, Y E ~,

Ilu-vll

~

0 implies that

n

- "d*I(-6K L lJ l' < 1':l - v' c.l - Y < - 0) i= l

~ SJ(Y, u) - SJ(Y , v) n

~

L dtJI(O < Yi -

V'Ci -

Y ~ 6Kd·

i=l

This in turn implies (using the fact that a ~ b ~ c implies b2 ~ a2+c2 for any reals a, b, c)

{ SJ(Y, u) - SJ(Y, v) } 2

< {

t

dtJ I (0 < Yi - Y - v'c,

l=l

+

{t l=l

~ 0Ki) }

dtJI( -OKi < Yi - Y -

2

V'Ci

~ 0)}2

5. Minimum Distance Estimators

178

<

2

{t d~I

(-<5Ki < Yi

- Y-

V'Ci S; <5Ki)}2 ,

z=1

for all 1 S; j S; P and all Y E lit Now use the fact that for a, b real, (a + b)2 S; 2a2 + 2b2 to conclude that , for all 1 S; j S; P,

ISTu - stl~ <

4

J{t, d~

[I( -OKi < Y; - Y - v' Ci S OKi) - Pity, v , oj]

r

x dGn(y )

J{t, d~

+4 4 {Ij

+ II j } ,

Pi(Y, v,

OJ}

2

dGn(yj

(say),

wh er e Pi(Y, v , <5) = Fi(y + v' ci<5 Ki) - Fi(y + V'Ci - <5Kd · But (d~)2 S; drj for all i a nd j im plies that p

p

n

L / L(dm 2pi(Y, v , <5)(1 - Pi(Y, v , <5))dGn(y) j=1 i=1

ELIj j=1

< /

t

l:

IldiIl 2 pi(Y, v , <5)dGn(y)

i=1

<

n (/

1n(Y+S)dG n(Y)) ds ,

by (c) and Fubini , where an = (-b - <5) maxi Ki, b., = (b + <5) maxi Ki a nd where "[ri is defined in (f). Therefore, by the assumption (1) , p

ELIj = 0(1). j=1 From the definition of IIj above and the assumption (g), p

lim sup

n

L I t, S; k<5

2

.

j=1

From the above bounds, we r eadily obtain p

(5.5 .8)

limsupE

n

sup

L

lIu- vll:S" j=1

IYj Ou

-

Yj~I~ S; 40k<5 2 .

5.5. Asymptotic uniform quadraticity

179

Thus if we choose 0 < 0 :S (E/40k)1/2, then (5.5.5) will follow from (5.5.8), (5.5.7), (5.5.6) and (5.5.4). This also completes the proof of (5.5.3) . 0 To state the next theorem we need to introduce KD(t)

L J{Y/(y , 0) + t'Rj(y) + mj(Y)} P

:=

2

dGn(y)·

j==l

In (5.5.9) below, the G in KD is assumed to have been replaced by the sequence G n , just for extra generality. Theorem 5.5.1 Let Yn 1 , ' " , Ynn be independent r.v. 's with respective d.f. 's F n 1 , ' . . , Fnn . Suppose that {X , Fni, H ni , D , Gn} satisfy (a) - (j) . Then, for every 0 < b < 00 , (5.5.9)

E sup IKo(Au) - Ko(Au)1 = 0(1). IlullSb

Proof. Writ e K , K etc . for Ko , Ko etc . Not e that K(Au)

L J[SJ(y , u) P

J-l~(Y) + mj(Y)] 2 dGn(y)

j ==l

L J[Y/(y , u) - Y/(Y) + Y/(Y) + U'Vj(Y) + m j(Y) P

j==l

+J-l~(Y, u) - J-l~(Y) -

U'Vj(y)f dGn(y)

where yjO(y) == yjO(y , 0) , J-l~(Y) == J-lJ(Y,O) . Expand the quadratic and use the C-S inequality on the cross product terms to obtain

(5.5.10)

IK(Au) - K(Au)1 P

<

L

{IY~Fu - Yj~I~ + IJ-lJu -

J-lJ -

u'vjl~

j ==l

+ 21Yjou - Yj~ln [IYjO + U'Vj + mjln + IJ-lJu - J-lJ - U'Vjln] + 21Yjo

+ U'Vj + mjln 'IJ-l~u - J-l~ -

U'Vjln} .

5. Minim um Distance Estim ators

180

In view of Lemmas 5.5.1, 5.5.2 and assumpt ions (h) and (j) , (5.5.9) will follow from the above inequ ality, if we pr ove p

(5.5.11)

sup L lIull:SB j= l

°

IfLJu -

fLJ - u'vj l; = 0(1).

°

2 l c . .I n VIew . Let <'JU . - IfL ju - fL j - U "1 v J n' :S J' :S P, U E T!llp IN>: . 0 f th e compac t ness of N (b) and th e assumption (i), it suffices to pr ove that V E > 0, 3 a 0 > 0 3 V v E N(b ),

(5.5.12)

lim sup n

sup s~p ~ju Ilu- vll:S8 j = I

- ~jv l

:S

E-

But

I~ju - ~jvl < 2{lfLJu-/-lJvl; + Ilu - vll 2 l1 vjll; + ~;~2 [1J-lJu + IfLJu -

fLJvln + Ilu -

VIl IlVj lln]

fLJvlnllu - VIIIIVjlln}.

Hen ce, from (5.5.7) and t he ass um ption (i) , the left hand side of (5.5.12) is bo unded above by

2 { 4k02 + 02(a + 2k 1 / 2 a 1/ 2 ) } = k10 2 where a = lim sup., L~=l IIvjll; . Thus up on choosing 02 :S Elk1 one obtains (5.5.12), hence (5.5.11) and t herefore t he Theorem . 0 Our next goal is to obtain an analogue of (5.5.9) for Ki) . Before stat ing it rigorously, it helps to rewr ite Ki) in terms of standardized processes {Y/} and {fLJ } defined at (5.5.1). In fact , we have

Ki)(Au ) =

P

L

j= l

!

]2

n

[SJ(Y,u) - Ldij +SJ (-y, u ) dGn(y) i= l

P

=

L ! [yjO(y, u ) - yjO(y)

+ Y/( - V, u ) -

Y/( - V)

j= l

+ fLJ(y , u) - fLJ(y) - U'Vj(Y) + fLJ( - V, u ) - fLJ( - V) - U'Vj (Y) + u'vj (y) + W/(y) + mj(y)f dGn(y),

5.5. Asymptotic uniform quadraticity

181

where

n

mj(y)

.-

Ldij{Fi(Y) - 1 + Fi(-y)} i=l

n

=

p,J(y)

+ p,J(-y)

- L

dij ,

Y E JR, 1

~

j

~ p.

i=l

Let p

(5.5.13)

ki)(Au) = L

j[w/ +mj +u'vj]2dGn,

u E IW.

j=l

Now proceeding as in (5.5.10) , one obtains a similar upper bound for IKi)(Au) - ki)(Au)\ involving terms like those in R.B .S. of (5.5.10) and the terms like IYj~ -lj~l-n, Ip,Ju - p,J -u'Vjl-n , Ilvjl!-n, IYjOI- n , where for any function h : W+1 ~ JR,

It thus becomes apparent that one needs an analogue of Lemmas 5.5.1 and 5.5.2 with G n (-) replaced by G n ( _ .) . That is, if the conditions (e) - (j) are also assumed to hold for measures {G n ( _ · )} then obviously analogues of these lemmas will hold . Alternatively, the statement of the following theorem and the details of its proof are considerably simplified if one assumes Gn to be symmetric around zero, as we shall do for convenience. Before stating the theorem, we state Lemma 5.5.3. Let Y n1, ' " , Y nn be independent r.v. 's with respective d.f. 's Fn1,'" , Fnn · Assume (a) - (d) , (f) , (g) hold , {G n} satisfies (5 .3.8) and that (5.5.14) hold, where

(5.5.14)

j t IldniI1 2

{ Fni(-y)

+1-

Fni(y)}dGn(y) = 0(1).

i =l

Then, p

(5.5.15)

E

L j=l

IYj~l=-n

= 0(1),

5. Minimum Distance Estimators

182

and \:j 0 < b <

00 , p

L

E sup IYj~ - Yj~I=-n = 0(1). Ilull:Sb j=:1

(5.5.16)

o

This lemma follows from Lemmas 5.5.1 and 5.5.2, since under (5.3.8), LHS's of (5.5.15) and (5.5.16) are equal to those of (5.5.2) and (5.5.3) , respectively. The proof of the following theorem is similar to that of Theorem 5.5.1. Theorem 5.5.2 Let Yn1, ' " , Ynn be independent r.u. 's with respective d.f. 's Fn1, ' " , Fnn. Suppose that {X, Fni , D , G n} satisfy (a)(d), (f) - (i), (5.3.8) for all n 2': 1, (5.5.14) and that

L J{mj(y)}2dG n(y) = 0(1), p

(5.5.17)

j=:1

Then,

\:j

0

< b < 00 , E sup IKri(Au) - kri(Au)1 = 0(1). Ilu} :;b

(5.5 .18)

Remark 5.5.1. Recall that we are interested in the asymptotic distribution of A -1(/3n - (3) which is a minimizer of Kn({3 + Au) w.r.t. u . Since i3n satisfies (5.3 .22), there is no loss of generality in taking the true {3 equal to O. Then (5.5.9) asserts that (1/2)Kn satisfies (5.4.Al) with (5.5.19)

(Jo

0,

s; =

A -I ,

s, A-I :Tn, W n = ABnA , a; '- - rn(Y){Y~(Y) + mn(y)}dGn(y) ,

e; .-

J Jrn(y)r~(y)dGn(Y) ,

yg

where rn(y) = AX'A*(y)D ,A* as in (4.2.1), := (y10 , .. . , YpO) and m6 := (ml ,' . . , m p ) . In view of Lemma 5.5.1, the assumptions (e) and (h) of this section imply that EKn(O) = 0(1) , thereby ensuring the validity of (5.4.A4).

5.5. Asymptotic uniform quadraticity

183

Similarly, (5.5.18) asserts that (1/2)Ki!; satisfies (5.4.Al) with (5.5.20)

= =

(Jo

Sn

:J~ .B+n

.-

0, an = A-I ,

A-I :J+ W n = AB~ A , n'

-Jr~(y){W~(y) + Jr

m o(y)}dGn(y) ,

+' (y)dGn(Y), n(y) r n

where r~(y) := AX'A+(y)D ,A+(y) := A*(y) + A*( -y) ,y E ffi.P , WtJ' := (wt ,··· , W:) and (mt , · · , ,mt) ·

mt::=

In view of (5.5.12), (5.5.14) and (5.3.8) it follows that (5.4.A4) is satisfied by Ki!;( O) . Theorem 5.4.1 enables one to study the asymptotic distribution of /:Jo when in (1.1.1) the actual error d.f. F n i is not necessarily equal to the modeled d.f. H ni ,1 :::; i :::; n . Theorem 5.4.2 enables one to study the asymptotic distribution of f3tJ when in (1.1.1) the error d.f. F n i is not necessarily symmetric around 0, but we model it to 0 be so, 1 :::; i :::; n. So far we have not used the assumptions (k) and (l). They will be now used to obtain (5.4.A5) for K o and Ki!; . Lemma 5.5.4. In addit ion to th e assumptions of Theorem 5.5 .1 assume that (k) and (l) hold. Th en, 'i E > 0, 0 < z < 00, 3N (dep ending only on E) and a b E (0,00) (d epending on E, z) such that,

(5.5.21)

P( inf Ko( Au) 2 z) lIull >b

> 1-

E,

'in 2 N ,

(5:5.22)

P( inf K o( A u) 2 z ) > 1 Ilull>b

E,

'in 2 N,

P roof. As before write K , K etc . for K o , K o etc. Recall the definition ofI'n from (k) . Let kn(e):= e'I'ne, e E ~p . By the C-S inequality and (k) , p

(5.5.23)

sup Ikn(eW :::; Ilell=I

Ilr nll2

:::;

L

j =I

II vj ll; · Ign!;

= 0(1).

5. Minimum Distance Estim ators

184 Fix an

E

> 0 and a

z E (0,00) . Define, for t E IRP , 1 ~ j ~ P,

~(t)

.-

!

Vj( t)

.-

! [Vjd(Y, t ) -

{Y/

+ t'Rj + mj }gndGn,

t

dnij Hni(Y)] gn(y)dGn(y) ·

i=I

Also, let , ,

'

V := (VI , '"

,

,

,Vp ) , V := (VI, '"

Write a u E IRP with t he C-S inequality,

2

.

, Vp ) , "[n := Ignln, T := lim sup jc . n

lIuli > b as u =

re ,

Irl > b, lIell

= 1. Then , by

inf K (Au ) Ilull>b

>

inf (e'V(r A e))2 l t« , Irl>b.llell=I

inf K( A u) Ilull>b

>

in f (e'V(rA e))2 h Irl>b,lIell=I

n,

It thus suffices to show t hat B abE (0, 00) and N 3 , V n 2: N , (5.5.24) (5.5 .25)

p( Ir l>b,lIell=I inf (e'V (r A e))2 h n 2: z) p( Ir l>b,llell=I inf (e'V(r A e))2 hn 2: z)

>

1-

E,

> 1-

E.

But V UE IRP , IIV(Au) - V (Au)11 P

~ 2Tn

I: {IYj~ -

Yj~ l; + IflJu -

flJ- u'vjl;}·

j =I Thus, from (k), (5.5.3) and (5.5.10), it follows t hat V B E (0, (0) , (5.5.26)

sup IIV (Au) - V (Au)1I = op( l). IlullSb

Now, let T j := ! {Yl and rewrite

+ m j}g ndGn,

1 ~ j ~ Pi T' := (TI

e'V(rAe) = e'T

+ r k n( e) .

, '"

,Tp ) ,

5.5. Asymptotic uniform quadraticity

185

Again, by the C-S inequality, Fubini, (5.5.3) and the assumptions (j) and (k) it follows that :3 N; and bl , possibly both depending on 10, such that

P(IITII :::; bt}

(5.5.27)

~ 1-

(E/2),

Choose b such that (5.5.28) where a is as in (k). Then, with an := inf{lkn(e)l ; IIeil = I},

(5.5.29)

p(

,

2

inf (e'V(1'Ae)) Irl=b,llell=l "[n. P(le'Y(1'Ae)1

>

~

z)

~ (z"Yn)l /2,

P(lle'TI - 11'llkn(e)11

V Ilell = 1,11'1

= b)

~ (z"Yn)l /2, Vllell

= 1,11'1 = b)

> p(IITI:::; -(z"Yn)l /2 + ban) >

p(IITII:::; _(z"Y)l/2 + ba)

> p(IITII:::; bl ) ~ 1- (E/2),

Vn

~

Nl

·

In the above, the first inequality follows from the fact that Ildl-lell :::; jd+cl , d, e real numbers; the second uses the fact that [e'T] :::; IITII for all IIell = 1; the third uses the relation (-00 , _(z"Y)l /2 + ba) c (-00 , - (Z"Yn)l /2+ban) ; while the last inequality follows from (5.5.27) and (5.5.28) . Observe that e'Y (1' Ae) is monotonic in r for every II e II = l. Therefore, (5.5.29) implies (5.5.25) and hence (5.5.22) in a straight forward fashion . Next , consider e'V(1'Ae) . Rewrite e'V(1'Ae) =

J

i)e'dd[I(Yni :::; Y + 1'X~iAe)) - Hni(y)]gn(y)dGn(y) i=l

5. Minimum Distance Estimators

186

which, in view of the assumption (l), shows that e'V(r Ae) is monotonic in r for every lIell = 1. Therefore, by (5.5.26) :3 N 2 , depending on E, 3

P (

(e'V(rAe))2 >

inf Irl>b,lIell=1

/'n

z) P ( inf (e'V(rAe))2 2: z) - (E/2), Irl=b,lIell=1

> P(

(e'V(rAe))2 >

inf

Ir/=b,llell=1

>

z)

/'n

-

"[n.

>

1-

2: N 2

\j n

E,

V N1,

by (5.5.29). This proves (5.5.24) and hence (5.5.21) . 0 The next lemma gives an analogue of the previous lemma for KiJ. Since the proof is quite similar no details will be given . Lemma 5.5.5. In addition to the assumptions of Theorem 5 .5.2 assume that (k+) and (l) hold, where (k+) is the condition (k) with

T'n replaced by

r;t

:=

(vi , ' " ,vt ) and where {vt} are defined just

above (5.5 .13) . Then , \j E > 0, 0 < Z < 00, :3 N (depending only on (depending on E, z) 3 \j n 2: N ,

Z)

> 1 - E,

P( inf kiJ(Au) 2: z)

> 1 - E.

P ( inf KiJ(Au) 2: lIull>b

lIull>b

E)

and a b

o

The above two lemmas verify (5.4.A5) for the two dispersions K and K+. Also note that (5.5.22) together with (e) and (j) imply that IIA -1(A - ,8)11 = Op(I), where A is defined at (5.5.31) below . Similarly, Lemma 5.5.5, (e) , (5.5.17) and the symmetry assumption (5.3.8) about {G n } imply that IIA -1(A + - ,8)11 = Op(l) , where A + is defined at (5.5.35) below . The proofs of these facts are exactly similar to that of (5.4.2) given in the proof of Theorem 5.4.1. In view of Remark 5.5.1 and Theorem 5.4.1, we have now proved the following theorems. Theorem 5.5.3 Assume that (1.1.1) holds with th e modeled and actual d.f. 's of the errors {eni' 1

:s: i :s: n}

equal to {Hni , 1

:s: i :s: n}

187

5.5. Asymptotic uniform quadraticity

and {Fni, 1 ::; i ::; n}, respectively. In addition, suppose that (a)(l) hold. Then

(5.5.30) where

A satisfies

the equation

(5.5.31) If in addition,

(5.5.32)

J3 n -1 exists for all n ~ p,

then ,

(5.5.33)

:rnand B n

are defined at (5.5.19) . 0 Theorem 5.5.4 Assume that (1.1.1) holds with the actual d.j. 's of the errors {e ni ,l ::; i ::; n} equal to {Fn i , l i S n} . In addition, suppose that {X, Fni , D, Gn } satisfy (a) - (d), (1) - (i), (5.3.8) for all n ~ 1, (k) , (I) and (5.5.14). Then , where

<

(5.5.34) where ~ + satisfies the equation

(5.5.35) If, in addition,

(5.5.36)

(J3~)-1 exists for all n ~ p,

then,

(5.5.37) where

a: and B~ are defined at (5.5.19) .

0

Remark 5.5.2. If {Fi } are symmetric about zero then mt == 0 and f3t is consistent for f3 even if the errors are not identically distributed . On the other hand, if the errors are identically distributed,

5. Minimum Distance Estimators

188

f3ri

but not symmet rically, then will be asymptotically biased . This is not sur prising becau se here t he symmetry, rather t han t he identically distributed nature of the errors is relevant. If {Fi } are symmetric ab out an unknown common point then that point can be also est imated by the above m.d . method by simply augment ing t he design matrix to include t he colum n 1, if not 0 pr esent alrea dy. Next we turn to t he K and f30 (5.2.17) an d (5.2.18). F irst we state a theorem giving an ana logue of (5.5.9) for K Let Yj, J-Lj be Yd,J-Ld of (2.3.1) with {dnd replaced by {d nij} , j = 1, .. · ,p, X ni replaced by Y ni and Cni = Al (Xni - x n ), 1 ::; i ::; n , where Al and x n are defined at (4.3. 10). Set

o

o'

n

R j (s ) := I ) dnij - dnj(S))( Xni - Xn )qni (S), i :::1

where, for 1 ::; j < p , dnj(s) := n - I 2:~1 dnij eni(S), 0 ::; S ::; 1, with {lnd as in (3.2.34) and qni == f ni( H- I ) , 1 ::; i ::; n . Let

In t he assumptions of t heorem be low, L in KlJ is supposed to have been replaced by L n. Theorem 5.5.5. Let Yni , ' " , Ynn be independent r.v. 's with respective d.f. 's Fnl , '" , Fnn . Assume {D , X , Fnd satisf y (a)-(c) , (2.3.5) , (3.2.12), (3.2.34) and (3.2.35) with ui; = dij , 1 ::; j ::; p, 1 ::; i ::; n . Let {L n } be a sequence of d.f. 's on [0, 1] and assume that (5.5.38)

LP 10t J-L]( s , O)dL n( s) = 0( 1). j :::1

Th en, for every 0 < b <

0 00 ,

sup IK o( A u) - K D(A u)1 = op( I) . lI u ll :Sb Proof. The pro of uses the AUL result of T heorems 3.2.1 and 3.2.4. 0 Det ails are left out as an exercise.

5.5. Asymptotic uniform quadraticity

189

This res ult shows t hat t he dispersion K (5.5.39)

80 W

o satisfies (5.4.Al) wit h

an= A l l , s, = A l l :T~ ,

0,

A~B~A l ,

n

-1r~ (S){YD (S) + 1

:T~

.-

B~

.-

1r~(s)r~

IlD(s)}dLn( s) ,

1

(s)d Ln(s) ,

wher e r~ ( s ) = A~XcA ( s )D ( s ) , D (s ) := ((dnij - dnj( s)) , 1 ::; i ::; n , 1 ::; j ::; p; A(s) as in (2.3.30), 0 ::; s ::; 1; x, as in (4.2.11); Y D := (Yl , ' " , Yp),Il~ = (Il l ,' " , J-Lp) with 1j(s) == Yj(s , O) ,llj(s) == Ilj( s , O). Call t he cond ition (k) by t he na me of (k*) if it holds when (F n, Gn) is replaced by (f~ , L n) . Analogous to Theorem 5.5.4 we have Theorem 5.5 .6 As sum e that (1.1.1) holds with the actual d.f. 's of the errors {eni' 1 ::; i ::; n } equal to {Fni , 1 ::; i ::; n }. In addition, assume that {D , X , Fnd satisf y (N X *), (b), (c), (2.3.5), (3.2.12), (3.2.34)' (3.2.35) with ui, = dij , 1 ::; j ::; p , 1 ::; i ::; n , (k*) and (I). Let {L n} be a sequence of d.f. 's on [0, 1] satisf ying (5.5.38). Then

where

a * satisfi es the equation

(5.5.40)

If, in addition, (5.5.41)

( B~)- l exists for n ~ p,

then, The proof of this theorem is simi lar to t hat of Theorem 5.5.3. The details are left out for int erested readers. 0 Remark 5.5.3. Discussion of the assumptions (a) - (j). Among all of these assumptions, (g) and (i) are relatively harder to verify.

190

5. Minimum Distance Estimators

First, we shall give some sufficient conditions that will imply (g), (i) and the other assumptions. Then, we shall discuss these assumptions in detail for three cases, v.i.z ., the case when the errors are correctly modeled to be i.i.d. F , F a known d.f., the case when we model the errors to be i.i.d . F but they actually have heteroscedastic gross errors distributions , and finally , the case when the errors are modeled to be i.i.d . F but they actually are heteroscedastic due to difference in scales . To begin with consider the following assumptions. For any sequence of numbers {ani , bnd , ani

(5.5.42)

< bni, with

max (bni - ani) ---+ 0,

l~i ~n

limnsu p

!~!l~xn bni ~ ani l:~i J{fni(Y + z) -

fni(y)}2

xdGn(y)dz = O. (5.5.43) Claim 5.5.1. Assumptions (a) - (d), (5.5.42), (5.5.43) imply (g) and (i) . Proof. Use the C-S inequality twice , the fact that (d~)2 ::; dlj for all i , j and (b) to obtain P

L

J[Ld~{Fi(Y + c~v + o~d - + < t Ii Jt O~i fbi + < mfx(2o~d-1 l~i J + n

j=l

Fi(y

Ci V -

O~i)}] dGn(y) 2

i =l

dil12

2

z= l

z=l

4p202

f;(y

z)dzdGn(y)

fl{y

z)dGn(y)dz ,

at

where a; = -~iO +
+ c~v , 1 < < (k

which shows that (g) holds.

(by Fubini) ,

n . Therefore, by

= lim sup max IIiI;), n

z

5.5. Asymptotic uniform quadraticity

191

Next, by (b) and two applications of the C-S inequ ality, t he

L.R.S. (i)

~ f [t, d;j{ F;(y + c;u) - F; (y) <

pit

{ Fi(y +

i= l

=

[t u

p{f E;

(J;(y

f

+ Ei [<

2p

c~u) -

f

(J;(y

c~UJi(Y)} 2 dGn(y)

Fi(Y) -

+ z) -

C;Uj;(y))] 2 sa; (yl

f ;(y))dZ] 2 dCn(y)

+ z) -

j;(Y))dZ] 2 dCn(y)}

I [Etc~u I
~

< [m?-x - \I

t:

ciu . - 1C; ul

1

Ji(y)}2dz]dGn(Y)

•

l {h (Y + z ) - h(y)} 2dGn(Y)dz] n

x 4p 2)c~u) 2 , i= l

where Et (Ei) is th e sum over those i for which c~ u 2: O( c~ u < 0). Since E~=l ( c~ u)2 ::; pB for all U E N(b ), (i ) now follows from (5.5.42) and (a). 0 Now we consider th e three sp ecial cases mentioned above. Case 5.5.1. Correctly modeled i.i.d. errors: Fni _ F H ni , Gn == G. Suppose t hat F has a density f w.r.t. A. Assume t hat 2dG (5.5.44) (a) 0 < fdG < 00 , (b) 0 < f < 00 . (5.5.45)

I

(5.5.46)

(a)

I

I

F(l - F)dG <

(b)

lim

I I

z-tO

lim

z-t O

f(y

00 .

+ z )dG(y ) =

f2(y

+ z )dG)y ) =

I I

fdG , f 2dG.

192

5. Minimum Distance Estimators

Claim 5.5.2 . Assumptions (a) , (b) , (d) with G n == G, (5.5.44) (5.5.46) imply (a) - (j) with G n == G. This is easy to see. In fact here (e) and (J) are equivalent to (5.5.44a) , (5.5.45) and (5.5.46a) ; (5.5.42) and (5.5.43) are equivalent to (5.5.44b) and (5.5.46b) . T he LHS (j) is equa l to O. Note that if G is abso lute ly cont inuous then (5.5.44) implies (g). If G is p ure ly di screte and f cont inuous at the p oints of jumps of G then (5.5.46) holds. In part icul ar if G = bo, i.e. if G is degen er ate at 0, 00 > f (O ) > 0 and f is cont inuous at 0 then (5.5.44), (5.5.46) are t rivially sa t isfied . If G (y ) == y, (5.5.44a) and (5.5.46a) are a priori sat isfied whil e (5.5.45) is equivalent to assuming that El e1 - e21 < 00 , e1, e2 i.i.d . F . If dG = {F (1 - F) }-1 dF , the so called Darling - Anderson measure , th en (5.5.44) - (5.5.46) are satisfied by a class of d.f. 's that inclu des normal , logistic and double expo nent ial dis t ributions . Case 5.5.2. Heteroscedastic gross errors: H ni == F, Fni == (1 bndF + bniFO· We sha ll also ass ume t hat G n == G . Let f a nd f o be cont inuous densities of F and Fo . Then { Fnd have densities f ni = f + bni(JO - 1) , 1 :::; i :::; n . Hence (c) is satisfied. Cons ide r t he ass umption (5.5.47)

o :::; bni :::;

(5.5.48)

J!Fo-

max bni -+ 0,

1,

z

Fl dG <

00 .

Claim 5.5.3. Suppose that fo , f , F satisfy (5.5.44) (5.5.46), and (5.5.45), and suppose that (a ), (b) and (d) hold. Then (5.5.47) and (5.5.48) imply (e) - (i). Proof. The relation

Vj -

Ii == f + bi(JO- 1) implies that

n

n

i=1

i=1

L dijcd = L dij cibi(Jo -

1) ,

and n

"[n. -

L i=1

n

2

II d i l1 f

=

L i=1

IIdi l12 bi(Jo -

1) .

5.5. Asymptotic uniform quadraticity

jbn(Y+x) -

193

tlldi I12 f (y + x )]dG(y) i=l

< pmrxoi IjUo(y + x ) - f(y + X)]dG(y)1' Therefore, by (5.5.47) , (5.5.44a) and (5.5.46a) , it follows that (J) is satisfied. Similarly, the inequality

~J

n

Vj -

L dijcd

2

dG

i=l

ensures the satisfaction of (h) . The inequality

j

t

Ildi ll 2 { Fi (1 -

Fd - F(l - F)}dG

i=l

~ 2pmrxoi j!Fo -

FldG ,

(5.5.45) , (5.5.47) and (5.5.48) imply (e). Next ,

j {Ji(y

+ x ) - Ji(y)}2dG(y)

< 2(1 + 2of) j U(y + x) - f(y)}2dG(y) +4of j Uo(y

+ x) -

fO(y)}2dG(y).

Note that (5.5.44b) , (5.5.46b) and the continuity of

f imply that

lim jU(y + x ) - f(y)}2dG(y) = 0

x--+o

and a similar result for fo. Therefore from the above inequality, (5.5.46) and (5.5.47) we see that (5.5.42) and (5.5.43) are satisfied. By Claim 5.5.1, it follows that (g) and (i) are satisfied. 0

194

5. Minimum Distance Estimators

Suppose that G is a finit e measure. Then (Fl) implies (5.5.44) - (5.5.46) and (5.5.48) . In particular these assumpt ions are sat isfied by all those 1's that have finit e Fisher inform ation. The assumpt ion (j), in view of (5.5.48) , amounts to requiring that

(5.5.49)

But

n

p

n

n

n

L(L dij od = L L d~OidkOk :S (L Ild 2

j=1 i=1

i=1 k=1

d2 .

illo

i=1

This and (b) suggest a choice of Oi == p-1/21Id ill will satis fy (5.5.49). Note that if D = XA then IIdl1 2 == X~(X/X)-1xi '

/3x

When studyin g the robustness of in the following section, 0; == p- 1x~(X/X) -1xi is a natural choice to use. It is an an alogue of n -1/2- contaminat ion in the i.i.d. set up. 0 Case 5.5.3. Heteroscedastic scale errors: H ni == F, Fni( y) == F( TniY) , G n == G. Let F have continuous density f . Consider t he condit ions

(5.5.50) (5.5.51)

Tni == IJni + 1; IJni > 0, 1 :S i :S n ; m~XIJni ---+ 0. lim

s~ 1

JIylj

fk( sy)dG( y) =

JIylj

t

f k(y)dG( y) ,

j = 1, k = 1, j = 0, k = 1,2.

Claim 5.5.4. Under (a ), (b), (d) with G n == G , (5.5.44) (5.5.46) , (5.5.50) and (5.5.51) , the assumpt ions (e) - (i) are satisfied.

Proof. By the Lemmas 9.1.5 and 9.1.6 of t he App endix below, and by (5.5.23), (5.5.27) , and (5.5.31), (5.5.52)

lim lim sup max

x~o

lim

x~o

n

J

If(y

t

J

+ x) -

If(T1( y + x )) - f( y + x W dG(y) = 0,

f( yW dG( y) = 0, r = 1,2.

5.5. Asymptotic uniform quadraticity

195

Now,

! t IIdIl

2{F (1 - Fd - F (l - F )}dG i

i= l

-r] W(TIY) - F (y)ldG(y) 2p mrx i ! lylf(sy)dG(y)ds = 0(1),

< 2p

Ti

<

by (5.5.30) and (5.5.31) wit h j = 1, r = 1. Hence (5.5.45) implies (e). Next ,

!

'Yn(Y + x )dG(y) -

t

<

2 Ii dill Ti

i= l

!

t ! Iidi ll 2

fdG

~= l

{ lf h (Y + x )) - f(y + If(y

+ x )1

+ x ) - f(y)1 }dG(Y) +

mrx ow ! fdG .

Therefore, in view of (5.5.30) , (5.5.52) and (5.5.44) we obtain (1). Next , consider

!

U i(Y + x ) - Ji(y)} 2dG(y)

< 4Tl ! { !f(Ti(Y + x )) - f(y + x)J2 + [j (y + x ) - f(yW

+ [j (TiY) - f(y)J2 } dG(Y)

T herefore, (5.5.50) and (5.5.32) impl y (5.5.21), and hence (g) and (i) by Claim 5.5.1. Note t hat (5.5.32) and (5.5.23b ) imply (5.5.22). F inally, n

p

L! I v j - L j=l

mrx ! {Tilh y) - f(y)}2dG(y) 2p2 mrx Tl B i 9 [! Uhy ) - f(y)}2dG(y) + ! f 2dG] = 0(1),

< p2 <

dij cilll2dG

i= l

5. Minimum Distance Estimators

196

by (5.5.50) , (5.5.46b) , (5.5.52) . Hence (5.5.46b) and the fact that

o

implies (h). Here , the assumption (j) is equivalent to having n

p

j[Ldij{F(TiY) - F(y)}fdG(y) = 0(1) .

L

i= l

j=l

One sufficient condit ion for this, besides requiring F to have density

f satisfying lim j(Yf( sy))2dG(y) = j(yf(y))2dG(Y) <

(5.5.53)

00 ,

s-+l

is to have n

Lor = 0(1) .

(5.5.54)

i= l

{od

or

One choice of satisfying (5.5.54) is == n- 1/ 2 and the other choice is = x~(X'X)-lXi , 1 ~ i ~ n . Again, if f satisfies (Fl) , (F2) and G is a finite measure then (5.5.44) , (5.5.46) , (5.5.51) and (5.5.53) are a prior i satisfied. Now we shall give a set of sufficient conditions that will yield (5.4.Al) for the Q of (5.2.9). Since Q does not satisfy (5.3.21), the distribution of Q under (1.1.1) is not independent of f3 . Therefore care has to be taken to exhibit this dependence clearly when formulating a theorem pertaining to Q. This of course complicates the presentation somewhat. As before with {H ni }, {Fnil denoting the modeled and the actual d.f.'s of {end , define for 0 ~ S ~ 1, y E JR, t E W ,

or

m n(s , y)

.-

n- 1/ 2

ns

L {Fni(y -

x~if3) - Hni(y - x~if3)} ,

i=l ns

Hn( s , y , t)

.- n- 1 L Hni(y - x~it) , i= l ns

M 1n( s, y)

.-

n- 1/ 2 I)I(Yni ~ y) - Fni(Y - x~if3)} , i= l

do:n( s , y) .- dLn(s)dGn(y)

197

5.5. Asymptotic uniform quadraticity Observe that

Q(t) =

/[M1n (S, y) - n

+ mn(s , y)

1/2 -

-

2

{H n (s, y , t) - H n (S , y, ,8)}] dan (S , y) ,

where the integration is over the set [0, 1] x lit Assume that {Hnd have densities {hni} w.r .t. A and set, for S E [0,1], Y E JR, ns

(5.5.55)

Rn(s , y)

.-

n- 1/ 2

L Xnihni(Y -

x~i,8) ,

i=1 n

h~(y)

.-

n- 1

L h~i(Y -

x~d3) ,

i=1

v; .-

ARn,

B in := / vnvn,da n·

Finally define, for t E W ,

Theorem 5.5.7. Assume that (1.1.1) holds with th e actual and th e modeled d.f. 's of the errors {eni ' 1 S; i S; n} equal to {Fni ' 1 S; i S; n} and {Hni , 1 S; i S; n}, respectively. In addition, assume that (a) holds, {Hni , 1 S; i S; n} have den sities {hni' 1 S; i S; n} w.r.i. A, and the following hold. Ih~ln

(5.5.56)

vv

E N(b) , V 6

= 0(1).

> 0,

where ani = -6/'l,ni - c~iv , bni AXni' 1 S; i S; n .

=

6/'l,ni - C~iv , /'l,ni

= IIcn il/,

Cni =

5. Minimum Distance Estimators

198

vu

E

N(b),

f{

n 1/2 [H n(s,y ,{3 + Au) - H n(s ,y,{3)]

+u'vn } 2 dan(s, y)

f t x~i{3)(1 f m~(s,y)dan(s,y) n- 1

Fni(y -

- Fni(y -

= 0(1).

x~i{3))dGn(Y) =

0(1).

i=l

= 0(1) .

Then \i 0

< b < 00 , E sup IQ({3 + Au) - Q(Au)1 = 0(1). lIull:Sb

The details of the proof are similar to those of Theorem 5.5.1 and are left out as an exercise for interested readers. An analogue of (5.5.34) of 73 will appear in the next section as Theorem 5.6a.3. Its asymptotic distribution in the case when the errors are correctly modeled to be i.i.d. will be also discussed here . We shall end this section by stating analogues of some of the above results that will be useful when an unknown scale is also being estimated. To begin with, consider Kn of (5.2.19). To simplify writing, let K~(s , u) := Kn((1

+ sn- 1/ 2 ) , Au) ,

s E IR, u E JRP.

Write as := (1 + sn- 1/ 2 ) . From (5.2.19),

where Hi is the d .f. of e. , 1 ::; i ::; n, and where J-lJ , Yjo are as in (i) and (5.5.1)) , respectively. Writing J-lJ(y) , YjO(y) etc. for J-lJ(y , 0), Yi(y, 0)

199

5.5. Asymptotic uniform quadraticity etc., rewrite K~(s, u) p

L j=l

=

j {Y/(ya s, u) - YjO(y) + J-L~(yas) - J-L~(Y) - syv;(y) + YjO(y) + U'Vj(y) + syv;(y) + mj(y) + J-L~(yas , u) - J-L~(yas) - J-L'Vj(ya s) + u'[Vj(ya s) - Vj(y)]} 2 dGn(y)

where Vj are as in (h) and v;(y) := n- 1 / 2 E~=l dnijfni(y) , 1 :::; j :::; p. This representation suggests the following approximating candidate:

k~(s , u)

p

:=

j {Y/ + U'Vj + syv; + mj}2dGn.

L )=1

We now state Lemma 5.5.5 . With In as in (J) , assume that 'lisE JR, (5.5.57)

+ sn- 1/ 2)y + x)dGn(y)

lim limsupj,n((l

x---+o

n

= limnsup

j

In (y )dGn (y)

< 00 ,

and

(5.5.58)

limlimsupj lyl,n(Y

s---+o

n

= limnsu p

+ zy)dGn(y)

j IYiJn(y)dGn(y) <

00

Moreover, assume that V(s, v) E [-b, b] x N(b) = : C1 , 0 and '118 > 0 p

n

j [Ld;ij{Fni(ya s + C~iv + j=l i=l

lim sup L

n (5.5.59)

-Fni(ya s + CniV

-

8(n -

1

/

00,

2 IYI + "'nd)

8(n - 1/ 2IYI + "'ni))}

for some k not depending on (s , v) and 8.


r

dGn(y) :::; k8 2,

5. Minimum Distance Estimators

200 Th en , V 0

< b < 00 ,

L J{Y/ (l + sn p

(5.5.60)

Esup

I 2 / )y, u )

j =I

_Y/(y)} 2 dGn(y) = 0(1), where th e supremum is tak en over (s , u ) E CI . P roof. For each (s, u ) E CI , with as = 1 + sn-

I 2 / ,

p

E.L J {Y/(ya s, u ) -

Y/(y)} 2dGn(y)

j=I

<

i:nnJ

'"Yn(ya s + z )dGn(y )dz

J

+ ibbnn IYhn (y + zy)dGn(y)dz where E n = bmaxi IIKill, bn = bn (5.5.58), for every (s, u ) E CI ,

I 2 / .

Therefore, by (5.5.57) and

p

E .L J {yjo(ya s , u ) - YjO(y)}2dG n(y) = 0(1). j=I

Now pro ceed as in the proof of (5.5 .3), usin g the monotonicity of Vjd(a, t ), f.L~ (a , u ) a nd the compactness of CI to conclude (5.5 .60). Use (5.5 .59) in place of (g). The details are left out as an exercise. D

The proof of the followin g lemma is quite simi lar to that of (5.5.10) . Lemma 5.5.6. Let G~(y) = Gn(y /a T ) . Assume that for each fixed (r , u ) E CI , (h) and (i) hold with G n replaced by G~ . In add it ion, assume th e following : p

L J(yvj (y))2dGn(Y) = 0(1) . j=I

t J{f.L~(yas) j=I

-

f.L~(y) -

ryvj(y)}2dG n(y) = 0(1), V

lsi :s; b.

5.6. Asymptotic distributions, efficiency & robustness

201

< b < 00,

Th en , V 0

t!

sup

{J-LJ(ya s , u ) - J-LJ(aTY)

(S,U)ECI j=I

-U'Vj (aTY)} 2 dGn(y ) = 0(1), p

sup L

!{J-LJ(ya s )

-

J-LJ(y ) - Tyvj (y)}2 dGn (y) = 0(1).

Isl:Sb j = I

Theorem 5.5.8 Let Y n I , .. . , Y n n be in depen dent r.v. 's with respective d.f. 's Fn I , ' .. , Fn n . A ssum e (a) - (e) , (h) , (j) , (5.5.57) - (5.5.59) an d th e con ditions of Lem ma 5.5.6 hold. M oreover assume that for each lsi ::; b p

L ! Ilvj( ya s )

-

v j( y) 11 2dGn(y ) = 0(1).

j=I

Th en, V 0

< b < 00 ,

where the supremum is tak en over (s , u ) E CI . The proo f of t his th eorem is qui te similar to that of Theorem 0 5.5.1.

Asymptotic Distributions, Efficiency &

5.6

Robustness 5.6.1

Asymptotic distributions and efficiency

To begin with consider th e Case 5.5.1 and th e class of estim ators {,8 D}' Recall that in this case the err ors {end of (1.1.1) are correct ly mod eled to be i.i.d. F , i.e., H n i == F == F n i . We sha ll also take Gn == G, G E JD)I(lR). Assume that (5.5.44) - (5.5.46) hold. The various quantities appearing in (5.5.19) and Theorem 5.5.3 now take t he following simpler forms.

r n(y ) = AX'Df (y ), y E lR,

(5.6.1)

e;

=

AX'DD'XA! f dG ,

a; =

-AX'D !

Y~fdG .

202

5. Minimum Distance Estimators

Not e that 8 ;;1 will exis t if and only if the rank of D is p. Not e also that

(5.6.2)

1 8n ..,/T n

- (D'XA)- 1 J

Y~fdG/ (J f 2dG)- 1

(D 'X A J f 2dG )-1

t

d i [1j1(ei ) - E 1j1 (edJ,

i=1

where 1j1(y) = J~oo f dG , y E ~. Because Gn == G E JI))I(~ ) , th ere always exists agE L ; (G) such that 9 > 0, and 0 < J g2dG < 00 . The conditi on (5.5.k ) with gn == 9 becomes (5.6.3)

lim inf inf le'D'XAel 2:: n Ilell=1

0,

for some

0

> O.

Cond ition (5.5.l) implies th at e'D'XAe 2:: 0 or e'D'XAe < 0, VileII = 1 and V n 2:: 1. It need not imply (5.6.3) . The above d iscussion together wit h t he L-F Cra mer-Wold Theorem lead s to Corollary 5 .6a.1. Assume that (1.1.1) holds with the error r.v. 's correctly modeled to be i.i. d. F, F known . In addition, assume that (5.5.a), (5.5.b), (5.5.l) , (5.5.44) - (5.5.46), (5.6.3) and (5.6.4) hold, where (5.6.4)

(D 'X A) - 1 exists for all n 2:: p.

Then,

(5.6.5)

A-I (!3D - (3)

(D'XA J f 2dG) -1

t

i=1

If, in addition, we assum e (5.6.6)

then

(5.6.7)

d ni[1j1(end - E 1j1(endJ+ op (l ).

5.6.1. Asymptotic distributions, efficiency & robustness

203

whe re

~D

'=

.

(D'XA)-1D'D(AX'D )-1

2 ,T

= Va7"lp( ed

(J j2dGF'

0

For any two square matrices L 1 and L 2 of the same order, by L 1 ~ L 2 we mean t hat L 1 - L 2 is non-negative definite. Let L and J be two p x n matrices such tha t (LL')-1 exists . The C-S inequ ality for matrices states that

(5.6.8) with equality if and only if J ex: L. Now note that if D = XA then ~D = I p x p . In general, upon choo sing J = D' , L = AX' in this inequality, we obtain

D'D

~

D'XA . AX'D or

~D ~ I p x p

with equality if and only if D ex: XA . Fr om t hese obse rvat ions we deduce Theorem 5.6a.l ( Opt imality of fJx ). Suppose that (1.1.1) holds with th e error r. v. 's correct ly modeled to be i.i. d. F . In addi tion, assume that (5.5 .a) , (5.5.d) with G n == G , (5.5.44) - (5.5.46) hold. T hen , among the class of estim ators {fJ D; D satisfying (5.5.b), (5 .5.l) , (5.6.3) , (5.6.4) an d (5.6.6) }, th e estim ator th at minimizes the asymptotic variance of b ' A - 1(fJn - (3), f or every b E IRP , is fJx- the fJn with D = XA . 0 Ob serve that under (5.5.a) , D = XA a priori sati sfies (5.5.b), (5.6.3) , (5.6.4) and (5.6.6). Conse que ntl y we obtain Corollary 5.6a.2. (A sym pt ot ic n ormalit y of fJx) . A ssume that (1.1.1) holds with th e er ror r.v. 's correctly modeled to be i.i. d. F . In add it ion, assu me that (5 .5.a) an d (5.5.44) - (5.5.46) hold. Th en , A

- 1

((3x - (3) A

-'td

2

N( O, T Ip x p ) .

o

Remark 5.6a.1. Wri te fJn (G) for fJn to emphasize the dependen ce on G. The above t heorem proves the op t imality of fJx (G) among a class of estimators {fJn (G) , as D varies}. To obtain an asy mpt ot ically efficient est imator at a given F amo ng t he class of estimato rs

5. Minimum Distance Estimators

204

{.8x (G) , G varies} one must have F and G satisfy the following relation. Assume that F satisfies (3.2.a) of Theorem 3.2.3 and all of the derivatives that occur below make sense and that (5.5.44) hold. Then, a G that will give asymptotically efficient .8x (G) must satisfy the relation

-fdG

(I/I(J) . d(j / J),

I(J) :=

J

(j / J) 2 dF.

From this it follows that the m.d . estimators .8x(G), for G satisfying the relations dG(y) = (2/3)dy and dG(y) = 4doo(y), are asymptotically efficient at logistic and double exponential error d.f. 's , respectively. For .8x(G) to be asymptotically efficient at N(O,I) errors, G would have to satisfy f(y)dG(y) = dy . But such a G does not satisfy (5.5.58) . Consequently, under the current art of affairs , one can not estimate f3 asymptotically efficiently at the N(o , 1) error d.f, by using a .8x (G) . This naturally leaves one open problem, v.i.z., Is the conclusion of Corollary 5.6a.2 true without requirinq f f dG < 00,0 < f f 2 dG < oo? 0 Observe that Theorem 5.6a .l does not include the estimator .81 - the .8D when D = n 1 / 2 [1, 0"" ,O]nxp Le., the m.d. estimator defined at (5.2.4) , (5.2.5) after Hni is replaced by F in there. The main reason for this being that the given D does not satisfy (5.6.4). However, Theorem 5.5.3 is general enough to cover this case also . Upon specializing that theorem and applying (5.5.31) one obtains the following Theorem 5.6a.2. Assume that (1.1.1) holds with the errors correctly modeled to be i.i.d. F. In addition, assume that (5.5.a), (5.5.44) (5.5.46) and the following hold. (5.6.9)

Either

n-l/2elx~iAe 2: 0 for all 1 ~ i ~ n, all

lIell

= 1,

or

n-l /2elx~iAe ~ 0 for alII ~ i ~ n, all lle] = 1.

(5.6.10)

5.6.1. Asymptotic distributions, efficiency & robustness where

xn

(5.6.11)

is as in (4.2a.11) and

n

1/2-'

xnA . A

-1

fit

205

is the first coordinate of O. Then

A

_

({31 - {3) -

Zn J j2dG + "» ( 1) ,

where n

Zn

= n- 1 / 2 I)1/J(e nd - E1/J(end} ,

with

1/J as in (5.6.2) .

i=1

Consequently, nl/2x~(,Bl - {3) is asymptotically a N(O, 72) r.v. 0 Next, we focus on the class of estimators {{3i;} and the case of i.i. d. symmetric errors. An analogue of Corollary 5.6a.1 is obtained with the help of Theorem 5.5.4 instead of Theorem 5.5.3 and is given in Corollary 5.6a.3. The details of its proof are similar to those of Corollary 5.6a.1. Corollary 5.6a.3 . Assume that (1.1.1) holds with th e errors correctly modeled to be i.i.d. symmetric around O. In addition, assume that (5.3.8), (5.5.a), (5.5 .b), (5 .5.d) with G n == G, (5.5.44) , (5.5.46), (5.6.3), (5.6.4) and (5.6.12) hold, where

1

00

(5.6.12)

(1 - F)dG <

00

Then ,

(5.6.13) =

A -1({3i; - {3)

-{2AX'D

J

f 2dG} -1

J

W +(y)f+(y)dG(y)

+ op(l) ,

where f +(y) := f(y) + f(-y) and W+(y) is W+(y,O) of (5.5.13). If, in add it ion , (5.6.6) holds, then

(5.6.14)

o

Consequently, an analogue of Theorem 5.6a.1 holds for {3j{ also and Remark 5.6a.1 applies equally to the class of estimators {{3j( (G) , G varies} , assuming that the errors are symmetric around O. We leave it to interested readers to state and prove an analogue of Theorem 5.6a.2 for {3t .

5. Minimum Distance Estimators

206

o}

Now consider the class of estimators {,B of (5.2.18) . Recall the notation in (5.5.39) and Theorem 5.5.6. The distributions of these estimators will be discussed when the errors in (1.1.1) are correctly modeled to be LLd. F, F an arbitrary d.f. and when L n == L. In this case various entities of Theorem 5.5.6 acquire the following forms:

fni(s) == 1; D(s) == D , under (5.2.21) ; D q(s ), q = f(F- 1);

0;

A 1X c

= B*n

=

-AIX~D /

YDqdL

(AIX~DD'XcAd/

= AIX~D

t

dniepo(F(end ;

i=1

q2 dL ,

where X, and Al are defined at (4.2.11) and where (5.6.15)

epo(u)

l

:=

u

q(s)dL(s), O:S u :S 1.

Arguing as for Corollary 5.6a.1, one obtains the following Corollary 5.6a.4. Assume that (1.1.1) holds with the errors correctly modeled to be i.i. d. F and that L is a d.j. In addition, assume that (Fl) , (NX c ), (5.2.16), (5.5.b), and the following hold. liminf inf le'D'X cA 1 el 2: a > O.

(5.6.16)

lIell =1

n

(5.6.17)

Either

e'dni(xni -

x n )' A 1e 2: 0, VI :S i:S n, V [e] = 1,

or

(5.6.18)

(D'X c A 1 )

-1

exists for all n 2: p .

Then,

(5.6.19)

1

A1

CBo- 13)

1 1

(D'X cA 1

o

q2dL)-1

t

i =1

dniep(F(end)

+ op(l) .

5.6.1. Asymptotic distributions, efficiency & robustness

207

If, in addition, (5.6.6) holds, then

(5.6.20) where ~D = ( D/Xc Al) -l D'D(AlX~ D)-l , and 2 _

(5.6.21)

ao -

Varrpo(F(ed) (fol q2dL)2 '

with rpo as in (5.6.15) . Consequently, (5.6.22) and {,8x e } is asymptotically efficient among all {,8D' D satisfying o above conditions .

Consider the case when L(s) 2

J J[F(x /\ y) -

ao =

== s. Then

F(x)F(y)]j2(x)f2(y)dxdy (J P(x)dx)2

It is interesting to make a numerical comparison of this var iance with that of some other well celebrated estimators. Let alad' a~ and a;s denote the respective asymptotic variances of the Wilcoxon rank, the least absolute deviation, the least square and the normal scores estimators of ,8. Recall, from Chapter 4 that

a;,

a:; ~ als

=

{12(J f'(X)dX) ' } a2;

a;s

- 1

a1'd ~ (2f(OW' ;

= {[J f 2 (x )/ rp ( - l (F ))Jdx } -2 ;

where a 2 is the error variance. Using these we obtain the following table.

F Double Exp. Logistic Normal Cauchy

Table 1 a 02 a w2 1.2 1.333 3.0357 3 1.0946 tt /3 2.5739 3.2899

2 a/ad 1

4 7f/2 2.46

2 a ns 7f2 7f 1

a2 2 7f2/3 1 00

208

5. Minimum Distance Estimators

It thus follows that the m.d. estimator j3xJL), with L( s) == s, is superior to the Wilcoxon rank estimator and the LAD estimator at double exponential and logistic errors, respectively. At normal errors, it has smaller variance than the LAD estimator and compares favorably with the optimal estimator. The same is true for the m.d. estimator x (F) . Next , we shall discuss 13 . In the following theor em the framework is the same as in Theorem 5.5.7. Also see (5.5.82) for the definitions of ti.; BIn etc. Theorem 5.6a.3. In addition to the assumptions of Theorem 5.5.7

/3

assume that (5.6.23) lim inf inf n

11011 =1

J

I v~danOI ~

a , for some a > O.

Moreover, assume that (5.6.9)) holds and that 8 1;

(5.6.24)

exists for all n ~ p.

Then, (5.6.25)

A -1(13 - 13)

JJ

-8 1;

vn( s , y){Mln( S, y)

+ mn(s , y)}dan(s, y) +op(l) .

Proof. The proof of (5.6.25) is similar to that of (5.5.33), henc e no 0 det ails are given . Corollary 5.6a.5. Suppose the conditions of Theorem 5.6a.3 are satisfied by Fni == F == Hni' Gn == G, L n == L , where F is supposed to have continuous density f . Let

JJ11 1

C

=

o

ns

1

[{An-

0

1

nt

L L x ixjfi(y)fj(y)A}(s l\t) i= l j = 1

X

(F(Y

1\

z) - F(y)F(z))] dai« , y)da(t , z) ,

where Ii(y) = f(y - x~j3) , and da( s ,y) = dL(s)dG(y) . Then the asymptotic distribution of A - 1(13 _ 13 ) isN(o, '£0(13)) where '£0(13) = 1CB - I B0 In In '

209

5.6.2. Robustness

Because of the dependence of ~o on f3 , no clear cut comparison between f3 and iJx in terms of their asymptotic covariance matrices seems to be feasible. However, some comparison at a given f3 can be made. To demonstrate this, consider the case when £(s) = s, p = 1 and /31 = O. Write Xi for XiI etc . Note that here, with = I:~=1

T;

BIn

=

T;2 inr n-

1

n

ns ns L Xi L xjds . i=1

1

C

xl,

T;211 o

0

X

1

J2

j dG ,

j=1

ns nt n- 1 LXi L Xj(s i=1

1\

t)dsdt

j=1

J J[F(Y 1\ z) - F(y)F(z)]d'IjJ(y)d'IjJ(z).

Consequently

L;o(O) =

Recall that T2 is the asymptotic variance of Tx(Sx - {3). Direct integration shows that in the cases Xi == 1 and Xi == i , Tn -+ ~~ and ~~ , respectively. Thus, in the cases of the one sample location model and the first degree polynomial through the origin, in terms of the asymptotic variance, Sx dominates 7J with £(s) = s at /3 = o. 0

5.6.2

Robustness

In a linear regression setup an estimator needs to be robust against departures in the assumed design variables and the error distributions. As seen in the previous section, one purpose of having general weights D in iJn was to prove that iJx is asymptotically efficient among a certain class of m.d. estimators {iJn,D varies}. Another purpose is to robustify these estimators against the extremes in the design by choosing D to be a bounded function of X that satisfies all other conditions of Theorem 5.6a.1. Then the corresponding iJn would be asymptotically normal and robust against the extremes in

5. Minimum Distance Estimators

210

the design, but not as efficient as fJx . This gives another example of the phenomenon that compromises efficiency in return for robustness. A similar remark applies to {,sri} and {,sD}' We shall now focus on the qualitative robustness (see Definition 4.4.1) of fJx and ,si . For simplicity, we shall write 13 and ,s+ for fJx and ,si, respectively, in the rest of this section. To begin with consider 13. Recall Theorem 5.5 .3 and the notation of (5.5.19) . We need to apply these to the case when the errors in (1.1.1) are modeled to be i.i.d. F, but their actual d.f.'s are {Fnd, D = XA and G n = G. Then various quantities in (5.5.19) acquire the following form.

rn(y)

where for 1

,

on

AX'A*(y)XA,

13 n = AX'! A*IIA*dGXA,

!r

+ An(y)]dG(y)

~

n(y)AX'[on(Y)

=

z, + b.,,

say,

i ~ n, y E JR,

.- (a nl,an2 , '

"

, ann), A~:= (.0. nl,.0.n 2, · · · ,.0.nn );

II .- X(X'X)-lX';

bn

:=

! r n(y)AX' An(y)dG(y).

The assumption (5.5.a) ensures that the design matrix X is of the full rank p . This in turn implies the existence of 13;;-1 and the satisfaction of (5.5.b) , (5.5.1) in the present case. Moreover , because c; = G, (5.5.k) now becomes (5.6.26)

liminf inf kn(e);:: T ' for some T > 0, n Ilell=l

where

kn(e) :=

e'AX'

!

A*gdGXAe,

°

Ilell

= 1,

and where 9 is a function from JR to [0,00], < J gr dG < 00, r = 1,2. Because G is a (J - finite measure, such a 9 always exists. Upon specializing Theorem 5.5 .3 to the present case, we readily obtain Corollary 5.6b.1. Assume that in (1.1.1) the actual and modeled d.J. 's of the errors {eni , 1 ~ i ~ n} are {Fni , 1 ~ i ~ n} and F ,

5.6.2. Robustness

211

respectively. In addition, assume that (5.5.a), (5.5.c) - (5.5.j) with D = XA, Hni == F, Gn == G, and (5.6.26) hold. Then A -1(/3 - {3) = -B~l{Zn

+ b n} + op(l) .

0

Observe that B~lbn measures the amount of the asymptotic bias in the estimator /3 when F ni i= F . Our goal here is to obtain the asymptotic distribution of A - 1(/3 - {3) when {Fnd converge to F in a certain sense. The achievement of this goal is facilitated by the following lemma. Recall that for any square matrix L , IILlioo = sup{IIt'LIl :s: I}. Also recall the fact that (5.6 .27)

IILlloo

:s:

{tr .LL'}1/2,

where tr. denotes the trace operator. Lemma 5.6b.1. Let F and G satisfy (5.5.44). Assume that (5.5.e) and (5.5.j) are satisfied by G n == G,{Fnd ,Hni == F and D = XA. Moreover assume that (5.5.c) holds and that (5.6.28)

Then with I = I pxp, (i) IIB n - If f 2dGIIoo = 0(1). (ii) IIB~l - I(f f 2dG)-1I1oo = 0(1). (iii) Itr.B n - p f f 2dGI = 0(1). (iv) I Z=~=1 f II vjll2dG - p f j 2dGI = 0(1). (v) IIb n - fAX' An(y)j(y)dG(y)} = 0(1). (vi) IIZn - fAX'O:n(y)j(y)dG(y)1I = op(l) . (vii) sUPllell=llkn(e) - f jgdGI = 0(1). Remark 5.6b.1. Note that the condition (5.5.j) with D = XA, G n == G now becomes (5.6 .29)

Proof. To begin with, because AX'XA == I, we obtain the relation rn(y)r~(y) - j2(y)I

= AX'[A*(y) - j(y)I]XA ·A X ' [A *(y) - f(y)I]XA AX'C(y)XA· AX'C(y)XA 1)(y)1)'(y) ,

y E IR,

5. Minimum Distance Estim ators

212

where C(y ) := A *(y ) - If (y ), 'D(y ) := AX'C (y )XA , y E lit Therefore, II S n

(5.6.30)

I

-

J

f 2dGli oo

<

sup Il tl l~ l

= 'D'D'.

1It''D(y)'D'( y)lIdG(y)

J

< where L

J

{t r.LL'}1/2dG

Note t hat , by t he C-S inequa lity,

tr.L L ' = tr.'D'D''D''D ::; {t r.'D'D'}2 .

Let Oi =

Ii - t,

1 ::; i ::; n . Then n

(5.6.31)

Itr·'D'D'1 =

tr.

n

2:: 2:: AXiX~A . AXjxjA .s.s, i=l j=l

n

n

i= l

j=l

n

n

2:: 2:: OiOj (xj A A x d2 <

2:: 2:: IOiOjl ' lIx~ A I1 2 . II x jA II 2 i=l j=l

=

(t II

AXiIl 21&il) 2 = p"

Consequent ly, from (5.6.30) and (5.6.31),

IIBn

-

I

f

f'dG II 00

<

f (t II

AXi Il)

' If; -

f l'dG

~ 0(1),

by (5.6.28). This proves (i) while (ii) follows from (i) by using the determinant and co-facto r form ula for th e inverses. Next , (iii) follows from (5.6.28) and the fact that Itr.S n

-

P

J

f 2dGI =

I

J

tr .'D'D'dG I ::; Pn ,

by (5.6.31) .

To prove (iv) , note t hat with D = XA ,

2:: JII v j ll 2dG p

j=l

=

t t Jx~AAxkX~AAxdi (y)fk(y)dG(y) . i=l k= l

5.6.2. Robustness

213

Note that t he R.H.S. is p J f 2 dG in t he case

~

I II

v j ll' dG - p

I

I'dG =

I

Ii == f . Thus

tr.v v 'dGI S Pn·

This and (5.6.28) prove (iv) . Sim ilarly, with d j (y) denoting t he j-th row of 1) {y ), 1 ::; j ::; p,

Ilb

n -

!

AX'

~nfdGl12

= =

I ! 1)AX~dGII 2 t {! ~n (y)dG(y) d j {y)AX'

}2

j= l

< Pn ! IIAX' ~n (y) 11 2dG(y)

(5.6.32) and (5.6.33)

II Zn -

!

A X 'O:n(y)f (y )dG(y )112

::; Pn

!

II A X' O:nIl 2dG .

Moreover , (5.6.34)

E!

II AX'cx n l12dG =

! t Ilx~A11 2

Fi (1 - Fi)dG .

1= 1

Cons equently, (v) follows from (5.6.28) , (5.6.29) and (5.6.32) . The claim (vi) follows from (5.5.e), (5.6.28), (5.6.33) and (5.6.34). F inally, with 1)1/ 2 = AX' C 1 / 2 , V eE W ,

Therefore,

1I ~\l~, Ik,,(e) - I I9dGI

<

I {t, II

< Pn

A x; II' lh

{! g2dG}

1/ 2

- I I} gdG

= 0(1). by (6). 0

214

5. Minimum Distan ce Estimators

Corollar y 5.6b.2. Assume that (1.1.1) holds with the actual and the modeled d.f. 's of {eni, 1 ~ i ~ n } equal to {Fni , 1 ~ i ~ n} and F , respectively. In addition , assum e that (5.5.a) , (5.5.c) - (5.5.g), (5.5.i) , (5.5.j ) with D = XA , H ni = F, Gn = G; (5.5.44) and (5.6.28) hold. Then, (5.5.h) and (5.5.b) are satisfied and (5.6.35)

where Z n .- / AX'o n{y )d'ljJ {y ) =

A t x ni['ljJ (end - / 'ljJ {x )dFni{x) ], i == l

bn

.-

/ A X 'A n{y )d'ljJ (y ) = / t A x ni[Fni - F]d'ljJ , t== l

with 'ljJ as in (5.6.2). 0 Consider Zn. Note t hat wit h (T~i := Var {'ljJ {eni)lFnd, 1 ~ i ~ n ,

One can rewrite

By (5.5.44), 'ljJ is nondecreasin g and bounded. Hence max i IlFni Flloo -7 0 readily imp lies th at maXi(T~i -7 (T2 , (T2:= V ar{'ljJ{ e) IF}. Moreover , we have t he inequ ality

It t hus readily follows from t he L-F CLT th at (5.5.a) implies t hat 2

'

Zn -7d N{ O, (T I p x p ) , If max i IlFni - F ll oo -7 O. Cons equently, we have A

5.6.2. Robustness

215

Theorem 5.6b.1. (Qualitative Robustness). Assume the same setup and conditions as in Corollary 5.6b.2. In addition, suppose that (5.6.36)

max t

(5.6.37)

IlFni - Flloo IIAlloo

=

0(1),

=

0(1).

/3

Then, the distribution of under rr~l Fni converges weakly to the degenerate distribution, degenerate at (3. Proof. It suffices to show that the asymptotic bias is bounded. To that effect we have the inequality

From this, (5.6.35) , and the above discussion about {Zn}, we obtain that V 7J > 0, :J KTJ such that pn(ETJ) --t 1 where P" denotes the probability under rr~l Fni and ETJ = {IIA -1(/3 - (3)11 :::; KTJ} ' Theorem now follows from this and the elementary inequality 11/3 - (311 :::;

IIAlloollA -1(/3 - (3)II.

0

Remark 5.6b.2 . The conditions (5.6.28) and (5.6.36) together need not imply (5.5.g), (5.5.i) and (5.5.j) . The condition (5.5.j) is heavily dependent on the rate of convergence in (5.6.36) . Note that

IIbnl1 2

(5.6.38)

:::;

min {1j;(oo) /

II AX'a II 2 d1j;,

(/ f 2 dG) /II Ax 'a Il2dG} . This inequality shows that because of (5.5.44), it is possible to have IIbn l12 = 0(1) even if (5.6.29) (or (5.5.j) with D = XA) may not be satisfied. However , our general theory requires (5.6.29) any way. Now, with ip = 1j; or G, (5.6.39)

/

IIAX'a11 2 d
=

/

t t x~AAxjtljtljd