0,
f
IY(x)((o)I.f(x)dµ(x)
a
f fdµ f LeXp (Y(x)2\ - l]dlt(x) a2 /
+a
f f(x)
/
1/2
f
Ilog I 1 + .f(f.xd) µ)I
dµ(x).
This fact, Proposition H.2 of Appendix H, and the definition of z then imply, since z((o) > 0, (2.7.21)
f
IY(x)(w)I f(x)dg(x)
< z1 f f(x) dlt (x) +
f
[log .f (x)
f(x)
(l + I f da)]
1112
dIL(x)
For u > 0 let F(u) := u[log(1 + (u/,B))]1/2. Then F is increasing and
Gaussian Measures and Processes; Sample Continuity
70
convex on [0, oo) by derivatives. Jensen's inequality (RAP, 10.2.6) then gives (log 2)1/2 f
fdg =
F
(f f d i)/// < fF(f)dt fx)
1/2
f(f
f f(x)
['og 11+
dµ
\l
dµ(x).
I
This, with (2.7.21), gives part (b) except for bounding Ez. To do that, we have for any u > 0 and p > 1 by definition of z and then the Chebyshev-Markov inequality that l
(
P(z > u) < P
if
< 2-"E
exp (y(x)2/u2) dµ(x) > 2 } dµ(x)JP
([f exp (Y(x)2/u2)
)
.
By Holder's inequality, multiplying the latter integrand by 1 (or Jensen's inequality), we have
P(z > u) < 2-PE For 1 < p < u2/2, the
\
f exp (PY(x)2/u2) dµ(x))
.
quantity` on the right is finite and, by the Tonelli-
2pu-2)-1/2. For Fubini theorem, bounded above by g(p, u) := 2-P(1 u > uo := (2 + (log 2)-1)1/2, if we set p := u2/2 - 1/(2 log 2), then p > 1
and
g(p, u) = u(e log 2)1/2 exp (- u2(log 2)/2). For any t > 0 and random variable X with law P,
l
00
00
P(X > u) du
f fuo"
dP(x) du = 100
JX du dP(x)
t
00
(x - t) dP(x) = E(Xlx>t) - tP(X > t). It follows that if X > 0, then EX < t + fto° P(X > u) du. Thus
Ez < uo + (e log 2) 1/2
f
2_U2/2du
00 u
up
uo + (e log 2)1/2(log 2)-1 exp (- ua(log 2)/2) < 2.5 as stated, which finishes the proof of the lemma.
2.7 Majorizing measures
71
Now to continue with the proof of Theorem 2.7.18, recall that we have chosen
a jointly measurable version of L()(). It will be shown that on T can be well approximated by its averages over small balls with respect to A. Recall that B(x, r) :_ {y: d(x, y) < r}. For each k = 0, 1, and t E T let
A(B(t, D/2k)),
µk(t) (2.7.22)
Pk(t, )
0-pk(t))-11g(t
D12k) ('),
and Mk(t) := Mk(t, (o) f pk(t, u)L(u) d s(u), which exists and is finite a.s. by the Tonelli-Fubini theorem since for any x, to E T, E I L (x) I < 1I to II + D. For each t E T, we have B(t, D) = T, so M0(t) = f L d p doesn't depend on t. We also have E I L (x) - Mk (x) I < D/2k for each x and k. It follows that for each x in T, a.s. as k -+ oc, (2.7.23)
Mk(x) - L(x).
The rest of the proof will bound the increments of the processes Mk and their rate of convergence to L.
Let Y(u, v) := (L(u) - L(v))/d(u, v) if u v and 0 if u = v. Then Y is jointly measurable on H x H, EY(u, v) = 0, and EY(u, v)2 < 1 for all , letting pi (u) := p; (t, u), u, v E H. For each t E T and k = 0, 1, 2, k
Mk(t) - MO = T, MM(t) - Mj-1(t), j=1
and
IMj(t) - Mj-1(t)I = f Y(u, v)d(u, v)pj(u)pj-l(v)dtc(u)d1t(v)
< 3D.2-i. f I Y(u, v)1pj(u)pj-1(v) dl-L(u) dlt(v) Applying Lemma 2.7.20(b) to W := T x T with f(u, v) := pj(u)pj_1(v)
for j = 1, 2, .. and s x µ gives a random variable z > 0 with Ez < 5/2, not depending on n or t, and a set 21 C 0 with P(21) = 1 such that for all
c0E221,n EN,n> 1, and t E T, I Mn (t, (o) - Mn-1(t, (0)1 <
9D2 ((o)
[log
I+
)] 1/2
lcn (t)µn-1(t)
21/29Dz 2n (co ) clog (1
+
1
t-1n (t)
)]1/2
72
Gaussian Measures and Processes; Sample Continuity
since An (t) < An_1(t) for all n > 1 and t while log(l + x2) < 2log(1 + x) for all x > 0. Now, letting h(t, r) :_ [log(1 + µ(B(t, r))-1)]1/2
log (1 + µk(t)-1) < h(t, r)2
(2.7.24)
for r < D/2k. Applying this for r > D/2k+1 for each k > 1, we get 00
D/2
E I Mn - M,,-11(t, w) < 18z(c))21/2 n=1
h(t, r) dr.
J
For x > 1, we have (log(l +x))112 < (log(2x))1/2 < (log2)1/2 + (log X)112. So for co E 521, the series M0 + Fool Mn(t, (o) - Mn_1(t, (v) converges absolutely and converges to a limit
M(t, (v) := M(t)(c)) := limk,oo Mk(t, (0) Thus by (2.7.23), for each t, M(t, w) = L(t)(w) a.s., so M(., ) gives a version of Then we have D/2
(2.7.25)
IM(t, w) - Mo(cv)I < 18.21/2z(c))
h(t, r)dr.
J0
Next, for any s and tin T, take the unique integer k = k(s, t) > 0 such that (2.7.26)
D/2k+1 < d(s, t) < D/2k.
Then
IM(s) - M(t)I < IM(s) - Mk(s)I + IMk(S) - Mk(t)I + IM(t) - Mk(t)I. The proof of (2.7.25) yields (2.7.27)
I (M - Mk) (t, w) I
< 18.2 1/2Z(CO)
I
2-k-1 D
h(t, r) dr
and likewise for s in place of t. We have (2.7.28)
Mk(S) - Mk(t)
= f Y(u, v)d(u, v)pk(s, u)pk(t, v)dis(u)d,c(v). By the definition of pk (2.7.22) and (2.7.26), where the integrand in (2.7.28) is nonzero, we have d(u, v) < 3D/2k. Thus for all w, by Lemma 2.7.20(b) again,
I Mk(S) - Mk(t)I <
9Dz
[log I 1 +
1
111/2
µk(S)µk(t)l J
2.7 Majorizing measures
73
It follows as in (2.7.24) that
IMk(S) - Mk(t)I D/2k+1
[108(1
++l0) 1+
1/2
g
µ(B(t, r))
µ(B(s, r)) )
For r > 0 let µ(r) := µs,t(r) := min (It (B (t, r), B(s, r))). Then D/2k+1
1/2
1
[log 1+
I Mk(s) - Mk(t)I < 18.2 1/2Z I
)1dr.
For any nonincreasing function f > 0 and S > 0 we have S
S/2
f(r)dr.
(1. f
f(r)dr < 2 fo
Recalling (2.7.27) for s and t, we then have for w E 521,
1
f D/2k+2
GM(s) - M(t) I
< 154zJ
[log (1
1 /2
+ µ(r))J
dr.
Since d(s, t) > D/2k+1 by (2.7.26), for r < D/2k+2 the balls B(t, r) and B(s, r) are disjoint and so µ(r) < 1/2. For x > 2, we have log(1 + x) (log 3) (log 2) -1 log(x) by first derivatives. Thus
[log (1 +
< [log2 log
/t(r))J
log 3
log2)
log 3
1/2
1
i/2
A(r)
1/2
1
Llog
1/2
1
\lt(B(t, r)))]
1
)]1/2
+ [log \it(B(s, r))
Then (2.7.19) follows with Y(w) := 194z(w). Since Ez < 2.5 from Lemma 2.7.20, we get EY < 485 as stated. So, Theorem 2.7.18 is proved. Now to prove Theorem 2.7.1, by Theorem 2.7.18 there is a version M of L such that inequality (2.7.19) holds, and therefore letting t be fixed and s vary,
EL(C)* = EM(C)* and (2.7.29)
EM(C)* <
f
D
1
[log (/2(B(x, r)) )]
1/2
dr.
Since EY(w) < 485, EM(C)* < 970y (C) and the proof of Theorem 2.7.1 is complete. Theorem 2.7.18 also directly implies a sufficient condition for the GC property if the majorizing measure condition can be strengthened, as follows.
Gaussian Measures and Processes; Sample Continuity
74
2.7.30 Corollary Let H be a Hilbert space and C C H. Suppose that there exists a probability measure µ on C such that 1
b
limo1 suPxec Jo
to
g \µ(B(x, r))
1/2
dr = 0.
Then C is a GC-set in H.
2.8 Sample continuity and compactness This section will show that for a Gaussian process Xt indexed by a compact metric space, or other suitable parameter space such as an open or closed set in a Euclidean space, sample continuity reduces to that of the isonormal process on
some subsets, and continuity of the nonrandom function t H EXt (Corollary 2.8.5).
Let (T, T) and (W, U) be two topological spaces. Let {Xt, t E T) be a stochastic process defined over a probability space (0, B, P) with values in W, meaning that for each t c- T and Borel set B C W, Xt 1(B) E 13. (Recall that the or-algebra of Borel sets is generated by the open sets and that it's equivalent to assume Xt 1(U) E 13 for each U E U.) Let {Yt, t E T) be another process with values in W, possibly defined over a different probability space (7', B', P'). Recall that the processes {Xt } and {Yt } have the same laws iff for every n = 1, 2, and t1, t" E T, the law of on the product or-algebra in W" is the same as that of {Ytj} j-1 Recall that two processes with the same laws are said to be versions of each other. A process {Xt}tET will be called version-continuous if there is a process Y with the same laws such that for all co' E S2', t H Yt (w') is continuous from T into W. (Equivalently, continuity need only hold for almost all co' because then without changing the laws, for the set of measure 0 of values of co' for which Yt ((o') is not continuous, one can replace it by a fixed continuous function, say having a constant value in W.) Now let W = ll8 with usual topology and suppose {Xt, t E T) is a Gaussian process. Suppose also that (T, e) is a metric space with the metric topology on T. We have 2.8.1 Theorem A Gaussian process { Xt } indexed by a metric space T, defined on a probability space (0, P), is version-continuous if and only if both
(a) the nonrandom function t H EXt is continuous, and (b) the process {Xt - EXt} is version-continuous. Then, t i--
is continuous into L2(P).
2.8 Sample continuity and compactness
75
Proof "If" is clear. To prove "only if," suppose Xt is sample-continuous. For any sequence t, -+ t E T, sample continuity implies that Xtn -+ Xt almost surely and therefore in probability. For jointly Gaussian random variables, convergence in probability is equivalent to convergence in L2(P), since for the Gaussian variables Y := Xt - Xt to converge to 0 in probability, the means EY must converge to 0, and so must the variances. Thus EXtn -+ EXt. Since T is a metric space, (a) follows. Then by subtracting the continuous function EXt, (b) follows.
So, in studying sample continuity or version-continuity of Gaussian processes, we may as well restrict ourselves to processes with mean 0. Let Xt be such a process, t E T. Each Xt is an element of a Hilbert space H, namely L2(P). Consider the isonormal process L on this H. Then since L is Gaussian, has mean 0, and preserves covariances, we see that L(Xt) has the same laws as Xt. If h(.) is a continuous function from T into a Hilbert space H, with range C :_ {h(t) : t E T}, and if L restricted to C is version-continuous, then the process L o h is clearly version-continuous. Conversely, if (T, e) is compact and h is 1-1, then h is a homeomorphism (RAP, Theorem 2.2.11). Then version continuity of L on C and of L oh on T are equivalent. So for (T, e) compact and t i-* Xt one-to-one, version continuity of the Gaussian process Xt reduces to that of L on a subset C of H. (Theorem 2.8.2 and Corollary 2.8.5 will show that the 1-1 assumption is not actually necessary.) If T is locally compact, for example an open or closed subset of some Rk, then continuity is equivalent to continuity on each compact subset.
Let T be a set and d a pseudometric on T: for all x, y, z E T, d(x, y) _ d(y, x) andd(x, z) < d(x, y)+d(y, z), d(x, x) = 0, butpossiblyd(x, y) = 0 for some x y. Recall that for a set S C T, the diameter (with respect to d) is defined by diam S := diamd S := sup{d(x, y) : x, y E S}. The next fact holds for general, not necessarily Gaussian, processes.
2.8.2 Theorem If (T, e) is a compact metric space, h is a continuous function from T onto a metric space K, and Y(x, w), x E K, co E S2, is a stochastic process on K with values in a complete separable metric space S, then Y o h is version-continuous on T if and only if Y is on K.
Remark. If Y(x, w) - Y(x), so Y a nonrandom function, then the result is a known fact in general topology (RAP, Theorem 2.2.11). The difficulty in the proof here is that if Y o h is version-continuous, it is not clear that the
Gaussian Measures and Processes; Sample Continuity
76
corresponding sample-continuous process X can be written as Y' o h for a process Y' on K. Proof "If" is obvious. Conversely, let Y o h be version-continuous and take a process X on T with the same laws as Y o h and t i-+ X(t, co) := Xt(c)) continuous on T for all co. Let p and be the metrics on K and S, respectively. Let d be the pseudometric on T defined by d(s, t) := p(h(s), h(t)). Let A be a countable dense subset of T, and B := h(A) := {h(a) : a E A), so B is a countable dense subset of K. For y, 8 > 0 and any countable set F C T, define a random variable by
D(F, 3, y) := sup { (Xs, Xt) : s, t E F, d (s, t) < 8, e(s, t) < y). This is measurable since F is countable and S is separable (RAP, Proposition 4.1.7). Let D(F, 8) := D(F, 8, 1 + diame T), and
UC := n00U "0(D(A, 1/m) < 1/n), n=1 m=1
so that UC is measurable. If P(UC) = 1, then the sample functions t H Xt (w) are almost surely uniformly continuous with respect to d on A. Since A is countable and Y o h has the same laws as X, Y o h also has sample functions
almost surely uniformly continuous for d on A. Equivalently, Y has sample functions almost surely uniformly continuous for p on B. For any x E K let Y'(x) := lim{Y(u) : u -+ x, u E B). Almost surely, all these limits exist (by uniform continuity and since S is complete), and x i-+ Y'(x) is continuous. Since X and Y' o h both have continuous sample functions on T and have the same law on A, they have the same laws on T, and so does Y o h by choice of X. It follows that Y' has the same laws as Y on K, so Y is version-continuous as desired. Otherwise, P(UC) < 1. Then for some s > 0,
infs>oP(D(A, S) > 3e) > 3e. Then by inclusion (monotone convergence, with 8 = 1/m), (2.8.3)
P n {D(A, 8) > 3s) I > 3s.
\ s>o
/
On the other hand, continuity oft H Xt and compactness of T imply that for some y > 0 and any countable set C C T, (2.8.4)
P {D (C, 1 + diamdT, y) > e) < E.
2.8 Sample continuity and compactness
(Otherwise, take a countable union of countable sets for y = 1, 2,
77
1/n, n =
, to get a contradiction.)
T is a finite union of e-open sets Tj with diame Tj < y. For each i j, if there are s E T, and t E Tj with h(s) = h(t), then we say (i, j) E G, and let us choose and fix such s = s(i, j) and t = t(i, j). Let C be the union of A and the set of all s(i, j) and t(i, j). Since G is finite, we can assume that XX(i, j) (w) = Xt (t, j) (w) for all co and (i, j) E G. Let
J := n {D(C, 1/n) > 3s} f1 {D(C, 1 + diamd T, y) < e}. 0" n=1
Then (2.8.3) and (2.8.4) give P(J) > 3s - s > E. Fix an co E J and choose sn E C and to E C such that d(s, tn) < 1/n and (X,s,,, Xtn)((o) > 3e. By compactness, we can assume that the sequences sn and tn both converge fore and
hence also ford. Let sn -* s and to - t. Then d (s, t) = 0, XSn (w) -+ X5 (w), and Xt,, (co) Xt (co) as n oo. Lets E T, and t E Tj. If i = j we have, since
co E J, 3e < (X5, Xt)(co) < s, a contradiction. If i # j, then (i, j) E G. For n large enough, s, E T, and to E Tj, so
3s <
(Xn, XtJ (w) <
(XSn, XS(i,j)) ((0) +
(Xt(1,j), X,J ((0)
< s + s = 2e < 3s, again a contradiction. Now recall that a totally bounded set C in a Hilbert space H is called a GC-set if L restricted to C has a version with uniformly continuous sample functions.
2.8.5 Corollary A Gaussian process {Xt, t E T } with mean 0 on a compact metric space (T, e) is version-continuous if and only if both t H Xt EH L2(P) is continuous and its range K is a GC-set.
Proof Apply Theorem 2.8.2 with h(t) := K := h(T), p the usual metric in H, and S = R with its usual metric; again has the same laws as Xt. If Xt is version-continuous, then t i-* X, by Theorem 2.8.1, and the rest follows.
is continuous into H
Example. Let Xt be a Gaussian process defined for t E R, periodic of period 2n, so that Xt Xt+2n for all t, with E((Xt - X5)2) > 0 for Is - tI < 27r. Then we can write the process as Xt = Y(ett) where Y is a process indexed by the unit circle T1 :_ (z: I z I = 1) in the complex plane, which is compact. is 1-1 from Version continuity for X and for Y are equivalent, and z t-a
78
Gaussian Measures and Processes; Sample Continuity
Tt into H := L2(P), so version continuity is equivalent to that of L on the range of Y in H (without needing Theorem 2.8.2 and Corollary 2.8.5). On the other hand, any process indexed by R is version-continuous if and only if it is
so on each compact interval [-N, N], where in this example for N > n, the process is not 1-1 into H. Recall that a sample function of a stochastic process Xt is a function t H Xt (w) for a fixed w. The usual metric on a Hilbert space is the natural one for an isonormal process, but the GC-property holds for other metrics in the following sense:
2.8.6 Theorem Let C be a subset of a Hilbert space H. Then the following are equivalent:
(I) C is a GC-set; (II) L on C has a version with bounded, uniformly continuous sample functions; (III) there exists a metric p on C such that (C, p) is totally bounded, and the sample functions of the isonormal process L on C can be chosen to be p-uniformly continuous a.s.
Proof (I) implies (H) since by definition a GC-set is totally bounded, so a uniformly continuous function on it must be bounded. (II) implies (III) directly where p is the usual metric. Suppose (III) holds. Take a version of L such that on a set of probability 1,
the sample functions of L are p-uniformly continuous on C. Then L extends to a Gaussian process t i-± Xt on the compact completion M of C for p. Here Xt is version-continuous, and so by Corollary 2.8.5, C is included in, and thus is, a GC-set.
**2.9 Volumes, mixed volumes, and ellipsoids There will be no proofs in this section.
2.9.1 Volumes Any closed set C in a finite-dimensional Euclidean space Rk has a well-defined
volume, its Lebesgue measure V(C) := Vk(C) :_ Ak(C), invariant under Euclidean transformations (rotations, reflections, translations, etc.). Let C be a compact set in a Hilbert space H of dimension > k. Then Vk (C)
will be defined by Vk(C) := sup Vk(Jr(C)) where the supremum is over all
* *2.9 Volumes, mixed volumes, and ellipsoids
79
orthogonal projections 7r with k-dimensional range. Define the exponent of volume of C by
EV(C) := lim
(log V (C))/(n log n).
The exponent of entropy of C is defined by
r(C) := lim sup,10 (log log D(e, C))/(log(1/s)).
Thus C is a GC-set if r(C) < 2 by Theorem 2.6.1 and is not a GB-set if r(C) > 2 by the Sudakov-Chevet theorem (2.3.5). Milman and Pisier (1987) (see also Pisier (1989)) proved the following relation between r(C) and EV (C): Theorem (Milman and Pisier) Let C be a convex, symmetric, compact subset of
a Hilbert space. Suppose that for some 3 > 0 and M < oo, Vk(C) < Mk-'-' for all k = 1, 2, .. Then C is a GC-set and
EV(C) _
1
1
r(C)
2
2.9.2 Mixed volumes and mean widths
Let C be a convex set in a Euclidean space Rk and let B be the unit ball
B:={x: IxI <1}.Forr>OletC+rB:={x+ry: xEC, yE B). Then it is known that Vk(C + rB) is a polynomial of degree kin r, namely k
Vk(C+rB) = L.rk_ j(C) j=0
where ,Bj (C) is called the jth mixed volume of C (see, for example, Bonnesen and Fenchel (1934, section 29) or Leichtweiss (1980, Satz 15.4 p. 162); other normalizations of the mixed volumes can be found in the literature). We have fk (C) = Vk (C). Here if C is viewed as a subset of a space of higher dimension,
its mixed volumes may change, so more precisely one can write fj(C) = ,Bj,k(C). If C is a straight line segment, then 01 (C) is proportional to its length.
Let h1(C) := )81,k(C)/4 where k is such that for a line segment S, h1(S) equals the length of the segment for each k. Now if C is a compact, convex set in a separable, possibly infinite-dimensional Hilbert space and 2rm are finitedimensional projections increasing up to the identity, then h (C)) increase
up to a limit h1(C) < +oo.
Gaussian Measures and Processes; Sample Continuity
80
Let Sk-1 be the unit sphere in Rk, Sk-1 :_ {x E Rk : (x) = 1). For any convex set C in IRk and x E Sk-1, C has a width wx(C) in the direction x defined by
wx(C) := suPZEc(x, z) - infyec(x, Y) is the usual inner product. Let cok_1 be the unique rotationally where invariant Borel probability measure on Sk-1 (normalized (k - 1)-dimensional
surface area measure). Then the mean width of C is defined by w(C) := f wx (C)dwk_ 1(x). It is known that for each k, w (C) is proportional to h 1(C) or to #t (C) for convex sets C; see, for example, McMullen (1993, Theorem 5.5 p. 970), who refers for a proof to Hadwiger (1957). Sudakov (1971, Theorem 4, and 1973) discovered that:
Theorem (Sudakov) A compact, convex set C in a Hilbert space is a GB-set if and only if h 1(C) < oo. In fact,
h1(C) = (2n)1/2EL(C)*. Indeed, it is easily seen that for each k, EL (C)* for a convex set C in Rk is proportional to the mean width. Since for a finite-dimensional set C, EL (C)* doesn't depend on the dimension of the space in which C is imbedded, and neither does h 1(C), one can find the proportionality constant between h 1(C) and EL(C)* by evaluating both on a line segment.
2.9.3 Ellipsoids Let H be a separable, infinite-dimensional Hilbert space. Then a subset E of H will be called a (compact) ellipsoid if there exists an orthonormal basis {en }n> 1 of H such that for some sequence an 4. 0, we have 00
E = E({an})
jx=xnenEH: n=1
00
>(x,,/a,,)2 < 1
.
n=1
Then E is called a Schmidt ellipsoid if an2 < oo. Suppose for X E E we write L (x) xn Gn where Gn = L (en) are i.i.d. N(0, 1) variables, and then En xn Gn = En (xn /an )an Gn. We then have a.s. L(E({an}))* = En a2nG,2)112, where "<" holds by the Cauchy inequality and holds since for finite sums, the maximum over x is attained when xn/an is proportional to anGn. It follows easily that E is a GB-set if and only if it is a Schmidt ellipsoid. Approximating ellipsoids by finite-dimensional ones, it is also not hard to show that a Schmidt ellipsoid is a GC-set.
* *2.9 Volumes, mixed volumes, and ellipsoids
81
Let J be a possibly different separable, infinite-dimensional Hilbert space and let A be a bounded linear operator from J into H. Let B1 be the unit ball of
J, B1 := {x E J: IIxII < 11. Let A(B1) := {A(x) : x E B1}. Then A is called a compact operator if A(B1) is compact. The compact ellipsoids are exactly the sets A(B1) for compact operators A. The operator A is called a Hilbert-Schmidt operator if A(B1) is a Schmidt ellipsoid. Suppose that J = L2 (Y, v) and H = L2 (X, s) for some measure spaces (X, A, µ) and (Y, B, v). One such representation is always possible where X and Y are countably infinite sets (orthonormal
bases) and t and v are counting measures so that J and H are f2 spaces. The operator A is a Hilbert-Schmidt operator if and only if it can be written as an integral operator with a kernel K E L2(X X Y, g x v), namely A(f)(x) := f K (x, y) f (y) d v (y). If { fk }k> 1 and {gj }j> 1 are any orthonormal bases of J and H, respectively, the above criterion is equivalent to >k II A fk 112 < oo or to
Ek (A fk, gj)2 < oo. The latter two sums do not depend on the choices of the orthonormal bases. Further, these criteria do not require finding an orthonormal basis {e } of H in which the ellipsoid is diagonalized, as in the definition of Thus, the question whether an ellipsoid is a GB- or GC-set, in other words a Schmidt ellipsoid, is most easily settled directly rather than by computing its metric entropy or finding a majorizing measure. Still it can be interesting to see how the different kinds of criteria fit together. By Theorems 2.7.1 and 2.7.2, an ellipsoid has a majorizing measure on it if and only if it is a Schmidt ellipsoid. Ledoux and Talagrand (1991, pp. 448-453) construct explicit majorizing measures on Schmidt ellipsoids. For metric entropy, there is a specific criterion
for ellipsoids. Let H(s) := log D(s, E). Then the ellipsoid E is a Schmidt ellipsoid if and only if f0 s2dH(s) > -oo. Sudakov (1969, (4)) stated this criterion. It follows from more general results of Mityagin (1961) for ellipsoids in Banach spaces, as Marcus (1974) showed. Integrating by parts and using the Sudakov-Chevet theorem (2.3.5), one can see that an equivalent criterion is
f0 sH(s)ds < oo. Suppose that H(s) = log D(s, C) is of the order l/(s2[log(1/s)]`) ass j, 0 for some constant c. Then for general compact sets C in Hilbert space, c > 0 is necessary for the GB or GC property by the Sudakov-Chevet theorem, c > 2 is sufficient by Theorem 2.6.1, and both conditions are sharp for some families of sets. For ellipsoids we see that c > 1 is necessary and sufficient for the GB and GC properties. This is strictly intermediate between the best necessary (2.3.5) and best sufficient (2.6.1) conditions on metric entropy for the GB property. It follows that on some Schmidt ellipsoids, there is no probability measure m satisfying the approximate homogeneity condition in Theorem 2.7.4.
Gaussian Measures and Processes; Sample Continuity
82
**2.10 Convex hulls of sequences Two facts proved by Talagrand (1987 (5), Theorems 2 and 3), not explicitly in terms of majorizing measures, will be stated without proof, then some related, easier facts will be proved. Let co(A) denote the closed convex hull of a set A (smallest closed, convex set including A), and Lx := max(1, logx).
2.10.1 Theorem (Talagrand, 1987) C is a GB-set if and only if for some M < oo and gn E H with Ilgn II < M for all n,
C C co ({gn/(Ln)1I2}n>1) . 2.10.2 Theorem (Talagrand, 1987) C is a GC-set if and only if the same holds with IIgnII -* 0.
The next fact, an easy part of Theorem 2.10.1, will be proved.
2.10.3 Proposition If C = {gn}, where Ilgnll = O((logn)-1/2), then y(C) < oo, and C and J(C) are GB-sets. Proof Since constant multiples of C preserve y (C) < oo, we can assume Ilgn II < (Ln)-112 for all n. Let mn := c/(n(Ln)2) where c is such that En mn = 1. Let m :_ En mn8gn. To show that m is a majorizing mea-
sure, we need to consider m (B(gn, e)) for each n. Fix n and let c(j) := cj := (Ln)-t/2 + (Lj)-1"2. Then Ilgn - gj II < cj. Ifs > cj, then m(B(gn, e)) >
F-i> j c/(i(Li)2) > c/Lj. Break up the range of integration [0, oo) as the union of the intervals [0, CO, [cn, cn-1), c(n)
fo
[log (1/m (B (gn,
8)))11"2ds
, [c4, c3), [c3, oo). We have
< cn ( log [n(Ln)2/c])1/2
< 2(Ln)-1/2(Ln + 2LLn - log c)112 which is bounded in n. Next, the integral from c/+1 to cj is bounded by (cj -
cj+1)(L(L(j + 1)/c))1/2. By the mean value theorem, we get cj - cj+t = (Lj)-1I2 - (L(j + 1))-1/2 < 1/(2 j(Lj)3/2). The sum from j = 3 ton - 1 of (LL(j + 1) - logc)1/2/(2j(Lj)312) is bounded in n. The integral from c3 to oc is the integral from 2 to oo, which is 0, plus the integral from c3 to 2, which is bounded above by 2(L3/c)1/2 for all n (using logy < y for all y), so m is a majorizing measure.
To show {gn } is a GB-set, Proposition 2.2.1 gives P(I L (gn) I > M) < exp(-M2Ln/2), which is summable inn for M > 21/2, and the Borel-Cantelli
Problems
83
lemma applies. The set Y of finite convex combinations with rational coefficients of the gn is countable and dense in co({gn}), and L is a.s. bounded on Y, so co({g, }) is a GB-set by Lemma 2.5.1 and its proof.
Let en be orthonormal in H and let an
2.10.4 Proposition
,{
0, C =
{anen }n>1. Then the following are equivalent:
(I) C is a GB-set;
(II) an = O((Ln)-1"2);
(III) y(C) < 00.
Proof (I) implies (II) by Theorem 2.3.5. (II) implies (I) and (III) by Proposition
2.10.3. To show that (III) implies (II), let m be a majorizing measure and mn := m{anen}. Then for each n, 00
00 > ym(C) ? f (log(1/m(B(anen, 8))))1/2 de. 0
The integral is bounded below by the integral from 0 to an, which is at least
Let J
Ym(C)2 < oo. Then mn > exp(-J/an), E. m, = 1, and exp(-J/an) is nonincreasing inn, son exp(-J/an) < 1, a(log(1/mn))1/2.
and an <
(J/Ln)1/2.
Problems 1. If X and Y are i.i.d. N(0, 1), evaluate E max(X, Y). 2. Evaluate E exp(a II X 112) (finite or infinite) as a function of a > 0 if
(a) G(X) = N(0,1) in R, IIXII = IXI; (b) G(X) = N(0, C) in R2, C = (0 ), and II (xl , x2) I =
(x1 +x22)1 2.
3. Let H be a Hilbert space with orthonormal basis {ej}j>1. Let Gn be independent with laws N(0, (a) Under what conditions on does F_ Gnen converge almost surely in the norm of H? Hint: Apply the 3-series theorem (RAP, 9.7.3) to suitable real-valued random variables.
(b) If G = En Gnen in H as in (a), where the sum converges almost surely, find for what a > 0 we have E exp(a II G II2) < 00. After doing this directly, compare it with the results of Section 2.2.
4. Let Gn be i.i.d. N(0, 1) variables and an > 0 for each n. Under what conditions on an is Y-n an I Gn I < oo a.s.? Hint: Use the 3-series theorem.
Gaussian Measures and Processes; Sample Continuity
84
5. (a) Show that for any set A in a real vector space V and for any vector space W of linear forms on V, the polar A*1 of A, defined by A*1 := {w E W: w(v) < 1 for all v E A}, is convex in W. (b) Let C be a set and D its convex hull, that is, the smallest convex set including C. Show that D*1 = C*1.
(c) In I[82 let C be the unit square {0 < x < 1, 0 < y < 1). Evaluate the polar C*1.
6. Let H be a Hilbert space with orthonormal basis {en }n> 1. For c > 0 let E((cnIn>1) :_ (En>1 xnen : Y_n>1 xn/cn < 1), an infinite-dimensional ellipsoid. Show that E is a GB-set if and only if n c,2 < oo. 7. With notation as in the previous problem, let C {en /(log n) 1/2 : n > 21. Show that C is not a GC-set (it is a GB-set as shown in the example before Theorem 2.3.7).
8. Let 1/i be a real-valued characteristic function on R, where 1/i(t) _ °0 00 e`xt dP(x) for some symmetric law P on R, P(A) = P(-A) for
f
all Borel sets A. Show that there exists a Gaussian process Xt, t E R, with mean 0 and covariance EXSXt = 1/i(s - t) for all real s, t.
9. (a) For each t > 0 let flit be a "triangle function" on R with *t (0) = 1, and for some t > 0, Tit(s) = 0 whenever Isi > t, while flit is linear on each interval [-t, 0] and [0, t]. Show that 1/!t satisfies the conditions of the previous problem. Hint: Find its (inverse) Fourier transform and show that it is a probability density.
(b) Let 1/i be a continuous real-valued function on R which is even, 1/i(-x) = 1/i(x), 1/i(0) = 1, and on [0, oo), 1 is nonincreasing, nonnegative, and convex. Show that 1 is a characteristic function. Hint: Use a mixture of triangle functions. First consider the case that 1/i is piecewise linear and is 0 outside some finite interval. Take increasing limits of such piecewise linear functions.
10. Force > 0let 1/i(x) = 1 - (log(l/Ix1))-' forx in some neighborhood of 0 (piecewise linear elsewhere). (a) Show that there exists such a 1/i satisfying the conditions of problem 8, assuming problem 9. (b) What can be said about sample-continuity of the Gaussian process
Xt for different values of a? 11. Let p be a majorizing measure on a metric space (T, d). (a) If T is a subset of a Hilbert space H with usual metric, then T is a GB-set by Theorem 2.7.1 and is totally bounded by Theorem 2.3.5. Prove directly, in general, that (T, d) is totally bounded. Hint: If for some r > 0 there are infinitely many disjoint balls B(x1, r),
Problems
85
then inf1 µ(B(x;, r)) = 0, and l/p (B(x1, s)) is nonincreasing in s for each i.
(b) Let p be another probability measure on (T, d). If 0 < A < 1, show that A s + (1 - .k)p is also a majorizing measure on (T, d). 12. Let Xt (w) Fn=_,,o G, (co)etnt where Gn are independent random variables with laws N(0, Q,2) and >n Qn < oo. Show that the process t i-+ Xt is sample-continuous if and only if {Xt () : 0 < t < 2ir } is a GC-set in
L2(0) 13. Let z be as in Lemma 2.7.20. Show that E(z2) < 100/9. 14. Let e be orthonormal in H and C :_ {an (log n)-112en }n>2 where an --> 0 asn --> co. (a) Show that every such set is a GC-set.
0 slowly enough, show that for any r > 0 there (b) By taking a exist GC-sets C with D(e, C) > exp[1/(s2J log ej'')] for s > 0 small enough.
15. (A further extension of problem 10). For c > 0 let fi(x) := 1 (log(1/IxI))-1(log log(1/lxI))-c forx in some neighborhood of O (piecewise linear elsewhere).
(a) Show that there exists such a f satisfying the conditions of problem 8, assuming problem 9. (b) What can be said about sample continuity of the Gaussian process Xt for different values of c? (c) Show that for any r < 2, there exist GC-sets C with D(e, C) < exp(1/(s2l logejT) for s > 0 small enough. Compare this with problem 14, part (b), to see that for 0 < r < 2, one cannot tell from D(e, C) whether C is a GC-set or not. 16. Let vk be the Lebesgue volume of the unit ball in Rk. Then it is known that . For any c, > 0, the ellipsoid Vk = nk/2/ l,(1 + (k/2)) f o r k = 1 , 2, [X: rk1 x2/c? < 11 C Rk has volume VkC1 ck £k E({c; }k1)
for any ci > 0, i = 1,
,
k. For s > 0 let m := D(s, E).
(a) Show that m > cic2 . . . ck/sk.
(b) If cj > s for j = 1, , k, show that m (s/2)k < 2kc1 c2 . . . ck. (c) If c j = j -" f o r j = 1 , 2, , give upper and lower bounds for D(e, E) as s . 0 and k - oo. Hint: Recall Stirling's formula k!/[(k/e)k(22rk)112]
1 ask - oo (Theorem 1.3.13).
17. If g is a Young-Orlicz modulus, x2 = o(log g(x)) as x -> +oo, and Y is a N(0, 1) random variable, show that 11 YIIg = +oo.
86
Gaussian Measures and Processes; Sample Continuity
18. If g is a Young-Orlicz modulus, log g(x)) = 0(x2) as x -+ +oo, and Y is a N(0, 1) random variable, show that II YIIg < oo.
19. Let f(x) = r-xfor0<x
f ^- c + Y, an sin(nx) + bn cos(nx) n=1
in L2((-7r, 7r)) and use it to prove Yn_1 n-2 = n2/6.
Notes
Notes to Section 2.2. Proposition 2.2.1 improves on RAP, Lemma 12.1.6(b), by a factor of 2. Lemma 2.2.5 was first proved by Landau and Shepp (1971), then by Femique (1970), whose note giving a much shorter proof appeared in print earlier. The main theorems 2.2.2 and 2.2.8, giving the best possible upper bound for a, were then proved independently by Marcus and Shepp (1971) and Fernique (1971). The current exposition is based mainly, though not entirely, on Fernique (1975). The Darmois-Skitovic theorem says that if X1, , X, are independent real random variables where for some constants a 1, , an is independent with a,b1 $ 0, then for that i, Xi has a normal distribution; see Darmois (1951) and Skitovic (1954). C. R. Rao (1973, pp. 158-163 and 218) lists various characterizations of normal distributions. Notes to Section 2.3. Slepian (1962, Lemma 1) proved his inequality. Sudakov
(1969) announced a result somewhat weaker than Theorem 2.3.5. Sudakov (1971, Theorem 5) announced a stronger result, corrected in Sudakov (1973) to be exactly Theorem 2.3.5. Lemma 2.3.6 is essentially from Chevet (1970). Fernique (1975, Theoreme 2.1.2, Corollaire 2.1.3) proved Theorem 2.3.7. The proof given is as in Fernique (1997, pp. 59, 63-67), who mentions an idea of Kahane; see Kahane (1986).
Notes to Section 2.4.
Theorems 2.4.1 and 2.4.4 and Corollaries 2.4.5 and 2.4.6, and essentially Corollary 2.4.7, are due to T. W. Anderson (1955). These
facts were to some extent rediscovered by L. Gross (1962). Both used the Brunn-Minkowski inequality in their proofs. See also Borell (1974, 1975a,b), Y. Gordon (1985), and Kahane (1986).
Notes to Section 2.5. Lemma 2.5.2 is essentially due to Anderson (1955). Lemmas 2.5.3 and 2.5.4 are straightforward. In Dudley (1967, Theorem 4.6),
Notes
87
parts of which are due to Gross (1962), most of the conditions in Theorem 2.5.5 were proved equivalent for convex, symmetric sets. Feldman (1971) proved part (a'), about the symmetric convex hull of C.
Notes to Section 2.6. In apparently the first public use of a metric entropy hypothesis to obtain sample continuity of Gaussian processes, V. N. Sudakov, in a talk at the International Congress of Mathematicians in Moscow, 1966, announced that if log N(e, C) = O(e-') as e . 0 for some r < 2, then C is a GC-set. His first publication on the topic was Sudakov (1969). Specific bounds on metric entropy implying sample continuity were suggested by previous work
of Fernique (1964) for processes indexed by a real parameter. The fact that f o" (log N(e, C))1t2ds < oo implies C is a GC-set was given (with an equivalent
series instead of the integral) in Dudley (1967). Theorem 2.6.1 and its proof were given in Dudley (1973). Theorem 2.6.2 and its proof are based (with some changes) on Ledoux and Talagrand (1991, Section 11.1), where the theorem is extended, with different functions go, to a large class of non-Gaussian stochastic processes. Komatsu's inequality, used in the proof of Lemma 2.6.7, is quoted,
with hints for the proof, in Ito and McKean (1974, p. 17), citing as original source Komatsu (1955).
Talagrand's original proof (1987) of Theorem 2.7.2 was quite long. Talagrand (1992) published a proof of the majorizing measure theorem (2.7.2) which is much shorter, although some steps at the end were left to the reader. Talagrand also gave other reasonably short proofs of Theorem 2.7.2. Building on these, Fernique (1997) gave a proof which has largely been followed here. Specifically, Lemma 2.7.6 is Lemme 3.4.5 of Fernique (1997), Lemma 2.7.9 is his Lemme 5.3.5, and so on, up to changes in constants. Specifically, some constants were allowed to become larger to make the proof more self-contained. Fernique (1975) first proved Theorem 2.7.1, namely that existence of a majorizing measure implies the GB property. The proof given here is from Fernique (1997, pp. 100-101, 104-111) with minor changes. Thus, for example, Theorem 2.7.18 is Fernique's Theoreme 5.2.6 and Lemma 2.7.20 is his Lemme 5.2.7. As given here, the proofs of Theorems 2.7.1 and 2.7.2 turn out to be about equally long. The author wrote out the proofs of the much easier facts: Proposition 2.7.3, Theorem 2.7.4, and Corollary 2.7.5.
Notes to Section 2.7.
Notes to Section 2.8. I have no reference for Theorem 2.8.2. The facts in this section on Gaussian processes were to some extent stated, but not proved, in
88
Gaussian Measures and Processes; Sample Continuity
Dudley (1967, 1973). Andersen and Dobric (1988, Lemma 8, (4.6); preprint, 1985) first published proofs of Corollary 2.8.5 and Theorem 2.8.6 according to Gine and Zinn (1986, p. 58).
Note to Section 2.9. The section incorporates its own notes. Notes to Section 2.10. Talagrand (1987) proved Theorems 2.10.1 and 2.10.2. The author (without claiming any priority) proved the easier Propositions 2.10.3 and 2.10.4.
References *An asterisk indicates a work I have seen discussed in secondary sources but not in the original. Andersen, Niels Trolle, and Dobric, Vladimir (1988). The central limit theorem for stochastic processes II. J. Theoret. Probab. 1, 287-303. Anderson, Theodore W. (1955). The integral of a symmetric unimodal function over a symmetric convex set and some probability inequalities. Proc. Amer.
Math. Soc. 6, 170-176. Bonnesen, T., and Fenchel, W. (1934). Theorie der konvexen Kdrper. Berlin, Springer; repr. Chelsea, New York, 1948. Borell, Christer (1974). Convex measures on locally convex spaces. Ark. Mat. 12, 239-252. Borell, C. (1975a). Convex set functions in d-space. Period. Math. Hungar. 6, 111-136. Borell, C. (1975b). The Brunn-Minkowski inequality in Gauss space. Invent. Math. 30, 207-216. Chevet, Simone (1970). Mesures de Radon sur Rn et mesures cylindriques. Ann. Fac. Sci. Univ. Clermont#43 (math., 6° fasc.), 91-158. Darmois, Georges (1951). Sur une propriete caracteristique de la loi de probabilite de Laplace. Comptes Rendus Acad. Sci. Paris 232, 1999-2000.
Dudley, R. M. (1967). The sizes of compact subsets of Hilbert space and continuity of Gaussian processes. J. Funct. Anal. 1, 290-330. Dudley, R. M. (1973). Sample functions of the Gaussian process. Ann. Probab.
1, 66-103. Dunford, N., and Schwartz, J. T. (1958). Linear Operators, Part I: General Theory. Interscience. New York; repr. Wiley, New York (1988). Feldman, Jacob (1972). Sets of boundedness and continuity for the canonical normal process. Proc. Sixth Berkeley Symposium Math. Statist. Prob. 2, pp. 357-367. Univ. of Calif. Press, Berkeley and Los Angeles.
References
89
Fernique, Xavier (1964). Continuite des processus gaussiens. Comptes Rendus Acad. Sci. Paris 258, 6058-6060.
Fernique, X. (1970). Integrabilite des vecteurs gaussiens. Comptes Rendus Acad. Sci. Paris Sir. A 270, 1698-1699. Fernique, X. (1971). Regularite de processus gaussiens. Invent. Math. 12, 304-320. Femique, X. (1975). Regularite des trajectoires des fonctions aleatoires gaussiennes. Ecole d'ete de probabilites de St.-Flour, 1974. Lecture Notes in Math. (Springer) 480, 1-96. Fernique, X. (1985). Sur la convergence etroite des mesures gaussiennes. Z. Wahrscheinlichkeitsth. verw. Geb. 68, 331-336. Fernique, X. (1997). Fonctions aleatoires gaussiennes, vecteurs aleatoires gaussiens. Publications du Centre de Recherches Mathematiques, Montreal.
Gine, Evarist, and Zinn, Joel (1986). Lectures on the central limit theorem for empirical processes. In Probability and Banach Spaces, Proc. Conf. Zaragoza, 1985, Lecture Notes in Math. (Springer) 1221, 50-113. Gordon, Yehoram (1985). Some inequalities for Gaussian processes and appli-
cations. Israel J. Math. 50, 265-289. Gross, Leonard (1962). Measurable functions on Hilbert space. Trans. Amer. Math. Soc. 105, 372-390. Hadwiger, H. (1957). Vorlesungen fiber Inhalt, Oberflache and Isoperimetrie. Springer, Berlin. Ito, Kiyosi, and McKean, H. P. Jr. (1974). Diffusion processes and their sample paths. Springer, Berlin, New York. Kahane, Jean-Pierre (1986). Une inegalite du type de Slepian et Gordon sur les processus gaussiens. Israel J. Math. 55, 109-110. *Komatsu, Y. (1955). Elementary inequalities for Mills' ratio. Rep. Statist. Appl. Res. Un. Japan. Sci. Engrs. 4, 69-70. Landau, H. J., and Shepp, Lawrence A. (1971). On the supremum of a Gaussian process. Sankhya Ser. A 32, 369-378. Ledoux, M., and Talagrand, M. (1991). Probability in Banach Spaces. Springer, Berlin. Lehmann, E. L. (1986). Testing Statistical Hypotheses, 2d ed. Wiley, New York; repr. 1997. Leichtweiss, K. (1980). Konvexe Mengen. Springer, Berlin. Marcus, M. B. (1974). The e-entropy of some compact subsets of V. J. Approximation Th. 10, 304-312.
90
Gaussian Measures and Processes; Sample Continuity
Marcus, Michael B., and Shepp, L. A. (1972). Sample behavior of Gaussian processes. Proc. Sixth Berkeley Symp. Math. Statist. Prob. (1970) 2, 423441. Univ. of Calif. Press, Berkeley and Los Angeles. McMullen, P. (1993). Valuations and dissections. In Handbook of Convex Geometry, vol. B, eds. P. M. Gruber, J. M. Wills, North-Holland, Amsterdam. Milman, V. D., and Pisier, G. (1987). Gaussian processes and mixed volumes. Ann. Probab. 15, 292-304. Mityagin, B. S. (1961). Approximate dimension and bases in nuclear spaces. Russian Math. Surveys 16, no. 4, 59-127. Pisier, G. (1989). The Volume of Convex Bodies and Banach Space Geometry. Cambridge University Press, New York. Rao, C. R. (1973). Linear Statistical Inference and Its Applications, 2d ed. Wiley, New York.
Skitovic, V. P. (1954). Linear combinations of independent random variables and the normal distribution law. Izv. Akad. Nauk SSSR Ser. Mat. 18, 185-200 (in Russian). Slepian, David (1962). The one-sided barrier problem for Gaussian noise. Bell Syst. Tech. J. 41, 463-501. Sudakov, V. N. (1969). Gaussian measures, Cauchy measures and s-entropy. Soviet Math. Dokl. 10, 310-313. Sudakov, V. N. (1971). Gaussian random processes and measures of solid angles in Hilbert space. Soviet Math. Dokl. 12, 412-415. Sudakov, V. N. (1973). A remark on the criterion of continuity of Gaussian sample function. In Proc. Second Japan-USSR Symp. Probab. Theory, Kyoto, 1972, ed. G. Maruyama, Yu. V. Prokhorov; Lecture Notes in Math. (Springer) 330, 444-454. Talagrand, M. (1987). Regularity of Gaussian processes. Acta Math. (Sweden) 159, 99-149. Talagrand, M. (1992). A simple proof of the majorizing measure theorem. Geom. Funct. Anal. 2, 118-125.
3
Foundations of Uniform Central Limit Theorems: Donsker Classes
3.1 Definitions: convergence in law Let (S, 13, P) be a probability space, to be called the sample space. Examples
to have in mind for S are Euclidean spaces such as the plane R2. To form empirical measures, we'd like to take variables X1, X2, , i.i.d. with law P. To do this, take a countable product S°O of copies of (S, 13, P) (RAP, Theorem
8.2.2) and let X; be the coordinates on the product. A product may be taken with another probability space. Throughout the rest of this book, X, will be defined as such coordinates unless something is said to the contrary. This way of defining Xi will be called the standard model. An example showing that use of the standard model makes a difference will be given at the end of Section 5.3. Recall that the product or -algebra is the smallest for which all the coordinates a r e measurable, and that 8 (A) = lA(x). Then, we can form the empirical measures Pn
- rn
x1
i=1
and the empirical process
vn := n1"2(Pn - P). So Pn is a probability measure on S, defined on B and actually on all subsets of S, for any values of X1, , Xn. Each vn is a finite signed measure of total charge 0. As n -* oo, for any set A E 13, vn(A) converges in law to a normal law
with mean 0 and variance P(A)(1 - P(A)). Next, a general analogue of the Brownian bridge will be defined. Let GP be a Gaussian process indexed by the set G2(P) of measurable, square-integrable functions for P, having
mean 0 and covariance EGp(f)Gp(g) = f fgdP - f f dP f gdP. To see 91
92
Uniform Central Limit Theorems: Donsker Classes
that such a Gaussian process exists, we need to check that the given covariance is nonnegative definite, which holds since it is just the covariance for P, in other words
EGp (.f) Gp (g) = (.f g)o, p
ov p (.f, g) Cf(f_ffdP)(g_fgaP)dP ,
and such a covariance is always nonnegative definite.
Let G2(P) be the set of all functions f E L2(p) such that f f dP = 0. Let L2(P) be the set of all equivalence classes of elements of L2(p) for equality a.s. (P). On L2(P), the covariance for Gp reduces to the ordinary G2 semiinner product. In other words, restricted to the subspace L2(P), Gp is an isonormal process. Let C be the one-dimensional space of constant functions c as a subspace of G2(P). Then Gp is 0 on C, while the spaces C and L2(P) are orthogonal complements of each other (RAP, Theorem 5.3.8) in G2(P). For
any f and gin G2(P) and c E 1t8, Gp(cf + g) = cGp(f) + Gp(g) a.s., since as is easily checked, Gp(cf + g) - cGp(f) - Gp(g) has mean and variance both 0.
The covariance for P defines a pseudometric on G2(P) by pp(f, g)
(E(Gp(f) -
Gp(g))2)112. Then pp equals the usual L2(p) pseudometric on
L2 (p). Moreover, with respect to the semi-inner product ( , -)0, p on L2 (p), Gp is the isonormal process and pp the usual pseudometric. Thus, the results of Chapter 2 apply to sample continuity of Gp with respect to pp. The Brownian bridge process yr is a special case of the Gp process where P is Lebesgue measure on [0, 1] and yt = Gp(1[0,(1); it is easily checked that this has the right covariance. The Brownian bridge process can be taken to have continuous sample paths, in other words to be continuous as a function of t for each w (RAP, Theorem 12.1.5). But Gp is not sample-continuous on the whole space G2(P). Recall
that an atom of a measure space (0, S, s) is a set A E S with µ(A) > 0 such that for all S E S, µ(S fl A) = 0 or it (A). A measure having no atoms is called nonatomic. In fact, unless P is concentrated in finitely many atoms, the spaces G2(P) and L2(P) are infinite-dimensional, in the sense that they contain infinite orthonormal sets, and we saw in Section 2.6 that an isonormal process on an infinite-dimensional Hilbert space H is not sample-continuous.
We will be concerned then with suitable subsets of £2(P). A class F C G2 will be called pregaussian if a Gp process (f, (o) i-4 Gp(f)(w) can be defined on some probability space such that for each w, f i-+ Gp (f) (w) is bounded and uniformly continuous for pp from .F into R.
3.1 Definitions: convergence in law
93
Given f e G2(P), let rro(f) := f - f f dP. Then 7ro(f) E G4(P). Given F C G2(P), let Jro(.P) be the set of all functions 7ro(f), f E .P. For any f E G2(P), pp(f, no(f )) = 0, and Gp(f) = Gp(7ro(f )) a.s. I claim that F is pregaussian if and only if rro(.P) is: if iro(.P) is pregaus-
sian, then f r- Gp(Iro(f)) has the desired properties on .P, while if .P is pregaussian, then for any g E 7ro(.F), select arbitrarily (by the axiom of
choice!) an fg E F with Iro(fg) = g. Then g H Gp(fg) has the desired properties.
Now recall the definitions of GB-set and GC-set from Section 2.5. Note that if a set C in a Hilbert space is a GB-set, then it must be totally bounded by Theorem 2.3.5. If C is also a GC-set, then uniformly continuous sample functions of the isonormal process on C must be bounded. Thus since Gp is isonormal on L2 (P), any set 9 C G4(P) is pregaussian if and only if the corresponding set in L2(P) is a GC-set. Recall the notion of prelinear function (Lemma 2.5.3). A Gp process Y on a class F C G2 (S, 8, P) for a probability space (S, 13, P) will be called coherent if for each (o E S, the function f F-+ Y(f) (w) on.F is bounded, pp-uniformly continuous, and prelinear.
3.1.1 Theorem Given a probability space (S, B, P) and F C £2(S, 8, P), .P is pregaussian if and only if there exists a coherent Gp process on F.
Proof "If" follows from the definition of pregaussian. For the converse, note that no above is linear and Gp is isonormal on Go (S,13, P), and apply Theorem 2.5.5.
Now, if a class F C G2(P) is pregaussian we can ask whether v converges in distribution, or in law, to Gp with respect to uniform convergence over F. Recall that for random variables Y, with values in a separable metric space S, convergence in law of Y to Yo is defined classically to mean that Eg(Y,,) Eg(Yo) as n -+ oo for every bounded continuous function g on S. But empirical processes, even empirical distribution functions as treated in Section 1.1, take values in nonseparable metric spaces, and for an arbitrary bounded continuous g, g(v,) may not be measurable. A new definition of convergence in law is needed to take care of this nonmeasurability. The definition will involve the notion of upper integral. Let g be a real-valued, not necessarily measurable function defined on a space X where (X, S, µ) is a measure space. Let R be the set [-oo, oo] of extended real numbers. Then the upper integral is defined
Uniform Central Limit Theorems: Donsker Classes
94
by
f gdµ := inf if h dµ : h > g, h measurable and R-valued I
,
which will be undefined if there exists a measurable h > g with f hdµ = oo-oo undefined, unless there is also a measurable i/r > g with f *dµ = -oo, in which case f * gdµ will also be defined as -oo. There always exists at least one measurable h > g, namely h - +00. We will be dealing often with compositions of functions. If f is a function whose domain includes the range of g then either f (g) or fog will denote the
function such that (f o g)(x) - f (g(x)). A function, which may not be measurable, from a probability space into a metric space will be called a random element. Now here is a definition of convergence in law, where only the limit variable necessarily has a law:
Let (S, d) be any metric space. Let (Q,, A,z , Q,) be probability spaces for n = 0, 1, 2, .. , and Yn , n > 0, functions from Stn into S. Suppose that Yo takes values in some separable Definition (J. Hoffmann-Jorgensen).
subset of S and is measurable for the Borel sets on its range. Then Y, will be Yo, if for every said to converge to Yo in law as n -+ oo, in symbols Y, bounded continuous real-valued function g on S,
fg(Yfl)dQfl
- fg(Yo)dQoasnoo.
For g bounded, f * g(YY) dP is always defined and finite. Then, here is a general definition of when the central limit theorem for empirical measures holds with respect to uniform convergence over a class F of functions. The metric space S will be the space £°O(F) of all bounded real-valued functions on F, with the metric given by the supremum norm IIHII,, := sup{IH(f)I :
fc.T}. Definition. Let (S2, A, P) be a probability space and F c G2(P). Then F will be called a Donsker class for P, or P-Donsker class, or be said to satisfy the central limit theorem (for empirical measures) for P, if F is pregaussian
for P and vn = Gp in Later on, a number of rather large classes F of functions will be shown to be Donsker classes for various laws P. The next few sections develop some of the needed theory.
3.2 Measurable cover functions
95
3.2 Measurable cover functions In the last section, convergence in law was defined in terms of upper integrals. The notion of upper integral is related to that of measurable cover. Let (7, A, P) be a probability space. Then for a possibly nonmeasurable set A C S2, a set B is called a measurable cover of A if A C B, B E A,
and P(B) = inf{P(C) : A C C, C measurable). For A C B E A with P(A), a measurable cover of A is n,, B,,. If B and C are measurable
covers of the same set A, then clearly so is B fl C. It follows that B = C up to a set of measure 0, in other words P(BOC) = 0 where A denotes the symmetric difference, or equivalently P (1 B = 1 c) = 1. For any set A C S2 let P*(A) := inf{P(B) : B measurable, A C B}. Then for any measurable cover B of A, clearly P*(A) = P(B). Let r0 := &A A, P, 9) denote the set of all measurable functions from S2 into R. Then G° is a lattice: for any f, g E G°, f v g := max(f, g) and f n g := min(f, g) are in G°. But this LO is not a vector space since we could have, for example, f = +oo and g = -oo, so f + g would be undefined. The map y i-* tan-1 y is one-to-one from ]l8 onto [-7r/2,7r/21. Then a
metric on R is defined from the usual metric on [-ir/2, r/2] by d (x, y) := I tan-1 x - tan-1 yl. On Lo we have the Ky Fan metric (RAP, Theorem 9.2.2) defined by
d(f,g) := inf{E>0: P(d(f(x),g(x))>s)<E}. Then d (f, g) = 0 if and only if P (f = g) = 1. For any set j C Ge(S2, A, P, R), a function f E Lo is called an essential
infimum of J, or f := ess.inf J, if for all j c- ,7, f < j a.s. and for any g E LO such that g < j a.s. for all j E ,7, we have g < f a.s. If f and g are two essential infima of the same set ,7, then clearly f = g a.s. A set ,7 of functions will be called a lower semilattice if for any f, g E .7, we have
min(f, g) E J. 3.2.1 Theorem For any probability space (0, A, P) and class ,7 C Go(S2, A, P, R), an essential infimum of ,7 exists. If for some function f : S2 i-+ R we have j = [j E Lo : j > f everywhere), then f * := ess.inf ,7 can be chosen so that f * > f everywhere. Also, f f * dP and E*f := f * f dP are both defined and equal if either of them is well-defined (possibly infinite), for example if f * is bounded below. P r o o f Let j i be the class of all functions min (fl ,
, f , , , ) for fl,
, f E .7
and m = 1, 2, . Then JI is a lower semilattice. For f E G°, f = ess.inf ,71 if and only if f = ess.inf J. So we can assume ,7 is a lower semilattice. For
96
Uniform Central Limit Theorems: Donsker Classes
each j E J, tan-1 j is a measurable function with values in [-7r/2, r/2]. Take jn E J such that f tan-1 j f tan-1 j dP. Then min(J1, , j,,,) is in Ji and decreases to g as m - oo for some g E Ge(S2, A, P, ]R). For any h E J, min(h, j1, , jm) J, min(h, g) so f tan- Imin(h, g) = f tan '(g) and g < h a.s., so g satisfies the definition of ess.inf J. If 3 = (h E LO : h > f) then J is a lower semilattice and g > f everywhere. By the definitions, f * f dP < f f * dP if either side is well-defined, and the inequality is an equation by the definition of essential infimum.
Here f * is called the measurable cover function of f. Recall that in Chapter 2, L (A)* was the essential supremum of L (x) for x E A, and so the essential infimum of random variables Y such that for each x E A, Y > L (x) a.s. - a different, although related, notion. If f is real-valued and bounded above by some finite-valued measurable function then f * is a measurable real-valued function. But whenever there exist nonmeasurable sets A 4. 0 with P* 1, as for Lebesgue measure (e.g., RAP, Section 3.4, problem 2), let f := n on An \
Then f is real-valued but f * _ +oo a.s. The next two lemmas on measurable cover functions are basic.
3.2.2 Lemma For any two functions f, g: S2 H (-oo, oo], we have
(a) (f + g)* < f * + g* a.s., and (b) (f - g)* > f - g* whenever both sides are defined a.s. Proof (a) We have -oo < f * < +oo and -oo < g* < +oo everywere, so f * +g* is an everywhere defined, measurable function > f+g, and (a) follows. For part (b), on the measurable set where g* = +oo, where by assumption f is finite a.s., the right side is -oo and the inequality holds. Where g* is finite,
g is also finite and f = (f - g) + g. Then f * < (f - g)* + g* by (a), so f * - g* < (f - g)*, since this holds where (f - g)* < oo and where
(f -g)*=oo. 3.2.3 Lemma Let S be a vector space with a seminorm II II. Then for any two functions X, Y from 0 into S,
IIX+ YII* < (11XII + IIYII)* < IIXII* + IIYII* a.s. and IIcXII* = Ici IIXII* a.s. for any real c. Proof The first inequality is clear, the second follows from Lemma 3.2.2, and the equation is clear (for c = 0 and c # 0).
3.2 Measurable cover functions
97
Next, in some cases of independence, the upper-star operation can be distributed over products or sums.
3.2.4 Lemma Let (2j, Aj, Pj), j = 1,
, n, be any n probability spaces. Let fj be functions from Q j into R. Suppose either
(a) f >0, j= 1, - - , n, or (b) fl = 1 and n = 2. Then on the Cartesian product IInj_1(S2j, Aj, Pj) with x := (xl,
f(x) :=
, xn), if
fj(xj), we have f*(x) = nj=1 f.*(xj) a.s., where 0 oo is set
equal to 0.
(c) Or, if fj(xj) > - ooforallxj, j = fl (XI) + ... + fn (xn ), then g* (xi, ... , xn) = fl* (x i) + .
+ fn (xn )
a.s.
Proof First, for (c), by induction we can assume n = 2. Clearly g*(x, y) < f l* (x)+ f2 (y) a.s., and if equality does not hold a. s., there is a rational t such that
on a set C of positive probability in the product space, g* (x, y) < t < fl* (x) +
f2 (y), and there exist rational q, r with q + r > t such that C can be chosen
with fl*(x) > q and f2 (y) > r for (x, y) E C. Let Cx := {y: (x, y) E C}. By the Tonelli-Fubini theorem there is a set D C 01 with PI (D) > 0 such that
P2(Cx) > 0 for all x E D. If fi < q on D then fl* < q a.s. on D, but for any x E D and y E Cx # 0 we have fl* (x) > q, a contradiction. So choose and fix an x E D with fl (x) > q. Then for any y E Cx, q + f2 (y) < fi (x) + f2 (y) < g* (x, y), so f2 (y) < g* (x, y)-q and f2 (y) < g* (x, y)-q for almost all y E C. For any such y, q + f2 (y) < q +r and f2 (y) < r, a contradiction. So (c) is proved. Now for products, in case (a) or (b), clearly f *(x) < nj=1 f.* (xj) a.s., with 1 * = 1. For the converse inequality we can assume n = 2, by induction in case (a). Suppose f *(x) < fl (xl) f2 (x2) with positive probability. Then for some
rational r, f * (x) < r < fl*(x1) f2 (x2) with positive probability. If fj - 1, this gives f (x) < f * (x) < r < f2 (x2) on a set of positive probability. Then by _
the Tonelli-Fubini theorem, for some x1, f2(x2) < f *(xl, x2) < r < f2 (x2) on a set of x2 with P2 > 0, contradicting the choice of f2*' So assume fi > 0 and f2 > 0. Then as in case (c), there are rationals a, b
with ab > r, a > 0, and b > 0, such that on a set C in the product with positive probability, fl*(xi) > a, f2 (x2) > b, and f*(x1,x2) < r. Again by the Tonelli-Fubini theorem, there is a set D C 01 with PI (D) > 0 and P2(Cu) > 0, u E D, and there is a point u of D where fl (u) > a. Then for any v E C, f2(v) < f *(u, v)/a, so f2 (v) < f *(u, v)/a for almost all
98
Uniform Central Limit Theorems: Donsker Classes
v E G. For such a v we have aft (v) < ab and f2 (v) < b, a contradiction, finishing the proof.
For the next fact here is some notation: given two functions f, g and a a-
algebra S on the range of f, let (f, g)(x) := (f (x), g(x)) and f -1(S) { f -'(A): A E S}. 3.2.5 Lemma Let (0, A, P) = II3_1(521, Si, PI) with coordinate projections 111: 111(x1, x2, x3) := xi, i = 1, 2, 3. Let Si ®S2 denote the product a-algebra on S21 x 522. Then for any bounded real function f on 01 x 523 and 9(x1, x2, x3) := f (x1, x3), conditional expectations of g* satisfy
E(g*1(1-11, 112)-1(S1 (9 S2)) = E(g*I11-1(S1)) a.s.for P.
Proof By Lemma 3.2.4(b), for 22 X (S21 x 523), g* equals P-almost surely a measurable function not depending on x2, thus independent of 11-1(S2). Let S be the collection of all sets A E (111, f12)-1(S1 0 S2) such that g* and h := E(g* 111-1(Si )) have the same integral over A. Then S contains all finite
disjoint unions of sets (Hi, n2)-1(B1 x B2) = IIi 1(B1) n n21(B2), Bi E Si, i = 1, 2, since both g* and h are independent of 17121(S2). Now S is easily seen to be a monotone class, so it equals all of (111, H2)-1(S1 0 S2) (RAP, Theorem 4.4.2).
3.2.6 Lemma Let X be a real-valued function on a probability space (0, A, P). Then for any t E R, (a) P* (X > t) = P(X* > t);
(b) foranye>0, P*(X>t)t)
t-e). Proof Clearly {X > t} c {X* > t) and {X > t) c {X* > t}, so we have "<" in (a) and the first inequality in (b).
Take a measurable cover A D {X > t), so P*(X > t) = P(A). Then X* < t a.s. outside A, so P(X* > t) < P(A), proving (a). Thus for 0 < 8 <
e, P(X* > t - 8) < P*(X > t - s). Letting 8 4, 0 proves (b). Let (0, A, P) be aprobability space. For a function f from Q into [-oo, oo]
let f* f dP := sup{ f gdP : g measurable, g < f}. Let f* be the essential supremum of all measurable functions g < f. Then just as for f*, f* is well-defined up to a.s equality and f* f dP = f f* dP whenever either side is defined, as in Theorem 3.2.1.
3.2 Measurable cover functions
99
It's easy to check that f* = -((- f)*) and that f* f dP = -(f * - f dP). So the convergence in law Y, = Yo as defined in Section 3.1 implies that f* g(Y-,) dQn - f g(Yo) dQoNext, here is a one-sided Tonelli-Fubini theorem for starred functions:
3.2.7 Theorem Let (X, A, P) x (Y, B, Q) be a product of two probability spaces. For a real-valued function f > 0 on X x Y, define f * with respect to P x Q. For each x E X let (E* f)(x) := f * f (x, y) dQ(y). Then
EiE2f(x,Y)
(3.2.8)
ff*(x,y)d(PxQ)(x,y),
where Ei = E* with respect to P. Also, if Q is purely atomic, so that >j Q({yj)) = 1 for some yj E Y, and (3.2.9)
f - dQ, then
Ei E2f(X, Y) < E*f(X, Y) = E2Ei f(X, Y).
Proof By the usual Tonelli-Fubini theorem, we have (3.2.10)
f
f*(x, y)d(P x Q)(Y) = ff f*(x, y)d Q(y) dP(x),
and f f *(x, y)d Q(y) is measurable in x. For each x E X let f (y) := f (x, y). Then f(x, y) < f *(x, y), which is measurable in y, so f,, (y) < f *(x, y) for Q-almost all y. Thus since E* fX = Efx by Theorem 3.2.1,
(E*f)(x) =
f
f,*(Y)dQ(Y) <
f
f*(x,Y)dQ(Y),
and (3.2.8) follows. Next, to prove (3.2.9), note that on Y, since Q is purely atomic, all real-valued functions are measurable (for the completion of Q), so E2 - E2, the left inequality follows from (3.2.8), and the equality from (3.2.10), so (3.2.9) is proved. We also have a one-sided monotone convergence theorem with stars:
3.2.11 Theorem Let (S2, A, P) be a probability space and let f be realvalued functions on Q such that f f f, that is, f (x) T f (x) for all x E Q. If
E* fl > -oo then E*fj T E*f as j -+ oo.
Proof By Theorem 3.2.1, E*fj = Ef * for each j, and as j -+ oo, f increases a.s. up to some function g where clearly g < f * almost surely. If
g < f * on some set A with P(A) > 0, then f < g on A implies f < g and so f * < g on A, a contradiction. The theorem follows.
100
Uniform Central Limit Theorems: Donsker Classes
Recall that for any measure space (X, A, µ) and any C C X, the outer measure is defined by s*(C) := inf(µ(A) : A E A, A D C). Note that there exist subsets Aj := A(j) of [0, 1] with outer measure k*(Aj) = I for all j and Aj J, 0 (RAP, Problem 3.4.2). Letting f := IA(.i), we have f 4, 0, and E* f = 1 for all j, so that the monotone convergence theorem fails for E* on decreasing sequences. Next is a Fatou lemma with stars.
3.2.12 Theorem Let (Q, A, P) be a probability space and f any nonnegative real-valued functions on 0. Then E* lim infs.,, f < lim infj,0 E* f j. Proof For 1 < k < m we have infra>k f, < fm and so E* fn < E* fm, thus E* infn>k fn < infra>k E*fm. Taking the supremum over k on both sides and using the previous monotone convergence theorem gives the result.
3.3 Almost uniform convergence and convergence in outer probability In Section 3.1 the definition of convergence of laws was adapted to define convergence in law for random elements which may not have laws defined. In this section the same will be done for convergence in probability and almost sure convergence. Let (0, A, P) be a probability space, (S, d) a metric space, and fn functions
from S2 into S. Then f, will be said to converge to fo in outer probability if
d(f, fo)* - 0 in probability as n - oo, or equivalently, by Lemma 3.2.6,
for every e > 0, P*(d(f , fo) > E) -k 0 as n -+ oo. Also, f, is said to converge to fo almost uniformly if as n - oo, d(fn, fo)* -k 0 almost surely. The following is immediate:
3.3.1 Proposition Almost uniform convergence always implies convergence in outer probability.
If fn are all measurable functions, then clearly convergence in outer probability is equivalent to the usual convergence in probability, and almost uniform convergence to almost sure convergence. Now some definitions will be given for Glivenko-Cantelli properties, which are laws of large numbers for empirical measures.
Definition. If (X, A, P) is a probability space and .P is a class of integrable real-valued functions, F C Gt (X, A, P), then F will be called a strong (resp.
3.3 Convergence without measurability
101
weak) Glivenko-Cantelli class for P if as n -+ oo, 11 P, - PII.r --+ 0 almost uniformly (resp. in outer probability). In the following proposition, part (C) is what is usually called "Egorov's theorem" for almost surely convergent sequences of measurable functions (RAP, Theorem 7.5.1).
3.3.2 Proposition Let (S2, A, P) be a probability space, (S, d) a metric . Then the following space, and f, any functions from 2 into S for n = 0, 1, are equivalent:
(A) fn - fo almost uniformly. (B) For any e > 0, P*{sup,,>m d(fn, fo) > e} 4, 0 as m -* oo. (C) For any 8 > 0 there is some B E A with P(B) > 1 - 8 such that fn -* fo uniformly on B. (D) There exist measurable hn > d(fn, fo) with hn -+ 0 a.s. Proof (A) implies that (SUpn>m d (fn, f0))* < Sup,,>m (d (fn,
f0)*)
4, 0 a.s. I
as m -+ oo,
which implies (B).
Assuming (B), for k = 1, 2, , let Ck := {supra>m(k) d(f, fo) > 1/k), where m (k) is large enough so that P* (Ck) < 2-k. Take measurable covers B k f o r Ck, so C k C Bk, Bk E A, and P(Bk) < 2-k. For r = 1 , 2, , let Ar := S2 \ Uk>r Bk. Then P(Ar) > 1 - 2-' and fn fo uniformly on Ar, so (C) holds. Now assume (C). Take Ck E A, k = 1, 2, , such that P(Ck) T 1 and f n -+ f o uniformly on Ck. W e can take Cl C C2 C . Take Mk such that
d(f, fo) < 1/k on Ck for all n >_ mk. Then d(fn, fo)* < 1/k on Ck, so d(fn, fo)* --> 0 a.s., proving (A). Clearly, (A) and (D) are equivalent.
Example. In [0, 1] with Lebesgue measure P let A 1 D A2 D . . . be sets with P*(A,) = 1 and fn°_1 An = 0 (e.g., RAP, Section 3.4, problem 2; Cohn, 1980, p. 35). Then 1An -* 0 everywhere and, in that sense, almost surely, but not almost uniformly. Note also that lAn doesn't converge to 0 in law as defined in Section 3.1. To avoid such pathology, almost uniform convergence is helpful.
3.3.3 Proposition Let (S, d) and (Y, e) be two metric spaces and (S2, A, P) a probability space. Let f, be functions f r o m 0 into S for n = 1 , 2,
, such
that f, - fo in outer probability as n -+ oo. Assume that fo has separable
Uniform Central Limit Theorems: Donsker Classes
102
range and is measurable (for the Borel or -algebra on S). Let g be a continuous function from S into Y. Then g(fn) g(o) in outer probability.
Note. If f G (fo) dP is defined for all bounded continuous real G (as it must be if fn fo, by definition) then the image measure P o f0 1 is defined on all Borel subsets of S (RAP, Theorem 7.1.1). Such a law does have a separable support except perhaps in some set-theoretically pathological cases (Appen-
dix Q. , let Bk :_ {x E S: d(x, y) < 1/k implies P r o o f Given e > 0, k = 1, 2, e(g(x), g(y)) < s, y E S}. Then each Bk is closed and Bk T S as k -+ oo. Fix k large enough so that P(f01(Bk)) > 1 - e. Then {e(g(fn), g(.f0)) > s) (1 f0 1(Bk) C {d (fn, .fo)
1/k}.
Thus
P*{e(g(fn),g(f0)) > e} < e+P*{d(fn, fo) > 1/k} < 2s for n large enough.
3.3.4 Lemma Let (S2, A, P) be a probability space and {gn }n o a uniformly bounded sequence of real-valued functions on Q such that go is measurable. If gn -* go in outer probability then lim supn,, f * gn dP < f go dP.
Proof Let I g, (x) I < M < oo for all n and all x E 22. We can assume M = 1. Given s > 0, for n large enough P*(I gn - goI > e) < e. Let An be a measurable set on which Ign - gol < e with P(S2 \ An) < e. Then
J
gndP < e+ J*gndP < 2s+ f A
A An
< 3s+ J $odP.
Letting s 1. 0 completes the proof. On any metric space, the a-algebra will be the Borel or -algebra unless something is said to the contrary.
3.3.5 Corollary If f, are functions from a probability space into a metric space, fn --* fo in outer probability, and fo is measurable with separable range, then fn = fo. Proof Apply Proposition 3.3.3 to g = G for any bounded continuous G and Lemma 3.3.4 to gn := G o fn and also to gn = -G o fn.
3.4 Perfect functions
103
3.4 Perfect functions For a function g defined on a set A let g[A] :_ {g(x) : x E A}. It will be useful that under some conditions on a measurable function g and general real-valued
f, (f o g)* = f * o g. Here are some equivalent conditions: 3.4.1 Theorem Let (X, A, P) be a probability space, (Y, l3) any measurable space, and g a measurable function from X to Y. Let Q be the restriction of P o g t to B. For any real-valued function f on Y, define f *for Q. Then the following are equivalent:
(a) For any A E A there is a B E B with B C g[A] and Q(B) > P(A); (b) for any A E A with P(A) > 0 there is a B E 8 with B C g[A] and Q(B) > 0; (c) for every real function f on Y, (f o g)* = f * o g a.s.; (d) f o r any D C Y, OD o 9)* = 1 *D o g a. s.
Proof Clearly (a) implies (b).
To show (b) implies (c), note that always (f o g)* < f* o g. Suppose (f o g)* < f * o g on a set of positive probability. Then for some rational
r, (fog)* 0.Letg[A]DBEB with Q(B) > 0. Then f o g < r on A implies f < r on B, so f * < r on B a.s., contradicting f * o g > r on A. Clearly (c) implies (d). Now to show (d) implies (a), given A E A, let D := Y \ g[A]. Then we can take 1D = lc for some C E B: let C be the set where 10 > 1. Then D C C
andIDog=(lDog)*=Oa.sonA.LetB:=Y\C.Then BCg[A],and
Q(B) = 1 - Q(C) = 1 -
f 1* d(P o g '),
which by the image measure theorem (RAP, Theorem 4.1.11) equals
1-J 1*DogdP = 1- J(1Dog)*dP > P(A). Note. In (a) or (b), if the direct image g[A] E B, we could just set B := g[A]. But, for any uncountable complete separable metric space Y, there exists a complete separable metric space S (for example, a countable product N°O of copies of N) and a continuous function f from S into Y such that f [B] is not a Borel set in Y (RAP, Theorem 13.2.1, Proposition 13.2.5). If f is only required to be Borel measurable, then S can also be any uncountable complete metric space (RAP, Theorem 13.1.1).
104
Uniform Central Limit Theorems: Donsker Classes
A function g satisfying any of the four conditions in Theorem 3.4.1 will be called perfect or P perfect. Coordinate projections on a product space are, as one would hope, perfect:
3.4.2 Proposition Suppose A = X x Y, P is a product probability v x m, and g is the natural projection of A onto Y. Then g is P-perfect.
Proof Here P o g-1 =m. For any B CA let By := {x : (x, y) E B}, y e Y. If B is measurable, then by the Tonelli-Fubini theorem, for C := {y: v(By) >
0}, C is measurable, C C g[B], and P(B) < m(C), so condition (a) of Theorem 3.4.1 holds.
3.4.3 Theorem Let (0, A, P) be a probability space and (S, d) a metric , (Yn,13,,) is a measurable space, gn a space. Suppose that for n = 0, 1, perfect measurable function from 0 into Y,,, and fn a function from Y into S, where fo has separable range and is measurable. Let Qn := P o gn 1 on 13n and suppose fn o gn - fo o go in outer probability as n --* oo. Then fn = fo as n oo for fn on (Yn, 13,,, Qn)
Before this is proved, an example will be given. Recall that for any measure space (X, S, µ) and C C X, the inner measure is defined by /,t. (C)
sup{µ(A) : A E S, ACC}. 3.4.4 Proposition
Theorem 3.4.3 can fail without the hypothesis that gn be
perfect.
Proof Let C C I := [0, 1] satisfy 0 = X*(C) < X*(C) = 1 for Lebesgue measure A (RAP, Theorem 3.4.4). Let P = A*, giving a probability measure on
the Borel sets of C (RAP, Theorem 3.3.6). Let 0 = C, fo = 0, Yn = I, fn :_ II\C, and let gn be the identity from C into Y,, for all n. Then fn o gn = 0 for all n, so f, o gn -+ fo o go in outer probability (and in any other sense). Let 13,, be the Borel or-algebra on Y,, = I for each n. Let G be the identity from I into IR. Then f * dQn = f * fn da, = 1 for n > 1, while f G(fo) dQo = 0, so fn does not converge to fo in law.
After Theorem 3.4.3 is proved, it will follow that the gn in the last proof are not perfect, as can also be seen directly from condition (c) or (d) in Theorem 3.4.1.
fo o go. Let H Proof of Theorem 3.4.3 By Corollary 3.3.5, fn o g, be any bounded, continuous, real-valued function on S. Then by RAP,
3.4 Perfect functions
105
Theorem 4.1.11,
JH(ffl(gfl))dP - f H(fo(go)) dP
f
H(fo) dQo
Also,
f *H(fn(gn))dP =
f H(fn(gn))*dP
by Theorem 3.2.1
f(Hofn)*(gn)dP
by Theorem 3.4.1
f(Hofn)*dQn
(RAP, Theorem 4. 1.11)
f * H(fn) dQn
by Theorem 3.2.1,
and the theorem follows.
In Proposition 3.4.2, X x Y could be an arbitrary product probability space, but projection is a rather special function. The following fact will show that all measurable functions on reasonable domain spaces are perfect.
Recall that a law P is called tight if sup{P(K) : K compact) = 1. A set P of laws is called uniformly tight if for every e > 0 there is a compact K such that P(K) > 1 - e for all P E P. Also, a metric space (S, d) is called universally measurable (u.m.) if for every law P on the completion of S, S is measurable for the completion of P (RAP, Section 11.5). So any complete metric space, or any Borel set in its completion, is u.m.
3.4.5 Theorem
Let (S, d) be a u.m. separable metric space. Let P be a
probability measure on the Borel or -algebra of S. Then any Borel measurable function g from S into a separable metric space Y is perfect for P.
Note. In view of Appendix C, the hypothesis that S be separable is not very restrictive.
Proof Let A be any Borel set in S with P(A) > 0. Let 0 < s < P(A). By the extended Lusin theorem (Theorem D.1, Appendix D) there is a closed set F with
P(F) > 1-s/2 such that g restricted to F is continuous. Since P is tight (RAP, Theorems 11.5.1 and 7.1.3), there is a compact set K C A with P(K) > s. Then C := F fl K is compact, C C A, P(C) > 0, and g is continuous on C,
so g[C] is compact, g[C] C g[A], and (P o g 1)(g[C]) > P(C) > 0, so the conclusion follows from Theorem 3.4.1.
106
Uniform Central Limit Theorems: Donsker Classes
Let (0, A, P) be a probability space and g a measurable function from S2 into Y where (Y, 8) is a measurable space. Let Q := P o g 1 on B. Call g quasiperfect for P or P-quasiperfect if for every C C Y with g -'(C) E A, C is measurable for the completion of Q. Then the probability space (0, A, P) is called perfect if every real-valued function G on 0, measurable for the usual Borel or -algebra on R, is quasiperfect.
Example. A measurable, quasiperfect function g on a finite set need not be perfect: let X:= {al, a2, a3, a4, a5, a6}, U := {al, a2}, V :_ {a3, a4}, W {a5, a6}, A := {0, U, V, W, U U V, U U W, V U W, X), P(U) = P(V) _
P(W) = 1/3, Y := {0, 1, 21, g(al) := g(a3)
0, g(a2) := g(a5) :=
1,
g(a4) := g(a6) := 2. Let B := {0, Y). For C C Y, g-1 (C) E A if and only if C E B, so g is quasiperfect. But P(U) > 0 and g[U] does not include any nonempty set in 13, so g is not perfect.
3.4.6 Proposition Any perfect function is quasiperfect.
Proof Let C C Y, A := 9_1(C) E A. By Theorem 3.4.1 take B C g[A]
with B E 8 and Q(B) > P(A). Then B C C, so Q(B) = P(g 1(B)) < P(g 1(C)) = P(A), and Q(B) = P(A). Thus the inner measure Q,,(C) = P o g 1(C). Likewise, Q. (Y \ C) = (P o g 1) (Y \ C), so Q*(C) = (P o g 1) (C) and C is Q-completion measurable.
U
3.5 Almost surely convergent realizations First let's recall a theorem of Skorokhod (RAP, Theorem 11.7.2): if (S, d) is a complete separable metric space, and Pn are laws on S converging to a law Po, then on some probability space there exist S-valued measurable functions X, such that P, for all n and X -+ Xo almost surely. This section will prove an extension of Skorokhod's theorem to our current setup. Having almost uniformly convergent realizations shows that the definition of convergence in law for random elements is reasonable. The realizations will be useful in some later proofs on convergence in law.
Suppose f, = fo where f, are random elements, in other words functions not necessarily measurable except for n = 0, defined on some probability spaces (52,,, into a possibly nonseparable metric space S. We want to find random elements Y "having the same laws" as f, for each n such that Yo almost surely or better, almost uniformly. At first look it isn't clear Yn what "having the same laws" should mean for random elements fn, n > 1, not
3.5 Almost surely convergent realizations
107
having laws defined on any nontrivial or-algebra. A way that turns out to work is to define Y, = fn o gn where gn are functions from some other probability space 0 with probability measure Q into Stn such that each gn is measurable and Q o gn1 = Q, for each n. Thus the argument of fn will have the same law Qn as before. It turns out, moreover, that the gn should be not only measurable but perfect. Before stating the theorem, here is an example to show that there may really be no way to define a or -algebra on S on which laws could be defined and yield an equivalence as in the next theorem, even if S is a finite set.
Example. Let (Xn, An, Qn) = ([0, 1], B, A) for all n (), = Lebesgue measure, 13 = Borel a-algebra ). Take sets C(n) C [0, 1] with O = A*(C(n)) < .X*(C(n)) = 1/n2 (RAP, Theorem 3.4.4). Let S be the two-point space {0, 11 with usual metric. Then f,, := lC(n) -* 0 in law and almost uniformly, but each "law" On := Qn o fn 1 is only defined on the trivial a-algebra {0, S). The only larger or -algebra on S is 2S, but no On for n > 1 is defined on 2S.
3.5.1 Theorem Let (S, d) be any metric space, (Xn, An, Qn) any probability . Suppose fo has spaces, and fn a function from Xn into S for each n = 0, 1, separable range So and is measurable (for the Borel a -algebra on So). Then fn = fo if and only if there exist a probability space (S2, S, Q) and perfect , such measurable functions gn from (S2, S) to (Xn, An) for each n = 0, 1, fo o go almost uniformly that Q o gn 1 = Qn on An for each n and fn o gn as n -+ oo. Notes. Proposition 3.4.4 and the "if and only if " in Theorem 3.5.1 show that the hypothesis that gn be perfect can't just be dropped from the theorem.
Proof "If" follows from Proposition 3.3.1 and Theorem 3.4.3. "Only if" will be proved very much as in RAP, Theorem 11.7.2. Let S2 be the Cartesian product fl 0Xn x In where each In is a copy of [0, 1]. Here gn will be the natural projection of 0 onto Xn for each n. Let P := Qo o fo 1 on the Borel or -algebra of S, concentrated in the separable subset So. A set B C S will be called a continuity set (RAP, Section 11.1) for P if P(dB) = 0 where 8B is the boundary of B. Then,
3.5.2 Lemma For any e > 0 there are disjoint open continuity sets Uj, j = 1, , J, for some J < oo, where for each j, diam Uj := sup{d(x, y) :
x,yEUj)<e,andwith 1:;J= 1P(Uj)>1-e.
108
Uniform Central Limit Theorems: Donsker Classes
Proof Let {xjf°° 1 be dense in So. Let B(x, r) :_ {y E S: d(x, y) < r} for 0 < r < oo and x E So. Then B(xj, r) is a continuity set of P for all but at most countably many values of r. Choose rj with e/3 < rj < e/2 such that B(xj, rj) is a continuity set of P for each j. The continuity sets form an algebra (RAP, Proposition 11.1.4). Let
U j := B(xj, rj) \ U{y: d(xi, y) < r,}. i<j
Then Uj are disjoint open continuity sets of diameters < e with
1, so there isaJ
1
P(Uj) _
1P(Uj)> 1-e.
Now to continue the proof of Theorem 3.5.1, for each k = 1, 2, - -, by the last lemma take disjoint open continuity sets Ukj := U(k, j) of P for j = 1, 2, .. -, Jk := J(k) < oo, with diam(Ukj) < 1/k, P(Ukj) > 0, and J(k)
E P(Ukj) > 1-2 -k.
(3.5.3)
j=1
For any open set U in S with complement F, let d(x, F) := inf{d(x, y) y E F). For r = 1, 2, - -, let Fr := {x : d (x, F) > 1/r}. Then Fr is closed and Fr T U as r -* oo. There is a continuous hr on S with 0 < hr < 1, hr = 1 on Fr, and hr = 0 outside F2r: let hr(X) := min(l, max(0, 2rd(x, F) - 1)). For each j and k, let F(k, j) := S \ Ukj. Take r := r(k, j) large enough so that P(F(k, f)r) > (1 - 2-k)P(Ukj). Let hkj be the hr as defined above for such an r and Hkj the her. For n large enough, say for n > nk, we have
jhki(fn)dQn > (1-2-k)P(Ukj) and
f* Hkj(fn)dQn < (1+2-k)P(Ukj) for all j = 1, - - -, Jk. We may assume n1 < n2 < ... . For every n = 0, 1, - , let fkjn := (hkj o fn)* for Qn, so that by Theorem - -
3.2.1,
fhki(fn)dQn =
ffkJndQn,
0 < ,fkjn < hkj(fn),
and fkjn is An-measurable. For n > 1 let Bkjn :_ (.fkjn > 0) E A. Let Bkjo
1(Ukj) E Ao. For each k and n, the Bkjn C f-1
for j= 1,---,Jk,andHkj(fn)=Ion Bkjn, soforn>nk, (3.5.4)
(1-2-k)P(Ukj) < Qn(Bkjn) < (1+2-k)P(Ukj),
are disjoint
3.5 Almost surely convergent realizations
109
and Qo(Bk jo) = P(Ukj). Let Tn := X, x In. Let it, be the product law Q, x .X on Tn where A is Lebesgue measure on the Borel or-algebra B in In. For each
k> 1, Ckjn
:= Bkjn x [0, F(k, j, n)] C T,,
Dkjn
:= Bkjo x [0, G(k, j, n)] C To,
where F and G are defined so that
An(Ckjn) = Ito(Dkjn) = min(Qn(Bkjn), Qo(Bkjo))
Then for each k, j, and n > nk,wehave by(3.5.4),since 1/(1+2-k) > 1-2-k,
(3.5.5) 1-2 -k < min(F, G)(k, j, n) < max(F, G)(k, j, n) = 1. Let J(k) Ckon
7'n \ U Ckjn,
J(k)
DkOn = To \ U Dkjn.
j=1
j=1
For k = 0let Jo := J(0) := 0, Coon Tn, Doon := To, and no := 0. For each n = 1, 2, , let k(n) be the unique k such that nk < n < nk+1 Then for n > 1, Tn is the disjoint union of sets Wnj Ck(n)jn, j = 0, 1, (3.5.6)
, Jk(n). We also have
If j > 1 and (v, s) E Wn j then V E Bk(n) jn so fn (v, S) E Uk(n) j
Next, To is the disjoint union of sets Enj := Dk(n) jn. Then µn (Wnj) = Ito(Enj)
for each n and j, and if j > 1 or k(n) = 0, then Ao(En j) > 0. For x in To and each n, (3.5.7)
let j (n, x) be the j such that x E En j .
Let L := (x E To : so(Enj(n,x)) > 0 for all n). Then To \ L C IJi En(i)o for some (possibly empty or finite) sequence n (i) such that po(En(1)o) = 0 for all i. Thus µo(L) = 1. For X E L and any measurable set B C Tn, in other words B is in the product
or-algebra An 0 B, let j := j(n, x), recall that Itn(Wnj) = t o(Enj), and let (3.5.8)
Pnj(B) := An(B fl Wnj)/Ito(Enj),
Pnx
Pnj(n,x).
Then Pnx is a probability measure on An 0 B. Let px be the product measure 1 -100 Pnx on T := IIn° 1 T, (RAP, Theorem 8.2.2).
Uniform Central Limit Theorems: Donsker Classes
110
3.5.9 Lemma For any measurable set H C T (for the infinite product a algebra with A® ® 13 on each factor), x i-+ px(H) is measurable on (To, Ao ®13).
Proof Let H be the collection of all H for which the assertion holds. Given n, Pnx is one of finitely many laws, each obtained for x in a measurable subset
En j. Thus if Y, is the natural projection of T onto Tn and H = Y,- 1(B) for some B E A®®13 then H E H. If H= ITEM Ym(i)(Bi), where Bi E An(i) 0 13 and M is finite, we may assume the m (i) are distinct. Then I
px(H) = niEMpx(Yn(i)(Bi)), so H E R. Then, any finite, disjoint union of such intersections is in 7-l. Such unions form an algebra. If Hn E 7-1 and Hn T H or Hn ,. H, then H E 7-1. Since the smallest monotone class containing an algebra is a a-algebra (RAP, Theorem 4.4.2), the lemma follows.
Now returning to the proof of Theorem 3.5.1, S2 = To x T. For any set C C S2, measurable for the product a-algebra, and x E To, let C, :_ l y E T : (x, y) E C}, and Q(C) := f px (Cx) dµo(x). Here x r-+ px (Cx) is measurable
if C is a finite union of products Ai x Fi where Ai E Ao ® 13 and Fi is product measurable in T. Such a union equals a disjoint union (RAP, Proposition 3.2.2). Thus by monotone classes again, x i-+ px(Cx) is measurable on To for
any product-measurable set C C Q. Thus Q is defined. It is then clearly a countably additive probability measure.
Let p be the natural projection of T onto Xn. Recall that Pnx = Pnj for all x E En j, by (3.5.7) and (3.5.8). The marginal of Q on X,,, in other words Q o gn 1, is by (3.5.8) again J(k(n))
E lso(Ej)Pnj o p-1 = An o
P-1
= Qn
j=0
Thus Q has marginal Qn on Xn for each n, as desired. By (3.5.3), 00
J(k)
E Q0 X 0 \ U f 1(Ukj) < 1: 2-k < 00. k=1
j=1
k
l
So Qo-almost every y E X o belongs to U f0 1(Uk j) for all large enough k. Also if t E 10 and t < 1, then by (3.5.5), t < G(k, j, n) for all j > 1 as soon
3.6 Conditions equivalent to convergence in law
111
as 1 - 2-k > t and n > nk. Thus for µo-almost all (y, t), there is an in such that (y, t) E U (k(n)) En j for all n > m. If x := (y, t) E En j for j > 1, then y E Bk(n) jo, so fo(y) E Uk(n) j. Also, by (3.5.8), Pnx = Pn j is concentrated in Wnj. For (v, s) E Wnj, fn (V) E Uk(n) j by (3.5.6). Since diam(Ukj) < 1/k for
each j > 1, Q*(d(fn(gn), .Po(go)) > 1/k(n) for some n > m)
< µo({(y, t) E Eno for some n > m}) -+ 0 as in -* oo, so fn (gn) -* fo (go) almost uniformly. Lastly, let's show that the gn are perfect. Suppose Q(A) > 0 for some A.
Now Q(A) = f px(Ax) dµo(x). Firstletn > 1. Thenforsomex, px(Ax) > 0. If µo(Eno) = 0, we take x Eno. Now T = Tn x T(n) where T(n) :_ ff1<(0nT,. Then on T, px = Pnx x Qnx for a law Qnx = fImonPmx on T(n). Let A (x) := A. By the Tonelli-Fubini theorem, px (Ax) = ff 1A(x) (u, v) dPnx (u) dQnx (v) Thus for some v, f 1 A(x) (u, v) dPnx (u) > 0. Choose and fix such a v as well as
x. Now Pnx = Pn j for j = j (n, x) with µo(En j) > 0. Let u = (s, t), s E Xn, and t E In. Then since Pn j = Qn x A restricted to a set of positive measure and normalized,
0 < ff1A(X)(stv)dQfl(s)dt. Choose and fix a t with 0 < f IA(x)(s, t, v)dQn(s). Let C IS E Xn (s, t, v) E Ax}. Then Qn(C) > 0. Clearly C C gn[A], so gn is perfect for n > 1 by Theorem 3.4.1. To show go is perfect, we have µo = Qo x ,., and px (Ax) > 0 for x = (y, t)
in a set X with Ao(X) > 0. There is a t E Io such that Qo(C) > 0 where C := {y: (y, t) E X). Then C C go[A], so go is perfect, finishing the proof of Theorem 3.5.1.
3.6 Conditions equivalent to convergence in law Conditions equivalent to convergence of laws on separable metric spaces are given in the portmanteau theorem (RAP, Theorem 11.1.1) and metrization theorem (RAP, Theorem 11.3.3). Here, the conditions will be extended to general random elements for the theory being developed in this chapter. For any probability space (S2, A, P) and real-valued function f on S2 let
E*f := f * f dP, E* f := f* f dP. If (S, d) is a metric space and f is a
112
Uniform Central Limit Theorems: Donsker Classes
real-valued function on S, recall (RAP, Section 11.2) that the Lipschitz seminorm of f is defined by
IIflIL
Sup{If(x) - f(y)Ild(x, y) : x 0 y)
and f is called a Lipschitz function if II f 11L < oo. The bounded Lipschitz := sup., If(x)j. norm is defined by II f II BL II f II L + II f II where Iif II Then f is called a bounded Lipschitz function if II f II BL < oo, and 11 II BL is a norm on the space of all such functions.
The extended portmanteau theorem about to be proved is an adaptation of RAP, Theorem 11.1.1, and some further facts based on the last section (Theorem
3.5.1). The proof to be given includes relatively easy implications, some of which consist of putting in stars at appropriate places in the proofs in RAP.
3.6.1 Theorem Let (S, d) be any metric space. F o r n = 0, 1 , 2, , let (Xn, An, Q,) be a probability space and fn a function from Xn into S. SupQo o f0 1 on S. pose fo has separable range So and is measurable. Let P Then the following are equivalent: (a) fn = fo;
(a') lim supn,, E*G(fn) < EG(fo) for each bounded continuous realvalued G on S; (b) E*G (fn) --* EG (fo) as n -+ oo for every bounded Lipschitzfunction
GonS; (b') (a') holds for all bounded Lipschitz G on S;
(c) sup{IE*G(fn) - EG(fo)I: IIG1IBL < 1) - Oasn - oo; (d) for any closed F C S, P(F) > lim supn,oo Qn (fn E F); (e) for any open U C S, P(U) < liminfn,oo(Qn)*(fn E U);
(f) for any continuity set A of P in S, Q* (fn E A) - P(A) and (Qn)*(fn E A) -* P(A) as n -* oo; (g) there exist a probability space (S2, S, Q) and measurable functions gn from S2 into Xn and hn from S2 into S such that the gn are perfect,
Qogn1 = Qn and Qohn1 = Pforalln, andd(fn ogn,hn) --). 0 almost uniformly.
Moreover, (g) remains equivalent if any of the following changes are made in it: "almost uniformly" can be replaced by "in outer probability"; we can take hn = fo o yn for some measurable functions yn from S2 into X0, which can be taken to be perfect; and we can take yn to be all the same, yn = yl for all n.
3.6 Conditions equivalent to convergence in law
113
Proof Clearly (a) implies (a'). Conversely, by interchanging G and -G, (a') implies
liminfn*cE*G(fn) > liminfn,,,,E*G(fn) > EG(fo), and (a) follows, so (a) and (a') are equivalent.
Clearly (a) implies (b), which is equivalent to (b') just as (a) is to (a'). To show (b) implies (c), let T be the completion of S. Then all the fn take values in T. Each bounded Lipschitz function G on S extends uniquely to such a function on T, and the functions G o fn on Xn are exactly the same. So we can assume in this step that S and So are complete. Let E > 0. By Ulam's theorem (RAP, Theorem 7.1.4), take a compact K C So with P(K) = Qo(fo E K) > 1 - E. Recall that d (x, K) := inf {d (x, y) :
y E K) and KE := Ix: d(x, K) < E}. Let g(x) := max(O, 1-d(x, K)/e) for 1 + 1/E < oo. Clearly 1K < g < 1KE. Since E*g(fn) -). Eg(fo) as n -+ oo, it follows that x E S. Then g is a bounded Lipschitz function with II9IIBL for n large enough (3.6.2)
(Q,,). (f, E K5) > E*g(fn) > Eg(fo) - E > I - 2E.
Let B be the set of all G on S with II G II BL < 1. Then the functions in B are uniformly equicontinuous, so their restrictions to K are totally bounded for the supremum distance over K by the Arzela-Ascoli theorem (RAP, Theorem 2.4.7). Let G I, , G j for some finite J be functions in B such that for each
G E B, supXEK I(G - Gj)(x)I < E for some j = 1, , J. Next, for any G E B, choose such a j. Then (3.6.3)
I E*G(fn) - EG(fo)I < IE*G(fn) - E*Gj(fn)I + I E*(Gj(fn)) - E(Gj(fo))I + I E(Gj(fo) - G(fo))I,
a sum of three terms. For the last term, splitting the integral into two parts according as fo c K or not, the first part is bounded by E and the second by 2E since I G j - G I < 2 everywhere and Qo (fo K) < E. The middle term on the right side of (3.6.3) is bounded above by
maxj<j IE*Gj(fn) - EGj(fo)I, which is less than E for n large enough by (b). The first term on the right in (3.6.3) is a sum of two parts, one over a measurable subset An of Xn where f, E KE with Q, (A,) > 1- 2E by (3.6.2), and the other over the complement of A. Since G and G j E B, and I G - G j I < E on
K, we haveIG - GjI <3s on K6. It follows that I (Go fn)*-(Gjo fn)*I <3E
Uniform Central Limit Theorems: Donsker Classes
114
on An, so the first part is bounded above by 3e. The second part is bounded by 2(1 - Qn(An)) < 46. So for n large enough the left side of (3.6.3) is less than 116 uniformly for G E B, and (c) follows. Clearly (c) implies (b), so (b) and (c) are equivalent. Next, (b) implies (d) by taking bounded Lipschitz functions gk decreasing down to 1 F as k - oo, specifically the functions g in the proof of (b) implies (c) with F in place of K
and 6 = 1/k. (d) and (e) are equivalent by taking complements. (d) and (e) together easily imply (f). For any bounded continuous function G, all but countably many of the sets
f G < t) are continuity sets of P. It follows that G can be approximated uniformly within e by a simple function F_ ti 1 Ai where Ai are continuity sets {ti-1 < G < ti}. So (f) implies (a), and (a) through (f) are all equivalent. Theorem 3.5.1 says that (a) implies (g), in the strongest form where hn
h 1 - fo o yl and yi is perfect. The weakest form of (g), with convergence in outer probability and hn depending on n and not necessarily a composition of fo with any (perfect) function,
will be shown to imply (b'). Given 6 > 0, take n large enough so that
Q*{d(.fn o gn, hn) > 6) < 6. n So on a measurable set An with Q(An) > 1 - e, we have d(fn o gn, hn) < e, and thus for any G E B (in other words II G II BL < 1) we have
IG(fn(gn)) -G(hn)I < 61A
+2. IA'-
It follows that
E*G(fn(gn)) < EG(hn)+36 = EG(fo)+3e (since hn has the same distribution as fo) and
E*G(fn (gn)) = E(((G o fn) o gn)*) = E((G o fn)* o gn) by Theorem 3.2.1 and since gn is perfect, and where all the expectations are with respect to Q. The latter integral, by the image measure theorem (RAP, Theorem 4.1.11), equals E*(G(fn)) (for Qn). Letting e 4. 0, (b') follows, so all of the forms of (g) are equivalent to any of (a) through (f). Theorem 1.1.1, in a special case, proved a form of convergence which in that case is now easily seen to imply the conditions in Theorem 3.6.1: the one in (c), for example. It will be shown that convergence in law is also equivalent to convergence in some analogues of the Prokhorov and dual-bounded-Lipschitz metrics, which
3.6 Conditions equivalent to convergence in law
115
metrize convergence of laws on separable metric spaces as shown in RAP, Theorem 11.3.3. We will have analogues of metrics, rather than actual metrics, because of the asymmetry between the nonmeasurable random elements f, and the limiting measurable random variable fo.
Definitions. Let (X., Am , Qm) be probability spaces, m = 0, 1, and let (S, d) be a metric space. Let fm be functions from Xm into S, m = 0, 1, such that fo is measurable and has separable range. Let P := Qo o f0 1. Then let
P(.fi, fo) := sup{IE*G(.fl) - EG(fo)I :
IIGIIBL <- 11,
and for F ranging over nonempty closed subsets of S,
p(.fi, .fo) := inf{e > 0: SUPF[P(F) - (Qi)*(fi E FE)] < e}. 3.6.4 Theorem For any metric space (S, d), probability spaces (Xm, Am, Qm) and functions fm from Xm into S, where fo has separable range and is measurable, the following are equivalent: (i) .fm
fo;
(ii) $(fm, Jo)
0 as m
(iii) p(fm, fo) - 0 as m
oo; oo.
Proof (i) and (ii) are equivalent just as (a) and (c) are in Theorem 3.6.1. To show that (ii) implies (iii), we have:
For any fl and fo as in the statement of Theorem 3.6.4, p(Ji, .fo) < min(l, (20(ft,
3.6.5 Lemma
.fo))1/2).
Proof Let F be any closed set in S and P = Qo o f01. Given 0 < s < 1, let g(x) := max(0, 1 - d(x, F)/e) as before. Then 11811 BL < 1 + 1/c, and
(Q1). (f, E FE) > jg(Ji)dQi >
fg(Jo)dQo_(JiJo)(l+) > P(F) - s cases,
if (fi, fo) < r2/2. Clearly p(fl, fo) < 1 in all (2p(.fi, .fo))1/2 when the latter is < 1, the lemma follows.
so letting s
Now to continue the proof of Theorem 3.6.4, applying the lemma to fm in place of fl for each in shows that (ii) implies (iii). To show that (iii) implies , let F k := {x : d (x, y) < 1/k (i), let U be an open set in S. For k = 1 , 2,
116
Uniform Central Limit Theorems: Donsker Classes
implies y E U). Then Fk are closed sets and Fk T U as k -f oo. For n large enough, p(fn, fo) < 1/k, and then P(Fk) < (Qn),k(fn E F1Ik) + k < (Qn)*(fn E U) + k, SO
P(Fk) < liminfn->oo(Qn)*(fn E U) + k Then letting k ->. oo gives P(U) < lim infn oo(Qn). (fn E U). This is (e) of Theorem 3.6.1, so the proof of Theorem 3.6.4 is complete. Next is a version of Theorem 3.6.4 where we have two indices.
3.6.6 Theorem
Let (S, d) be a metric space and (S2, S, Q) a probability
space. Suppose that f o r each m, n = 1 , 2, , fmn is a function from Q into S, and fo is a measurable function from Q into S with separable range. Then the following are equivalent:
(i) fm,n = fo, that is, for every bounded continuous real function G on S,
I*G(fm,n)dQ
f G (.f0) dQ asm,n -+ oo;
(ii) f(fmn, fo) - O as m, n -> oo; (iii) p(fmn, fo) -+ 0 as m, n -+ oo. Proof For a double sequence amn of real numbers, let lim SUpm
n,oo amn =
infk SUpm>k,n>k amn,
lim infm, n-#oo amn = SUpk lnfm>k, n>k amn
Then the steps in the proof of Theorem 3.6.1 showing that (a) through (f) are equivalent extend directly to convergence as m -+ oo and n -+ oo if lim supra,,, is replaced by lim Supm,n,oo (not by lim supm,o lim Supra,,, or the iterated lim sup in the reverse order, which may be different), and likewise for lim inf. Thus, the proof of Theorem 3.6.4 also extends.
We have the following:
3.6.7 Continuous mapping theorem Under the conditions of Theorem 3.6.4, if (T, e) is another metric space, G is a continuous function from S into T, and fo, then G(fm) G(fo) .fm Proof This follows directly from the definition of convergence in law =.
3.7 Asymptotic equicontinuity and Donsker classes
117
We also have
3.6.8 Theorem Suppose fm, m > 0, are measurable random variables taking values in a separable metric space S, so that laws G(fm) exist on the Borel or -algebra of S. Then convergence fm
fo is equivalent to convergence
of the laws G(fm) -+ G(o) in the usual sense. Proof For any bounded continuous real-valued function G on S, the functions G(fm) are measurable, so upper integrals reduce to integrals, and the result follows from the definitions.
3.7 Asymptotic equicontinuity and Donsker classes Recall from Section 3.1 the definitions of the empirical measures P, empirical process v,,, pseudometric pp, and the Gaussian process Gp. Recall that a set .F C 'C2 (p) is called pregaussian if a Gp process restricted to F exists whose
sample functions f i-+ Gp (f) ((o) are (almost) all bounded and uniformly continuous for pp on F. Such a Gp process is called coherent if, in addition, its sample functions are prelinear on .F as in Lemma 2.5.3.
3.7.1 Proposition For any probability measure P and pregaussian set F C G2(P), the symmetric convex hull sco(.F) of.F is also pregaussian, and there exists a Gp process defined on the linear span of .F and constant functions which is 0 on the constant functions and coherent on sco(.F). Proof A Gp process is isonormal on the space L2 (P) for the semi-inner product
(f, g)o,p := P(fg) - PfPg, and Gp is 0 on constant functions. So Theorem 2.5.5 on isonormal processes applies and gives the result. For a signed measure v and measurable function f such that f f d v is defined,
letv(f) := f fdv. A class ,F of functions will be said to satisfy the asymptotic equicontinuity
condition for P and a pseudometric r on F, or F E AEC(P, r) for short, if for every e > 0 there is a 3 > 0 and an no large enough such that for n > no,
f,gE.F, t(f,g)<S}>e} < E. Then.F E AEC(P) will mean.F E AEC(P, pp).
Uniform Central Limit Theorems: Donsker Classes
118
3.7.2 Theorem Let .T' C G2(X, A, P). Then the following are equivalent: (I) F is a Donsker class for P, in other words .F is P-pregaussian and vn
Gp in £'(F);
(II) (a) F is totally bounded for pp, and (b) F satisfies the asymptotic equicontinuity condition for P, F E AEC(P); (III) there is a pseudometric r on .F such that .F is totally bounded for r and
.T' E AEC(P, t).
Proof If F is a Donsker class and e > 0, then since F is pregaussian, it is totally bounded for pp by Theorem 2.3.5, so (a) holds. Take 0 < 3 < s/3 such that for any coherent Gp process,
Pr {sup{IGp(f) - Gp(g)I: pp(f, g) < 8} > s/31 < s/2. By almost surely convergent realizations (Theorem 3.5.1), for n > no large enough we can assume Pr*{Ilvn -GpII.p > s/3} < 8/2. If IIvn -GpII,p < s/3 and I Gp (f) - Gp (g) I < s/3 then I vn (f) - vn (g) l < s. So the asymptotic equicontinuity holds with the 3 and no chosen, and (b) holds, so (I) implies (II).
(II) implies (III) directly, with r = pp.
To show (III) implies (I), suppose F is r-totally bounded and F E AEC(P, r). Let UC := UC(F) denote the set of all real-valued functions on F uniformly continuous for T. Then UC is a separable subspace of t°O(F) for II Il.p since F is totally bounded and UC equals, in effect, the space of continuous functions on the co