This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
PsoCP> PSz
= 1. Proof(1):If
o = 1
cP\302\253
ii>\302\253
-
p\037
_\302\253
Fromt 1.7we obtainthat) d Y H o Psipy dO' 9=&0)
iscontinuous on [0,(x].It followsthat there issomej E [0,(X] such that d
_
=
Ps
d8
9=90)
p,
Thisprovesthe assertion.
0)
:
for all (x, p E [0,1] there donotexisttests 13.2Remark. Let 90 E e.Ingeneral, = p. Hencethere neednot existtwocP E Y such that PsoCP= (x, dO' PalfJ 9
sidedS-testssatisfying IX
E
.. theseconditions. But for
d.
90
[0,1]there are two-sidedS-tests
Pc.= 90q>
IX,
d8P9q>)
cp* E
every
fF satisfying
80 E e and every
=0.)
8=80)
lemma choosing This followsfrom the preceding qJ =:
(X.)
13.3Lemma. Supposethat E isan exponential experimentofrank I. Let8 90
- 00< + k Sf ' (h ) a.e.
S-testsatisfying and letq>* be a two-sided k 1 e R, k2 e litsuch that
..-
cP
_
I if fa > k 1fao 2 90 {o if fa < k 1f90 + k 2 Sfao')
t1
\037
=+=
t2
<00.Then there are
v _
ProofWe have to show that there are kj E n, k 2 En, such that I if t > k'iebS + \"2SeIlS, _ (Ilv )_a.e. q> {0 if 1 < \"Ie\"s+ k'2 Se\"5, where b = .90 9 * O. the system of linearequations))) Case1:Assumethat '1< 2 , Consider
-
'
13.Two.sidedtesting for exponential experiments:Part 2
ebll + It ebll = I, It.eb/2 + k;12 ebl2 = 1 The system has a uniquesolution since) e'\" t 1 ebll ..12)(t 2 = el'111 D.= evIl) .. k't
65)
k\037
.)
-
I
12ebrz
t
I)
) > O.
The solution is)
- ( e - II') e ) I-D 1 .1t - -D(e e '2) 1
k'
t 2 bl2
bll
bl
2
lo
.)
we notethat sgn b sgnk\037. Then,there is no intervalwhere Moreover, g:t 1-+k'li\" + t eb, , I E R, \037
k\037
isconstant; in addition, the property) g'(t)(s t) 0 if g(l) g(s) that g isa quasi-concave function. Thisimplies that {g::::: of implies I}consists = at mosttwo elements and {g>1} (/ 1,/2) since{II'12} {g= 1}.Hence 1> k'ieb, + k\037tebl impliest < II or t> 12 , and 1
-
\037
\037
\037
The system has a uniquesolution since)
Ie .- be (:1 D'0 + t)e blo
CblO
blo
b10)
is) The solution
-1
(blo + l)eb'O, k'i ::::: D) k'z
=
1
be'\"\302\260.
D)))
= e2b'O(bto +1
-bl > o)
o.)
Chaptcr2:Elementary Theory of TestingHypotheses)
66
notethat sgn b * sgn and argue as in case1 : h'. t k'I e'\" + k' ((I' t E IR ,) isa quasi-concave Thusa function,and there isno intervalwhereh isconstant. localextremumof h isa strictglobalmaximum,and from h(to) = 1,h'(1o)= 0 and h\" (/ 0) t' 0 we get that 1> k 1 e + kiteblis equivalent to t * 10'This provesthe assertion. 0)
2'
We
k\037
\037
bl
13.4Theorem.Supposethat E is an exponential experimentof rank I. Let 90 e andcp .'F.If cp* E.'Fis a two-sided S-testsatisfying \342\202\254
\342\202\254
= Psocp. d p, * do sCP I}'
PSolp,)
-
= d
df:)
9\03790
P\037cP
9\03790')
then)
Pscp.2:PsCP if 9 * 90 , classof testsfor (H,K).) S-Iestsisa complete Thus, the setof alltwo-sided
'
S-testwith t l = -00.Letfo.=fso ProofCase1:Assumethat cp* isa two-sided and fl'= Sj\037. We observe,that lp* isan upperS-testand from Theorems(11.6) i t follows that is a N P-test the for (!0'!1) in senseof cp* generalized (10.4) The same is true of cp, and cp* and cp coincide Definition(10.2). (hv)-a,e,on {ft * t 2fo}.Hencecp is(hv)-equivalent to an upperS-testand from (9.9)(2) it followsthat the power functionsof cp and cp* coincide. is provedsimilarly. Case2:If t 2 + 00,then the assertion there Case3:Assumethat 00< t l t 2 <00.Let 9 * .90 , By Lemma(13.3) are k I E !R, k 2 E IR, suchthat
::
* cp
1 {o
=
if if
-
\037
fs>klfso+k2 Sfso. (hv)-a.e. fs < ktfSo+ k 2 Sfso'
we obtain From(10.3)
Pscp.
\037
PsCP if
9 =F 90 ,
0)
13.5CoroUary.Supposethat E isan exponential experimento.frankl.lfvalue
at a pointof e S-testscoincide andslopeof the powerfunctionsof two two-sided on S-testsare admissible then they coincide Moreover,two-sided for (H,K).)
e.
is an exponential experimentof rank I. Let = Assume that cp.E '\037a(H, K) {90},K {.9:[) .90}.
13.6Corollary.Supposethat (1.E
[0,1],80 E
\342\202\254o),
and H =
is a two-sided S-test.Then)))
E
=1=
13.Two-sidedlesling for exponcnlial cxperiments:Part 2 P\037tp*
= sup{Pscp: cpE .rp;a.(H, K)} if 9 =f 90
67)
,)
= 0 sincethe power K) satisfies !Fa.(H. PsoCP= (I.\037 Ps'P 3=80) has a localminimumat 90 , 0
ProofEvery
cp E
13.7Corollary.SupposethaI E is an exponential experiment01rank I. LeI (X
E
[0,1],90 E e, and H = {.90},K = {.9:9 .90}.If a lestcp E !Fa.(H,K) i.'i =4=
K) at a single optimalin S;a.(H, poill/.9 =t: .90 then it is uniformly optimalin \037a.(H, K).)
the assertion istrivial in case = 0 or = 1.Let ProofSinceE ishomogeneous E (0,1), Further let cp E !F,AH, K) and 9 E K be the pointwhere cp is optimal. a two-sided i .e. E be S-test, cp. :Fa.(H, K) I jf S < t 1 or S> t 2' v-a.e. . cp = {o If t 1 < S < t 2 . (X
(X
(X
.
-
'
(0,1),Lemma 9.7 implies 00< 'I :::::; 2 < + oc. lienee,Lemma 13.3 k l E 1R, k 2 E !R such that) givesconstants As
(X
E
* cp
- {0 _ I
This shows that
if
I> k1/80+ k 2 S/80
if f< kd8.0+ k 2 Sfso ' tp*
is a generalizedN P-testfor
(j\037o'
Sf90;
j\037).
From the
with tp* optimalityof cp it followsby Theorems10.4and 13.4that cp coincides v-a.e.on {Is k1fso + k2Sj\037}.Accordingto the proofof Lemma (13.3) the and coincide v-a.e. where t S t teststp S 1 and tp* 2' Thus,tp isa two-sided S-testand thereforeuniformlyoptimalin (H,K). :f=
=+=
=+=
0)))
!F\302\253
Chapter3: Binary Experiments)
The power and simplicityof the lemma of Neyman and Pearsongives riseto the development This theory is a of a general theory of binary experiments.
caseofthe generaldecision seemsto simple special theory,but itsisolation in severalrespects. beadvantageous Generaldecision o n relies theory powerful In contrast,the theory of binary experiinstruments of functionalanalysis. ments can be basedexclusivelyon the lemma of Neyman and Pearson.A betweenexperiments, the so-caHed decision-theoreticaHy meaningfuldistance deficiency,is easy to handlefor binary experiments. In Section 14we definethe errorfunction of a binary experiment. Section 15 containsthe discussion of the natural semi-ordering of the classof binary In particular. we show it can be expressed experiments. by the pointwise The topological structureis treatedsimilarly, orderingof errorfunctions. and convergence of binary experiments Finally,it is provedthat semiordering can alsobe expressed functions o f by power optimallevel-oc-tests. Sections 16and 17 arc concerned with equivalence classesof binary experiments and their characterization. It turns out that besides the errorfunction boththe distribution of the likelihood ratioand itsMellintransform determine A the equivalence classes iscompactness of the spaceof uniquely. consequence binary experiments. It is convenient to introduceat this point the conceptof contiguous of probabilitymeasures. theseare sequences of binary sequences Essentially, The whoseaccumulation experiments. experiments pointsare homogeneous essential in Section 18. facts are presented the of 3 is Theconcept ofcontiguity is whole well-known. Practically chapter very
dueto LeCam[1960].)
14.The error function) Consider a binary experimentE = (D,.9I, (P,
Q\302\273.)
14.1Definition.The errorfunctionof the binary experimentE is the function g:[0,1] [0,t] which is definedby g(oc)= inf [(1 oc)Pcp+ ocQ(1 cp)]t oc E COt 1].
-.
-
-
.F\037
00 we definefor reasons In the followingassertion ofconvenience: = 1.The t +00)))
14.The error function
69)
to Q and of Q with respect to Pare ofP with respect Lebesgue decompositions denotedby and M) (\037; . N).) (\037\037.
14.2Theorem.Let g be the errorfunction of E. if 0 :5k \037.
areequivalent: e:F then the followingassertions
(1)
\037.
(2)
g(IX) = (1
E
'\302\245J.;(P,
-
Q) for
\037
IX, 0
\037
IX
\037
- - 1.
= 1
k
-
<X
+ exQ(l
ex) p\037.
\037.).)
Proof (1) (2):If k = 0 then Q\037* = 1andtherefore \037
+
C 0) Pcp\" C 0) Q(I \037
-
= o.)
rp\")
\037
On the otherhand we have)
g
C ) \037o
=g(l)
Q(I-cp)
\037
O.)
\037
:\037\037
If k = 00then p\037. = 0 and therefore
C:'00)
Pcp\"+
C:00) Q(I-rp*)=
O.)
On the otherhand we have)
1 g 1+ 00 = g(O) = inf ( )
-.
P\037
= O.
tPE\037)
Let 0 < k ex
<00and =
1
l+k For any iF we have dQ dQ J dp dP+ M'r-.N' J (l-IX)-IXdP 1 [ IX
\037dP
(1-
+ ex! = (1
-
\037)dQ
IX)
The infimum isattainedfor
1 if \037.
\037E
=
0 if
+
(1- L ex)
P
(x
( 11;11 ).
- 11;11
-
seethat the preceding isnotdiflicultto assertion isalsovalid if W = 1 1ro.c), l'> O. The charm of the argument liesin the fact that it restsonly on the It
Neyman-Pearson lemma in the version of Lemma 28.1.The proof of the in its fulJ generalityrequires assertion someelementaryBayesiantechniques.
Thereforewe postpone it toSection 40.
resultsto a slightly moregeneral situation. Finally we extend the preceding = For tEL and <1 > 0 let Pr.u !f(<1id+ tl Nil)' In the followingwe consider the = teL.<1> O}). experimentF (H,fR(H). {.P,.u: For every <1 > 0 the experiment (H,fII(H),{P,.C1: teL})is of the same type as the experiment E if the vector spaceH is endowedwith the inner 1 1= <., It is natural to calJ an estimate() e .1t(F, product<.,.)\" \037
f\037
.). median-unbiased for F ismedian-unbiased for every R(e,t, (1)= II W(lx -f(t) I) e(',dx)dP'.C1'teL, i
\037)
(]
iff it
f\037,
<1
(1
>
> O. We denote) O.)))
166
Chapter 7: Theory of
Estimation)
34.6'Ibeorem.Every medianunbiased estimate (!off satisfie.'i for every tEL, (J > 0,)
-
[/(1) <,f(I)+ d]) P'..\037(., and)
R(q,t,
0\")
\037
:;; (,,:,, ) <1>
<1>
Clf\037
)'
< > 0,d> 0,
v O,1S1 (ds).) J W(lIfll'ls!)
Proof.Apply Lemma 34,2and Theorem34.3to F;,(J > O.
0)
34.7CoroUary.The estimate K. = foPL isa medianunbiased estimate off and estimates optimalamongallmedianunbiased off)
35.Mean unbiasedestimation) that e 0 is an arbitrary setand E = (fl,.flI,{Ps:.9 E e})an experiSuppose ment in S(8).Denote = {Ps: 8 E 8). For the purposeof thissectionthe of n !R2 (Q,d,Ps) are calledestimates. elements =*=
(fjJ
91'9)
35.1Definition.A
e
admitsan unbiasedestimateK if K = 9 E e. The set of all (mean)-unbiased estimates off is satisfies f(9) Ps(K), denotedby H(f).) function f
\037!R
An esti35.2Definition.Supposethat f e -+ R admitsunbiasedestimates. mate K* E H(f) iscalledan unbiased estimate ofWliformly minimalvariance
-
f(9\302\2732)
PS\302\253K.
= inf \"E
Ps\302\253K
if)
-
f(9\302\2732),
ll(f))
From Steiner'sformula it followsthat for K2 E HCn,and every 8 E
e
-
f(8\302\2732)
\037\302\253Kl
\037
PS\302\253K2
35.3Lemma. If K and K2 K = K2 Ps-a.e., 8E e.) 1
in
-
f(8\302\2732)
iff
9E
e.
estimatesK 1 E H(!),
any two
Ps(KD
\037
Ps(K\037).)
H(f) are of uniformly minimalvariance then
1
ProofSince (K1 + K2) E H(f) and) \037
J
K1
(
1 2 1 2 + K2 2 K dP 9E e,))) dPs KldPS+ J 2 J 2 s, 2 ) 2 \037
35.Mean unbiasedcslimation it
167)
followsthat this inequalityis even an equality and that
-
j(KI K2)2dPs= 0, ge
e.
0)
35.4l.emma.Supposethat .9I .91is E-sufficient.Forany K e H(f) let ge E(Kldl ) = Ps(Kldl)Ps.-a.e., Then E(KldI) e J/(j) and t
\302\243;
e.
2 ge e.) t )2) $ Ps(K ), Pa(E(KI.9I
of Jensen's inequality. ProofThis is an immediateconsequence
0)
if for every estimate K An experimentE iscalled 2-complete
e.
9E 0, ,9 E e impliesK = 0, Ps-a.e., Thisisa slightly weakercondition than completeness, definedin Theorem16.4.) Ps K =
and Lehmannand Scheffe[1950]). 35.5Theorem(Blackwell[1947]. Suppose
d)
is2-complete. that .r.() .r.(isE-sufficientand that EI If H(f) i.'t an unbia.'ted estimate 0/ uniformly minimalvarianceK*, and K* \037
K*
=
* 0 then there satisfies
Ps(Kldl ) Ps-a.e.,ge e,KE H(f).)
of H(f).Since Proof Let KI and K 2 be elements if gee, f(E(Klldl)-E(K2Id))dPs=0 of d l implies that E(Ktldl ) = E(K2 Idl ) Ps-a.e., ge completeness there is a function K* E H(f) satisfying
e.Hence
= E(KI..r.f 9E e, whenever KE H(f).) t) Ps-a.e., to the preceding lemma,K* is of uniformlyminimalvariance, According K*
0)
estimateK is an unbiasedestimateof In otherwords,every .9I1-measurable unifonnly minimalvarianceof itsmeanf: 91-+PS(K),9 E e.) Then 35.6Corollary.Supposethat the conditions are .';atisfied. of Theorem35.5 the unbiased estimateof minimalvarianceK* e H (f) satisfies)
s J W(K)dPs, 8 E J W(K*)dP \037
for every convexfunctionW: R
-
IR
H(f),) which is bounded frombelow.)
Proof The assertionfollows from inequality. 0)))
e,
K*
K
=
E
Ps(Kldt ) Ps-a.e.,and Jensen's
168
Chapler7: Theory of Eslimalion)
35.7Example.Let (H,( ., . bea Euclideanspace,L H a linearsubspace. \037
\302\273
sufficient mapping for the experiment E PL is a = (H,(JI(H),{NH*tQ:aEL}). Let us show that 9= {NIl*cQ:aEL} is 1 0 = = for the a-field.r41 pi Porthis,let g h PL besuch that 2-complete = for all L. This E that) 0 a \037g implies
Then
(\037(H\302\273.
o=
\037g
II h(x)e -re .-yIIxII 2
11/2
=
(21t)
=
C(a)f. hex)e<x,Q)e
dim
IIYII
2
e.
IIQIF
e<:r:.a)
2
dxdy
-\037 2 dx)
for every a E L. FromTheorem5.7we obtainthat) h+ (x)
e-u.;u:= h - (x) e-
l!!1I: 2
AI.-a.e.)
a E L. Henceevery function/0PL is an unbiased provesg = 0 \037.a.e., estimateof unifonnly minimal variance of its mean.If is linear then fo PL H(f).) which
/
\342\202\254
35.8Example.Keepingthe notationof the precedingexample,considerthe = id+ alNil)'It experimentF = (H,P.4(H),{\037.cJ:a E L, (1 > O})where x EN, is sufficientfor is again easy to show that S:x.-(pdx),!lxPI.(x)II), = a E (J > for and that is S-1 Therefore, F, L, O} 2.completc {\037.a:
-
\037
\037.cJ
.9\"\"'\302\253(1
(\302\243B(H\302\273.
estimates of unifonnly minimalvarianceare exactlythe estimates the unbiased K which dependon x E II through S. For short,Jet us callany unbiasedestimate of uniformlyminimalvariance Theorem35.5statesthat under the conditions imposedthere simply optimal. the setofall optimalestimates isa linearspace,which isgeneratedby a q-field. of Theorem35.5are necessary. It is our aim to show that the assumptions we information on the structure of the setof want to obtain some Moreover, in such caseswhen the assumptions of Theorem35.5are not optimalestimates satisfied.)
K* 35.9I.emma(Rao[1952]).An estimate optimaliff Ps(K*v) = 0for every i.\"
9 e e and every ve H(0).)
Proof An estimateK* is optimalifT for every 8 E e and every v E H(O) the function A.'+ ).V)2), e R, attainsits minimumat = O. This is the A.
\037\302\253K*
caseiff Ps(K*v) = O.
A.
0)
K such that there exists an optimalunbiased Let D be the setof all estimates H be a If K Diet 1t(K) version of the optimal estimateof 9 Ps(K), 9 \342\202\254
e.
estimate of 91--+Ps(K), .9 E e.)))
\342\202\254
35.Mean unbiased estimation
169)
35.10Lemma. ThesetD isa /inearspaceandn isa linearidempotentmapping. is a /inear space.too.) The.'letxeD)of alloptimalestimates in D and let i.., Ji E 'R. It followsfrom Lemma Proof Let \"1'\"2 be estimates that is an + optimal estimate.This proves the (35.9) ;,X(\"I) JlX(\"2)
assertion.
0) .rtf, n !{'2(!1,
In the followingwe topologize the spaceof estimates family
of semi-nonns K t-+ Ps(K2)1/2, 9E
e.
I\037)
by the
\037t:8)
35.11Lemma. The /inear spacex(D)isclosedand x: D -+ x(D);s uniformly
continuous.)
Proof FromLemma 35.9it isclearthat n(D)isclosed.Thus we need only Let \"I'K2 be arbitrary clements of D. provethat x is unifonnly continuous.
Then)
\037\302\253n(KI)
- n(K
2
\302\2732)
=
P9\302\25371(K
\037
P9\302\253K
which provesthe assertion.
1
1
-
K2\302\2732)
K2)2),
9 E e,)
0)
the class..w Now we introduce 0:={A E.9I:l A E n(D)}.It followsfrom Lemma
35.9that
A
E
.vI0 iff J vdPs =
0 whenever
vE
H(O), 9 E
e.
A)
35.12Lemma. (1).910 isa (I-field. (2) (3) (4)
e.\037timate is in 7l(D). Every.91 o-mea.\037urahle EI.9I0 2-complete. If .911 is E-sufficientthen .910 .91 for er;ery9 E e.) i.\037
\037
J
(P\037)
if A E .vI0 Proof (1) It is easy to seethat sf0 is a Dynkin system. Moreover, and a (I-field. .s.I0 ist1-stable then 1A V E H(0) if v E H(0).Therefore, usingLemma 35.9. (2) This is provedby induction, If P9 (K) = 0 for every 9 E e then estimate. (3) Let K be an .91 o-measurable K
E
H(O) and hence
J KdP9 = 0 whenever
A
E
.910 , 9E
e.
A)
e.
It followsthat\"= 0 P9-a.e.,9 E This meansthat EIsf0 is 2-complete. If A E.9I0 then 104 E n(D)and (4) Assumethat .911 isan E-sufficient(I-field. from Lemma 35.4we obtainthat alsoE(1A1.91 1) E xeD).lieneewe have I...= E(1AI.s.I 1)PS-a.e.,9E e.)))
170
Chapter 7: Theory of
Estimation)
Ifwe denoteB= {E(IA I.Iit.)= 1},then BE .Iit.and A = B
(J\037\037),
e.
.9E
0)
shows that the mapping1t is almosta conditional The followingassertion
expectation.)
35.13Theorem(Bahadur[1957]).Themappingn:D -+ n(D)has thefal/owing
properties: then Kh E D and 1t(Kh) (1) If KEDand if h is.91o-measurableand bounded. = 1t(K)h. (2)
If
KE D and
8E e.)
then n(K) = if n(K) is do-measurable,
P\037(KI.9Io)P\037-a.e.,
and bounded. Firstwe notethat) Proof(1) Let\"ED and h bedo-measurable s = 0, 8 E e.) J (Kh 1t(K)h)dPs= J (K
-
-
sinceK
- (,,) H(0).It follows 1t
E
g:9...... Ps(Kh). If v E
1t(K\302\273hdP
that n (Ie). h H(O) then hv E R(O) and
is a mean-unbiased estimate of therefore
0, 8 E e.) Thisimplies that n (K). h is optimal. s= J n(K)hvdP
(2) The propertyjustprovedimpliesthat for KEDand
A E
s = J n(K) l...dPs = J n(\" l...)dP s ;;;;; J \"dPs, 9 E J 1t(K)dP
A
.910 we have
e.
A)
it is a version of Hence,if n (K) is o-measurable, .QSf
\037
(K I.QSf 0)'
0)
The second assertion of Theorem35.13 leadsto the question, whethera given
is .91o-measurable.) optimalestimate
then it is 35.14Theorem(Bahadur[1957]).If an optimalestimateisbounded,
.r;Io-measurable.)
ThenKV E H(O)for every v E H(O). ProofLet K bea boundedoptimalestimate. we obtainthat K isoptimalfor all n E N. Ifp isa polynomial on By induction R then po\"is optimal. SinceK(D) iscompactthe Stone-Weierstrass theorem If U iii isopen impliesthat cp 0 \" isoptimalwhenevercp:!R -+ IR is continuous. 0 then it followsthat lu K is optimaland therefore{K E U}E .Iito. HenceK is .91o-measurable. 0) ft
\037
converseof Theorem35.5.) Now we arrive at the desired
35.15Theorem (Bahadur[1957]).Assume that E is dominated.If for every)))
35.Mean unbiasedcstimalion function which
admit.v
171)
a bountkdunbiasedestimatethere (\037xivt.v an optimal
estimate then ..cI 0 is E-sufficient.)
Proof We prove that .910 is pairwise sufficientfor E. The rest followsfrom Theorem (20.4).Let PI and Pz be arbitrary elements of r!J= {Ps:9 e 8}. such that Q:= L 2-1 Completethe pair(PI'P2) to a sequence (P,J\"\"N d is equivalentwith Then the densitiesfi , i = 1,2,can be assumedto be dQ bounded. \037
\037
\037
l\342\202\254N
\037.
We have
i=
Let h,= n , j = I, 2, are d.-measurable.
to show that \037ci
1,2.Then Ps(hiv) = 0
if
8 E e,
ve H(O).On the otherhand we have v E H(0).This implies \037
Q((h.-
\037ci
But hi
0,
VE
\037ci
).
\037(v)
= 0 and hence Q
v
dR
( dQ)
= 0,
H(O).
- :ciisitself
an elementof H(0) which implies)
Q((h,-
= O.
\037ci
''IT vye
)V)
(
ve H(O),and thereforeQ(hjv) = 0,
. 0btam
t h at
),)
. dfi.IS optima and I
measurable. 0) dQ
.
m view 0f ItS b ound ed ness sI0-
35.16Corollary.Assume that E is dominated. Iffor allA E sI the functions
9.-.P,,(A) admit optimalestimatesthen do isE-suffcient.)
to 35.15 i s estimate off Since\" bounded - -0 ProofLetf e By
R
Theorem
such that /lK.. co
K/lu
-
L /I\"..
,,=
and such that there is a boundedunbiased K off estimate we have show that there existsan optimalunbiased we may find a sequence of stepfunctions(K,J and (eventually by choosing a subsequence))
\"\"+l/1u
<00.
1)
By
for every Kn there is an optimalestimateK: of [) assumption,
9 E 8,n E N. It followsthat for every 9 E e /I
-
-
Ie: \":+1112,9 /I \".. \037
\037
and thus) t;I)
L 11=1)))
-\"..
\"\"+1/12,.9II \"..
.9Ee. /lK:-\":.llb,9<00,
+
ll1u,
n
EN,
\037
Ps (\"..),
172
Chapter 7: Theory of
Let Po E
Estimation)
C(9)such that 96 --Po.Then,obviously,) 00
L IIK\037-K\037+llb.Po
and it followsthat (,,\037) has a Po-a.e. limit \"*.Now, it
,,*is an optimalunbiasedestimate off
isnotdifficult toseethat
0)
36.Estimationby desintegration)
that (1',d) is a metric spacewhich is locallycompact and separable, Suppose the we a Let &f(T) be its BorelO'-field. For following, choose Borelmeasure < if K Tiscompact, and keepit fixed.Let us t hat means 00 p(K) plfj(T), = an experimentE (a,d, {p,: t E T})in 4 (1'). consider TheexperimentE iscalledf1I(T)-mea.\037urab/eif \037(A) is.1J(T)-measurab for every A E d. If E is .\037(T)-measurable then we may define the measure pep):A...... J P, (A) p(dt),A E ,rd.TheexperimentEiscalledweakly continuousif I H P'(A) iscontinuous for every A E d.Sincein this caseE is separable, the v with set9 = {p,:1E T} isdominatedand for every O'-finitemeasurev 1.91 the function (w, t) H (w), (w, I) E a x 1', can be chosens1x \037(T)\037
t.-
\037
\037
\037;
measurable (Lemma4.6).) Then E is p-inlegrable if for 36.1Definition.Supposethat E is measurable. function qJ E (Q,.91)such that every SETand every f; > 0 there is a critical PsCP< f,
-
and J P,(1 T)
!/
cp) p(tlt)<00.
if every testingproblem({s}, In otherwords,E is p-integrable sET, 1'\\{s}), admitsa testof arbitrary smalllevelwhosepower functiondiffersfrom oneby for every hounded a p-integrable function.It is clearthat E is p-integrahle
measureplat(T).)
definition 36.2Remark. ThecritiC<11functionof the preceding may bereplaced
criticalregion.Indeed.choosea criticalfunction cp such that < f.2 and P'(1 p(ti/)<00. ThenA = {q> > f,} satisfies and J P,(A')p(dt)<00. \037(A) < f, T by a
\037q>
-q\302\273
\037
to checkp-integrability criterion Thefollowingassertion isthe mostconvenient of
E.)))
36.ESlimation
36.3Theorem.A isa-jinile.)
by
desintc:gration
weakly continuousexperimentE isJ.l-integrable iff
t 73)
P(J.l)I,r# is
and let seT. From 36.2it followsthat Proof Assumethat E is J.l-integrable .s.Isuch that \037(AII) 11 and P(J.l)(An) < ex; for there is a sequence (AII)lIc N ,....,1,such that P(J.l) sf\" n is n N. Hence there a set isCTDJ E .YI, every E finite.Let S T be a countable and densesubset. Then D = U Ds issuch that seS SinceE is weakly continuous it \037(D) = t jf S E SandP(Jl)I.r#n D is a-finite. followsthat P'(D)= 1 for every t E T. HenceP(J.l)(D')= 0 which provesthe \302\243
I
\037(\037)
\037
\302\243
assertion.
Assume converselythat PC/L)I.ctis u-finite.Then there existsa sequence such that A\" T Q and P{Jl)(All) < 00, n EN. This provesthe (AII)\"\037N \302\243;.91
assertion.
0)
36.4Corollary.A
weakly continuousexperimentE isJ.l-integrable iff any
isfulfilled: followingconditions dp' (1)J dP. J.l(dt)<00\037-a.e.for every SE T. J
of the
dP.
f (2) Thereis a dominatingmeasurevl.ttsuch that J dv d\037
(3) J
tl(dt)< 00 v-a.e.
. .
J.l(dt)< 00v-a.e. .forevery dommatmKmeasurevl.r#.)
d:
Then assertions (3) and (2) follow Proof Assume that E is J.l-intcgrablc. immediatelyfrom) J
dp'
d; (d/) = J.l
-
dP(tl) dv
v-a.e.)
sinceP(J.l)isCT-finite.Similarlyit isshown that (1)isvalid.Nowassumethat (2) d issatisfied. To provethat E is J.l-integrablelet C,,:={J P'.J.l(dt) n},n E N, dv n v < n N. Then A :=CIIn DII' and let D,.i fl, DII E .ct, E N, such that (Dn) 00, E < 00,n E N, and v(Q\\ U A,,) = O. HenceP(J.l)is CTn E N, satisfy P(J.l)(A,,) \037
II
liE '1/
finite. be a probabilitymeasure which is that (1)holds.Let \\' 1.91 Finally, assume -k t E T}and isof the form v = L 2 Ps\" where(SjJkfN isdense equivalentto {p,: te in T.Then we have
-
N)
-
dp, k dp' J dv J.l(dJ) 2 J - J.l(dt)< 00 \037
dPs\
kEN.))) \037Ic-a.e.,
174
Chapter7: Theory of Estimation)
It followsthat N = {J p(dt)= oo}satisfies PSIc(N)= 0 for every kEN and \037; hencev(N) = O. 0) in subsequent sections let Although much moregeneralcaseswill bediscussed us illustrate theseconcepts some by typicalsimple examples.)
.
36.5Examples.Suppose that (H.< . . is a Euclidean space,L H a linear \302\243;
\302\273
and n = dim H.k = dim L.The Lebesgue measureofH isdenoted subspace by A.
H
.
.
(1) Let P )'H be a probabilitymeasureand denoteh = d; . Consider dAB the shift E = (II,f!J(H),{\037: a E L}),where = P.Choosep = L. Then) \037
\037
\302\243\"
-
dP(A.J(x) = dA. 1I I hex a)A.dda) = hex pdx)+ a\302\273).dda), I
-
A.
XE H.)
Since)
J f hey LJ.i) it
+ a) A.dda) L1(dy) = P(R)= A.
1
followsthat dP()'L)/dA. H < 00 )'H-a.e. This provesthat E is L-integrable. In the specialcasewhere L = H we A.
obtaineven P().II)= ).11' For laterpurposeswe notethat a t-+\037. a E L. is continuous for the variational distance in any case.This is a consequence of Remark 3.1.
Now.assumethat dimL < dim H and considerthe shift with scale a E L.a > O}). where P\".rJ = !R(aid+ alP). parameter F = (H.PA(H).{Pa.o: 1 a E L. a> O. Choose @ da.Then p(da,da) = ).,.(da) (J) (2)
-
\037\037(Jl)
(x) =
dAH
f
R+
JL -i-rr a
h
X
(
= J f }+1h x R+La
(
-a a
-
) Adda)da
PL(X) + a a)
dP(p) = 1__ (x) IIx - pdx)II\"-k
Now, we put y =
\037,
ex
=
\037
a)
(J
IIx PL(X)II.which leadsto
dA.H
k
-
f f '!II\"'cx\"
1
X
h(cx
) )'L(da)du, x Ell.
- pdx) +Y)AL(dy)dcx,xeH.)))
llx_pdx)1I
36.ESlimation
by
175)
dcsintegration
on S. We want Let S = {zE H:IIzll= I} and let t be the uniform distribution we that < for this show and to show that dP(J.l)/dA.H 00 lH-a.e.
l j.i
ex
n
- -1h(exz + y) k
A.
L (dy) dexr:(dz)
= 1.)
is almostimmediatesince(ex, z, y) 1--+ n k h(exz+ y) is a version of It followsfrom Remark (3.1) @ t @ AL)' HenceF is J.l-integrable. dP/d(J.'R' that for shifts with scaleparameter the mapping (a,0-)1--+\037.(1' (a,0-)E L x +, is for the variationaldistance.) continuous But this
I
ex
\037
36.6Definition.A measurablefunction f T --. is (E,J.l)-integrab/eif E is (If J.l)-integrable. IR
I
I
then every boundedfunction is (E,J.l)-integrable.) If E is J.l-integrable.
36.7Remark. Let f T IR be a measurablefunction.Theorem 36.3and are equivalent: Corollary36.4imply that the followingassertions functionf T --.IR is (E,jl)-integrablc. (1)The measurable \037
isa-finite. A E.>1, A 1--+J P'(A)I/(I)IJ.l(dt), (2) The measure P(I/IJ.l): (3) S I/(t)I
dp' J.l(dt)< 00\037-a.e.for every sET. dP. s dp'
measurevld suchthat J 11(t) J.l(dt)< 00 v-a.c. (4) Thereisa dominating dv 1
(5) J 1/(t)I \037; J.l(dt)<:x>v-a.e.for every
and f T If E isJ.l-integrable
.- dP(J.l)) dP(J.l))
then the RN-derivative) is (E,p)-integrable
----
\037
dP(IJ.l) dP(f+J.l)
measure vi d.) dominating
\037
dPCIJ.l) dP(J.l))
ofIcouldbe iswell defined.Theideaof using this RN-derivativeasan estimate to the measure with respect calledestimation by desinteRration J.l1\037.)
36.8Examples.(1) Considerthe shift of Example 36.5(1).Let p> 0 and that P(II.I/')< co.Then) suppose
-
-
AL) dP(II.I/' (x) =
+ Ilia pdx)I/'h(x p,,(x) dAH 2'lUx pdx)+ all'hex PL(X)+ a) A.dda) + 2'I/xll'J hex pdx) + a)A.dda),x E
\037
-
-
a\302\273).,,(da)
-
H.)))
Chapter 7:Theory or Estimalion)
176
Thesecond term isfinite AII-a.e. (1).Thefirst term isalsofinite by Example36.5 smce)
J L\037
f
II
< 00, + all\"h(y + a) Adda)).v. (dy) = P(II.II\")
y
I.)
Hence11.11\" is (E.AL)-integrablc.
(2) Assumethat dimL < dim 1/.For the shift
is valid.Let p > 0 and J IIx
\037\037r;:.\037nasscrtion
\037P
F with scaleparameterthe IIxll' P(dx)< 00, Then we _ PL(X)II\"
Q!.!I'Jl) (x) dlH X 1 pdx) + a . S J n+l-lix-pdx)+all\"h IR-L(J ( (1 ) A.L(da)d(J
-
\0372\"
x t 2\" R.Lu\" J J \037 IIxll'h (
_
PL(X) + a (T
) lL(da)d(J
- +yl/\" J San-k-1-\"lItXPL(X)I! IIx - pdx)lIn-k 2'
x .PL(x)
A+
-
\\Ix
-- -
lx pdx)
II
2'I\\xl\\'
+. 11
XE
L
x PL(X) . + Y)A.ddy)d(X _
h\302\253(Xi
We
-
/i\"=k
PL (X ) 11
X
J R\"
I
(X
n-k-11 I(
X
(X
II
X
- pdx) + ) . (d ) d - ) PL ( X
y
A.
L
Y
(X,
II
II.)
know from Example36.5(2) that the secondterm is finite i.H-a,e. For the
first term thisisalsovalid since) S f J (Xn SR+ L)
=
It-l--,
\\ltXZ
+ yll'It(tXZ + y)lddy)d(X!(dz)
xII' P(dx)<00. J--x--------pdx) II
1\\')
\\I
Hence11.11' is (F,Jl)-integrable. (If we only assumethat POI.II')< 00 then followsthat (a.a)
..... aII)'.(a.a) II
E
(3)
If we specialize P=
We have))) (E..A.,,)-integrable.
N H in
is (F.I')-integrable.) every linearfunctionf L -t iii is
L x R+
(\037
(1) then
it
36.Estimation \037P(fAL)
dP(A,J
dP(fl.J
=
-
dA H
-
--
desintcgralion
117)
dP{A.J
/
dAll
- llix-a1l )
ff(a) ,. exp(
by
1
2
--.
AL(da)
J
J,.)exp( 2 11x aW) l,.(da)
-
and from IIx all2 = IIx
- p,Jx)
11
2
-
+ IIpdx) all2 it followsthat
-
dP(fA.,) f o PL') dP(A,J If we specialize P = Nn in (2) then it is not difficult to seethat the conditionis satisficdiff n k > p. If k > 1 then every linearfunction and we obtainin a similar f L IR if (F,p)-integrable way as for (3)
-
(4)
-.
,,-
dP(fp)
dP(P)=foPL')
The basicideaisas follows. The distributions. Now, let us introduce posterior
!
mixture)
A
x B\037 P'(A) p(dt),
A
E.9I,Be &I(T),
x isdecomposed along(0,sI) by a kernel F:!1
\302\243i(T)
JB P'(A) }l(dt)= J F(w, B) P(p)(dw), A
- [0,1]
such that
E.\",BE(}I(T).
A)
Any kernel with this
distribution.) propertywi1l be calledposterior
-
A 36.9Definition.Supposethat E is weakly continuous and jl-intcgrablc. x lI(T) [0,1]iscalleda posterior stochastic kernel F:!1 distribution (with
respectto p)
if)
. dP(1BP) F(B):=F(.,B) E dP(p) , BEfJ (T).) 71lenthere that E isweakly continuousand JJ.-inteKrable. 36.10Lemma. Suppose
existposterior distributiolls.)
Proof Let v d be a dominatingmeasure and let Q114be an arbitrary measure. Let) probability I
N=
{
f
;
l'(dOj
(O,W)})))
178
Chapter 7:Theory of
Then P(j.t)(N) =
Estimation)
O. For every BE(JI define)
F(B)= JB
/
dp,
j.t(dt) dv-
I Q(B) Thenwe have for every A
E
dp' J dV\" II(dt) on Q\\ N, on
N.)
sI with P(j.t)(A) < 00and every BE14(T))
d = J r P' JI(dt)dv JA F(B)dP(j.t) dV A\\N' = = J P,(A\\N)p(dt)
P(1BP)(A).
B)
Thisprovesthe assertion.
0)
The precedinglemma also gives us a methodfor computingposterior
distributions.)
36.11Examples.(1) Consider the shift of Example 36.5(1).It is easy to see that a
distribution is) posterior
_dP.
JB d
A.
- a)
(x
-
H
F(x,B) = dP J
(x
AL(da)
(x,B)E H x .\037(L).
,
a) )'L(da))
dA.H
If we specialize P = Nil then we obtainthat F(x,.)= Epr.(JC) * Nt\" x E H. the shift with scaleparameter of Example(36.5) (2) Now, consider (2).Then
a
distribution is) posterior
-
1 dP X a d(J L (da) H nrr d B (J I\\H ( (J ) , (x,B) E H x rill (L x F(x,B) _ x a H (J}+1 d H ( ) (/(JJ..,.(da)
-
\037
A.
\037
+ \037
).
d\037
(J)
A.
to P = NH doesnot leadto a considerable Specializing simplification. which doesnot rely on a distributions We needa representation of posterior in Theorem36.14 after somepreparations.) measure. It isobtained dominating
36.12Lemma. Supposethat E is weakly continuous. Then) \037
JB
{
dp'
I p.(dt)< 2 j.t(B)
II
\037
d\037
}
4\037\037\037
\037
- P'II, BE
tM,
sET.)))
36.Estimation
by desintegralion
179)
is trivial if J.L(B)= n. If Il(B)> 0 then we have) ProofThe assertion Ps
JB
{
=
I Jl(dt)< 2 Jl(B)
dp' dPs
}
dp' dp, { dp' {, dPs
1 2
J.L(dr)> J.L(B)
J.L(B) r
\037
\037
\037
\037
\037
r
\037
Recall,that
1
J.L(dr)> J.L(B)
t
, --
d\037J.L(dt)
d\037
0
P, II.
4 supII P,
}
\"2
2 :r 1 dp' J Jl(B) uB)
}
if r....... definitionE is continuous for the p, is continuous
by
variationaldistance.)
36.13Corollary.Supposethat E is continuousand Jl(U) > 0 for every U c T, U 0. Then) dp' [ dp,J.L(dt)> 0 \037-a.e.
open
=*=
fo.revery S E U and every openU c T, U:+:0.) Let U T be open, S E U, and B(s,c):={tE T:II P,II < c} U.Then)
Proo!
\302\243;
\037
P,
{!
\037;,
\037
\037
-
choose t > 0 such
that
\302\243;
I'(dl)=
o}
dp' < _1 J.L(B(s, 4c. J.L(dt) 2 {B(s,e) dP. } S
e\302\273
\037
o)
oS)
36.14Tbeorem.Supposethat E is continuous, Jl-integrah/eand that J.L(U) > 0 distribution F satisfies) for every open U c T, U * 0. Then every posterior dp'
_
. J - .J.L(dt)
p.s-a.e.,B E /A (T), S E T. F(B)= !,_d\037 dP, J dP.J.L(dt) s)))
\037
J
80
Chaptcr7: Thcory of Estimation)
ofa posterior distribution we have for every ProofLel BE91(T). By definition
Ae.91)
= P(IB Jl)(A) = J P,(A)Jl(dt). J F(B)dP(Jl) B
A
dp' P'(N) 0 and P,(A\\N) = AJ d - dp', t e T, The P, existence of such a setN followsfrom weak continuityof E. Let t E T. From >0 Thus we obtain it follows that J Corollary(36.13) dP,) Jl(dt) p'-a.e. Let
Ned be such that
o
'\"\"'
\037l}
J
A\\N
!.
dP'
=J J Jl(dt)d\037 F(B)dP(Jl) 8A\\N dP. s = J
J8
A\\N
f
dp'
dp'Jl(dt) dp,Jl(dt)dp,. dp' dt J dp' Jl ( )
dP. s)
-
. followssince . dP(Jl) dp' --- - on Q\\ N. The asserlton Jl (dt) = J dp.s)
dP,
o)
36.15Corollary.Supposethat the conditions aresatisfied. A of Theorem36.14
measurablefunctionf T -+
\037
is (E,Jl)-integrable iff
SE T.) flf(t)1F(dt)< 00 p'-a.e.,
In this case)
dP(hl)= dP(Jl))
sET. SfdF f's-a.e.,
and 36.14. combine36.7(3).36.13 The second ProofFor the first assertion
followsfrom Lemma 36.10.
0)
lemma.) Finally,we prove a useful approximation
36.16Lemma. Supposethat E is continuous.Let
...,
K
\037
T be compact and
K -+ R !?I(T)-measurable and bounded.Then for every t > 0 there are a measurablepartitionBt, BM ofK,pointst i e Bi . 1 i M.and <Xi E lit 1 i
f
\037
M, such that) \037
-
dp'Jl(dt) lcf(t)dp' f
\037
M (J.j j\037l
\037
dp' Jl(Bj) < E;, SET.))) dP;
\037
37.GeneralizedBayesestimates
un)
If Jl(K) = 0 or IIfllu:=supIf! = 0 then Proof Let c> 0 and K s; T compact. K the assertion is trivial.Therefore,we may assumethat Jl(K) > 0 and IIfllu > O. SinceE iscontinuous there exists{) > 0 such that) dp\" \037
dPs
- - -P. 1< whenever Itt - < dP,2 dPs
\037
II P\"
/2
I
I;
2JL(K)IIfllu)
t21
f>,
t..
t2E
K, SE T.)
!of
Sincefisboundedthere isa stepfunctiong = i=L !Xi lBI such that IIgll.. IIfllu and IIf gllu < ' The sets Bi , 1 I. M, may be chosensuch that 2Jl(K) diam BI < f>, 1 ; M, and {HI:1 ; M} is a partitionof K. We obtain) \037
-
I
\037
\037
-
\037
\037
dp'
(t) d
K
=
-
\037
-
M
\037
dP'.Jl(BI) Jl(dt) ,-1 i .L d Ps
If
\037
\037
ex
\037
dp, (f(t)Jl(dt)+ I}If(t) g(t)ldp'Jl(dt) r k
\037
M
d\037
i\037t
M
\037
+
i\0371
\037
\037
. Jl(K) +
2Jl(K)
\037
.
2Jl(K)
M
dp' ( ) Jl(dt) dP. dP. Ig(t)1 dPsJl(dt)) g(t) ki r
get)\037
\037\037
Jl(B,)< E;.)
dP,
dPs
d\037
d\037
d\037
o)
i\0371
37.GeneralizedBayesestimates) and assumptions 36. of Section keepthe notations In the followingwe considera loss function W = (U0/ET' given by IE T, suchthat (s,t)...... (s,I)e Tx T, is \037(T)x \037(T)J-Y,: T\037 [0,(0), measurable and W(O) = O. If E Jl(E,T) isan estimate then the risk of (!with respectto W is given by We
\302\273-;(s),
\037
tE T.) U(ro,ds)P'(dro), H the notations) Forconvenience let us introduce \302\273-;UP,:=
\302\273-;(!:
\302\273-;(S)
w...... J
\302\273-;(s)
(!(w,ds),
WE
fl, t E T,)))
182
Chapter 7:Theory of
and) l?
ESlimation)
p,:Bt-+ f (!(w, B)
\037
(dw),
BE94, t E T.
discussfirst Bayes estimates assumingthat IIIgJ(T) is a finite measure. Generalized are the extension of this conceptto non-finite Bayes estimates t he Recall that F denotes measures w.r.t.Jl.) distribution 1l1\037(T). posterior
We
37.1Definition.Supposethat JlIEl(T)is finite
is Jl-intcgrable. An to Jl and W) if estimate e*E fJl(E,T) iscalleda Bayesestimate (with respect P'Jl(dt) for a1l e Yt.(E, T).) J J.v,q*P'Jl(dt) J \037
J.v,\037
and
W
\037
37.2Theorem.Supposethat E is weakly continuous, JlI5l(T)isfiniteand W is thenfor every SET) (!*e {Jt (E,T) isa Bayesestimate If an estimate Jl-integrable. J
(J\302\245,U*)
(w) F(w, dt)
\037
J J.v,(s)F(w, dt) P(Jl)-a.e.)
Proof Assumethat there existsSoE T and A E .91with P(Jl)(A) > 0 such that f
Definean
< f(W;(!*)(w) F(w,dt) J.v,(so)F(w,dt)
if
WEA.)
estimate C!E fJt(E,T) by
(!(w, .) =
(!.(w, .)
if
w rf A,
I
ClJ
'f
{
\302\243\"0
EA.)
Thenit is easy to seethat
<J
P'Jl(dt). 0) If E is weakly continuousand Jl(U)> 0 if U c:T is open, then the exceptional setis assertion P(Jl)--{p,:t E T}.In thiscasein the preceding J J\302\245,e\037Jl(dt)
J\302\245,(!*
of \037-measurezerofor every t E T. The assertion of Theorem37.2is the startingpoint for the definitionof generalized Bayesestimates.) if for every SETthe 37.3Definition.A lossfunction(U0u;Tis(E,Jl)-integrable
function / t-+ J.v,(s),/ E T, is (E,Jl)-integrable.)
and (Jt;)'ETis(E,Jl)-integrable. 37.4Definition. that E isJl-integrable Suppose to Jl and An estimate (!*e fJt(E,T) isa generalized (with respect Bayesestimate W) if)
(w) F(w,dt) J (J.v,(!.) for all .'IET and rE T.)))
::;J
J\302\245,(s)
F(w, dt)
\037-a.e.
37. Generalized Bayesestimates
183)
37.5Remark. Let usmake somecomments on the preceding definition. First we notethat the inequality may
be written as)
O.(w, ds')
f (f J-v,(s')F(w,
\037
dt\302\273
J
F(lt),dt)
J\302\245,(.\037)
r E T, sET.) p\"-a.e.,
Assumefor the moment that
W'-'inf J J-v,(s)F(w,d/),
Q,
WE
SE T)
isd-me:asurable. Letting) B(w) 1={s'E T:J J-v,(s')F(w, dt) = inf J '/fiT) we seethat
F(w, d/)},
J\302\245,(s)
WE
Q,
0.isa generalizedBayesestimateiff
o.(w,
B(cd\302\273
=
p'-a.e.,t E T.)
1
The preceding of generalized remark showsthat the existence Bayesestimates
depends on regularity properties of the posterior risks the:mostgeneralset J J-v,(s) F(w.dt),sET.w E Q,We: donotintend to state: of generalized of conditions the existence since which implies Bayesestimates, for most specialcasesexistence followsjust from a moment'sreflection. of which is useful in many cases. we will work a out set conditions However,
s.-.
First,letus consider examples.)
37.6Examples.(1) Let us consideragain the
shift on a Euclideanspace beenintroducedin Example 36.5(1).Define the loss function by \037(b)= lib a1l2, a,bEL, and assumethat P possesses finite momentsup to the order2.Then)
(H,< ., .
\302\273
which has
-
\".(x) = J aF(x,da) =
- a) (x - a)
:; dP
-T
Ja
J
(x
d)..H
is a generalized Bayesestimate. of (1)but let (2) Keepthe situation I if b a > C. = W a (b)
if
(da)
,
xE
H,
Adda))
a e L.
IIb-all
{o
-
AI.
II
The pertaininggeneralized are calledMaximum Probability Bayes estimates
Estimates.
Nil'Then it is easy to see that for bothtypes of lossfunctions,,*= PL is a generalized Bayesestimate. the caseunder (1)to P = (3) Now we specialize
It)))
184
Chapler7:Theory of Estimation)
will turn
outin Section38that this iseven vaJid for a considerably larger class
of lossfunctions.
a Gaussian shift with scaleparameter and define (4) Finally let usconsider
-
\037
as in Example 36.5(2).Definethe Jossfunction by , a(b,t) = (J2 I1b aUZ, a,bE H, (1, > O. Then it is not difficult to seethat K. = PL is a generalized Bayesestimate. existence theorem. Now we approachthe announced In the followingJet (H,< ., . bea Euclidean spaceand assumethat T = H. = W(s t) where W: T....., Moreoverlet Jl = All and [0,(0)is a measu= rablefunctionwith W(O) O. An experimentE E 8(T)is\037-integrableoforder p 0 if it isJ\037-integrableforf t r+ t liP, t E T. If W isoforderp and E isAIIintegrable ojorderp then (Jt;)u;H is (E,AH )-integrable.) \037
'\037
or
-
\302\273
J\302\245,(s)
II
\037
37.7 Lemma. Suppose that E;scontinuous and ;,w;ntegrable oforderp.If W is riskfunctions of orderp then the posterior
st-+fW(s-t)F(dt),sEH, arecontinuousPr-a.e. Jorevery r E H.)
-
Proof ChooseS E Hand r E H. Let B(s,b l ) H be such that W(u t) CJ \" IliP + C4 if u E B(s,01)and IEH. SinceE is AII-integrab]eof orderp it \037
\037
followsthat
JiltjlPF(dt)<00 J:,-a.e.,rEH.) of r E H. Now Jet > 0 Theexceptional setN E .91can bechosenindependently arbitrary and fix somew N. Thereexistsat 0 such that) \302\243
\037
\037
II tll f IJrJIP>a.
P
- -. I;
. F(w, dt) < 3C
J)
Define u-(c),=min{W, CJa + C4 }.We may find bE (O,bl ) such that) C
II u-(t)(U
-
t)\302\243(w,
dt)
-
- J u-(t)(s-
I)F\302\253(I),
dt)1 <
\037
3)
Hencewe wheneverII u s\" < O. Thisfollowsfrom continuityof convolutions. obtainfor every II E B(s,b)) I
J W(u \037
- t) F(w, dt) - J W(s -
-
+ J (W \302\243
\037-
3
-
\037t\302\273(u
F(w,dt)1
-I - t)F(w, - F(w, + J - -
If \037C)(u t)F(w,dt) t)
t)
dt)
(W
+2CJ f II/JlPF(w,dt)\037\302\243. 11'11' >41.)))
dt)1
\037C)(s
\037t)(s t) F(w, dt)
0
37.Generalized Bayescslimates
HS5)
37.8Theorem.Supposethat E i...continuousand l,,-integrable of orderp. there Assumefurther that W is separating, level-compactand of orderp. 111en existgeneralized Bayesestimates.)
of Remark 37.5issettledby Lemma37.7.If ProofThemeasurabilityquestion we can provethat B(w)* 0 on a set A E d with \037(A) = 1,S E H, then the assertionfollowsfrom the existencetheorem on measurable selections,
Theorem6.10.
1SI step:Let.'iE Handt > O. Firstwe notethat there existsa b < supW such
that) \037{J W(s
- t) F(dt)> b}<
To provethiswe
\342\202\254.
setK integrablethere existsa compact r
\037{
is not bounded.SinceW is (E,AH)Hand l> > 0 such that
first assume that W
W(s
-
\302\243
t) F(dt)> l>} < t.
H\\K
is boundedon K. -.) isboundedand separating. Let c< sup
The assertion followssinceW(s
W besuch that that W Now, assume = B {tET: W(s-t) c} is a neighbourhood of s. From Lemma 36.12we obtainthat there isan 11> 0 such that
::;
< 11} < t. \037{F(B) The number 11can be chosensuchthat c< supW
-
t) F(dt) = supW' (1
f W(s
-
\037
= supW
-
We
ohtainthat)
supW' F(B')+ (supW '7)F(B) + supW' F(8) '7F(B)
-
F(8\302\273
11F(B).
Henceit followsthat)
-
- 11.
W(s
t) F(dt)> supW
-
2 '7 }
{F(B)< 11}< E. 2nd step:Let.rE Hand > O. We showthat thereexista compact setK Hand a number 0 < l> < such that) inf f W(x t)F(dt)> inf J W(y t)F(dt)+ l>} > 1 c. \037(f
\037
\037
E;
\302\243)
\037{
\037
-
-
XE II\\K
)I\037
-
H)
Since > 0 isarbitrary thisimplies to the first that B(w) * 0 \037-a.e.According W a b < such there exists that) step sup \302\243
\037(J
W(s -t)F(dt)
\037
b} 1 \037
- 2'l: -
Let 0 < l>
C c H satisfying)))
Choosea compactset
186
Chapter7:Theory or Estimation)
-o}< 2 - t)F(dt) b}rI{F(C) 1-o}.)
<1 Ps {F(C) and let)
A,.::::; {f W(s
\302\243
')
\037
Then we have \037(Ac) > 1
\037
- t. Since
is level-compact there existsa compact setKc H such that tEe and x rt Kc imply W(x t) > (b + 0)/(1 0).Then we obtainfor every x rt Kc that on the setAI:)
-
W
\302\243;
J W(x
- t) F(dt)
\037
> b+b F(C)
1-=b.
\037
\037
cJ W(x b
-
t)
-
F(dt)
-
+ 0 J W(s t) F(dt)+ 0 \037
inffW(y-t)F(dt)+o. D
yeH)
proofof the precedingtheoremshowsthere existgeneralized Bayes which are even non-randomized estimates estimates.) As the
38.Fun shift experimentsand the convolutiontheorem)
l,
Let G be a locallycompact groupwith countablygeneratedtopology.Let = ).tif and At be a pairof right and left invariant measures such that Z(O'IJ.,)
u:x.-.X-I, X E G. The Borel-u-field of G isdenotedby &I(G).The modulus function of G isdenoted by A.)
. .
38.1Definition.Let PltI(G)be a probabilitymeasure onG.Thenthe experiP:t e G})iscalledthefullshift experimentgenerated ment E = (G,{M(G), {\302\243,
by
P. For shortwe denotePc:=
\302\243,
P, t E G.)
A measurable 38.2Remark. The followingfonnulasare immediate. function
is P-integrable iff it is Pc-integrable and JfdP,= Jf(lx)P(dx), (E G. we have Pc(A) = P(t-I A), t E G, A E 14.If P )'1then In particular, \037
tE G, and)
-
dp' _ dP -I (t x) (x) dJ..;
A.r a \"
.c.,
t
E
Pc
\037
I'b
G.)
d)'\037
38.3Lemma. (Torgersen[1972]).Supposethat E is the full shift experiment
areequivalent:))) generated by P. Then the followingassertions
3M. Full shift
experimentsand the convolution theorem
187)
(1) P l\" t E G} is dominated, (2) {p,: IE G} l\" (3) {p,: (4) E is weakly continuous, (5) E is continuous.) \037
\"\"
Proof'(1) (2):Let A e \037(G).Then (I:,+ P)(A) = J 1A (Ix) P(dx) = P(t 1 A), \037
t
E
G.
If AtCA) = 0 then AAt -t A) = 0 and hence(t,* P)(A) O. t e G}.If measuresuch that Q {p,: (2) => (3):Let QIEM(G)bea probability 1 A E &iJ(G) and t E G then Q(t- A) = 0 implies \037(t.-I A) = 0 for every s E G. = Hencewe obtainP'\037(A) 0 for all s E G and there Q(A) = O. It followsthat Q(A) = 0 iff
-
\037
0= J Q(t-lA) A(dt)
= J I lA (IX)Q(dx)A,(dt) = I A,(Ax-l) Q(dx)= l,(A)jL1(x I) Q(dx),)
where L1
is the modulusfunction of G. Since > 0 we obtainQ L1
(3) => (1):Obvious.
(5) \037 (4) \037 (2):Obvious. (3) =- (5):It is a well known fact that
limII/(x)
'-e)
-/(t- x)1).tCdx)= O.
.
Since)
for every IE
dee, - P)
- l(.
!e(G,.'!I(G), It) 1
1
dP_ (I 1 x),
=
d;.,--(x) d).,
the assertion follows.
XE G, Ie G,)
0)
that G isAbelianand let G be the charactergroupof G,i.e.the group Suppose of all homomorphisms x: G {zE c:Izi = 1}.The Fouriertransform of a measure PIBI(G) is probability
-.
P:
X
t-+ J XdP.
X
E
G+.
It iswell known (seee.g.Parthasarathy [1967]) that two probability measures iff their Fouriertransforms on fM(G) coincide coincide.)
38.4Lemma. Supposethat G is Abelian.If p(x)* 0,X E G*,then the fullshift measures.) experimentE consists 01pairwisedifferent probability ProofAssume that
\037
=
\037,
s,lEG.Thenit
foHowsthat
for every X E G*)))
188
Chapler7: Theory of Estimalion)
=
f x(x)P'(dx) f
(x) \037(dx) and thereforeXes) = X(t) for every X E G*.This provess = X
t.
0
Estimationfor full shift experiments dependson the methodsdevelopedin 36and 37.) Sections
38.5Lemma. Every fullshift experimentE is A,-integrahle.) ProofLet usshow that J P,A,(dt) = l,.If A E ll(G)then we have = f P'(A) A., (dt) ffl A (tx) P(dx)A, (dt) = HIA (tx)A.,(dt)P(dx) = A,(A). a)
38.6Example.Themain examplein the presentsectionwill bea full shift ona
Euclideanspace(H.<'. For thiscasethe precedingassertioncan be improved to the followingfact:If P has finite p-th momentsthen E is J.H integrableof orderp. This is the particularcaseof Example 36.8(1),where '\302\273.
H=L.
Throughoutthe followingwe supposethat E is a dominatedfuJi shift measure experimentgeneratedby the probability Let us compute the posterior distribution given A,. A particularcaseof the PI\302\243f(G).
in Example36.11 followinglemma iscontained (1).)
38.7Lemma. The posterior with re.'IpeCl to A, is F(x,.)= ex.P, distribution = XE G, where P .!f(uIP).) Proof Let A E \037(G).Thenwe have
!
dP
dP
d)'\037
d)'\037
(t-Ix) A., (dt) = J 1,4(/-1) dP =A(x)J 1A (xt -1) d).
(IX)At (dl)
\037(t\302\273).(dt)
= A (x>J1,4(xr ) P(dt)= L1 (x) 'p(x-I A). Henceit foJlowsfrom the proofof Lemma 36.10 that 1
F(x,A) = J(x)
A) =
:(\037-I
P(x-I A).
0)
thispointthe readershouldrecallDefinition32.1where amenablegroups of certain areintroduced. Amenabilityis the reasonof the optimum properties We have beenconcerned beforein Section 32. situation with a similar estimates. distributions.))) w e statesome lemmas for First, approximation posterior At
38. Full
shift
cxpcrimcnlsand
convolulion lheorem
the
189)
38.8Lemma. Supposethat G isamenable.Theilfor every probability measure (K\ QI\037(G)and every summingsequence Jim
*
At) , J (ex
\"-00
Q) (K\037)
K\
K\"
I.,(dx)= o.
followsfrom) ProofThe assertion
i (ex
*
Ar (\037II)
Q) (K\037)
1,(dx)
\"
1
= A,
1 (1 J.r(dx) (K ) J KJx) J -lKJxt\302\273 Q(dt) II
1
= J Q(dt)
J OKJX)
A,(K,,)
-l Jx) l K
KJxt\302\273
A,
(dx)
1
\037JQ(dt)A
,(K) J 11,,(x)-l,,(xt)IA,(dx).0 K
K
K\
II
Let F bethe posterior distribution 38.9Lemma. Supposethat G isamenable. of E given J.,.Thenfor every summingsequence) lim \"....00
l/,
KII)
= O. HKJIF-F(.IK\lIdP,A,(dt)") )
-
ProofSince F F(.IK ) = F(K\037). n E N, and sinceF(x,.)= II
II
II
only show that)
lim
1 \037 JJ \"-00\"', ( \"K\ )
(\302\243.1
\302\267
P)
(K\037)
P)(K\037) J (ex.P)(K\037) (e, P)(dx)= (E, and thereforewe may apply Lemma (38.8) with Q = Every
.
P we need
;.,(dt)= o.
.P.
.
We have)
P, (dx)
\302\243;(
P.P.
0)
measure QI9I(G)definesan estimate probability e e \037(E, G) by (l(x, Q. XE G. .)I=\302\243X.
Such estimates satisfy e p, =
P, *
Q, t E G.)
38.10DefinitiOD. An estimatee E Yl(E,G) is a convolution kernel if e(x,.) = tx * Q.x E G, for someprobability measure Q E 91(G).) 38.11Remarks. (1) An estimate(l E gr(Et G) is a convolutionkernel
iff)))
190
Chapler7:Theory of Estimation)
the condition is necee(tx,tA) = e(x,A), X e G, t e G, A E gI(G).Obviously, we define which To show ssary. t sufficiency, Q:=e(e,.) impliese(x,A) = Q(x- A) = (ex.Q) (A), x e G, A E \037(G). of a convolution kernel satisfy P, t E G, the distributions (2) Sincep, = = t E latter G. The an invariancecondition, eP, namelyeP, propertygives riseto the followingdefinition.)
.
\302\243,
\302\243,.
38.12Definition.An estimate e E gr(E,G) isequivarianrif eP,=
\302\243,.
eP, t E G.)
38.13Remarks. (1) An estimatee E 9t(E,G) is equivariant iff (eP,)(tA) = (eP)(A), t e G, A E the special casewhereG = (H,< ., . isa Euclidean (2) Consider spaceand K: H P = NH . In this casea non-randomized estimate Hisequivariantiffit is of the form K(X) = x + a P-a.e.for somea E H. To see this note that \302\2438(G).
\302\273
equivarianceimplies SI(K(x\302\273P,(dx)
= SI(K(s+ x)
-.
- s)P'(dx)
for all .'1,tE H and every IE ljjb(H).Sincethe Gaussian shift experiment is = it follows that K(S+ x) K(X) + s P-a.e., .'Ie H. An application of complete Fubini'stheorem yields p2{(.'I, x):K(S+ x) = .'I + K(X)}= o.
Hencethere existsXo E H such that K(S+ xo) = S + K(Xo) for P-almost every S E H. Now, taking a = K(XO) Xo provesthe assertion. The last rcmark showsthat sometimes non-randomized equivariant estikernels.Can a similarassertion mates are almostconvolution be provedfor
-
arbitrary equivariant estimates? In the followingtheoremit is shown that even the distributions of nonare \"in the mean\"convolutions.) equivariant estimates
38.14Theorem.Supposethat G isamenableand let(K,,)bea summingsequence. measures Qn I\037(G),n E N, e E 91(E,G) there existprobability
Then lor every .'iuchthat)
lim II ,,\037oo
,(
A
r
\302\267
) \"in)
\037
\302\243,-1
Proof Recall that for every
h:G2
-.
IR
-
eP'J.,(dt) P.Q\"II = O. boundedfunction
.\037(G) @ .\037(G).measurable
we have)
= H hex,t)F(dtIK,,)(x)Ps(dx)A,(ds),) hex,t)P,(dx)A,(dt) LJ L n E N. We obtainfor every A E 84(G))))
Ful\\ shift
3\037.
x,(
experimentsand the convolution thcorcm
191)
(aP')(tA)i.,(dt)
f
) \"K\"
\037
=
J If e(x,tA)F(dtIK,,)(x)\037(dx)i.,(ds)
) ( \"K\"
I\037 r \037
=
l)K,,)L If e(x,lA)
* (\302\243;1;
P) (dt) \037(dx);',(ds)+ R,,(A))
where) IR,,(A)I
\037
l,(\037,,)
By Lemma 38.9it
LJ
IIF- F(.IK,,)/IdJ:;',(dt).)
followsthat
lim sup IR,,(A)I = O. ,,-\302\253t
If
A..aI(G))
we denote)
Q,,:A
1
\037
l,(ds), J e(x,xA) l's(dx) l,(X\"K\ ) f .
A
E \037(G),n
EN,
then we obtain)
1
A
,(K) inr (eP')(tA)l,(dt) \"
= R,,(A) + 1 A
xtA) P(dt)f>s(dx);',(ds)
,(K) Ie\f If e(x, \"
= R,,(A) + J Q,,(tA) fl(dl)= R,,(A) + which provesthe assertion. A
first
(P.Q,,)(A),)
0)
is the convolution theorem.) importantconsequence
38.15Theorem.(Boll[1955]).Supposethat G is amenable.Then for every eE equivariantestimate that Qp, = p, * Q, t e G.)
91(\302\243,
a probability measureQ19I(G) such G) thereexists
of Proof SinceQ is equivariant we obtainfrom Theorem38.14a sequence measures probability Q\"I&f(G),n E N, such that
- P.
lim lIeP
Q\"II=
O.
\"\"'<10)
Let QI\037(G)be a vague limit pointof
(Q,,),,\302\253:\037.
Then for every fe \037oo(G)
we)))
192
Chapter7: Theory of Estimation)
have)
foP = ffd(P+ Q).)
Since(}pis a probabilitymeasurethe same is true for P.Q and Q.
0)
38.16CoroUary.Supposethat G is amenable.Then for every equivariant estimate there existsa convolutionkernel with the same distribution. Let W: G [0,(0)be a lower semicontinuous function and definethe loss function by \037(x):=W(t- x), x G, t e G. Then the risk of an estimate G) is given by o
-
I
\342\202\254
\342\202\254
91(\302\243,
\037(}p,
= Sf W(t-Iy ) e(x,dY)P'(dx),tE G.
kerneldefinedby If (!isa convolution
QI\302\243\302\245(G)
then
tEG. \037(!P,=SWd(P.Q), theoremforamenablegroupsthe risk of any cquivariant By the convolution
estimate isalsoof thisform.The maximalrisk of arbitrary estimates can be boundedfrom below by the risksofconvolution kernels.)
38.17Theorem.Supposethat G is amenable.Then for QE
!Jt(\302\243,
.
such that measureQI.1J(G) G) there is a probability
\037(! P, sup IE (i)
\037
f W d(P Q)
functionW: for every level-compact
G
every
estimate
- [0,(0).)
of probabilitymeasures Proof FromTheorem38.14we obtaina sequence
11E N, such that Q\"I\037(G),
lim supJ fd(P* Q,,),IE \037,,(G). sup \"6f(}P' \"-'00) \037
If the right hand sideis zerofor all fE \037oo(G)then it followsthat sup \037QP,= W(oo) 'fiG) and the assertion is valid with arbitrary Q.Otherwise,let Qol\037(G)bea vague limit point of (Q,,),,( N, Qo O. Then Qo is a vague limit point of =t=
p.
(P.Q\UN.")Let Q = Qo1(G). Qo.Then QI\037(G)is a probabilitymeasure.If fE fCt (G) then we have lim inf f fd(P.Q,.)
\037
\"\"'00)
f
fd(P.Qo)
\037
f
fd(P.Q)
and thereforefor every levelcompact functionf E fCb (G)we have (since)))
38.fuJl
shift
experimentsand thc convolution thcorcm
193)
supf-IE'Go
(G\302\273
lim SUpJ fd(P* Q,,) J fd(P* Q)
,,-
\037
co)
We
obtainthat)
supf(?P' J fd(P* Q) I( for every levelcompactfunctionfE\037,,(G) and the proofis finishedby Lemma \037
G)
6.4.
0)
and suppose that 38.18CoroUary.Let W: G -+ [Ot 00)belowersemicomillUOUS there is a pointYo E G such that)
J W(x'yo)P(dx) J W(x.y)P(dx) if yE G.) Then the non-randomized estimateK: x \037 X. YOt x E Gt \037
ha.\037
the following
properties: (1) K is optimal(for W) amongallconvolutionkernels. (2) If (i isamenablethen K isoptimal(for W) amongallequivariantestimates. (for (3) If G isamenableand W islevel-compactthen K isa minimaxestimate W).)
and (3) followsfrom Proof (1) is obvious,(2) followsfrom Theorem38.15
Theorem38.17.
0)
Now we turn to the casewhere G isa Euclidean space,denotedby (H,< ., .
\302\273.)
38.19Example.Let W(x) = IIx1l2, X E H. Thenthe assumption of the preced-
ing corollaryis fulfilled with Yo = J xP(dx),providedthe first moment exists. Hence,K: x \037 x + Yo' x E H, is a minimax estimatewhatever PIPA(H) looks
like.If PIEI(H)iscentrallysymmetricthen the first moment iszeroif it exists and hence,the identity is minimaxfor quadraticloss.This easy fact can be extendedvery far as we shalldo below.) 38.20Lemma. Let C, D 5 H be convexsetswhich are centrallysymmetric. Then) l\302\253C+
y)r-.D) l(Cr-.D),YE H.) \037
It is that C and D arc closed. Proof In view of Theorem(6.1)we may assume clearthat we may even assumethat C and D are compact.For any convext compactsetA II let SeA) A + ( A). Then it is easy to see that \302\243
-
I\"\"\"
\037
Cr-.D,using that S\302\253C+y)r-.D)!;;
\037
C and D are convex and centrally sym-)))
194
Chapter 7: Theory of
Estimation)
it issufficientto show that metric.Hence,forthe assertion ).(A) for convexsetA 5 H. But this isan easy consequence of the Brunnany compact, MinkowskiTheorem6.2.For this,put k:=dim H and notethat ).(S(A\302\273
1 ).i\"
=
(S(A\302\273
11
).1(2A + 21(
-
1
A\302\273
\037
11-
1
2).i(A) + 2 ).k(
38.21I\037mma. (Anderson[1955]).LetP <( ,t.Supposethat W:
H -+ [0,(0)aresubconvex.Then) f WdP
\037
f W(x + y)
A)
=
\037
1 ).i\"
(A).
0
- dP di and
P(dx), ye H.)
ProofLet C 5 H beconvexand centrallysymmetric.Thenfrom Lemma38.20 it
foltowsthat for every ex
P(C+ y) =
\037
0,y E H)
1.t(C+ y)n {
>
\037\037
.(
;:;;I Cn Putting C = {W
=
{\037\037
>a})da
P},P 0, we obtain P{W>P} P{W(.+ y) > P}, \037
da
a})
P(C), ye H.
\037
\037
{J
\037
0,ye H,)
and therefore) QO
QO
P{W>{J}d{J f0 P{W(.+ y) > {J}d{J =fW(.+y)dP, yeH. 0) At this point we couldapply Corollary38.18 to obtainthat under the of Lemma 38.21 the identity is the optimalequivariant estimate. assumptions J WdP = fo
\037
However,for the verificationof the minimaxpropertywithout assuminglevelof Wwe have to go back to Theorem38.14.) compactness
- dP
38.22Theorem.LRt P <{)..Supposethat di..and W: H -+ [0,(0) are subconvex.Then Ie:x....... x, X E H, is a minimaxestimate (for W).) Let QE .\037(E, H) we may assume that Wis lower semicontinuous. ProofW.l.g. of Then from Theorem(38.14) we obtaina sequence be an arbitrary estimate. measures n E N, suchthat probability Q\"I\302\243\302\245(H),
sup H'\"e IIEH
\037
;;;; lim
supJ (W (\\ a) d(P* Qn)
n-QO)))
38.Full for every
a> O. Sincewith
W
experimentsand
shift
the
convolution theorem
195)
and also W r.a, is lower semicontinuous
that it follows subconvex, w,.eP\" J (W r.a)dP, sup heR) \037
a> O.
From)
J WdP
--=
sup J (Wr.a)dP 0>0)
the assertion follows.
0)
Thequestion whetherthe identityisthe only equivariantestimate with minimal risk is answeredby the followingcorollary.)
38.23Corollary.Let P
\037
A
and p(z)
=+=
0,Z E H.
.\\\037uppose
that
W:
H -+ [0,oc)is
andsuch Ihat) level-compact J W dP < J W(x + y) P(dx) if y * O. Then an equivariantestimale (l E H) has minimalrisk (for W) 91.(\302\243,
= 1,4(x) P-a.e.,A
E
(ff
(l(x,A)
&f(H).)
measureQI\037(H)it followsthat Proof.Since(lP= P * Q forsomeprobability J WdP = J Wd(P*Q). First,we show that Q = f:o.For C( 0 let DII = {xe H:!Ix C(}.Define A(.I= inf J W(y + z)P(dy). \037
II
\037
z.D.)
Theseinfimaare attainedsincelevel-compactness implies Jim z..\037)
J W(y + z) P(dy)= supW.
HenceMil > J W dP for every > O. We have \037
J Wd(P.Q) = H W(y + z)P(dy)Q(dz)
+ Q(D(J)Mo + Q(D(J)Mo.) It followsthat Q(DIl)= 1 for every C(> 0 and thereforeQ = to. that for every t E If and Thus,we obtain(!p= P. Equivarianceof (!implies \037
=
A
E
fM(H)
\037
(1-
Q(D(J\302\273Mil
-
P(dx)= peA t). II be compact. SinceshiftsareA-integrable J P(K t) )'(dt)< 00,))) J (l(x+ I, A)
Let K
Q(D\037)Ma
-
196
Chapter 7: Theory of
Estimation)
and hence)
-I(z,r -x) J (}(I,K) P(dx) (dx) = J e-I(z,,) ZE H. P(K
Je
-
A.
t\302\273)'(dt),
Fubini'stheorem thisyields p(z)J e-i(z,r)e(t,K)/.(dl) = p(z>Je-i(z,r) which provesthe assertion. 0) By
lK(t\302\273)'(dl))
The idea of the followingproof has been used also by Ibragimovand
Has'minskii [1981].)
38.24Discussion. Lemma). If P = Nil then (Equality in Anderson's J WdP = J W(.+ y)dP impliesy = 0,) for every subconvexfunction W: H -+ [0,(0)which islevel-compact, separating and of finite order.It shouldbenotedthat noof theselatterconditions can be omittedwithout compensation. Lemma 38.21. In To provethis,let y 0 and recallthe proofof Anderson's W view ofthisproofit issufficientto show that there issome < such that) Po sup =t=
P{W>P}< P{W(.+ y) > P} for all P Po. ChoosePo in such a \037
way that {W
\037
0 as inner Po} contains
sinceW is separating. Let Po P < supWand denote point.This is possible = C {W P}.Then we have \037
\037
P(Il\\C) < P(H\\(C iff)
J.
-y)n
((C
>
{
\037\037
-
y\302\273)
_})
d (cn{
\037\037
>_}))
for someex > O. This is the inequalitywe shallprove below. Let
_.=sup
C S;
{Y:
{
>
\037\037
Then
r}}.
it
followsthat C s;;
{
\037\037
Moreover,we show that
oCn
{
\037\037
For thislet
=
,,}
\037
0.
b.!0 and choosex.
E
Cn
{
\037\037
<\"+
n E
b.}.
N. Then)))
;<;
,,}.
3M. Full shift
(X
(X,,)< a
\037
+ b\",
nE
experimentsand the convolution lhcorem
N, and there is an
197)
accumulation point Xo of (x,,)
\037\037
a. Since C is dosed we
(xo) =
satisfying
have
Xo E
C, and since
\037\037
t
we have Xo E ac.Thus,we obtain
>
\037
\037},
{\037\037
XOEacn{
\037\037
=+
Next, Jet us show that \037
a}).)
\037 Xo\037s\302\253c+y)n{
Assumingthe contrary,namely)
y)n
XOE S\302\253CI
\037 {\037\037
dP
1
=2(C+Y)11 {d)'
\037(X
dP 1 + 2(C-Y)fl d)' it
{
would followthat) Xo
a}) }
\037
(X
}
-
,
= 1 (Xt + y) + 1 (X2 y) = 2 2
where x,.X2 E C,x, + Y E
1
;;;
{
a}.x,
\037\037
(X2 + X2),
\"2
-
YE
{
SinceC
\037
\037\037
\037
\037}.
and sinceXo isa boundarypointof the strictlyconvexset
;;; {we obtain a} that \037\037
Xo
= necessarily XI
+Y E
\037
{
\037\037
a}
{\037\037
X2 = Xo. But
and
Xo
-
Y
E
{
\037
\037\037
\037
a}
+
isa contradiction to the fact that Xo is a boundarypointof the strictlyconvex dP set dl 2.a .
{
-}
.)
It followsthat for somet > 0
>
B<Xo,8)\037
(s\302\253c+y),,{ a}))'.))) \037\037
198
Chaptcr7: Theory of Estimation)
On the otherhand Xo E acf'I
=
{
\037\037
Therefore
a}.
xoetf'l{ > a}.
(1-\037)
\037\037
This is due to the fact that 0 E (; and 0 E
(1 n
separate
>
{
\037\037
Xo
onecould
a}.(Otherwise
a contradiction to Xo E from (;andproduce
clearthat the openset U,\037 B
((1 i)xo.n satisfies =0. >a})
Uf'lS(C+Y)f'I{
\037\037
C).Now. it is
but
uus(c+Y)f'I{>a})\302\243cf'l{>a}.) \037\037
\037\037
This implies)
>
'(S(C+Y)f'I{ >a})d(cf'l{ a})).) \037\037
\037\037
Another application of the Brunn-Minowski theoremasin the proofof Lemma 38.20provesthe asserted inequality.)
solutionof the estimation Now, we are in a positionto presenta complete with linearrestrictions. problemfor Gaussianshift experiments and consider the experimentE = (H,1I(H), Let L Hbe a linearsubspace {NH 6,:tEL}),This is not a full shift experimentif L 4= /I. We dealwith the L IRk wherewe assume a linear function! problemofestimating w.l.g.thatf issurjective. Thismeans, that the decision spaceisD = and the lossfunction is of the fonn W; (y):=W (y [0,(0).We y E IRk, tEL. where W: iii\" estimateK = f 0 PL is the best possible shallprovethat the non-randomized
.
\302\243;
-
-
\037k
f(1\302\273.
-
choice.)
we shallfrequently 38.25Remark. In the proofsof the followingassertions
usea reductionof the problem. Let F = (H, {Nil.6,:t L kerf}) and letG:=K.f: To describe the experimentG we endowR k \302\243f(H),
with the inner
product) \302\253Yl'Y2\302\273:=(X
.X2)
1
jf
y,=f(xi),xi.lker.f.;=1,2.)))
38.Full With
shift
experimentsand the convolution theorem
199)
thisinner productwe have \037(KINH.6,) = NGlIc.6/(,), t
1.kerf,
and thus
G = (RJ:,EI\", {NRIe.6/(1):t 1.kerf}).) The pointis that G is a full shift experimentwhere the theory of the present sinceK = f 0 P,.isa suffcientmapping,G and can beapplied. section Moreover, F are statistically whether E and G are (Thequestion isomorphic experiments. makes no sensein ourframeworksince and G have isomorphic statistically \302\243
differentparameterspaces.)
Forconvenience Our first resultisconcerned with the convolution theorem. kerf) Nil. IE L, Q,= NRIl.Cf(')'1.1
we denote.P, =
C\"
38.26Theorem.Let e E g,(E,IRk) be an estimate H..hichis \"equivariantfor the in the estimation off\" followingsense: * (ePo), tE L. eP,= Then there existsa probability measureR such that e.P,= (f 0 Pd'p')* R, tEL.) \302\243/(t)
I \037k.
!i'
ProofSincee E fJt(E,IR\") and 9t(F,IRk) = 9t(J::,[Rk) we have e E !1i(F,IRk). The F and G are statistically and thereforethere exists experiments isomorphic t E 9t(G,IR\"} such that tQ,= eP\" 1.1 there existsRial\" kerf By Theorem38.15 such that
eP,= tQ,= Q,.R, t.lkerf, or in otherwords) (ePo)= NAIe. * R. t 1.kerf. SinceIE L occurson bothsidesonly throughf the equationis valid even for \302\243/(1).
every
\302\243/(')
IEL.
0)
Next, we show that
K
= f 0 P,.is a minimaxestimate.)
38.27Theorem.Assume that inf \"F\037(F..R\")
Proof Let e E Then)))
W
i.v
lowersemicontinuous and subconvex.Then)
sup JYreP,= J W(foPL)dN\". ,pl.)
91(\302\243,
H\")
and
choose't E 9t(G,IR\"} as in the precedingproof.
200
Chapter 7: Theory of
sup \037e\037 r L
\037
Estimation)
SUp
r.l ker f
E
= SUp
\037e\037
r 1 ker f)
\037tQ,
= J WdQo = J W(f\302\260PL)dN H
This provesthe assertion.
.)
0)
the uniqueness FinaJly,we consider problem.)
-
38.28Theorem.Assume that
the subconvexfunction W: IRk [0,(0)is leve/compact,separatingand of finite order.Then every equivariant estimate (!E 91(E,1R ) (in the senseof Theorem(28.26))with 1t
\037eP,= J
W(fopJdNll , tE L,)
satisfies U (x,B) = 1B (f 0
p,.(x\302\273
A.-a.e.,BE fill<
.)
as in the proofof Theorem38.26. ThenCorollary 38.23and Discussion 38.24imply that Proof.Chooset E 9t(G,
\0371)
'[(.,B)=1 8 Qo-a.e.,BE\037k, or in otherwords) J (1(.,B)dNH = J IB KdNH O
A
A)
for all BE gr, A c:K-1 (81\.")If we put
(1(.,B)dNll = NH(K-
=
A
1
f 1(--1(81)
(B\302\273,
,,-1(8) then it followsthat BEgr,
and hence)
e(x,B)= 1
if
fopdx)EB
A.H-a.e.
0)
carriesover to the casewith an It is clearthat the precedingdiscussion
scaleparameter (J > O. additional
since If P NH then the reductionusedin Remark 38.25is not possible This isoneof the reasonsfor the \" = f 0 P\" neednotbe a sufficientmapping. of the theory of the next section.) development =*=
39.The structuremodel) In thisparagraphwe consider which generalizes the caseof full shift a situation 38.))) considered in Section experiments
39. Thc slructurc model
39.1Example.Let P
201)
and considerthe n-fold product E\" IE n E {(P t,)\": !R}), N, of the full shift generatedby P. The on the samplespacen = Ii\" by translation groupG = IR operates gw = (WI + g,W2 + g. w\" + g), g E G, wE Q. ofQ. Each orbitisisomorphic Theorbitsof the groupactionform a partition can we with the operatinggroup.Therefore, identify the pointsWEn by the Let M = {wE L Wi = O}.This is a crosssectionof followingprocedure: i-I each orbit in exactly onepoint.Each WEn can be orbits,i.e,it intersects = (R\",\037\",
\037
AI\302\243itl
*
...,
!1: /I
in the followingway:) decomposed W
=
-1\" .L n
+
Wi
1=1
Denoting
-1 i-L
(
WI
- 1\" .... - -1\" .r ) . .1: 1-1 .-1 WI'
n
w\"
WI
n
\"
T(w) =
Sew) = we obtainW
\037
n
and
Wi>
t)
- nl=1 .L 1
(
WI
- nl=l) -1 L /I
/I
Wi>\"\"
W\"
Wi
),
T(w) + Sew), where
T(g + w) = g + T(w),
w
\037
n, g E G,)
and)
S(g+ w) = Sew). we n, ge G.) In otherwordsTisequivariant and S is invariant. Let (n,\037)be a measurable spaceand let G bea groupoperatingon Q.We assumethat G is a locallycompact groupwith countablygeneratedtopology and the operation Let ).,. @ ..
(\302\243I(G)
q:g 1-+g 1,g E G.)
The followinganalysisis basedon onefundamentalassumption.)
39.2Assumption.There is an (.>1.
\302\243I(G\302\273-measurable
satisfies T(gw) = gT(w) wheneverg E G and WE Q. The followingexamplegeneralizes Example39.1.)
map T:Q
- G,
which
39.3Example.Supposethat Q = H where(H,< ., . isa Euclidean spaceand \302\273
let L H be a linear subspace. Then G = L operates on H by the operation xw :=X + w, x E G, WE H. It isclearthat the orthogonal T = PL projection satisfies the condition of Assumption 39.2.))) \037
Chapter 7: Theory of
202
Estimation)
39.4Example.As beforelet a = HandL s;;; H bea linearsubspace satisfying = x dimL < dim H. But now let the semidirect G IR. L product operateon H + by (a,s)w:=aw + s, (0',s) E R x L, WE H. If we define = idH P,. and (.)II, 1'2)satisfiesthe conditionof Assumption39.2for 1'2= PL then l'= (1111 WE H\\ L. To seethiswe have to show that) 1\037
T(O'w+ s) =
-
(0',s) T(w)
= (O',s)(lll1 (w)lI, T2 = (0'11(w) II, a T2 (w) + s).) (w\302\273
\037
But thisis immediate since)
111; (uw + s)1I= lIu11 (w) + 11(s)1I= ulll1(w)1I) and)
1i(uw+ s) = uT2 (w) + T2 (s) = uT2 (w) + S.) ForW E L the function Tdoesnotmap into1R+ x L However,for ourpurposes it doesnot matter to repJace Q = H by Q = H\\ L sinceL is a subsetof show that Lebesguemeasure zero.Denotek = dim L. Easy computations da @ ).L(ds) is a left-invariantmeasure on G and ).,.(du, A(da,ds)= ds) = da@ ).L(ds) is right-invariant.
!
ak\0371
0')
39.5Lemma. The mapping S:WH (1'W)-IW , under the operation ojG, i.e.it satisfies) (1) Sew)= S(gw), (2)
WE
WE
a, is maximal invariant
a, gE G,
S(wl)= S(W2) implies Wt = gW2 Jorsome g E G.)
Proof (1)This followsfrom
-1(gw) = -1(gw) = (Tw) -1g-Igw = (Tw) -1W = S(w).
S(gw)= (2) WI
(gT(w\302\273
(T(gw\302\273
If 8(wl ) = 8(w2) then (Tw1f WI = (Tw2)\"
= Tw I
I
W2 . '(T0>2)-I
I
W2
and therefore
0)
each orbit in exactly onepoint.Every 39.6Lemma. The set im S intersects (J) = g . S where g E G and S E im S,namely admitsa unique decomposition WE !1
g = T(w),S = Sew).)
Proof The definitionof S showsthat Sew) and ware on the same orbitfor that im S intersects eachorbitat mostat one))) every WE a.Property(1)implies
39.The structurc
model
203)
of the asserted point,i.e.im S is a crosssectionof orbits.The existence = w Q. To follows from W E let w T(w) Sew), proveuniqueness decomposition = gs,g E G, se im S.Thisimpliesthat Sew) = S(gs)= S(s)= s and therefore T(w)s= gs.Applying T we obtain) T(w) T(s)=
T\302\253Tw)s)
and thereforeT(ro)= g.
=
T(gs)= gT(s))
0)
39.7Corollary.The setimSisin.PIsinceimS= T-1 {e}.) T(w) = e. If conversely Proof If s = Sew) = T-1 (w)w then T(s)= T-1 (w). .1 . = = sET-I{e}then T(s) e and thereforeS(s) T (s) S = s. 0)
39.8Examples.In the caseof Example39.3the maximalinvariant is S:x x x PL(x),X E 1/.For Example39.4we have S:x........ PL (x) -'x E /I. In the x PL(X)) secondcaseim S = L1 {xE H:!Ix = I}.
-
(\\
_-II
II
\037
11
39.9Remarks. At
the first glanceAssumption 39.2might seem to be very Morefamiliarare thc followingtwo assumptions: restrictivc. (1)Thereis a crosssectionM of the orbitsof G onQ. This means that for every wED the setM (]{gw:g E G}contains exactlyoneelement. The of all WE Q coincide. (2) (Theisotropic isotropic subgroups subgroupof WE D is (gEG:gw= w}). Let If G bethe commonisotropic SinceH isa normal subgroup subgroup. we may replaceG by the factorgroup G/ /I. Then the commonisotropic subgroupof every WE Q is H = {e}. It isclearthat our Assumption 39.2implies conditions (1)and (2) with M = im Sand H = {e}.However, apart from measurabilityquestions alsothe converseis true sinceeachro e Q admitsa uniquedecomposition as w = g . s with g E G and s EM, Defining T(w) = g iff ro = g . s yieldsa mapping T:Q -+ G which satisfies T(gw)= gT(w),g e G,we o.) \037
39.10Definition,Let PI.saf be a probabilitymeasure.For p(g-IA), A E d. Then the experiment E = (D, let\037: A
every
gE G
g E (i}) is calledstructuremodelgeneratedby P if the groupoperation satisfies condition \037
.\037, {\037:
39.2.
The casesconsidered in Examples39.1, 39.3and 39.4are structuremodels. In the followingwe denotePs:=\037(SIP).It is clearthat Ps = \037(SIPg),
g E G, (by
(39.5\302\273.)))
204
Chaptcr 7:Thcory of
Estimation)
39.11Remark. We definethe measures) A.t * Ps:A H H l (gs) ;.t(dg) Ps(ds), A E.\037, and A, * Ps:A H H 1...(gs) ).,(dg)Ps(ds), A E .91.) A
are O'-finite sincel(and A, are O'-finite and P::;is It isclearthat bothmeasures finite.Moreover, we observethat) (l,* Ps)(A)= J t1 0 T del,* Ps), A E .tI. ...)
whereA isthe modulusfunction of the groupG.Sinceim * Ps.) that At * Ps '\"1..,
A
\302\243
(0,00)it follows
39.12Lemma. Every structure modelisJ.,-integrable sinceP(A,) = i.,* Ps.) Proof If A E .91and g E G then we have A) J p(g-1
A,
(dg) =
;.,(dg)P(dw) IS 1...(gw) = JJ 1... (gT(w).r) A., (dg) Fs(ds) = JJ 1... (gs)i.,(dg) P.r;(ds)= (A., * Ps)(A).
0)
39.13Lemma. Themeasurei.(* Ps is invariant, i.e. (A,.Ps)(gA) = (A.t * Ps)(A), g E G, A Ed.) Proof For gl E G and A E.c{we have
Ps)(g1A) = H l\"...(gs)A. t(dg)Ps(ds).) Since1\",... and sinceAI. is left invariant on G the assertion (g.\\') = 1...(gj'\"lgS) (A., *
follows.
0)
39.14Theorem.For
every structure
equivalent:
(1) P 4.A,. Ps, (2) (3) (4) (5)
.\037
i.\\'
dominated,
9 l, * P.r;, \037
E is weakly continuous, E is continuous.)
Proof (1)<:>(3):Let A E
d.
Then)))
model the following assertion.\\' are
39.The struclurc model = Peg-1A) = \037(A)
(hs) II 19-IA
205)
.
dP deA, Ps) (hs) A,(dh) PS(ds)
-
- (hs)i.t
d(I., Ps) dP -1hs\302\273).,(h)Ps(ds). = SI1A (hs) d .(g * P d(A, s))
'.
Thisimplies \037
\037
i'l * Ps.
-
Qld
measuresuch that Q {P,:gE G}.We be a probability (2) => (5):Let claim that Q Q(A,), Indeed,if Q(A,)(A) = 0 then there existsagEG such that Q(g-1 A) = 0 for every hE G and therefore A) = O. This impliesp,,(g-1 = PlICA) = 0 for every hE G.HenceQ(A) O. By Lemma 39.12we have Q(I.,)= A, * Qs and Remark 39.11implies * '\" * therefore At * Qs. A( * Qs A, Qs. It follows that Q i'l Qs and that * Qsis an invariant measureand therefore Lemma 39.13 implies \037
9
\037
\037
).\"
(w) =
d\037
d(\037(-\"Q)
dP
d().,* Q)
(g
. Q-a.e.,g 1.(.
-1w)
showsthat g\" The Lemma of Riemann-Lebesgue
.
hm \"\"'00
I
.
.dP .dPQs)(g-1hs) - d(A(.
G.)
-g implies
=0 hs) ;.,(dh) (g;1 Q.(})
-dCA,
E
for Qs-almost every S E im S.Hence
.
I1m \"... (0
. Qs) _ (A,d()., Qs) d(i., Qs) dPg
d\037\"
*
*
lemma (Remark and Scheffe's
proves(5).
(3.1\302\273
(5) \037 (4):Obvious. (4) \037 (3):Weak continuity implies P(A,). Since follows. PC),,)= A, - Ps'\")., Ps the assertion (3) ::>0(2):Obvious. 0) \037
\037
\302\267
we see immediately that condition(3) may be consequence to 9 A., * Ps sinceA, * Ps = P(A.,) f!J and A., * Ps\"'\"A, Ps.) strengthened 39.15Examples.(1) When we are dealingwith a situationlike Example39.3 dP P )'H'Let h = - -;-.It followsthat then we usually assume
-
As an easy
\037
\037
peA) = J h(PL(X) + x
- PI.
d\"'H
(x\302\273
A)
A.H(dx)
!
= J J hey + z) i.ddy) L (dz) = I hd(AL.;'L!) A A.
A)))
\302\267
206
Chapter 7: Theory of Estimation)
for all A
\342\202\254
we have \037(G).Therefore
dPs (y) = hex + y)J.ddx), YELl.. I
dIL\037
with Example39.4. Herewe observethat (2) Thingsare morecomplicated peA) =
=
!
-
x
-
h(lIx p,Jx)H _ PL(X) + Ilx PL(X)
)'H(dx)
pdx\302\273
II
z
+ J!h(lIzlI'lI\037
Y)
),L(dy)ALJ.(dz),
A
e 14.)
Let t be the uniformdistribution on {ze [): II z /I = I}.Usingtransformation to coordinates onesees that is with polar 1'-density ,91(II.III),L\037) a measure q...... C q\"-lc-I, q> 0, where n = dim Handk = dim L. Thenit followsthat) peA)
1 = JJ J CrL'h(as + y) k+T dat(ds)),L(dy), A
A E
oc)
a,
and therefore)
dPs
dt
(s) = SSCoc
1
\" h(oc.\037
+ y) (rc +l doc),r.(dy),) sEimS.)
39.16Lemma. Supposethat G is Abelian.If .!t>(TIP)(x) * 0, X E G*,then the SlrucluremodelconsislS mea!1'Ure.r.) 01pairwisedifferenlprobahi/ily Proof.Assumethat
\037I
=
\0371
for someKI'K2 E G.Thenit followsthat for every
XE G*) IX(T(w\302\273\037I(dw)
=
fX(T\302\253(I)\0372(d(l)))
and thereforeX(gl) = X(g2)for every X E G+.Thisprovesgl = g2'
0)
39.17Examples.Theassertion of the preceding lemmacan be verifieddirectly in particularcases.Let Ie(10 (H).If in Example 39.3there existsan aE L, a * 0, such that
= P then PM = P for all n EN which implies JI(x + na) P(dx)= JI(x)P(dx), \" EN. \037
-
-.
SinceI(x + na) 0,x E H, as n 00,it followsthat P = O. leadto) considerations As for Example39.4similar
u\"-1
II(a\"x + q _ .-a)P(dx)= f/(x) P(dx), n EN,) which admitsonly
P = 0, or P = 1;\"/(1 - a) if q *
1.)))
39.Thc structurc
modcl
207)
39.18Lemma. The invariant a-jie/d
d = {A Ed:gA = j
A
if g E G}
isgeneratedby the mappingS,i.e.A c .f1f, iff A = S-1(A).) Proof Followsfrom Lemma 39.5.
0)
which is provedin the followingtheoremisbasicfor the The decomposition
modelisa mixtureof full It isshownthat a structure rest of the presentsection.
shifts.)
structure 39.19Theorem.Supposethat E = (0,.>1, {\037:g E G})is a dominated a im exists there model.Thenfor every s E S fullshift = (G, that: E such {P;:g GJ) \302\243S
\302\243I(G),
(1)
.91i'
under distribution is a version of the conditional
\037
of T given
A, = {hE
G:hs E A},
(J) 1-4!'i(tIJ)
(2) Pg(A) = J J>;(AJPs(ds), g E S E im S.)
G,
A
E.>I, where
Proof First,we notethat for every CE 5B (imS))
-1dP
= P{SEC}= Ps(C). (hs)).Adh)Ps(ds) H C d(Af * Ps))
Thisimplies that
.1.(dPP ) (hs) ).Adh) = 1
J d(
\302\267
s)
Ps-a.e.
For thoses E im S for which the equality holdsdefine)
P;(B)=
!
(g-Ihs) ).,(dh), BE!M(G), g E G.)
d()':\037 Ps)
For the restdefine {P\":g E G} to be any shift experiment. It is then clearthat for every s E im S the experiments) ES:=(G,fM(G), {J>;: g E G}))
areshift experiments. and Ce:\037(im S) (1)We have to show that for every BE \037{TEB,SEC}= J P;(B)Ps(ds),gE G. C) Thisis easily seenfrom))) \302\243I(G)
208
Chapler7: Thcory of Eslimalion)
dP -1 (g hs) ).,(dh) t dO..,... Ps) = \037{h.\\': he B,.\\'EC}= Pg{TEB,Se C}. needonly be provedfor setsA e .91of the form assertion (2) The second A = {hs: hE B,sEC}, Be \037(G),CE\037(imS).
Ps(ds)= cJ J>;(B)
r r \037
But thishasbeendonealready under (1).
D
39.20Discussion (Fisher'sfiducial distribution). (1) Let us first discussthe caseof a full shift.HavingobservedWE Q onemight beinterested in particular . the covering of the randomset(J) B where Be 8d(G).It isobvious probability a moment's reftectionthat by = \037{WE Q:ge w. B} PCB), Be ii,ge G. Theinteresting feature of thisequation is the fact that the right hand doesnot dependon g e G. Therefore,PCB)can be interpretedas the probabilitythat w. B coversthe unknown parameterg E G. (2) Now, we turn
a similar to structuremodelsand attempt to establish
relation. The preceding Theorem39.19 showsthat
P:{hE G:g E h . B} PIl(B), B E
g E G,S E im S. Again, the right hand doesnot dependon g E G and thereforethe number of the random setT(w) . B,given Sew).We PS(WI(B)isthe covering probability into may put the covering probability =\"
PS(WI(B)=
\037\037:'I
\037,
(T(w). B).
has beencalled It is by R.A.Fisher. fiducialdistribution of the mostimportantcasein statistics wherea posterior an analysis experiment is possible without using a priordistribution. of fiducialdistributions and posterior distri(3) Thereis a strongrelation butions.Let us provethat,independentlyof the choiceof SandT the fiducial with the posterior distribution for Ar. distribution coincides = Choosew h.'iand B E (G).Then
Thedistribution
\037\037:\037
\037
J B
dP
d{;.,-.i>s)
-1
(g w) ).r(dg)= J s
= J I B (g -I)
dP
d().(.A)
-1
(g hs) )'r(dg)
dP
d(;',,,,Ps) (ghs) A(dg) dP
= L1(h)J 1s(hg1) (gs) A.,(dg) d(A,... Ps) =
.1
(1'(w\302\273
J ls(T(w)'g-I)
d(A\037=
Ps)
(g'S(w\302\273;'(dg).)))
39.Thc structure
model
209)
F for ).,is) distribution Hence,the posterior dP r w ).,.(dg) (g * f's) d(J.( F(w,B) = dP -I ' (g w ) 1_,.(dg) J
, -_.
I
\302\273
d(J.;-:-P.<;)
-
J 1B(T(w)g-1) d()'\037\037-P\037
_.-.-
J
d(
)'\037\037
(g'
(g.
S(w\302\273
S(w\302\273
A(dg)
).(dg)
\037<;)
= J 1B (T(w)g t)PS(I4)(dg) =
P\037\037c:J)(B),)
Be a. This leadsto a simplerformula for the covermg probabilities: p.<;(6))(B)= F(w, T(w) . B), WE !l, BE forA,. satisfy an invariancecondition, distributions namely (4) The posterior where we fl,
a.
. gB)= {>S(w) (Tw' B) = F(w, B), we fl, Be \037(G), ge G.
F(gw,gB) =
pStgW)
I
I
\302\253T(gw\302\273'
of the coveringprobabilities: This leadsto a further modification PS(C/)) (B)= F(S(w),B), wED, B E \037.
It must be stressedthat the fiducial methodis only a computational T satisfying Assumpestimator methodand doesnotdistinguish any particular choiceof T shouldbesuch that the coveringprobabilities tion39.2.A suitable T are large in somesense.E.g. if for someB E \037(G) there existsan estimator (5)
and satisfyingAssumption(39.2) F(w, T(w) . B) = supF(w, g . B), leG)
wED,
then the confidence regionl'(w).B hasmaximalcoveringprobability. of remarkscan beextendedliterally to the computation (6) The preceding risksgiven S(w).Since(4) implies conditional 1 J1B(h-g)P;(dh)= J1B(h)F(s,dh),SE imS, Be \037(G), ge G, risk with respect to a lossfunction W: G [0,00)is the conditional
-
I)F(s,dh),seimS, geG. fW(g-lh)PgS(dh)=JW(h In otherwords,given Sew) the conditional risk of T(w) is = J W(h- I . T(w\302\273F(w,dh).))) J W(h-I)F(S(w),dh)
210
Chapter7: Theory or Estimalion)
But the latterisnothingelsethan the posterior risk of T with respect to A., and lossfunction (W(g-l.\302\273gI!G' Thus,in caseof structuremodels,we have
the
risk as a conditional risk without of the posterior obtainedan interpretation In this sense, with Bayesestimates any needof a priordistribution. generalized conditional risk functions. can viewed be asestimates respectto A., minimizing In the followingit is our aim to find estimators which are optimalwith risk functions. It will turn outthat thisisthe same to the unconditional respect risks.) classwhich minimizesthe conditional
of estimates). Remark (Decomposition We keep the notationof (39.21) Then we have for every bounded,\037-measurablefunction Theorem39.19. fQ-+1R \037(fldi)= Jf(hS)\037s(dh), gE G.
another Consider an estimate e E (E,G).Forevery SE im S we may construct s x estimate e':(h.B) e(hs.B).(h, B)e G a.It isclearthat (le fJl (E G),and
.
\037
\037
we have)
ePg = Je'P;Ps(ds),ge G.
Thisfollowsfrom)
'
I';
= JJ e(hs,B) (dh) Ps(ds) (B)Ps(ds) J (e 1':> = J e(en,B)Pg(dw) = (e\037) (B), BE91.)
.
39.22Definition.An estimate e E (jt (E,G) is equivarianriff e\037 = eg (eP)for every g E G.)
39.23Definition.An estimate e E [Jt (E,G) isa convolutionkernelif e(gen. gA) =
\037(en,
A)
for all en eO,g e G, A e (G).) \037
-
It is clearthat every convolutionkernel is equivariant.A non-randomized estimate K: 0 G is a convolution kernel iff K(gen)= g K(en),g E G, WE Q.)
39.24Remark (Decomposition of convolutionkernels).If e E at (E,G) is a
convolution kernel then we may write e(en,B) = e(T(w)Sew),B) = e(S(w),T-1 B), (l) E Q, BEtM(G). If we denote Q'(B):= u(s,B). SE imS, Be \037(G) then we have e(hs,B) = Q'(h-1 B), hE G, se imS, Be a (G).It followsthat in this case(with the notationof Theorem39.19).) \302\253(1)
(B)= JJ e(hs,B) J>;(dh)Ps(ds)
(e\037)
\037
HQ\037(h-lB)J>;(dh)Ps(ds)
=J
(I';.Q')(B)Ps(ds).)))
39.The slruclurc model
211)
Thisexplains the term \"convolution kerne)\".Conversely, eachstochastic kernel defines convolution k ernel a to .\037EimS, .n-+Q$I\037(G), according e(w,B) 1 =
QS(W)
{T- (w).B}.
The followingis an extension of the convolutiontheorem to structure
models.)
39.25Theorem.Suppo.\037e Ihal G i.'I amenable.Then for every equivariallle.'Ilikernel s Ho QSIfJI (G),s E im S,such mate e E 9t (E,G) there existsa stochaslic that) Q\037
=
J(p;.QS)Ps(ds),gE G.)
Proof For Q e (Jt (E,G),se im S,define Q\037:
BHo A.
1
Be a(G), ne\037, ,(K,,) H e(hs,hB)P;(dh\302\273).,(dg), K...)
where (K,,) is a summingsequence. Then it followsfrom Theorem38.14 that)
.
hm \"-o\037
1
sup
1
BI!!I(G)A, (Kit )
j (e$P;)(gB)
A.,
..)
(dg)
- (ps..Q:)(B) = 0,
S E im S. Let IE
denote
AltU'.
q\302\273
= \037 J II ) \"1
A,(K
and)
\"..)
S
[JJ/(g-lh) (e P;)(dh)cp(s)Ps(ds)];',(dg)
(F.Q:)(dh) q>(s)Ps(ds).)
B,.U:cp) = JJf(h)
of bilinear. functionsand by By weak compactness
Theorem6.11there is an
ne N, accumulation pointQ:sHoQ$I\037(G)of the sequence QII:sH>Q\037I&f(G), t x on ttoo(G) L (imS,Ps) and a for the topologyof pointwise convergence such that) s; N subsequence No lim A,,(f,cp) = lim B,.(f,cp)
\"eNo
liE
=
1\\:10)
H/(h)(PS.QJ (dh) q>(S)Ps(ds))
for allIE ttoo(G)and q> E V (imS,Ps).Sincee is equivariant we have P, fE\037oo(G},nEN,) A,,(f,1)=le and therefore) P = Hld(PS * QI)Ps(ds), fE '6'00 (G).)))
le
212
Chaptcr 7: Theory of Estimalion)
Since)
1= 1q P = JJ 1d(?* QS)Ps(ds) it foHows that Q:s Q\" can be adjustedPs.a.e.to be a non-degenerate kernel.This provesthe assertion. 0) stochastic \037
39.26Corollary.Assume that G is amenable.Then for every equivariamestimate there existsa convolutionkernel with the same distribution. Let W: G -+ [Ot (0)be a lower semicontinuous functiont let the lossfunction be (W(g-l. and definethe risk of an estimate l!E fJl (E,G) by \302\273g6G'
W(g-ly) e(wt dy) \037(dw), gE G.) It isclearthat the risk of an equivariantestimate isindependent ofg E G.If e is S a convolution kerneldefinedby Q:s\037 Q I.1l(G) then \037e\037:=H
\037e\037
= II W(g-ly)(\037 * QS)(dy) Ps(ds) = H f W(g- 1xy) E>;(dx)Q\"(dy)Ps(ds) = f If W(xy) PS(dx)QS(dy)Ps(ds) = IS W(y)(ps QS)(dy) Ps(ds).)
.
39.27CoroUary.Supposethat G is amenable.Then for
every estimate
kernel Q:s QS (G) such that e E (jt (Et G) there isa stochastic supU-;e\037 JJ W(y) (PS * QS)(dy) Ps(ds) \037
I \037
\037
gEG)
fUllctionw:) for every level-compact
copythe first part of the proofof Theorem39.24.Then we obtaina possiblydegenerate S E im 5t satisfying) kernel s lim A,,(J: 1)= Hfd(}>\"* Ps(d.\037), fE rrJoo(G).
ProofWe
may
\037
Q\037t
Q\037)
\"t;
No)
Let QS:=
Q1\037G)
measure Qbt whereQb(G)> Ot and QS an arbitrary probability
on \037(G),elsewhere. Then it followsthat inf fe\037
\037
gEG
for every
lim A,,(f,1)
\037
\"eNo)
ffd(ps
fe fto(G)+.Hence,we
fE \037b(G) supf()\037 g.G)
\037
.
QS)Ps(ds)
have for every
Jfd(ps.QS)Ps(ds).
The proofis finishedby Lemma 6.4.
0)))
function level-compact
39.The structure:
model
213)
of the lossfunction W. Note that Q is independent in as Section 3 8 we conclude that in caseof an amenable groupan Similarly We have to discuss the optimalequivariant estimateis a minimax estimate. how to find uniformlyoptimalequivariant estimates.) question
39.28Lemma, Supposethat there is a measurablefunction tp: im S -+ G such that for every y E G
J W(xrp(s\302\273 PS(dx) J W(xy) PS(dx) \037
Then for the convolution kernel K:
m.-T(m)
Ps-a.e.) tp
(S(w\302\273,
WE
fl, the following
assertions holdtrue: K is (1) uniformly optimalamongallconvolutionkernels. (2) If G is amenable then K is uniformly optimalamong all equivariant estimates andisa minimaxestimate.) Proof (1)The risk of K is J
W(K(W\302\273
P(dw)= II
W(Xlp(S\302\273
PS(dx)Ps(ds).)
From the assumption it followsthat K minimizesthe risk over all convolution
kernels.
of (1)and of Corollaries 39.26and 39.27. (2) This is a consequence
0)
It is rather obviousthat the estimates definedin Theorem39.24are (\037)!HGunbiasedif \037:=W(g-I.),g E G.) The final assertion of thissectionwill be that uniformly optimalequivariant estimates can be obtained as generalized for A,. In view of Bayes estimates Discussion 39.20(4) and (6),theseare just the estimatesminimizing the conditional risks.)
39.29Theorem.If a non-randomized generalized BayesestimateK with respect to ..t,is a convolutionkernelthen it is uniformly optimalamong allconvolution kernels.Hence,if G is amenable then K is uniformly optimalamong all and minimax.) equivarianrestimates
we obtainthat a dominated structure modelis ).,Proof FromLemma 39.12 = Discussion 39.20 shows that F integrable. \037. Now, let\"bea non-randomized, generalized Bayesestimate. By assumption, K is even a convolution kernel.We notethat for all WE and hE G)
!l
f
W(gT-t (w)h)
(dg) = fW(g-th)Pf\037\037(dg) = J W(g-th)F(w,dg).))) pS\302\253(O)
214
Chapter 7:Theory of
Estimation)
The defining propertyof generalized impliesthat for every Bayes estimates
heG)
PS(tAJ) (dg) J W(g TJ W(gh) p\"\037(tAJ)(dg) &'-a.e. SinceK is a convolutionkernel,we have T- 1(w) K(W) = where II' the condition of Lemma 39.28. satisfies This provesthe assertion. 0) t
\037
K(\302\243O\302\273
\302\253(0)
Ip(S(\302\243O\302\273
Let us consider someexamples. the caseof Example39.3. 39.30Example.Consider DenoteII = dd:. 11
2 = 11.11 and that F satisfies appropriateintegrability Thenit followsfrom Example37.6(1),that conditions.
(1)Supposethat K(X)=
W
- a) J.dda) xEH,)
f ah(x
fh(x\037a)l.dda)-' isa generalized and a convolution kernel.Thereforeit isoptimal Bayesestimate in the senseof Theorem39.29. If W is notboundedthen assume that F (2) Let P = NHand W subconvex. satisfies conditions. In view of Example 36.11 appropriateintegrability (1),a estimate isobtained generalized Bayes by minimizing) z'
W(z' - a) e-'2llpd.x)-aIiZ A.L(da). z' 1
L. we seethat for every x E H. From Anderson's Lemma (38.21) \037
f
E
is a a i t estimate and convolution kernel. Therefore is Bayes optimalin generalized the senseof Theorem39.29. K
= PL
39.31Example.Now we consider . the caseof Example39.4.Let II = d d\037
2 and (1)Supposethat W = 11.11 conditions. Thenit followsthat)
=---( - a ) JIh
F satisfies appropriateintegrability
X a ---1 \037d(1).,Jda)
----------.
fJ ah
K2(X)
I.H
that
(J
X
-
0-\"+3
1
XE
H,
(--(1 ) (;n+.jd(1).dda))
and a convolution is a generalized kernel.thereforeoptimalin Bayesestimate the senseof Theorem39.29. If W isnotboundedthen assumethat (2) Let P = NHandW besubconvex. Then a generali7.edBayes F satisfiesappropriateintegrability conditions.
isobtainedby minimizing))) estimate
40. Admissibility Z'
-a
1
of estimators
215)
1
2 Z'H H W ---;;--e-a}IIP1.(Z)-1I11 a,,+1ldda)dG, z'eL, ( )
for every x E H. From Lemma 38.21we see that the integralover L is minimized by of (1 > O. Hence\"2 = PI. is a generalizedHayes z' = PI.(x), independently and thereforeoptimalin the senseof Theorem39.29.) estimate
40.Admissibility of estimators) In this sectionwe discussbriefly somewell-knownfacts concerning admissibility of estimates. Let E = (Q,.r:I,{Pc:tEL})bea dominated structure modelwherethe operatwe denotethe a Euclidean is For convenience ing group space(L,<., of L on by + \".Let W: L [Ot 00) be a subconvex function.) operation
!l ..
.\302\273.
\037
40.1DefinitioD.A generalized for (W (t Bayesestimate a Pitman estimate for W.)
-.
\302\273,
ELandA L iscalled
40.2Examples.The followingis a particularcaseof Example 39.29.Let Q = and L = IR. Definethe operation by \037\"
t
+ W = (t + Wit
..., t
+ w,,),
t E IR,
WE
R\".)
is) (I) If W(t) = t 2, t E R, then the Pitman estimate
K(W)
=
Jt
If
(ll)
(jAH
dP
S
- I)dl
-, -dP
d).H)
(w
wEQ, tE IR.
t)dt
=
Itl, tE R, then any Pitman estimateIS a median of the hencesatisfying) distribution, posterior (2)
Wet)
a(w)
dP
(w
JI)di;, + 0() dP
- I)dt
-f -(w-t)dt d).H)
1
2'
wEQ.)
0()
-
If Wet) = 1 1(0.()(I), IE IRt then the Pitman estimates are calledmaximum probability estimates sincethey satisfy))) (3)
216
-
Chapter 7: Theory of
J
dlH
x\302\253(J))-c
(4)
IfP =
- t)dt= uR J-CJ dl'dP,- - t)dt, !1. s+c
dP
X((J)+(
Estimation)
\302\253(J)
SUp
-
!
(J) E
\302\253(1)
H)
N R \" then in any
caseK(W) = n
i:.
i-t)
Wi>
WE
a, isa Pitman estimate.
40.3Remark. Extending the precedingexample,supposethat Q = H, If P = NH then (H,< ., a Euclideanspaceand L H is linearsubspace. of K = PL within the classof equivariant Theorem38.28provesadmissibility estimates and (takef= idL).Thisresultcouldbeextendedto P * NH if \037
.\302\273
-
:\037
of Corollary38.23,usingthe decomposition of assumptions But evenfor the Gaussian caseP = NH it isknown that K = PL Theorem39.19. isinadmissible assoonasdimL > 2,(Jamesand within the classofallestimates W
satisfy the
Stein [1960]). We are interested in this sectionin admissibility within the classof all estimates. In view of the precedingremark we confineourselvesto the case = dimL 1,i.e.L = lit)
40.4Definition. An estimate if for any further (l e .t1t (E,R) isalmo.rtadmissible (1E rJi (E,IR) the inequalities) estimate J.v,0'J; J.v,eJ;, tE!R,) \037
imply)
=
J.v,(1p,
all t E R.) 0P, for ).-almost
J.v,
40.5Lemma. Let W(t) = t 2, t E R. Supposethat (l E fJl (E, isalmostadmissibleand satisfies H (x t)2e(W, dx) P'(dw)< 00, t E IR. and uniquely n01lrandomized Then it followsthat (l E 9t (E,R), (l is admissible, determinedby itsriskfunction.) \037)
-
-
momentsare finite it followsthat e(.,{+ 00, co})= 0 ProofSincethe second (E R. Hence(!E 91(E,IR). Then p'-a.e.,
-
J (Jlx \037
e(w,
P'(dw) H (x 1)2e(w,dx) (w) < 00, tl
dX\302\2732
P,
that implies Jlxlu(w,dx)< co p'-a.e.,IE IR.) estimator))) Hence,the non-randomized
t E IR,
40. Admissibility
of tstimators
217)
,,=Jxe{.,dx) is well-defined. yield Easy computations -- W; K .p, = SUX2(l(W,dx) (f X(l(w,
-
W; Q.p,
P'(dw), t E\037,)
dX\302\2732J
we arrive at) and,since(l isalmostadmissible
p'-a.e.,tEfR.) Jx2Q{.,dx)-<JxQ(.,dx\302\2732=O This means,that Q is non-randomized sinceit coincides with K. Now, let (1E .rJt (E, be an estimator satisfying) \037)
W;O'.P,\037W;K'p'. tEIR.)
Then a is almostadmissible, too,and can be replacedby a nonrandomized estimator \"1:Q R. It followsthat
-
-
I(K1 -/)2d'p'= J
J
K1
(
+K 2
-I)
-.-!-.f
2
dPr + J
=
K
(
I
2 f (\"t
K
2 I
) dp'
- I) d'p'+ 2 s 2
I
(K
t)
2
dp',)
IEI1i.)
This implies of K and K l' that by admissibility S (K 1
whenceK 1 =
-,,)2dp,= 0, ).-a.e.,) K
Ie IR. 'p'-a.e.,
0)
The followingassertion containsthe basicideaof Blyth'smethodfor proving
-
Let h(t) = s(lt I).IE IR, be a positiveand continuous admissibility. probability such that s: i s w e Later, shallput density [0,00) (0,00) decreasing. h(t)
\037
n(1
\037
/2)
' IE R,
and h(/) =
vk
exr
( ), \037
40.6Theorem.Lei(!be the Pitmanestimatefor W If lim (J J.v,eP,' h(8t)dt inf J J.v,a P,'h(a)dt)= O. ,-0
-
tI (\037(E,R))
then Q is a/most admissible.)))
IE\037.)
21H
Chapter 7: Theory of
Estimation)
ProofAssumethat (1E tit (E,A) satisfies)
tER.
J.Y,G.p,\037 J.Y,u.P\"
If (!, denotesthe Bayes estimatefor Wand the prior distribution Jl\" : t 1-+th (tI), 1E R, then we have
\037'
J J.Y,eePrJl.(dt)J J\302\245,a.P,Jl.(dt) J J.Y,e\037Jl.(dt). \037
\037
the sets) Consider Ma.h =
{III< a, J.Y,eP,> J.Y,u P, + b},
Then)
a> 0, b > O.)
-
J J.Y,qP,JlI(d/) J J.Y,C!tPrJle(d/) \037
J.Y,(1.P,)Jle(dl) f(J.Y,e.P,-
\037
bJ..(MII,b)
\302\243h(\302\243a).
The assumption then implies that
l(Ma.b) = O. Hencethe assertion.
0)
Let us consider 40.7 Discussion the caseof Wet) = 12, tE lit If (Stein[1959]). and has a finite second is) moment,then the Pitman estimate PI\037l iscentered dP
K(x)=Jtdl (x-t)dt=x-JzP(dz)=x,xEIR.)
The Bayesestimates K. for the priorsJl.are given by) KC(X) =
J If;(x,dt), x E
IR,)
the posterior of E and /J.e' Thisgives) where f; denotes distribution J J\302\245,KPrh(\302\243/)dt- J J.Y,Kc.P,h(et)dt
- (dx)Jl. - -t1 JI (x) - (dx) 1 -2t(x= - H (x t 1 - 2/(x- f;(x,dt)Ps(dx)/J.,(ds) = - HI (x - \":(X) t 1 - - (x = ./J..(ds) t II (x = -1H (x P'(dx)/J.c(ds). = 1 JJ (x t
-
1)2.P,
2
(dr)
K;(X)
(K.
t)2 .P,
Ke(x\302\273)'p'(dx)/J.l(dt)
2
K.(X\302\273)
2
K;(X) 2K,(X)
Ke
(X\302\273
t)))
2
K,(X\302\273
\037(dx)
/J.,(dt)
of estimators
40. Admissibility
Since)
219)
..
-I)dP d).(x I)h(el)dl X-K\302\243(X)=J(X-I)\037(x,dl)= ---iP----.J (x -I)h(el)dl) J (x
.
d).
we obtain
J I-\302\245,KP'h(tl)dl-J I-\302\245,K,P,h(et)dl =H
- t) dP (x - l)h(t/)dl dP dJ. -------n. ) Jl (x - t)h(tt)dtdx dj>- J (x t)h(r.I)dt
J (x
2
-
(
fiX
h(t(z-
= J ($zh(e(z f
p(dZ\302\2732
x\302\273
P(dz))
x\302\273
dx.
in a suitable choiceof h such that the last Thus,an admissibility proofconsists tendsto zerofor t O.) expression
-.
Let Wet) = t 2, t E [R, and PI\037I such that 40.8Theorem(Stein [1959]). J z P(dz)= 0, J Z2 P(dz)< 00, Then the Pitman eSlimateK = idR is admissible and uniquely delermined by itsriskfunction.) the preceding discussion and notethat) Proof We continue
-
- -
= (Jz(h(e(z h(tx\302\273 (J zh(t(z h(tX\302\2732 P(dz).) J Z2 P(dz)'J (h(t(z x\302\273
p(dZ\302\2732
\037
- -
x\302\273
p(dZ\302\2732
x\302\273
for t Moreover, J
h(\302\243(z
\037
-
1 we have)
x\302\273
P(dz) P( \037
-1,+ 1).)
-I
h(tx+y).
inf \037)I'\037
1)
Now, we specifyh(x):=\037.' _l_-i'XE lit Since 1t
1+x
1+ (z = 1, 1+ +21)2 z) z-\037 lim
there issomec> 0 such that) inf
h(z+y) c'h(z), ZEUi. \037
-1\037)';:;1)
Hence,we may
estimate)))
220
Chapter 7: Theory of Estimation)
-
(J zh(r.(z dx J P(dz) J Const.J Z2 P(dz). JJ (h(t(z x)
-
h(\302\243(z
p(dZ\302\2732
x\302\273
- -
x\302\273
\037
h(tx\302\2732
/ h (tx)dxP(dz).)
we have) Moreover,
- -
J (h(e(z x) = Jh 2
h(ex\302\2732
(\302\243(z
h
x\302\273j
/ h(ex)dx
2 (ex) dx ----J
= J \"--
h(r.(x+
-
-
h(f;x)dx 2J h(f;(Z
x\302\273dx
+ J h(f;X)dx
h(\302\243x)dx
z\302\273
1 1 1 1+ e2 (x + Z)2 2 2 =it (1--t-e)y.dx-\037S-f+\037xidx 1 1 xz + Z2 dx = =1 dx = J 2 \"2-') Z2,t (1+ e2 2)2 n (1+ X )2 \"x
;
\302\2432
2\302\2432
\302\2432
\302\2432
Thus,we obtain) J
(J
zh(\302\243(z
-
h(e(z_
x\302\273
x\302\273
z.)
dx = e.Const.(J z 2 P(d
p(dZ\302\2732
P(dz)
Thisprovesthe assertion. we obtaina As a consequence
\302\243Z2
2
\302\273
0)
model.) generalresultfor an arbitrary structure
40.9Theorem(Stein[1959]).Let E = (fl,d, {p,:t E R}) be a structure model convolution kernel 10:Q R, and assume that there existsa non-randomized for W (t) = t 2 . t C IR, whichhas afinile fourth moment.Then the Pitmanestimate and uniquely determinedby itsriskfunction.) is admissible
-
Proof Let So be the maximal invariant basedon and F the posterior is (by Discussion for Then the Pitman estimate distribution dz) F(So(oo), dz) = J (z + T{oo)= J zF{oo, = 10(00) Po (lOld) (00). = J z\037o(W)(dz) + 10(00)
l.
\037
(39.20\302\273)
-
1O{oo\302\273
j
It followsthat T has a finite fourth moment,too. Let S be the maximalinvariant basedon T and denoteby p's,t E IR, s E im S, of T given S = s. We have to show that) distribution the conditional
-
-
lim (H(T 1)2dP,h(tt)dt H (K,
,-.0)
-
= t)2 dP,h(tt)dt)
for the prior#l\" where K, is the Bayesestimator
0,
d;.!(x) = eh(ex),x e IR.
We)))
40. Admissibility have)
-
of cstimalors
221)
-
H (T 1)2dp,h(8t)dl= J H (z t)2P'$(dz)h(t;t)dtPs(ds),) and)
H (Kt
-
= J H (Kt(Z + S) t)2 dP,h(r.t)dt
-
t)2 P,s(dz)h (tt)dtPs(ds).)
z H Ke(Z + .\\'), z E R is the It isclearthat for every fixeds E im S the estimator for the conditional t E R}) and Pl' experimentEs = (R,\037, {P'$: Bayesestimator 40.7to each of theseexperiments, it remainsto show Thus,applyingDiscussion that)
dx Ps(ds)- 0. PS(dz) J h(t;(z 1 1 and considering the proofof Theorem Specifying (x) = . l+x2' lim JJ
\037J
\037
-=
\037\037(\037
l\"'O
\037\302\273
\037$\037d\037\302\2732
x\302\273
h
X
E
IR,
7t
(40.8)it sufficesto provethat Ps(ds)< 00.) J (J Z2 But this is an immediateconsequence of 4 H Z4 PS(dz)Ps(ds)= J T dP < 00. P'(dl\302\2732
0)
casewhere Q = andP = Vo,,' In this case,the Now, we turn to the special result of Theorem40.8can beextendedto a considerably larger classof loss \037
functions.)
40.10Theorem(Blyth [1951]). LetE = (R, a, {VI.,: t E \037}),f= that W is separating and offinite order.Then the identity uniquely determinedby itsriskfunction.)
andassume is admissible and id\037
ProofThe proofis dividedintothree parts. of Theorem40.6is (1) In the first part we shallshow that the assumption 2 . = satisfiedfor h (x) 1/V2n exp ( x /2),x E Sincein the prescntcasethe we have to show that) identity is the Pitman estimator
-
\037.
-
= OCt). inf J U';Qp, v o.1!c(dt) J W(ltl)vo,ddt)(IE 91(F.,/II))
To compute inf
Vo , IfL (dt) J U';e.P,
(l'\037(E.A))
we consider the mixtures) Pr(A x B):=J B)))
V\"
I (A) Vo. \037_
(dt),
A
E
g4(R),
BE91(R),
Chapter 7: Theory of &timation)
222 F.
> O. The marginal distribution for the first coordinate is Jlt(A x R) = VO,I-t/t(A),
and desintegration of with J..lc
J..lc(A x
A
E
\302\243i(\037),)
yields) respectto the first coordinate
B) = J \037(x, B) J..lc(dx, R),
A E
.sf(R), BE .sf(\037),
A)
is given by) wherethe posterior distribution
\037(x,
B) =
JB f
dV,.1
d)' (x) Vo.t/l(dt) = v
dVr 1
di
it
X
E
IR,
(x) VO.1/c(dt))
\302\243i(IR).
(1+c).I/(1 +c)(B),
_
x.-.1x+t) , x E
The estimates Ke:
BE
xf
-
IR, t;
> 0, minimize
SI-+J W(ls tl)f;(x,de), SE R,) for every x E R and therefore inf
J J.\302\245,l!p'Vo,t/c(dt)
QE \037(E,A))
= J W;Kc1;vo.1It(dt)
= II
I: w( ) .0.1
=J
W
I
(dx).0.1{,(dl)
\302\243
x
( }I1+t)) vo..
Integrationby partsyields
-J ( = 2 J (4)( + - + 2q/l + - 1) J a'
VO. 1 (dt) J W(111)
f. 1 V\037.
W
!X)
4>(
+
) Vo,t
a}l1+t\302\273
W(d!X)
[O.a\302\273)
\037
f.
(O.a>
cp(+a) W(da) =
))
-
sincex> y > 0 impliesIcl>(x) identity isalmostadmissible.)))
cl>(y)j
\037
Ix
-
YI
O(e)
cp(y). This provesthat the
40. Admissibility
of estimators
223)
In the be such that supJY,eoP,\037JW(ltl)vo,,(dt). C?oE9t(E,R) fER = 0 A-a.e. secondpart of the proofwe shallshow that eo(x,{oo}) Thecasewhere W isunbounded isobvious. that W isbounded. Let Suppose = = 0> 0 be arbitrary and defineA {Oo(.,{oo})> c5}.We provethat A(A) O. For every x E A we obtain) (2) Let
JJ W (Iy ;;;;
-II)
+c),l/U+cl (dt) Qo(x,dy)
v x/U
.IW(co)+ (I
-
D)f
-
c5)J W
= 0 W(oo) + (1
w(
i:-.-I )
(dt) V'/lI+.\"1/1\"')
III
(V1+E) vo,ddl) ,)
which implies, again for x E A,)
III
JW HW(ly-ll)vx/U+C),l/U+c)(dl)Qo(x,dy)(VI .t't:) vo..
\037
\037
- III ) (vt\037 b(W(cc)- J W(ltl)vo,1 o(W(oo) J W
vO,!
(dl\302\273
-2 J
c5(W (00)
(0,
(d!\302\273
\037(-o:)
W(do:\302\273
=:017> 0,
1)
sinceW is separating. If we integratetheseinequalities by vO.1 + lIt then we obtain)
-
J JY,eoP,vo.tl,(dl)
( ) _'1 0,1+1/,)
> c5n . v
inf j.>u'<E.llh
J \037QP'VO,I/C(dt)
A
-
part (1)of the proofthat VO,t + lIt (A) -= OCt), t O. Hence = O. A{A) In the third part we may thereforeassumethat eoE Yf(E, R), For (3) simplicitywe denote) which implies by
g(C1,Z)::: J (\037(Z-C10:)+\037(-Z-C1:x)-2\037(-u<x\302\273W(d<x), (O,
where(1> 0,Z E IR, It iseasy toseethat this iswell-definedsincethe integrand is and forfixed(J > 0,Z E IR, dominated non-negative by a W-integrablefunction. calculation and Easy integrationby partsyield)))
224
Chapter 7: Theory of Estimation)
H W(I Y
-II)
V#n
=HW(
+').lIn+.)(d/) Qo(x,dy)
y--- t\037 ) x
1+8
V1+e
= J C?O(X,dy}g(Vl+\037, vT+ ey
-J W( I:. -) I
v./n +0,.I/('+0'(d/))
Vo.l(dt)(}O(X,dY)-JW I\037 VO'l(dt)
-
(Vl+e)
. \037)
XE
R.
1+\302\243)
If we integrateby J
J-\302\245,C?OP'
vo, I
\037
lIt
then we obtain)
-
inf
1/c(dt}
V O.
/?
E
tI(E.GI)
J
. (dt}
O 1/c
J-\302\245,(}p'v
= JJg(1f1+8, 1f1+8y-
\037) C?O(X,dY)VO'l+l/C(dX). 1+\302\243)
By part
(1)of the proofand Patou'slemma we arrive at
-
H lim infg(vr+e.V1+ ey c-O
+
1 J/TX
f.)
)
llo(x,
dy\302\273)'(dx)
=
o.
It is not difficult toseethat lim inf g(Vi+ r.,0\"+ ty
c-O
---
\037-
.co
)
V\"f+t
= g(1,y
- x)
continuous and that z 1-+g(1,z), Z E R, is a non-negative function which vanishesexactlyat z = O. This provesthe assertion. 0)
.
Now we return to the generalcase.Let E = (H,.\037(H), {p,: tEL})where H isa = Euclidean N and L a is linear function.) space,p, f., H , tEL, f \037
40.11Theorem(Blyth [1951]).Supposethat
\037
the lossfunctionW
and of .finite order.Then fo PL is admissible for determinedby itsriskfunction on (ker/)1.)
is separating
El(kcrf)I and uniquely
Proof Choosea unit vcctor e.lkerf, eEL, such that fee)> O. Define the F = (H,Y4(H), {NH * E IR}).It iseasy toseethat,,:= subexperiment fo PL eA\037:)'
is sufficientfor F. We have)
G1=\".(F)= (R,
\302\243f,
A e R}). {vAllfll.1I/1I2:
Thereforethere existst E 9t (G,
\037)
QA\302\253(}O(.,
such that
B}IK)= t(K,B), Be a,)))
40. Admissibilily of eslimalors
where Ql:=NH * Sle.-,A E
If W(ly \037
225)
lit The decision functionf satisfies)
-A.llflll)t(.,
dy)dv J W( yl) vO.llfll2 (dy), I
A
II/II,1I/1I2
;.E
\037.
it followsthat t ( ., B) = By Theorem(40.10)
BE .41,and we obtain 18;'-a.e.,
Il = J1B oKdN H Jeo(.,B)dN
A
A)
wheneverA J
E
K-1 (\037), BE
\037.
t particularlyA = K (B)it followsthat Choosing
(/0(.,B)dNH = N H (,,-I(B\302\273
1(-1(8))
and hence
(}(x,B)=l if fopdx)EB An-a.e.,BE\037.
0)
isobvioushow to generalize Theorem40.11 tothe situation considered at the endof Section 34.))) It
Chapter8: GeneralDecisionTheory)
decision in orderto classify Statistical theory has beenfoundedby Wald [1950] of the vast variety statistical methods The basicideaisto by generalprinciples. consider statistical as in the sense o f von Neumann's experiments games theory of games.In specialcaseswe have introducedthe ideasof decision theory 7 and 33.Early importantresultsof the theory were the already in Sections minimaxtheoremand the complete classtheorem. Decision theory becamea after it was possible to formulate general framework for statistical purposes importantideasof classicalstatisticssuch as sufficiency and asymptotic terms.Bothproblemshave bec:nsolvedby optimalityin decision-theoretic LeCam[1964 and 1972]followingpreviousattemptsby Blackwell[1951 and
and Hajek[1972]. 1953],
consider the problemof sufficiencyin Chapter9, and the foundations of i n 1 0. I n the w e asymptoticoptimality Chapter presentchapter only prescnt thosebasicfacts ofclassical decision which have already beenknown to theory Wald, but we do this as generallyas requiredlater on. Theobjects of statistics are statistical Toapply linearfunctional experiments. the of families have to beembedded m easures analysis underlying probability intosuitable linearspaces. In caseof a dominatedfamily thisisdonein Section 24by taking the L I-spaceof an equivalentC1-finite measure. Thegeneralization this of procedureto an arbitrary experimentleadsto the L-spaceofDefinition in Section 41.3.The basicfacts on L-spaces are presented 41. In Wald's decisiontheory the strategies of the statistician are given by kernels. functionsis not topologically However,this setof decision complete. it is convenient to i t Therefore, completc by considering arbitrary bilinear decision 42. functionsas generalized functions. This is the subjectof Section However, only decisionfunctionsin the narrow senseadmit a statistical 42.5and 42.7dealwith the roleof kernelswithin the Theorems interpretation. classof generalized decision functions. functionsare bilinear decision Generalized functionson the productof the continuous decision o fbounded functions on a space spaceandthe topological In thiscontextthe boundedcontinuous functions L-spaceof the experiment. Forstatistical however,it isnecessary play the roleof lossfunctions. purposes, to admit alsothe upperenvelopesof boundedcontinuous functionsas loss the class o f to the setof functions. h as tobeextended loss functions Therefore, In Section lowersemicontinuous functions. 43we showthat after thisextension the classof decision function in the narrow senseremains sufficiently large We
(Theorem43.5).)))
41.Expcrimcnlsand their J,-Spaccs
227)
Section45 containsa functionalanalytic version of the minimaxtheorem in Section 46 to the framework of decisiontheory.In which is specialized 47 we prove a versionof the complete classtheorem. Section
1b a
large extent we follow LeCam [1974].For the theory of Banach
In orderto avoidthe Mackeylatticeswe refer to Schaefer[1970and 1974]. topologyusedby LeCamin the originalproofof Theorem42.5we employ 45 we followHeyer [1969]. Lemma42.4.In Section of thischapteris devotedto the general theorem of Hunt The final section and Stein.Insteadofa transfonnation groupoperatingon the samplespacewe startwith a groupof stochastic on the L-spaccof the experiment. operators This generalizes the first situation,but will turn out later to be almost We it. to distinguish equivariant from strictlyequivariant decision equivalent In caseof shift experiments orstructure functions. models,the latterconcept with that of convolution coincides kernels. We provea versionof the theorem ofHuntandSteinwhich coversthe originalversion in testingtheory (Section of Sections 38and 39.Practicallythe whole theorems 32)and the convolution of this sectioniswell-known.Further information on thissubjectiscontained in Luschgy [1984].)
41.Experimentsand their lrSpaces) Let T:t=0 be an arbitrary
set and E = (Q,.t'/,9) with 9' = {p,:lET} an
experiment. t E T, assertions of decision Measuretheoretic theory often are valid p'-a.e., i.e.9'-a.e.If E isa dominatedexperimentthen by Lemma(20.3)there existsa measure Poldsuch that Po\"\"rJI.Thus,the expression rJI-a.e.may probability be replacedby Po-a.e.The suitablefunctionalanalytic frameworkis then the
pair L1 (Q,d, Po) and LCX)(Q,d, Po},i.e.the L-spaceand the M-spaceof section24.However, if E is not dominatedthen the questionfor a suitable functionalanalyticframeworkhasa morecomplicated answer. It is the aim of the presentsection to find the right generalization of the conceptof L-space. Let ca(Q,.flf) bethe setof all boundedsignedmeasures on (Q,d).We begin with a reinterpretation of our originalconceptof L-spaces for dominated experiments.)
41.1Lemma. Let EE 8(T)bea dominated experimentandlet \\lId bea q-finite \\'. Then) measuresuch that L(E)= {JlE ca(Q, \037
\"\"
d):Jl
Proof Obvious.
0)))
\037
v}.)
228
Chapter 8:General DecisionTheory)
we needa representation of L-spaces which does For a suitable generalization not rely on dominating measures. For this,we prove)
41.21beorem.LeiE E cf(T)bea dominated The'lJ1 e L(E) iff for experiment. every u e ca(D,.9I)
a 1. lET, implies u 1.JL) P\"
Proof Let Po E C(fJ)be such that Po u
1.1;,T, tE
iff
- 9.Thenwe have
u 1.Po')
Indeed,if u 1.Pothen there isA E .sIsuchthat Po(A) = 0 and u(A') = O. This implies1;(A) = 0 for every t E T and therefore.P,.lu for every t E T. If converselyp, 1.u for every lET then let AI Ed be such that 1;,(Ai) = 0 and 00 a: = :;: = i Then A where E 0 , N, Po(A) = 0 and a(A;) Po (Xjp',\" () A, satisfies i=1 ;-'1 u(A') =0.Hcnccu1.P o. we Therefore, needonly provethat J1 E L (E) iff (11.Po implies (11.Jl. If JlE L(E) then Jl Po,and u 1.Poobviouslyimplies (1.1Jl. If converselyJ1 satisfies the condition then let A E d be such that Po(A) = O. Since J1IA 1.Poit = followsthat J1IA 1-J1 which is only possibleif J1(A) O. HenceJ1 Po, i.e. \037
\037
\037
J1 E
L(E).
We
take the preceding assertion as startingpointfor our conceptof L-spaces.)
0)
41.3Definition(LeCam[1955]). Let Ee t!f(T)be an Then the L-spaceof E is
arbitrary
experiment.
L(E)= {,uE ca(D,.w'):(11.,uif (11.p',lET}.)
41.4Corollary.TheL-spaceL (E)ofan experimentE isa Banachlatticefor the
variationalnorm. if every subset which has an Recall,that a Banach latticeis ordercomplete upperboundhasa leastupperbound.It is a well-knownfact that baeD,.w') Banachlattices and ca(D,d) are ordercomplete (seeDunfordand Schwartz,
[1969]).)
41.5l..emma. The L-space of an experimentis ordercomplete.) Then it is clearthat Proof Let {Jli:i E l} L(E) and denoteJ1 = sup i,I ,ul' issuchthat u 1.p, for every t E Tthenwe have .w').If (1 e ca(D,.w') ,u E ca(D, \302\243;
to)))
41.Experimentsand
their
L-Spaccs
229)
showthat u .1p.SincePi E L(E),i E /, it followsthat (1.1Pi'i E /, i.e. (1 n PiI = 0, iE I. Now, from P- Pi- Ipi!.iE I, we obtainJl-nIO'I= O. On the I
\037
I
I
\037
otherhand) P
+
n 10'1= suppt v 10'1= suppt +
id
id)
I
(1 = P + + 10'1 I
+ 0) implies P 1-0'.Hence0'.1Jl. Our next aim isto obtainmoreinfonnationaboutthe relationbetweenL(E)
and ba(D,.PI).)
41.6Lemma. Each O'E baeD,.91)admits a
such that)
0'= unique decomposition
O't
+ 0'2
(1) O'IEL(E), u 2 EL(E)1, (2) if u 0 then U 1 0, (12 O.) \037
\037
\037
of any decomposition satisfying (1)isclear.To showthe ProofThe uniqueness of the decomposition we need only consider 0' O. We define) existence u 1 = sup{tEL(E):0 t O'}, \037
\037
\037
(12=0'-(11') It is immediate that 0'1 0, (12 O. We have to show that 0'21L(E).Let e + (11E L(E). P E L(E)and e = (12n pl.It isclearthat e E L(E)and therefore = But e + (11 (12+ Ut (1which implies by the maximalityof U 1 that e = O. Hence0'2 .1Jl. 0) \037
\037
I
\037
41.7Theorem.There existsa positive/inear operatorT: ha(!l,.flI) -.L(E) satisfying the followingconditions:)
(1) I'll= 1. (2) (Tu) (1)= 0'(1) if (1 (3) TIL(E)= idL(E)') II
\037
O.
Proof Let 7t E L(E)be arbitrary such that 7t 0, 7t(I) = 1.For0'E baeD,d) define T(O')= 0'1 + \"2(1)n, using the notationof Lemma 41.6.Then T is a For(1)we note (2) isobvious. positivelinearoperatorsatisfying (3).Condition \037
that) II
Tull= \037
+ u 2 (1)nl (1) 11'0'1 (1)= 10'1
10' (1)n(1)-Iul + 0'2 (1)= 10'1 1 (1)+ 10'21 (1)= 110'11 1
which implies II I'll
\037
1
1.Togetherwith (2) it follows(1).
a)
The topological dualof L(E)iscalledM-spaceofE and is denotedby
M(E).)))
230
Chapter8:GeneralDecisionTheory)
41.8Lemma. Supposethat E 8(T)isa dominated experimentand let a u-finitemeasuresuch that 1/ v. Then M(E)= L\302\253)(fl,.9I.v). \302\243
- definesan element
\037'
I,vi be
In general, to every f !Rb (D,d) cPf M (E) according = epr(a} a (f), a E L(E).If the:mappingf...... CPr issurjectiveontoM(E)then E iscalledcoherent. are Dominated experiments coherent. \342\202\254
We finish this
\342\202\254
the relationbetween sectionwith an assertionconcerning
for arbitrary experiments.) M(E)and !i',,(D,.PI)
41.9Lemma. LetE bean arbitrary experiment. Then) {cp/EM(E):0
\037
f l,fe!R,,(D, .PI)}) \037
isdensein) {q>EM(E):0
\037
cp(u)
\037
u(D) wheneveru
\037
0.UE L(E)}
for the L(E)-topology of M(E).) Proof If the assertionis not true then there existssomecP E M, 0 (1(D)if (1 0, (1E L(E),and (J E L(E)such that .PI).) u(f) S c t < c cp(u) if 0 t.fe !i',,(D, for someC E R, e > O. This implies q+ (D) C e < c cp(q) cp(q+) q+ (D) Hencethe assertion. 0) which isnotpossible. \037
\037
-
\037
-
\037
cp\302\253(1)
\037f\037
\037
\037
\037
\037
42.Decisionfunctions) (fl,.91,\037)be an experiment and let D be a topological space.The D We consider as spaceof possible \037o(D). Recallthat a Markov kernele:D x \037o(D)-+ [0,1]iscalleddecision decisions. functionsisdenotedby functionforE and D.The setof all decision D).) Let E =
Baireq-fieldof D isdenotedby
91(\302\243.
42.1Remark. Let e e 9t (E,D).Forconvenience we denote IfJJ.l= H/(x)l!(w. dx)J.l(dw). function e E rJt (E.D) definesa bilinear L(E).Every decision to functionP : \037\"D) x L(E) -+ R according if IE rc,,(D), J.l E Q
J.lEL(E).))) fJqU:J.l)=fQJ.l if fE<:&,,(D),
42. Dcci\037ion funclions
231)
42.2Definition. A generalized decision function for E and D is a bilinear function fi: \037b(D) x L(E) R satisfying the fonowingconditions:) (1) IP(f,Jl)! IIfilullJlIIif fe (G,,(D). Jl e L(E). if f 0, Jl O. (2) P(f,Jl) 0 if JleL(E).) (3)) P(1,Jl)= Jl(Q)) \037
\037
\037
\037
\037
The setof an generalized decision functionsisdenotedby 11(E.D).It isclear that for every (1 E D) we have fiQE 91(E, D). The weak topologyof 91(\302\243,
on \037b(D) x L(E).) dI(E,D) is the topologyof pointwise convergence
42.3Theorem.The set91(E,D) of allgeneralized decision functionsis convex and compact for the weak topology.)
@ L(E)., Proof.dI(E,D) isa convex,closedsubsetof the unit ballin \037,,(D). which is compactfor the (i,,(D)@ L(E)-topology of \037,,(D)+@ L(E)+. 0)
The next theorem isbasicfor the whole theory.For the proofof this theorem of Lemma41.9.) we needthe followingextension
42.4Lemma. Let E be an arbitrary experiment.Then the set of all linear functionson L(E)k which are of the form) Ie
(OJ)1
-iSk \037
i'\"
Ie
o /; 1, L It functions \037
:E (1i(/;)'
\037
\037
i.\037
\037
1)
1,hE !I' ,,(D,.cI),1
\037
i
\037
k, is densein
the setof aI/linear
1)
\" \302\253(1i)1
L
\302\253(1i),
(
M(E)\",
1)
\"
which satisfy
i
L =1
\037
Cl'j\302\253(1)
L(E)-topology of
(1(1)if (1 0, (1E L(E),(forthe k-foldproductof the \037
M(E\302\273).)
Proof First,we notethat
every
continuous linearfunction on L(E)\" is of the
fonn) \302\253(1')1
--
\037'\037k......
k
L
--) E M(E)\".
(
\037,\037\"
It isthen clearthat every linear we may identify M(E)\" = (L(E)\..") Therefore, function on M(E)k which iscontinuous for the k-foldproductof the L(E)topologyof M(E),is of the form)))
232
Chaptcr 8:Gcncral DecisionTheory) k
(
L CPi(O'i), (O'i)1i-I
\037iq
E
.)
L(E)\".
is not true then there exist(
i-I
\037
\037
c -c
k
L
S:i\037a
O'j(/,)
\037
Ie
\037
j\0371
L
cPj(O'j)
i=1)
\"
wheneverO
\037
f,\037
1 \037i\037k, 1,i-I L f, 1,f,e!R\"Q,d), \037
and for someceIR,
\"
6> O. Denoting0'0:=U = i
0';(1)S; c
-
O'i
\302\243
we obtainfrom Lemma (6.7)that
I)
Ie
\037
Ie
L cp/(O';) 0';(1) L cp/(O'j) i-I) i-I \037
\037
which is not possible. Hencethe assertion. 0)
42.5Theorem(LeCam[1964]).Theset{{3(/E 1I(E,D):(!E
91(\302\243,
1I(E,D)for the weak topology.)
D)}isdensein
{[I,lz,..'
...,
Proof Let fJ E (}I(E,D).Choosee > 0 and finite sets ,1m} rc,,(D) and {Jt),Jt2. Jt,,} L(E).We will construct some E 91(E.D) such that \037
\037
fJ
l\037i\037m, l\037j l1.)
if 1{3(f;,JtJ)-f,(1JtJ/
\037
n. By Lemma 6.6 \037j a finite partitionof the unity (p,,)I there exists on D such that each/;, 1 i p n, varies lessthan on the supportof the Pie' 1 k p. By Lemma (42.4)
W.l.g.we may assumethat JlJ(Q)= 1 and Jlj 0, 1 \037
\037
\037
\037,,\037
j
\037
\037
\037
choosefor every k E {I,2,..., p} somehIe E \037 (Q,d) such that) t
IP(p\",Jlj)-Jtj(h,)I<
if l\037j\037n.
m
3p(L 1=
IIhllu)
1)
This impliesthat)
-
p
L hit) JtJ(Q) Jlj(1e=1
-
p
= JlJ(1 L hit) 1e=1
<
\302\243
m
3
L
i'\"
Ilhllu
1)
to Lemma42.4the functionshIe may bechosensuchthat 0 According p p 1 k p, and hIe 1.Let us denote110= 1 L hIe' \037
\037
\037
k=1
\037
-
Ie-I)))
\037
hIe
\037
1,
42. Decisionfunclions
Let x\" e Supp(PIe),1 p
Q(w, B) =
L
\037
k
\037
P, and let Xu be arbitrary. Then we define)
h,,(w) I B (x,,), BE \037o(D),we Q.
1:=0)
it followsthat for every we have (!e 9P.(E,D).Moreover Obviously, \037
n,) I
233)
-
-
p
J\037)
1 \037j
p
L {J(hPI:' fJ(Ji,Jlj) Ji(1Pjl = \"=1 Pj) L .f,(x,J pj(h,,)! \"-0) I
-
p
I
\037
L ({J(J;P'\" Pj) {J(h(x..)p\", Pj)1 1e=1
-
p
P
max 1IJ;lIuJlj(1L hIe) + L 1J;(x..) -J;(x,,) P(P..,Jl) Jlj(h..)+ l;:Ii;:l'\" 1:-1) 1:=1 I
P
<
Ui-.f,
(x,,\302\273
IP(I:\037I
P
\342\202\254
I:\037I
Ie
Ie
1
+_
\342\202\254
I
e
1J;(x\IIP(p,,,Jlj)") Jlj(h,,) + 3)
P
P( L:;t Pt.Pj) + L= IJ;(xt ) 3
_ \037
-
P p\", pj)1+
\302\243
P
3 L J; \"=1) II
3
\037
I
e.
0
II\"
e \037(E, D):Q e [}teE. D)}are not identical. The followingassertions describe the mostimportantcaseswhere they are.) In general, the sets (E.D) and \302\243I
{P\037
42.6Lemma (Farrell[1967]). and D is a locallycompact If E i.'i dominated with b ase then countable e space for elJery P \037(E, D) there existsa kernel
- [0,1]
(1:Q x &I(D)
such that
P(!\"tp) = f(/}l
if
Ie
thisisTheorem6.11. ProofEssentially,
0)
42.7Theorem.Supposethat E is dominated and D is a locallycompact .t;pace with countable base.If P E \037(E, D) issuch that fl-+P(/' P,) isa Daniellintegral on \037,,(D)for every t e T, then thereexisl.5(/ E fJt(E,D) satisfying) P(!\"tJl) = feJl
if f E CC,,(D),Jl E L(E).)
Proof Apply Lemma42.6and notethat 1(1P, = sup{f(!p,:0 '5.f '5.1,f e
1)
for every t E T.Thisimp1ies (!(w,D) = 1 \037-a.e.Eventually improving() ona set of 9\"-measurezeroyields(/ E 9t(E,D). 0)))
234
Chapter 8:General DecisionTheory)
42.8Coronary.If E is dominatedand D tM(E, D) = {Ptl : (l E 9t(E,D)}.)
IS a
compactmetric spacethen
ProofThisfollowsfrom Theorem42.7sinceon a compact spaceevery Radon measureis a Daniellintegral on (jb(D). 0) of non-randomized thissectionwith a characterization decision function E fJt(E,D) is non-randomized if functions. Recall that a decision E t E T and all BE f or (D).) (l(.,B) to,I} \037-a.e. every We finish
f1
\302\243to
42.9Theorem.ut e E fJt(E,D).Thefol/owingassertions areequivalent: (1) (1 isnon-randomized. and E L(E). (2) f2 eJ.l= (fe)2J-lforevery fE .P,,(91 o (D\302\273
(3) f2(lp'= (f(1)2 p'forevery fE \037,,(D) and t E
J-l
T.)
Proof(1) (2):Firstwe note,that by assumption lET. f2(1\037 = (f(l)2P, if fE .P,,(9I o Fix fE .P and let A = {f2(1 (f(1)2> O}.Since[2(1-([(1)2 O it = followsthat P'(A) 0, t E T. Let E L(E).ThenJ.lIA.iP, for every t E T and by definition of L(E)thisimplies that J.l .iJ.L. HenceJ.L(A) = 0 and therefore f2eJl= (f(l)2J.l. (2) (3):Obvious. \037
(D\302\273,
-
,,(ato(D\302\273
\037
J-l
L\037
\037
(3)
\037
(1):It followseasily that
= (18(1)2 lET, (1B)2(1p' P\"
wheneverB is\037,,(D)-opcn. Thesystemof allsetsBE\037o(D)with this property is a Dynkin systemand thereforecoincides with Ylo(D), 0)
43.Lowersemicontinuity) Let f D
-.
IR
be a lower semicontinuous function which
is
boundedfrom
below.If D is a separable metric spacethen there is a sequence (
J.lEL+(E).) feJ.l= sup{q>eJ.l: q> \037f, q>E\037,,(D)}, This motivatesthe followingdefinition.)
43.1Definition.Iff D
-. islowersemicontinuous and boundedfrom R
below)))
43. Lower semicontinuity
235)
then)
P(f,J.l):=sup{{J(ep, J.l):ep \037f. ep E tjb(D)},Ji E L+ (E).) and D if a compactmetricspacethenfor every 43.2Theorem.If E dominated P E \037(E, D) there existsfl E rJt(E,D) such that P(f, J.l) = f (l functionf D ;R which is for every J.l E L+ (E) and every lowersemicontinuous i\037
-
J.l)
bounded frombelow.)
if D is a compactmetric spacethere existsa Proof Sincetj(D) is separable as in Theorem f f. If (!E 9t(E,D) ischosen (cp,,)S CC(D), \037f. sequence it a nd E then follows t hat (42.7) J.l L(E) ep\"
ep\"
= sup{J(ep\",J.l) = P(f,J.l). fflp.= sup ep\"flJ.l liE liE '\ I\\:
The lastequality holdssinceP(.,J.l) is a Daniellintegral on \037(D).
0)
The followingtheorem is relatedto Theorem42.7.)
43.3Theorem.Supposethat E is dominated andD is a locallycompactspace
-
countable base.Let P E 91(E,D). If there existunbounded level compact functions 1,:D IR such that 0< p(/\" P,) < 00 for every t E T then fJ = p\037for somee E Yt(E, D).) with
Proof In view of Theorem42.7we needonly show that 13(., P,) is a Daniell for every t E T. Let t E T and choosef. > O. Then integral on rc,,(D) 1 K:={I, - PC];,p,)} \037
\302\243)
iscompact and)
g.=-B 13(1\"P').(t - tIC) 1
\037
/\"
Sinceg is lower semicontinuous we obtainfJ(g, P,) P(1 1 , P,) E. Thisprovesthe assertion. 0)
-
IC
\037
{J(f\"
P,) which
implies
\037
In general, an element {J E rJI(E,D) neednot beof the form P = {J(/ for some (!E fJt (E,D). Nevertheless,onemay provean assertionrelatedto Theorem 43.2.Recall,that if D is locallycompactthen \037o(D)is the spaceof all)))
Chaptcr 8:General DecisionThcory)
236
continuous functionson D which vanish at infinity. We have \037o(D)= \037oo(D) for the uniform topology.)
that E isdominated 43.4(.emma.Suppose and D isa locally compactspacewith countablebase.Then for every {J E :1l(E,D) there existsa decision function E such that (J (E.D) P(f,J.L) fQJ.L if ferco(D)+,J.LeL(E)+.) \037
\037
Proof Let P E (E.D) and chooseQo accordingto Lemma 42.6.Let A = {weD:eo(w,D\302\273O} and let PlfB(D)be an arbitrary probability measure. \302\243I
-
Define)
-l!o(w, --..B) eo(w,D)
e(w,B) =
I PCB)
.
tf
OJE
A
,
if w, A ,
Be \037(D).Thenit followsfor fE CCo(D) \"
J.L
E
L(E)t,that
= (J(f,J.l)=f(}oJ.l fLf(x)eo(w,dx)J.L(dw) = Lf(x)eo(w,dX)J.L(dw) dx) Lf(x)Q(ro, J.L(dw) =feJ.L. 0 J Jf(x)Q(ro,dx)J.L(dro) \037
!!
\037
D)
43.5Theorem.Supposethat E is dominated and D is a locallycompactspace base.Thenfor every p E !fI(/::,D) there existsa decision with countable function (J E
\037
(E,D) such that feJ.L
\037
PU:J.L), /l E L(E)+,
for every level-compactfunctionf D
-
\037
IR.)
-
and in 'tfb(D).Then supf f is in ProofFirst,letf D IR be level-compact isan immediateconsequence of Lemma43.4.Now, rco(D)+and the assertion
be arbitrary tevet-compact. Let
= {gE
letf D
\037
R
.Yf
+)
J.L):g .Jf'} {J(f,J.L) sup{{J(g, sup{geJ.L: g E K} = fQJ.L) Levi-theoremis valid. sincefor eJ.L the generalized \037
\342\202\254
\037
0)))
44. Risk funclions
237)
44.Riskfunctions) Let T 0 bean arbitrary set.Assumethat D is a topological space.The space D will play the roleof the decision space.) =t=
= (J-v,),(T of functionsJv,: D -+ lET, which iscalleda lossfunclion. A are boundedfrom belowand aJo(D)-measurable. = W if is attributed to each has the (w;),,,T W;, lET, property.) property
44.1Definition.A
family
W
\037,
44.2Definition.Let {3E f:I(E,D) and let
be a lower semicontinuous loss function.The risk of p at lET with respectto W is p(W;, The risk of (!E fJt (E,D) is the risk of {3t1 E (JI (E,D) and is denotedby Jv,(!\037, lET. The functionsI P(W;, -Pr), lET,and I W;Q lET,are calledriskfunclions.) W
\037).
\037
\037
44.3Definition.If problem.The sets
W
P\"
is a Jossfunction then (T, D, W) is cal1eda decision
R(E,D, W) = {(PCW;,
T: p E
P'\302\273'E
.r:\037
(E.D)}
\037
IRT)
and)
Ro(E,D, W) = {(W;Q P,),c:T: Q E 9f (E,D)} IRT are calledthe risk selsof the experimentE for the decision problem(T, D, W).) \037
44.4Lemma. If W isa bounded continuous lossfunclion Ihen R(E, D, W) s; isconvexandcompaclandRo(E,D, W) = R(E,D, W).) 42,3and 42.5. ProofThisfollowsfrom Theorems
\037T
0)
Let us consider the behaviour of risk setsunder restrictions of the parameter T -+ IJ(To the projection.) space.Let To s; T and Jet PTo: R
44.5Theorem.Let W bea bounded conlinuous lossfunction.If To c T then) D, PTo(Ro(E, PTo
(R(E,D,
W\302\273
W\302\273
= Ro(ETo' D, W), and = R(ETo D, W).)
'
isobvious. It fol1ows that) D) the first equation ProofSince(}leE,D) = (}l(ETo' R(ETO' D, W) =
Ro (ETo'D, W)
2 PTo(Ro(E, D,
= P.rD(Ro(E, D,
W\302\273
W\302\273
= PTo(R(E,D,
W\302\273)
sincePTo iscontinuous. On the otherhand compactness of R(E,D, W) implies that PTo(R(E, is obtain))) and we D, too, compact, W\302\273
238
Chapter 8:Gcncral DecisionTheory)
D. R(El,o' D, W) = Ro(ETo,D, W) = PTo(Ro(E. = PTo(R(E,D, C PTo(R(E. D, Thisprovesthe assertion. 0)
W\302\273
W\302\273
W\302\273.)
45.A generalminimaxtheorem) In thissection we present a generalversionof the minimaxtheorem in a purely functional to decision analyticframework.Theapplication theory.in particular the minimaxtheorem of decision in the next paragraph. theory isconsidered Let T be a locallycompactspaceand let ..ItAo(T) be the spaceof all o n measures with compactsupport.Each m E probability dI(T) (T) definesa linear functionon fIl(T).The spacefIl(T) is topologized with the on compacts.) topologyof uniform convergence ..Jt\037o
45.1Example.Let T 0 be an
arbitrary = topology.Then ...If\037o(T) STand \037(T) = =t=
set endowedwith the discrete
\037'f.)
45.2DefinitioD.Let M
the function) m
H tpM(m):=inf{Jldm:IE M}, mE ...ItAo(T).)
45,3DefinitioD.Let M c
f.'\
45.4Remark. If M \037(T) is compactthen isclosedin CC(T).To see this,let g E (M).Then for every compactK T and every I:> 0 \037
e\302\253M)
(X
\037
g(/)+ if IE K} * 0.) The family of closedsets isdirected from belowand therefore n {Fx.c:Kcompact,> O} + (/). Fic,e -= {hEM:h(/)
\037
\302\243
FK..E
\302\243
This provesthat g E a.(M).
Recall that a set M c \037(T) is calledsubconvex if for 11E M, 12E M and e (0,1)there is13e M such that 13 (XII + (1 (1.)/2'Every convex setis if M isdirected from belowthen it is subconvex, subconvex. too.If Moreover, M is subconvexthen :x(M)is convex,)))
(X
\037
-
45. A general minimax theorcm
239)
45.5Remark. Forevery M 'I/(T)and m E JI\037o(T)we have \037
tpM(m) = tpa(M}(m).
isthe basicseparation theorem.) The followingassertion M2 S; <6'(1'). Assumethat M2 45.6Theorem(LeCam[1964]).Let M1 S; <6'(1'). ns areequivalent: issubconvex.Then Ihefollowinga.(}serlio (1)Forevery fE M1 there;ssomege fJ.(M2) such that g \037f
for every m E \"I(\037o(T).)
(2) tpM:(m) 1I'M.(m) \037
(m) for every Proof (1) => (2):Assertion(1) impliesthat tp\037(m) 1I'M. m E \"I(\037o(T).Remark 45.5 proves(2). => (2) (1): Assumingthe contrary t hereexistsfeM1 suchthat g \037ffor every ge(J,(M 2). Hencef\037fJ.(M2)' SincefJ.(M2) is convex and closedthere is a continuous linear functionL on <'G(T)such that) \037
L(f) < inf {L(g):g E (J,(M2)}.) Since(J,(M2) = M2 + (1')the linearfunctionL must bepositiveand henceis \037+
a Radonmeasure on \037oo(1'). in such a way that It can benormalized L(h) = J hdm, hE f({(1'), for somem
E \"I(\037o(T).Thus we
obtain
tpM.(m)< 1I'\302\253()f2)(m) = tpM2(m) which contradicts (2).
0)
f({(T),M2 WeT). Assume that M2 ;s subconvex and (M2) isclosed. Then the followingassertions are equivalent:
45.7 CoroUary.Let Mt
\037
\037
fJ.
(1)Forevery fe M there issomege M2 such that g \037f 1
(2)
1I'M2(m)
\037
tpMI
(11'1) for every
m E
(1').
.J(\037o
apply the precedingresultsto obtain a version of the well-known minimaxtheorem.) We
45.8Theorem (Minimax Theorem). LeI l' be a convex, compact subsetof locallycomJexspaceandlet Y bea convexsubsetofa vectorspace.Assume that the followingconditions: f l'x Y .....IR satisfies t-J. I is continuous andconcaveon l'for every y E Y. (1) f(t,y) (2) Y t-J.f(t, y) is convex011Y for every t e T. Then)
inf )leY
inf f(t,y). sup f(t.y) = sup leT leT )IcY)))
240
Chapter 8:Gcneral DocisionThcory)
Proof The inequality ex
E
( - 00,00]suchthat
..
inf f(l, y) sup ICT )'c
is obvious.To prove the reversed inequality let
\037\"
\037
a.
Y)
We have
to show tha
inf .HY
t)
SUp[(I,y)\037a. 1\302\2431\
If = 00nothingisto be proved.If a < 00 let M1 = {a} WeT) and ex
\037
M2 =
{[(., y):
Y
E
Y} <;; \037(T).
The setM2 issubconvex. For m E J(\037o(T)let 1m be the barycenterof m on T. Then we obtain = inf {Jgdm:g E M2} 1S'M:(m) inf f(1,\", y) a = 'PM.(m). \037
\037
)IE
Y)
of gEa(M2) such that g Theorem(45.6)implies the existence e > 0 there isYc E inf y\037
Y
suchthat
[(.
sup[(I,y) a + t. \037
r lET)
, Yc)
\037
\037
+ e and therefore
\037
a. For every
0
45.9Corollary.Assume Ihat the condition... Then o[Theorem45.8aresatisfied. there is to E T such that inf suplet,y) = inf [(to,y). lET YEY
y\"y)
on T. If there is ProofThe function t 1-+infY let,y) is uppersemicontinuous YI: If to E T such that inf [(to,y) = 00 then the assertionis immediate. )'f: r
< Cl), lET, thc:n the function is boundedon T and attainsits followsfrom Theorem45.8. 0) supremum.Now, the assertion inf f(t,y)
)'E Y
46. The minimaxtheoremof decisiontheory) sectionto the particularcasewhich of the preceding Now, we apply the results = t in Example45.1. Let E (U,sI, {p,: isdescribed T})bean experimentand an arhitrary subset.<11 .<1I(E, D a topological decision D) space.We consider which is weakly compact and convex.))) \342\202\254
\037
46.The minimax
46.1Lemma (LeCam[1964]).Assume that
function.Let To
\037
W
theorem of dccisiontheory
241)
is a bounded,continuousloss
T be an arbitrary subset.For every fE IRTo the following
assertions areequivalent: (1) ThereexistsP E such that f(t) {J(\037,P,)for every t E To. (2) Sfdm inf {JP(\037,p')dm:P E for every In E STo') \037
\037
\037
\302\243j}
ProofIt isobviousthat (1)=> (2).To prove(2) => (1),we defineM. = {f}and M2 = {(P(\037, that 'PM2(\037 ) 'PM1(m)for (2) implies J;)'ETO:PE \037}. Condition we obtainsome every m E STo'FromTheorem(45.6) g E (M2) suchthat g f -+ it followsthat M2 is Since[JI is compactand PTo:IRT (RTo is continuous compactand hence <x(M2) = <x(M2)' Thisprovesthe assertion. 0) \037
<X
\037
46.2Corollary.The assertion of Lemma46.1;salso(,alidif W isan arbitrary lowersem;continuous lossfunction.) (1)=> (2) isobvious.Toprove(2) => (1)let 1/ bethe set ProofThe implication of all bounded, continuous lossfunctionsV W. Condition (2) implies \037
Jfdm inf{J{J(V\"p')dm:PE fM} HenceLemma(46.1)impliesthat for every V E 1/) for every m E SToand V E \037
f.
{PE\302\243j:P(V\"P')\037f(t)if
Since iscompactand \302\243j
n
VE\"\
{PE
\302\243j:
tETo}*0.)
f isdirectedfrom abovewe
P(V\"
P,) \037f(t)
if
tE
1\037}
have)
*0
and any elementof thisintersection satisfies (1).
0)
46.3Theorem(MinimaxTheorem). Forevery lowersem;co1ltinuous lossfunction
W)
inf 6E9I
supP(\037,P')= sup inf IE T
..
ProofThe inequality we denote) ex
\037..
SP(J.\302\245\"p,)m(dt).
\"'EST 61:91)
isobvious. For the proofof the reversedinequality
= sup inf S fJ(\037. p')m(dt). \"'EST (te!!l)
If = 00 then nothinghasto be proved.If < 00then we put f(t) = ex, t E T, and apply Corollary46.2. 0) ex
rx
46.4CoroUary.For every lowersemicont;nuous lossfunctionW there is some Pw E
\302\243j
such that)))
242
Chapter8:General DecisionTheory.)
sup Pw (H-;,P,) = inf supP(H-;,P'). reT reT) 'c\037
Proof The assertionhas been actually provedIn the proof of Theorem
46.3.
0)
andD is a compactmetric space. 46.5Theorem.Assume that is dominated Then for every lower semicontinuous lossfunction W the followingassertions hold: inf supH-;\037l{= (1) SI(E,D) sup inf JH-;uP,m(dt). r r: T \"'E ST DI(E. (IE \302\243
(IE
(2) There issomeqw E H-;ewP, = sup re T
91(\302\243,
inf QE\037(E.D)
D))
D) such that) sup H-;ep,. IeT)
of Theorem43.2if ProofThis is an immediateconsequence Theorem46.3and Corollary46.4. 0)
it
is appliedto
46.6Theorem.Assume that E is dominated and D is a locallycompactspace countable base. T hen with for every level-compactlossfunction W the following hold:) assertions (1)
inf
sup H-;eP, = \"'EST sup inf
Qell(E.D)lET
(lEtl(E.D))
(2) Thereis someQw E
= sup h.T \037eW P,
91(\302\243,
inf
S H-;ep'm(dt).
D) such that) sup H-;ep,. T)
q..Iit(E,D)
I\037
..
Proof The inequalities are obviousin both cases.The oppositeinequalitiesfollow from Theorem43.5if it is appliedto Theorem46.3and Corollary46.4. 0) \037\"
47. Bayessolutionsand the completeclasstheorem) function PoE .f!4(E,D) is 47.1Definition.Let mE ST' A generali7.eddecision calledBayessolutionfor mE ST and W if -Pr)m(dt)= inf J P(Jt;,p')m(dt). J fio(H-;, e91(E.
,
D))
lossfunction Wand forevery))) It isobviousthat for every boundedcontinuous
47. Bayes Soil/lionsand
the
complete
theorem
cla\037\"
243)
mE ST there is a Bayes solutionfJo E :!I(E, D). This is due to the fact that on the compactsetf1I(E,D).) iscontinuous fJ t-t> J p(w\" p')m(dt)
lossfunction.Let 47.2Lemma. Assume that W is a lower semicontinuou.'i E mE ST' Thenfor every {J &feE, D) J {J(W\" where
p')m(dt)= VE'f\ sup J fJ(V\" p')m(dt)
f is the setofallbounded,continuow'lossfunctions
V
is obvioussincefor every I:> 0 we may ProofThe assertion p(w\" P,) P( P,) + for every t E supp(m). 0)
that
v\"
\037
\037
w.)
find
V
E
f such
\302\243
note the easy fact that for boundedloss functionsW the function m f P(w\" p')m(dt) iscontinuous on E A (T),for every fJ E [fI(E, D).) We
t-t>
(X
S\302\253,
47.3Theorem.For every lower semicontinuous lossfunctionWand every
me ST there existBayessolutions.)
lemma it followsthat the mapping Proof Fromthe preceding p t-t> J P(w\" p')m(dt),fJ E &j(E,D), is lower semicontinuous. Henceit attainsits infimum on the compactset f1I(E,D). 0)
47.4Corollary.For every lower semicontinuous lossfunction Wand every
meST)
inf
,.,9I(E.D)
J (J(W\"
- sup 1Ie!l(E.D)) inf J
\037)m(t)
fJ(V\"
VE1\"
p')m(dt),
where .y denote.'i the setof allbounded continuouslossfunctionsV
..
ProofThe inequality (X
= sup Ye'f\"
We
inf
\037\"
..
isobvious. To prove
J {J(V\"
\037\"
\037
W.)
let
p')m(dt).
6\342\202\254iI(E,D))
observethat for every
VE
\"j/'
the sets
{pe(}I(E,D):J {J(V\" p')m(dt)
\037
(X}
* 0.
Hencealsothe intersection of all thesesetsisnon-empty and therefore contains somePoE (}I(E,D) such that sup J Po(V\" p')m(d/) ex. YC'f\ \037
Now the assertion followsfrom Lemma47.2.
0)))
244
Chapter 8:General DccisionTheory)
--
If W is an arbitrary lower semicontinuous lossfunction then in general Ro(E,D, W) :f::R(E,D, W). The lower envelopesof Ro(E,D, W) and The preceding that however,implies assertion, R(E,D, W) neednot coincide.
-
at
leastthe followingis valid.)
47.5Corollary.Forevery lowersemicontinuous lossfunction andevery mE ST inf Jlf9l(E.D)
where
J fJ(W;,
p')m(dt)= Yet\" sup
inf
J V,ep'm(dt),
\037Etit(E.D))
l'denotesis tosetprove of allbounded class complete
continuouslossfunctionsV theorem.)
the
Our next aim
the
\037
W.
A generalized 47.6Definition. decision function fJo E fJd(E, D) is an extended Bayessolutionfor the lossfunction W if for every e> 0 there is m, E 81, sueh that)
J fJo(W;, p')mr(dt) inf {JP(W;, \037
+ e: P E 91(E,D)}.) p')ml(dt)
Thefollowing isa rather weakversionof the complete classtheorem. However, it
is valid for unboundedlossfunctions,)
47.7 Theorem.Assumethat
pE
W
isa lowersemicontinuous lossfunction.Forevery
D) there existsan extendedBayessolurionPoE gJ (t:,D) such that
14(\302\243,
Po(W;,P,)\037fJ(W;,P') if
tET.)
Proof Let C1E .
-
meS\",)
J (P(W;, P,)
(U0f\302\243
tp(m\302\273
-e)m(dt)
\037
tp(m) for every mE ST')
we obtainPoE \037(E, D) such that Now from Lemma (46.1) Po(W;,P,)\037P(W;,P')-\302\243if
tET.)
of e it followsthat) Fromthe definition inf
(J Po(W;,\037)m(dt)
-
tp(m\302\273
= O.
\",eST)
HencePo isan extendedBayessolution.
0)))
48. The generalizedtheorem of Hunt and Stein
245)
47.8Corollary.Assume Ihal W is a bounded, lossfunclowersemiconl;nuous lion.Thenfor every p E \037(E, D) and every E A (T) Ihere is a BayesSolulion Pa E 91(E, D) such Ihal Pa(W\" P,) fi(W\" P,) if IE t;X
t;X.)
\037
ProofThe proofof Theorem47.7remains vaJid if Sr is replacedby particular E A (T).We obtainplJ E (E,D) such that PaO'v', P(w\" P,) if t E rJ., and rJ.
S\037
for a
,\037
\037)
inf
-
\037
(JPIJ(\037' \037)m(d()
IIIt:S.)
tp(m\302\273
= O.
Theorem47.3it followsthat for boundedloss In view of the notepreceding and thereforethe infimum functionsW the function'I'isuppersemicontinuous is attainedon Sa' 0)
ClassTheorem.) The followingis the Complete lower 47.9Theorem(Complete ClassTheorem). Assume that W isa bou1lded. Ihere exisls a weak T hen E loss semicontinuous function. for every P f1I(E,D) limit of Bayessolutions PoE \037(E, D) such that Po(\037, .P,)
Proof For every
t;X
\037
(J (J.v\".p,)
EA
for every t E T.)
(1')there is a Bayessolution suchthat
(JIJ(\037' P,) \037 (J(J.v\"
fJlJ
P,) if
tE rJ..
Let 910 be the setof all Bayessolutions in \037(E, D).Hencefor every
t;X
EA
(T)
Iion {O'E tM(E, D):O'(\037,P,) P(\037,P,) if (E rJ.} =F 0.) For reasonsof lower semicontinuity thesesetsare closedand thereforetheir intersection is non-empty. 0) \037
48. The generalizedtheoremof Hunt and Stein)
d, {.P,:t e T})be an experimentand G a locallycompactgroup countablebasefor its topology.The Borel-a-field of G is denotedby
Let E = (D,
with
\037(G).)
-
48.1Definition.A positivelinear mappingM: L(E) L(E) is a siochastic operatorif
II
MO'l/ = I/al/
if aE L(E),0'
\037
O. A family (Mg)geGof stochastic)))
Chaptcr 8:Gencral DecisionTheory)
246
isa groupif Mg, 0 g2 = Mg,\"Mg1 for gl'g2E G and Me = idL(E)'It isa operators if g 1-+(MgJl)(A) isa (G)-measurable measurablegroupof stochastic operators for all Jl E L(E) and A
E
d.)
on L(E).The 48.2Definition.Let (Mg)gFGbe a groupof stochastic operators experiment is invariant under (Mg)gcGif MgP,E sET}for a11g E G and {\037:
tE
T.)
48.3Example.Let Ebea dominated experimentand G a groupofmeasurable transfonnations g:Q Q where (g,w) 1-+gw, (g,w) E G x D, is .
-.
E a G-invariantexperimentif p, 0 g-lE sET}forall t E T,g E G.Particular are the shift experiments of Section 38and the casesof G-invariantexperiments 39. structuremodelsof Section of Section 31isa specialcaseof the situation Let us show that G-invariance described in Definition 48.2.ForJl E L(E)defineMgJl:=Jl 0 g-l,g E G.Firstwe of Ghave to show that M,llE L(E),g E G. This is an easy consequence in varianceand Lemma 41.1.It isclearthat (M')'1; G is a groupof stochastic that is and E invariant under operators (Mg)gf:G' Since the operation Fubini'stheoremimplies (g,(0)1-+goo,(X, (0)E G x Q,is1I(G)@ d-measurable that (Mg)gcG is a measurable group. We introduce the notionof a G-invariantdecision problem.) {\037:
Let (T.D, W) bea 48.4Definition. Assumethat E isinvariantunder (M')'E(i' decisionproblemwhere G operateson D continuouslyfrom the left. The decision problem(T, D, W) iscalledG-illvarianlif P'=M9\037 implies \037(x)=\037(gx), xED, wheneverS, lET and g E G. thisconceptby the mostimportantexamples.) Let usillustrate
48.5Examples.(1) Let (H,K) bea testingproblemin the classical sense,i.e.a o f t E into K. To it into the tenns two setsHand of general put partition {p,: T} decisiontheory let D = {O,I} and \037(1) = IH (P'), \037(O) = l K (P'),t E T. If M,Hs;Hand M,Ks;K for a11g E G then obviouslythe decisionproblem (T, D, W) isG-invariant. Specifyingto the caseof Example48.3we seethat Ginvariant testingproblemsin the senseof Definition31.1are G-invariant decision problems. that p, = Mg Ps iff t = g.';,wheneverg E G, (2) Let T = D = G and assume t E G. This is the casewith estimation or problemsfor shift experiments structuremodels,Let \037 (h) t= W (g-th) where W: G IR is a EI(G)measurable function which is boundedfrom below.Then .p, = Mg\037 implies
-.
t
= gsand hence)))
48. The generalizedtheorem of Hunt and Stein \037(x) =
Thus (G,G, W)
247)
W(s-tx) = W(r-t gx) = J+;(gx), X E G.) is a G-invariantdecision problem.)
Assumethat E isinvariant under(k!,),e 48.6Definition. G and that G operates on D continuously from the left.A generalized decision functionP E .\037(E, D)is equivariantif) tE T, gE G.) (J(fog,P,) = (J(f,M,P'),fEfjb(D),
It isstrictlyequivariantif) P<J0 g,p) = (J(f,Mgp),f E
\302\253fb(D),
PE
L(E),g E G.)
betweenequivarianceand strictequivariancebecomes clearif The distinction we specialize to the caseof Example48.3.)
48.7 Example.Considerthe situationof Example (48.3).Let Q E 9t(E,D). ThenQ is equivariant in the senseof the preceding definitioniA') Hf(gx)Q(w,dx)P'(dw)= Hf(x)Q(gw,dx)\037(dw)) and structure models for all fE CCb(D),g E G and t E T. For shift experiments thisisexactlythe conceptof equivarianceconsidered there.Now, let vld'bea
u-finitemeasure such that fjJ..., v. A decision function E .\037(E, D) in the sense of Definition 4 8.6 iff) equivariant Hf(gx)(l(w, dx) hew) v (dw) = H f(x)Q(g(I),dx) hew) v (dw) \037
is strictly
for all fE fCb(D),g E G and hELl(v). Thisimplies Jf(gx)Q(w,dx) = Jf(x)Q(gw,dx) v-a.e.) for all fe ttb(D)and g E G. In Sections 38 and 39 a convolution kernel was definedby the condition) (!(w,B) = (!(gw,gB),
WE
fl,g E G,BEg.f(D).)
The followingassertiongives conditions where strictequivariance can be
this is a lifting theorembut we improved to the lanerproperly.Essentially, Thesituation presenta complete proof,notrelying on generallifting theorems. issimilarto that of Theorem31.7. of measurability.) First,we have to discussbriefly a question
48.8Lemma. Considerthe situationof Example48.3andassume chat D is a basefor topology. Then))) locallycompactspacewith countable it.fj
248
Chapter 8:General DecisionTheory)
(g.(0)....... e(gW, gB), (g,(0) e G x Q. istJI (G) @ ,r.(-measurablefor every fl e 91(E,G),Be
\037
(D).)
on D and !/u ProofForrc00(D)let Yp the topologyof pointwise convergence the uniform topology of D. Then ffp c ffu. but since every ball baseof fYy {fet'Coo(D): IIflly < a} p) and sincethere is a countable = of ballsit followsthat !M(.'T For every consisting \037(!Yy) on t'Coo(D). p) \302\243
\302\243f(!i
fe \037oo(D)the map
g.......f(g-I.) Fj: G\037t'Coo(D): is5\"p-continuous and hence ll(5\"p)-. and ll(5\"y)-measurable. Moreover, (w,!) ....... x Q is .91 dx), (w,f)e f6'oo(D), @ \037(!Yu)-measurable. Now, the Jf(x),,(00, that) ofmeasurable composition mappingsimplies is
\302\243I(G)
(g) w)....... Jf(g-lx)e{gw,dx),
(g,(0) e G x Q,
for every fe @ ,fI-measurable
(too(D).
Finally, a standardargument provesthe assertion.
0)
the situationof Example48.3and assumethat D is a 48.9Theorem.Consider 111ell basefor itstopology. locallycompactspacewith countable for every strictly D) there existsa strictly equivariant decision function equivariant e e e \037(E, D) such that (1) flo(W, B) = fl(W, B),BE:M(D), and (2) eo(gw,gB) = eo(w,B), g e G, Be tI(D) on a .\\'et A e .91with \037(A) = 1,t E T.) 91(\302\243,
\037o
is a u-finite measure being equivalent to = {p,: A is u-finitethere t E T} and let l be a right Haar measure of G. Since existsa probabilitymeasure A.o ,\037(G) which is equivalentto A.. This implies) Proof Assume that
\037
\\'I.!1Sf
I
A.o(B)=Oiff .A.o(Bg)=O,Be.<1I(G), geG. field Let 910 9I(D)be a countable Let e e {)I(E,D) be strictly equivariant. orintersections of familieswith which generates \037(D).In the followingunions with the are countable unionsor Be indices !N(D) equal corresponding on910 necessarily which coincide intersections with BE 910 , sincetwo measures coincide on $feD). \037
We have v
{I
lIe);(D))
{w E Q:e(w,B) '* q(gw,gB)}=
for every g e G which implies)))
0
48. The generalizedtheorcm of Hunt and Slein (v
249)
= O. U {(W,g)E D x G:(!(w,B) =F (!(gw,gB)}
@ A.o)
BE9I(D))
Hencewe obtain A.o
I
{gE G:e(gw,gB)=+ e(w,B)}= 0 v-a.e.
J
B.?t;(D))
setof \\I-measurezerowhere the equationdoesnot Let N be the exceptional hold.Define = f e(hw,hB)A.o(dh)A.o-a.e.}. A:= {WE Q:e(gw,gB)
n
BEDI(D))
soonwe have shown that) (1) AEd, (2) Q\\N A and hence
As
\302\243
(3) (4)
A
E
d,
v(Q\\A) =
0,
j
f e(hw,hB) A.o(dh)=
J e(hgw,hg B) i.o(dh)if
WE
A, g e G, Be .r?I(D)
the proofcan be completed as foJlows. Define
- { e(hw, e(w, B)
eo(w, B) _
f
hB\302\273'o(dh)
if
WE
if
w f A,
A,
BE 91(D),
Be fA(D).)
Inorderto showthat eoisstrictlyequivariantwe notethat WE
A
implies by (4)
that)
(lo(gw,gB)= J (l(hgw,hg B\302\273'o(dh) = J e(hw,hB)Ao(dh) = Uo(w,B)) for al1 Be .c;!d(D).The
we A\\N we have)
decisionfunction ao solvesthe problemsincefor
e(w,B) = e(gw,gB) A.o-a.e.,Be \037(D),) and therefore) = eo(w,B), Be 84(D).) {!(w,B) = Je(hw,hB)lo(dh)
It remains to proveassertions (1)-(4).
Assertion (1)foUowsfrom the fact that A isthe setwherethe d-measurable
functions) W
H JIJ e(hw,
hB\302\273)'o(dh)
- e(gw,gB)llo(dg),weD,)
BE910 , vanish. For the proofof (2) we notethat w ri N implies g He(gw.gB) isconstantA.o-a.e. andthereforewe A. Next let us prove(4).)))
250
Chapter 8:General DecisionTheory)
If (I) E A then we denote = II {kE G:e(kw,kB)+ J e(hw,hB)Ao(dh)} Nt)) B@)/(DI)
and observethat
= o.Since)
Ao(Na\302\273
= O. For arbitrary ge G it followsthat Ao(N(I)g-l)
t Nwg- = {kE G:kgE N\"J = U {kE G:e(kgw,kg B) * J (!(hw,hB) ;.0(dh)},
B@9'(D))
we obtain)
J e(kgw,kg B) ).0(dk) = J e(hw,hB)Ao(dh) for all BE\037(/).This proves(4). Finally, we show that (3) istrue.If WE for every g E G
A
then Ao(Nwg-1) =
0 implies that
e(kgw,kg B) = J e(h(l), hB))'o(dh),BE:!I(D),I'o-a.e.)
thiswith (4) we get for all g e G Combining e(kgw,kg B) = J e(hgw,hg B) Ao(dh), B E \037(D), Ao-a.e., B by g-I B and replacing e(kgw,kB)= J e(hgw,hB)Ao(dh) Be \037(D), )'o-a.e.
Thisimplies gw E A.
0)
48.10CoroUary.Assumethat E isa shift experimentor a structuremodel.Then
for every strictly equivarianl I!E riI (E,G) there existsa convolulionkernel eoe fit (E,G) such that l!o(w,B) = e(w,B), BE (D),on a set A E J;( with P'(A) = 1,t e T.) \037
ProofFromthe proofofTheorem48.9it isclearthat eocan bechosenon Q\\A kernel. 0) to be a convolution theorem. Now we approachthe proofof a general version of the Hunt-Stein if G G there exists E such that For convenience, callsand t connected by g classes are Thisisan equivalence relationon T and the equivalence .P, = M,Ps. calledthe orbitsof G on T. Let usdenotethe setof orbitsby TIG.)
48.11Lemma. Assume that E is invariant under (Mg )9EG and (T, D, W) is Ginvariant whereW
isbounded andcontinuous. If P E fA(E, D) isequivariantthen
I........ P'),1e T, is conslanton Ihe orbitsin PO'Y\"
TIG.)))
48. The generalizedtheorcm of Hunt and Slein
251)
Proof Let P, = M 9 Ps.Then = P(\037.Ps). P(Jt;,l{)= P(Jt;.M9\037) = P(Jt;og,.P,)
0)
The main technical 48.12Discussion. toolfor the followingis a fixed point
property.
vector spaceand ..Ita family of continuous linear (1) Let X bea topological mappingsq>: X X. The family hasthe fixedpointproperty if every compact, convexsetK X such that q>(K) K.qJ E At.contains a fixedpointof vii, i.e. -\037
\037
\037
a pointXo E K satisfyingqJ(xo)= Xo for evcry qJ E .II. of a groupG isa homomorphism (2) A representation g \037 1;of G intothe linearmappingT: X -+ X of a topological vector space spaceof all continuous X. of G has (3) ThegroupG has thefixedpointproperty if every representation
the fixedpointproperty.
It is well known and will sufficefor our purposes that every Abelian the fixed the has T his is Markov-Kakutanifixed group pointproperty. point theorem. For a proofseeDunford-Schwartz[1967. TheoremV. 10.6]. relatedto amenability. (5) Thefixedpointpropertyisclosely Amenabilityis a weakercondition than the fixedpointproperty.If G isconnected then both conditionsare equivalent to each other.For a survey discussion of the between various conditions relatedto theseconcepts. see interrelationships Bondarand Milnes[1981].) (4)
Theoremof Hunt and Stein).Assume that E is 48.13Theorem(Generalized
on D continuouslyfrom the left. invariant under a group(Mg),\037G and G operates then has the JfG fixedpointproperty for every fJ E \037(E. D) thereexistsa strictly
functionPoE equivariantdecision
11(\302\243,
D) such that)
SUpP(jog-l. MgJ1) Po(j,J1) ,f:G) \037
+ functionf' D fur every pEL(E) and every lowersemicontinuous
-
IR.)
ProofForevery g E G let Pg: (j,}J) P(jog-I,Mg}J),fE \037b(D),P E L(E).Let 91 K be the closedconvex hull of {P,,: in E (E,D). If X denotesthe g G} x L(E), vector spaceof continuous bilinearfunctionson rc,,(D) topological -+ X X definedby then for every g E G the map 1;: \037
1;(Y):(f,}J) y(jog-I.M,J1),u:J1)E \037,,(D) x L(E),I'EX.) of G and satisfies is continuous and linear.Hence,(1;)is a representation K contains a fixedpointPoof (1;).SincePo is Tg(K)S K,g E G.By assumption, a fixed pointit is strictJyequivariant and sincePoE K, it satisfies the asserted \037
condition.
0)))
Chapler8:General DecisionTheory)
252
48.14Remark. (I) The resultsof the previouschapterswhere we have used much easierproofif the condition of amenability amenabilityadmit a similar, isreplacedby the fixedpointproperty.Thisisparticularlyvalid of the classical Hunt-Steintheorem (Theorem 32.5)and of Bon'sconvolutiontheorem (Theorem38.15).
the preceding assertion can be provedunder the assumption Conversely, of amenability by the technical means of previouschaptersprovidedthat 0 1-+ g /3(/ g-I,M\"Jl),g e G, is &t(G)-measurable, Ie\037b(D), Jl E L(E).Let us sketchthe prooffor this case. Let (K,,) be a summingsequence of compacts G, n EN, and define K\" (2)
\037
A.\"
= Ar(.1Kit)' n E N. Denote)
if,Jl) 1-+J fJ(10 h-I, M J1)
L(E),n E FromTheorem42.3it followsthat there existsa weak accumulation point inequality is PoE (E,D) of (P\flEN'") If Po is equivariant then the asserted obvious. Let us provethat Po is even strictly equivariant. Thechoice of impliesthat for every g E G and allfeI'Gb(D),J1 E L(E), fJlt:
II
A\"
(dh), IE 'tb(D),J1 E
f\\J.)
\037
(..\\\II4IN")
t lim IJfJ(/og-loh=0. , M,.,Jl)i.,,(dh)Sp(foh-I,M\"J1\302\273)'\"(dh}1
,,-
co)
It followsthat
-
lim fJ,,(/cg-I,M,J1) fJ,,(f,Jl)1= 0 -. 1
\"
CIO)
for all g E G, IE't,,(D)and J1 e L(E). It is now clearthat every weak accumulation pointof (P\\"E") isstrictly equivariant. remark canbe appliedto Q E fJt (E,D),sincein thiscase (3) The preceding (/0g-I)(lM\",g e G. is \302\243B(G)-measurab)c for an Ie(lb(D). e L(E).For the proofconsider the function F definedby (g,h)eG2 .) F(g.h)=fog-I(}M,.J1, fit;
g.-
J\037
Ie
(D),J1 e L(E),e e D).Then it isclearthat g 1-+F(g,h), for for every he G,and h 1-+F(g,h), hE G, ismeasurable g E G, iscontinuous and dense,and let d be a distance every g E G. Let (g,,) G be countable for some
<6'\"
91(\302\243,
\037
the topologyof G.Then generating
{(g,h) E 02: F(g,h) ex} = n u (g,h) e G2 : F(g\",h) \037
It= I
,,=
1)
{
\037
ex
-l
+ k , d(g\",g) < k , \037
}
Sincefor every e R, which impliesthat F is 91(G) @ 91(G)-measurable. follows.))) D E \037 (G) @ !M(G)the set{gE G:(g,g) e D}is in !J4(G),the assertion
ex
48. The generalizedlhcorem of Hunl and Stein
2S3)
48.15CoroUary.Assumethat E isinvariant under(M,)geGand G operates on D
left.If G has the fixedpointproperty then for every D) there existsa strictly equivariantdecision function PoE \037(E, D)
continuouslyfrom the
PE such that) 91(\302\243,
P sup rEr
(J.Y\"
\037)
sup Po rEl)
(J.Y\"
\037
\037)
lossfunction W ofan invariant for every -;E T/Gandevery lowersemicontinuous deci.vionproblem (T, D, W).) IfE isdominated basethen the and D isa locallycompactspacewith countable be a ssertions can preceding improved.) and D isa locallycompactspace 48.16Theorem.Supposethat E is dominated with
countablebase.Considerthe caseof Example 48.3and assume that G
on D continuouslyfrom the left.Assumefurther that there existstrictly operates D).If G is amenablethen for every P E PA D) equivariantelementsin there existsa strictlyequivariamdecision functione E fJt (E,D) such that supP(fog-l,Mgp.)\037fllil 91(\302\243,
(\302\243,
leG)
for every
J.L
E
L (E)+ and every levelcompact functionf D-+ R.)
without lossof generalitythat ProofIn view of Theorem43.5we may assume of a strictly P E [Jt (E,D).ThenRemark 48.14(2) and (3),impliesthe existence E rM such that (E,D) cquivariantPo
supP(fog-l,M,Jl) Po(/'Jl), \037
Ji E
-
gEG)
L(E)\\
forevery lower semicontinuous fD R We prove that there existsa strictly equivariant decisionfunction 11E Yt (E,D) such that
Po(/'p.) \037fllP.,
J1 E
L(E)....)
D R For Pochoose a kernel110according forevery levelcompactfunction! to Lemma42.6.Let IIIbeany strictlyequivariantdecision functionin Y1(E,D). \037
Define)
-
e(w,B) 1=Ilo(w,B) + (1 ()o)(ro,D)el(ro,B),
ro E
Q, BEfleD).)
It is then clearthat 11E (E,D).It is alsoclearthat 11 is strictlyequivariant sinceIlo(w,D) = (!o(gw,D)\037-a.e.Iffe ((f,,(D)is levelcompactthen and for J1EL(E)+we obtain supf-fE\037;(D) .\037
-
-
-
P(supf /, Jl) = (supf f)eoJl (supf \037
f)UJl.)))
254
Chapter 8:General DecisionTheory)
This implies
P(f,Jl) \037feJ1. Now Lemma 6.4provesthe assertion. 0 It is obviousthat a similarconclusion as in Corollary48.15 is possible.)))
Chapter9: Comparisonof Experiments)
The main differencebetween the usual decisiontheory and the theory of
can be explained as follows.It is the aim of decision theory to experiments rule.In a fixedstatistical e.g.to find an optimaldecision investigate experiment, the relations considers contrastto decision theory the theory of experiments In this mostof the betweendifferentexperiments. sense, precedingtheory in thisbookcouldbe subsumedunder the labelof decisiontheory.However, of the sufficiency and invariancepointintothe direction problemsconcerning theory of experim\037nts. The startingpointof the theory of experimentswas the problemof comBlackwellintroduced initiatedby Blackwell[1951]. parisonof experiments, which a statistically natural concept forthe comparison of generalexperiments with binary experiextendsthe ideaswe have usedpreviouslyin connection 49 ments (Section (Section 15)andwith dominatedexperiments 22).In Section we discuss the relations betweenthe various definitions.
The essential is that the basicorder pointof the theory of experiments first definedin rather general terms, can be relationbetween experiments, In Chapter3 transformedin such a way that it becomes analyticallytractable. in an elementaryway, using this has already beendonefor binary experiments the Neyman-Pearsonlemma, only.Now, in the generalcase,things are more and we dividethe procedure intoseveralsteps. complicated Firstof all,by means of the minimax theoremthe orderrelationcan be in terms of the Bayesianrisksforpriorswith finite support. It iseven expressed iscarriedthrough in sufficienttoconsider only uniform priors.This reduction 49.At this pointthe orderrelation isstillbasedon all possible Section decision for a A suitable of s tandardization decision problems given parameter space. matters considerably without changingthe orderrelation. problemssimplifies It turns out that we needonly considerdecision spaceswhich are convex 50and 51).The polyhedra,and lossfunctionswhich arc projections (Sections next stepis to expressthe Bayesian risksfor uniform priorsand standard decision and to express problemsin terms of integralsover concavefunctions, the orderrelationbetweenthe experiments in terms of theseintegrals (Section52).This procedure of transforming the order relationgoesback to and 1953], It is Blackwe))[1951 and hasbeen generalized by LeCam [1964], alsodiscussed [1970]. by Torgersen In Section53 we show that the Hellingertransformscharacterize the classes As o f a we obtain that also equivalence experiments. consequence the)))
256
Chapter 9:Comparisonof Experiments)
of the likelihood distributions have this property,thus extending processes r esults o f Section 2 5. previous We distinguishtwo different order relations between experiments. The coarseroneconsiders testingproblemsonly whereas the finer one admits of experiments. In Section54 arbitrary decision problemsfor the comparison we presenta few resultsessentially back to Torgersen[1970]. Among going for binary experiments. coincide others,it is shown that bothorderrelations For arbitrary experiments relationsare only the corresponding equivalence identical. 24and 25 we have provedthat for dominatedexperiments In Sections the in terms ofstochastic orderrelation basedon testingproblemsean beexpressed This characterization is exL-spaces. operatorsbetween the corresponding tendedin Section55 to the randomization criterionwhich is the general of the ideaof sufficiency.Initiatedby Blackwell[1951] the final formulation version isdueto LeCam [1964]. with finite parameter spacethere Forevery equivalence classof experiments is a standardmeasure on the unit simplex.There is even a one-to-one and standardmeasures. This can betweenequivalenceclasses correspondence be shown in an elementaryway and is known sincethe papersof Blackwell [1951and 1953].In caseof infinite parameter spacesthe situationis more difficult.For every equivalence classof experimentsthere is a uniquely which corresponds determinedprojectivesystem of standardmeasures to a the time of It took some conicalmeasure, using terminology Choquet[1969]. until it was possible to give a correctproofof the converse. Our proof,given in we observethat Sections 56and 57 followsSiebert[1979]. As a consequence a limit. every projective system of experiments possessesprojective In Section 58we consider the ideaof invariant By means again experiments. of the preceding oneconceptof invariance theory it turns out that essentially all definitions considered before. subsumes on the spaceof experiments leadsto a definitionof a The orderstructure This isdoneby means of a distance. calleddeficiency. With natural topology. is a complete the spaceof equivalence classesof experiments this distance metric d iffersfrom the t he metricspace.Forinfinite parameter spaces topology to finite weak topologywhich is the coarsest topologysuch that all projections are continuous, Endowedwith the weak topologythe spaceof subexperiments is a This 59,goingback to experiments compact space. isthe contentof Section LcCam[1972].)))
49. Basicconcepts
257)
49.Basicconcepts) E E 8 (T)and FE S(T). Let T 0 bean arbitrary set.We consider experiments =t=
aredescribed Theseexperiments by) E= (!11 1, tE T})
,d F= (!1,d , {Q,:tET}).) {\037:
and)
2
2
-
of a topologdecision Furthermore we consider problems(T.D. W) consisting = W D t and a loss function where D icalspace R, E T. For any J-Y,: (JY,),& T'
lossfunction W II J-Y,II
we define)
= sup{IJ-Y,(x) I:xED},
t E T.)
49.1Definition.Let (T,D. W) be a decision problemwith a lower semicontinuouslossfunction Wand let t O. Thenthe experimentE ise-deficientwith respectto the experiment F for the decisionproblem(1',D, W) if for every fJ2 E 14(F,D) there is PI E Ee (E,D) such that \037
PI(J-Y,,\037)
\037
P20-y',Q,)+ ell J'Yr1l,
t E T.)
In this. casewe denoteE ;2 F. IO,W) definitionmakes only The readerobservesimmediatelythat the preceding sensefor E>0 if the lossfunction W isbounded. o If E ;2 F then we denoteE 2 F. The relation\" ;2 ,. is an order r;
(D, W)
relation.If E
(D, W)
(D.W)
2 F then E iscalledmore informativethan F for (1',D, W). If
(D.W)
E ;2 F and F ;2 E then E and F are calledequivalent and we denote (D.W)
E
..\"
(D, W)
F.
(D, W))
49.2Lemma. Assume that W isa bounded, lowersemicontinuous lossfunction. t Then E ::::> F holdsiff E ;2 Ffor every e > O. (D,W)
(D, W))
of the weak compactness of Proof This IS an immediate consequence \037(E,
D).
0)
In Chapters3 and 4 we have considered orderrelations betweenexperimenls to which are closely related the concept of the preceding definition. Let us recall
intoslightly more general tenns. t In Chapter3,Section15,we have considered the relation\"2n for binary
what we have doneand put
it
2)))
258
Chaplcr9:Comparisonof Expcrimcnts)
and in Chapter4, Section the orderrelation 22,we have considered experiments \" ;2 for general dominatedexperiments. Recallthe definitions: 2 Let t 0 and let E, Fe 4(T) bedominatedexperiments. Then the experiment E ise-deficient with respect to the experimentF for testingproblemsjf for every testingproblem(H,K) and for every lfJ2 E :F (!2 2 , .r;;/2) there is d I) such that) CPI e /Pil (.QI'
..
\037
P'([JIS; Q,([J2+ 2
if
tE H.
if
tE K.)
\302\243
P'CPI
\037
-t
Q,CP2
2
e
In thiscasewe denoteE 2 F. This relationis relatedto Definition49.1as follows. 2)
49.3Lemma. Let E 0 and let E.FE & (T) bedominated Then experimems. ( ( E ;2F iff E 2 Ffor every decision prohlemwherethe decision !ipaCeD contains \037
2
(V.W)
exactly two elements.) (
F, Let D = Proof.(1) Assumethat E;2 2
{O\037
I}and Wan arbitrary lossfunction.
We define)
If = {tET: J.v,(1) J.v,(O)}, K= {tET: J.v,(1)< J.Y,(O)}.) \037
In view of Coronary42.8we choosesomee2e fjt (F.D) and denotecpz ( = e2 1). Obviously,CP2 E !Ii'(Q2'd2) and accordingto E ;2F there 1S CPI E (QI\"\037I) such that)
(..
2)
\037
:::
\037CPI
Let el =
CPI
e
Q,CPl + 2 if
\037CPI
\037
Q,CP2
-
+ (1
\302\243\\
resp.O. We notethat) J-Y,\037I
-
E
2)
tE
H,
IE K.
if
CPt)f.o where
\302\2431
at 1 and to denotethe Diracmeasures
-
= J.Y,(l)P'cp. + J.v,(O)P'(1 CPt> = J.v,(O)+ P,([J.(J-Y,(1)
Q,= J.Y,(1)Q, + J-Y,(O)Q,(1 = + Q, (J-Y,(1)-
P,
J.v,(O\302\273
and)
J.Y,e2
CP2
1\302\245,(O)
CP2
CP2)
J-Y,(O\302\273.)))
49. Basicconccpls
It followsthat) Jv,Ul
-
\037
-
Jv,UzQ,-= (Jv,(1) I
\037
Jv,(1)
-
Jv,(O\302\273(\037q>1
259)
Q,q>2)
{;
Jv,(O) \\.
2
1;11Jv,1I.)
\037
I-
Hencewe have E ;2 F. (D, W)
e
problemwhere the (2) Assume converselythat E ..:J F for every decision (D.W) If (II,K) isan arbitrary testing decision spaceD contains exactlytwo elements. defineD and) {O,I} problemthen we \037
--21
Jv,(O) =
Jv,(1) =
1
+-
Let q>2 E !Ii(!J2 , 2)' Then for such that) \037Q/
+
Q2 =
q>z
-
i:1 + (1
2
if
IE H,
if
[EK.
q>z)i:othere is QI E 9t(E,D)
t
2' lET.)
11
Jv,U2 Q,
\037
1
if IE K,
2)
Jv,e..P,
+-21
IE H,
if
Denotingq>J = (>1(.,1)we obtain)
-2-. -P'III P'III '''t'l + -. 2 '''t'l (1
JoY,
eJ .p, =)
1 2 1
'p'q>1+
and)
J.Y,(>2Q,=)
1
-
2 (1
-
1
if
IE H,)
'p'q>.)) if
[e K,)
)
-:2(1 Q,q>2)+ 1 1 + Q,q>2 2 1 2
-
1 \"2
Q,lf>z if
(1-Q,q>2)
if)
[E H,) rE K.)
It followsthat) 1
-2:(1-
1
and)
- 21
'p'q>1+ r
ThisprovesE ;2F. 2)))
1
+\"2 P'epl
-2
1 i (1 P'q>I)
-2
\037q>I)
\037
\037
0
1
(1- Q,q>2)t 2 Q,q>2 2 jf [E H,) 1
I;
-+
1
Q,q>2+\"2
(1-Q,q>2)+\"2 \302\243
if
re K.)
260 We
Chaptcr 9:Comparison of Experimenls) c
dominated generalizethe notionE;2F to the caseof not necessarily 2)
experiments.
49.4Definition.Let t;
with O. Then the experimentE ist;-deficient
respectto the experimentFfor testingproblems (E 2 F) if E 2 F for every decision 2 (D.W) wherethe elements. decision contains two problem space exactly c The fonowingdefinition extendsthe spiritof the relation\" => \" to general decision problems. \037
c
t.
2)
49.5Definition.Let E O. The experimentE is t;-deficientwith respectto the problem(T, D, W) with a experimentF (E:;1f) if E :;1F for every decision (D,W) lossfunction W. bounded,continuous t t. is not true. It is obviousthat E 2 F impJiesE 22 F. In genera], the converse Later it will beshownthat the converse is at leastvalid for binary experiments (seeSection 54). o If E;? F then we denoteE;?F. The relation\" ;?\"is an orderrelation.If E ;?F then E iscanedmore informativethan F. If E :;1F and F:;1E then E and F are calledequivalentand we denoteE F. c It is obviousthat E:;1F iff E;2F for for every > O.) \037
t.
t.
\"'\"
E;
49.6Corollary.Let 8 (T, D, W)
with
C
O. If E;2F [hen E
t.
;2 F for every decision problem (D,W) a lowersemicontinuouslossfunctionw:) \037
problemwith a Proof Choose > 0 arbitrary and let (T, D, W) be a decision lower semicontinuous lossfunctionW. Let be the set of an bounded, V W such that inf t E T. It continuous lossfunctions Jt; J.v,
f-
\037
II
\037
JoY,
\037\"
II
\037
\037
JoY\"
f
isclearthat for every bounded, lossfunction V, W there is V E continuous suchthat VI V\037 W. Moreover,each Ve f\" satisfiesJt;1I J.v,1I (1+ <5), t E T. If {J2E (F.D) then E F implies) \037
\037
II
\037
II
II JoY, II
}
l1
\037
\037
n {pe :M(E,D):
P(V\"
lET)
1;)
\037
P2(J.v\" Q,)+ e(l +\037)
:+=
(/J
f6' Since isdirectedfrom aboveit fonowsthat) n n {fJ E fJI (E,D):fJ 1;) P2 Q,)+ t; J.v,II} :t 0. VE'fI'6 ,eT)
for every
VE
\"f'\037
(V\"
This provesthe assertion.
\037
(J.v\"
II
0)
for))) By means of the Minimax Theorema necessaryand sufficientcondition
49. Basicconccpts
261)
t
E ;? F can be established in terms of Bayesianriskswhich is the key of all (D.W)
followingconsiderations.) Assumethat (T, D, W) isa decision 49.7Theorem(Blackwell[1951]). problem I; F holdsiff for lossfunction.Let e O. Then E with a lowersemic:ontinuous \037
\037
(D, W))
every mE ST
inf fJl.\037(F.. D)
::: inf fJ2C
\037
(F, DI)
J PIOf;,\037)m(dt)
JP2(\037,Q,)m(dt)+efll\037lIm(dt).
Proof Apply Corollary(45.7)to M2 = {(PI(\037, PI E !M(E,D)}, and = 0) Mt {(fJ2(\037,Qr)+ell\037ll)rI'T:P2E\037(F,D)}. \037\302\273rET:
continuous lossfunction W. If (T, D. W) isa decision problemwith a bounded, theorem may be restricted to 9t(E,D) and then the infima in the preceding Yt (F.D),respectively. of E to be parameterspace(x. It turns For E A (T)let be the restriction of experiments outthat the comparison may be phrasedin terms of the finite (X
E\302\253
subexperiments, only.)
49.8Corollary.Assume that (T, D, W) is a decision problemwith a bounded continuouslossfunction.l.etr. O. Then the followingassertions are equivalent:) \037
c
(1)) E(D. F. \037
W)
t
(2))
\037
t.\037
(D.W))
f\037
for every
(X
E
A
(T).
Proof It isclearthat (E.D) = .(jt (E(I'D) and .rJt (F,D) = .rJt D) for every E A (T). Hencethe assertion is an immediate consequence of Theorem 49.7. 0) \037
F;\"
(X
m may bereplaced rinally,we:show that in Theorem(49.7)the priormeasures
by the unifonndistribution.)
Let c 0 and let D be a decision 49.9Corollary(Blackwell[1951]). space. \037
Then the followingassertions are equivalent: t
continuouslossfunction W. (1) E(D.=> Ffor every bounded W))))
Chapter 9:Comparisonof Experimcnts)
262
(2)
Forevery
ex
inf
continuous lossfunction W) (T)and every bounded inf L J-Y,QI P, L J-Y,Q2Q,+ t L J-Y,II. EA
II
\037
l/2c8t(F,D),CCl
l/1\0378t(E,D)'(:\302\253
'\037CI)
ProofIt isclearthat (1)=> (2).Let us prove(2) => (1).Let m E ST and let W be a bounded. continuous lossfunction.Let E A (1')be such that me We defineanotherlossfunction W by ex
S\302\253.
xED, tET. W(x.l)=lexlm{I}W(x.t), Then it followsthat inr l/If!Sf(E.D)
J J-v,Ul p'm(dt)=
_1 L -
.
Inf
\037
l/z5Sf(F,D)
I ex I
=
inf hf!\037(F,
D))
inf
\037
l/1\037Sf(E.D)
\037U2Q,+
u\302\253
-L 1
\302\243.
1
L Jf;Ul
\037
I ex I '@CI
ex 1
rTJ II yy,1I
u\302\253
0 J\037Q2Qtm(dt)+eJIIJ.-Y,lIm(dl).
50.Standarddecisionproblems) 50.1Deftnidon.A decisionproblem(1',D,L) is a standarddecisionproblem for a parameter spaceT if J) c; is a convex.compactsubsetand if ..... D is the the coordinate of HT, t E T. onto L,: projection \037T
It\"
IJ.I
It will be shown in the next paragraphthat comparison of experiments ofstandarddecision problemsonly.In this may bebasedon the consideration of sectionwe discussthe reduction arbitrary decision problemsto standard decisionproblems.Let us consideran experiment E E 8 (T) suchthat E=
(D,.flI.{p,:lET}).)
50.2Lemma. Let (T,D,L) be a siandarddecision problem.Then for every Q
eat(E,D) there is a nonrandomizeddecision function Qo e {Jt (E.D) such that
L,UoPr=
L,U\037,
leT.)
Proof For every we D letb(w)be the barycenter of Q(w,.)in D.SinceL, is decisionfunction affine-linear on D for every t E T the non-randomized
eo:W'-'b(w) satisfies = (L,Uo)(w). (L,e)(w) = J L,(x)e(w,dx)= is Sincew......(L,Qo)(w) It remains to show that (Jo is (d,\037o(D\302\273-measurable. of for every t E T and since{L,:t e T} separates .9/-measurable points (b(w\302\273,
the)))
SO.Standard decisionproblems
theorem impliesthat fo flo compactset D the Stone-WeierstraB measurable for everyfe \037(D).Henceflo is.ilo(D)-measurable.0)
IS
263)
,t;/-
A nonrandomized 50.3I>efinition. decisionfunction Uo E 9t(E,D) is called
simpleif it attainsonly finitely many values in
D.)
50.4Lemma. Let (1',D,L) be a standarddecision problem.Then the set is i s {(L,Uo1D,,, T:UO e 91(E,D) simplenonrandomized} densein R(E,D,L).) Proof It foHowsfrom Lemma 50.2that the set {(L,UoP')'ET: UO e tit (E,D) is coincides with {(L,UP,)UT: nonrandomized} (!e .\037(E, D)}which itselfisdense in R (E,D,L). Hencethe assertion followsby a standardargument usingthe fact that D is totally bounded.
0)
showsthat,given an experimentE e & (T),for every The followingassertion decision problem(1',Do,L) problem(T,D, W) there is a standarddecision such that the risk setsR(E,D, W) and R(E,Do,L) coincide. Thus,the analysis
of an experimentE may be basedon the consideration of standarddecision problemsonly.)
50.5Theorem(Blackwell[1951]). Let (D, W) be a decision problemwith
a
continuouslossfunction convexhull of bounded, If Do denotesthe closed, = R (E,Do.L).) {( T: xeD} then R (E,D. W) \037
J-Y,(X\302\273,c
Proof (1) Let us prove R (E,D, W) s;R (E,Do,L). By Lemma 44.4 it is sufficientto prove Ro(E.D, W) Ro(E,Do,L).If (!E Yl(E,D) then for every WE Q we defineUo(w,.)to be the one-point measurewhich isconcentrated at Then we have T' (J J-Y,(x)(!(w, dx)) J J-Y,(x)e(w,dx)= J L,(x)Uo(w, \037
dx\302\273,c
for every (I)E Q and tE Twhich provesthat (JY,(!P,)'.TE Ro(E,Do,L). Let show that us To this endwe show that R(E,Do,L) R(E,D, W). (2) Ro(E,D, W) isdensein R(E,Do,L) and apply Lemma44.4.Sincethe risksof the simple decision functionsare densein R (E,Do,L) it is sufficientto prove that every neighbourhood of the risk of a simpledecision function contains of Ro(E,D, W). Let E .91(E,Do) bea simpledecision elements funclionand \037
\037o
let)
if IE U={YERT:I>I-L,uoP,/
for some > 0, \302\243
!X
EA
C!o(w,B) =
(X}
(1').The decisionfunction flo isof the form) \"
L 1.4;(w)IB (x;), WE Q, BEfJI{D), '.0; 1)))
264
Chapter 9:Comparisonof Experiments)
...,
where (At.A 2 . partitionof!1and XjE Do,1 i Arc) is an .\037-measurable k. Fromthe definition of Do it followsthat thereare mj E SD, 1 i k, such
:;
that) I
J J-v,dm j
-
:; :;
< r.
(Xj),I
if
IE
ex,
\037
1 i k.) \037
\037
We define) rc
q(w,B) =
,-L l ,(w)1nj (B), we 0,
BE 91(D).
A
1)
U.Thisisthe (JE D) and it remainsto show that (J-Y,Q\037)'ETE Obviously, casesincefor every t e 91(\302\243,
rx)
I
-L,QoP'1=
J-Y,QP,
k I
k
L J-Y,dm i P, (Ai) i= I J \"
\037
L i=
- L (xi),P'(Ai)1
IS J.Yrdmj
i
-
t)
- (Xj),IPr(Aj)< E.
0
I)
It isconvenient to calldecision (T, D1, WI) and (T, D2 , Jt2) equivalent problems = if R(E.D1, Jt;) R(E,D2 , \"'2)for every experimentEE I(T).Thus we havc provedthat for every decision problem(T, D. W) with a boundedcontinuous lossfunctionthere isan equivalentstandarddecision problem.)
51.Comparisonof experimentsby standarddecisionproblems) Let us consider E = (QI' .rdI'{p,:t E experiments
T})and
F= (Q2.d2'{Q,:IE T}).The reductionwhich has been achieved m the
preceding paragraphyieldsimmediatelythe followingcriterion.)
51.1Theorem.Let E
\037
51.2Theorem.Let
\037
O. Then
C
c E;2F ho/d'iiff E ;2 F for every .'italldard
(Do.L) decision problem(T, Do.L). isconcerned with testingproblems.) Another consequence
\302\243
O. Then E
c \037
2
F iff E
Z
Ffor every standarddecision
(DQ.L) Xz},Xl E JRT, Xz E
problem(T, Do,L) with Do = co{Xl' Proof We know that E 2 F iff E
t \037
e\037T.)
2 F for every decisionproblem(T.D. W)
(D.W)
containsexactly two elements. By Theorem50.5for every such decision problem(Do,L) with problemthere isan equivalentstandarddecision = co x X E Let E Do {xl' 2}, x\\ 2 conversely (Do,L) be a standard))) where D
[\0371..
\0371'.
51.Comparisonof expc:rimcntsby
standard decisionproblems
265)
decision problemwhere Do= co{Xl'X 2 }.This is by Theorem50.5equivalent with the ordinary problem({XI'Xz},L).
0)
In the followingwe apply Corollary49.9tostandarddecision Let us problems. recallsomenotations.For every A (T) the set consistsof all points from R T to W isdenoted in Theprojection X E IRT with support contained by !X
IR\302\253
\342\202\254
IX.
A,.
We
testingproblems.) beginwith a simplefact concerning
51.3Theorem.Let
E;
O. Then
\037
c
F iff for every E;2 2
IX
EA
(T) and for every
standarddecision problem(T, J>a., L) where Da. = co{XI'X2},XI ERa, X2 E inf inf + 1; L IIL,I/. L L,(}Ip, L L,(}zQ,
\037:a)
\037
f4:a Qz.\037(F.D..)
f4ia QIf!91(E.D.)
fECI)
I
I
iff forevery :2 ProofWe knowfrom Corollary49.8that E :2F 2 z 49.9the secondassertion isequivalentto) By Corollary E\302\253
inf Qle\037(E;.DC.\302\273)
L
J.\302\245,lh
P,
L
inf
\037
/l2 E !I(F.D(.\302\273)
fC:\302\253
,e\302\253
\037(}2
\037
Q,+ 1;,ea) L
IX
E A
(T).
II J.Y,II
for every A (1')and every decision problem(/)(2), W) where D(a.) contains W is lossfunction.In view of two elements and a bounded,continuous exactly Theorem50.5for every such decision problem(D(QI;',W) there isaan equivalent standarddecision problem L) where = co{Xl'X2},Xi E n , j = 1,2,and = II J.v,1I, t E IX. 0) II L,II IX
\342\202\254
(D\302\253,
D\302\253
feature of the followingresultis the fact that only convex The interesting
to obtaina criterion for E polyhedraneedto be taken intoconsideration
SI.4Theorem.Let e
c
\037
F.)
c
F holdsiff for every C(e A (T) andfor every standarddecision problem(T,D:a,L) where Da. isa convexpolyhedronin \037
O. Then E
\037
RCI
inf /lIE\037(\302\243.D.,
L L,QlP,
\037
'Eal
inf Q2G\037(F,D..)
L L,QzQ,+ e L IIL,II. 'E\302\253
'Eal)
showsthat E 2 F iff ProofA similar argumentasin the proofofTheorem51.3 for every C(e A (T)and every standarddecision problem L) where c inf L L,(lJP, /l1(:9t(E.D..) '\037aI)
DCI
(D\302\253,
isconvexand compact)
\037
inf \0372f1:dt(F.D..)
jR\302\253
L L,(}2Q,+ E L L,II. II
,e\302\253
'4iCl)
of the presenttheorem is nt:Ct:ssaryfor E F. To prove Hencethe condition Forany > sufficiencylet C(E A (T)and let Da.S;; RCI be convexand compact. \037
0)))
\302\2431
266
Chapter 9:Comparisonof Experiments)
we choose a simple decision function e2E .'1t(F,Da) such that)
I:L,U2Q,
\037
rp\302\253
I:L,qzQ,+ 8 I: L,II.
inf
1
ht9l(F,D.) rt\302\253
,ta)
II
Let Da.lbethe convexhuH of the imageofQ2'Since 1 isa convexpolyhedron it follows from the condition of the presenttheorem that) D\037,
inf L Lrql P, 'u) QII9I(E,D.) \037
\037
inf L Lrql P, 'Ea) \",.\037(E.D...,)
L LrC?zQ,+ t 'ea) L IILrll D..,)rea inf L L,QzQ,+ (8 + 81)L IILrll. 0 rea inf
Q2_!lI(\"', \037
Q2e9l(F,D.)
u\302\253)
52.Concavefunctioncriteria) of is the setof all x E Let T * 0. The dual (IR\037)+
IR\037
R\037
Recallthe tenninology of Section 45.)
with finite support.
52.1Definition.Let D s;(iR be a boundedset.Then 'PD:Rr.--.R+:y.-.inf{(x,Y):XED}) \037)+
is caHedlowerenvelopeof D.
Notethat forany D s; we have 'PD = 'P\037D' Henceany lowerenvelope is the lower envelopeof a closedconvex set.The lower envelopeof a convex } is given by 'I'D(y)=min{<x;,y>:t\03 polyhedronD=cO{x l ,X2,''''X It iseasy to seethat) n},YE 'PD,(\\ 'PD2 = 'Pt.5(D,v D2)' 1 V'D, + A.2V'D2 = 'PA,DI+A2Dl' (\037\037)*
II
IR.\037.
\037
A.
for any
V'D(-Y)= V'(-D)(y),
YE\037T,)
setsD1, Dz, D
and
\302\243
(R\037).
A.
1
\037
0,
).z
\037
O.)
52.2Lemma. Any
lower envelopeis upper semiconrinllolls, concaveand positively homogeneous.)
Proof.Obvious.
0)
is even continuous.))) It isclearthat the lower envelopeof a convexpolyhedron
52.Concave function
52.3Definition.If lI'D is the D
\302\243
(IR.\037)*
criteria
267)
lower envelopeof a convex,compactsubset
then the norm of 'PD is defined by
II 1IJD II
= L supIx,I. 'frT x.,D)
-
It is easy to seethat I11J'DII= L max {-1J'v(e,), -lpv( e,)}.If (T,D,L) is a 'f!T) standarddecision problemthen 1111' D = 2: L,II. II
II
,.T)
52.4Lemma. The lower envelope11'0of a convex,compactsubsetD = (IR\037)* satisfies
-
-lI'D(12)1111IJDIIIYl Y2Ico,)iE 11I'D(YI) \037
Proof Easy.
Rt i = 1,2.)
0)
52.5Definition.Let E = .rd, t E T}) be Q ..... be such that) et E A (T)let 1;.: dp' (w) I.f tEet, = d 'L. \302\253(,1,
{\037:
an
For experiment.
every
R\037
$I'Q
(\037(w\302\273,
Io
\037
if
t
;
et
.)
Then = .!l'(1;.1
is calledstandardmeasure of = (Q,.r;I,{p,:tEet}) and of E. A(T) is the systemofstandardmeasures It isobviousthat the mapping1;.. is E(I-sufficientfor every et e A (T).) \037
(1\302\253
\037)
E\302\253
.sE\302\253
\302\253(1\302\253)\302\253\037
52.6l.emma.Let EE SeT)be an experimentandJet (T, D,L) be a standard decision problem.Thenforevery et E A (T)andevery nonrandomizedII E .
of given d(\037). Proof Let el be any version of the conditional expectation We will provethat P,{el;D}=Oif tEet. \037
Let M be the family of all affine-linearfunctionalst:
t(x) 0 if XE D.Then {e, D}= (r;,M) U {t0 (!I> O}. \037
;
If Mo c M iscountable and densein M then)))
R\037
.....R
+
satisfying
268
Chapter 9:Comparisonof Experiments)
;
{el D}= \"Mo) U {t0 fll > O}. notethat toele \037(toQI.Qt(1;\302\273 if IE and thereforeto(l1 0 \037-a.e.if 1E ex. Henceal can be adjustedto obtainqo. 0)
We
(X
\037
beenshown in Theorem51.4the comparison of experiments may be basedon the Bayesianrisksfor standarddecision These problems. Bayesian risksmay be expressed as integralsof the standardmeasures over concave As has
functions.)
52.7Theorem(Blackwell [1951]). Let Ee I(T) and let (O'(I)UA(T)be the associated Thenfor every C(E A (T)andfor every systemof standardmeasures. standarddecision problem(T, D(I'L) where D(I \302\243
R\037
= 1pD..d(1t,. inf L L,(!p, 1..) lEa QE.(E,D.)
Proo! It is
sufficient to considernonrandomizeddecisionfunctions D. (l e 9t(E,Da) which in the followingare viewed as mappings(l:Q ..... 0 = to Lemma52.6we may assumethat (} (lo It isthen clearthat According Q
1;.
L L,(lP,= J (eo(y),y)
C1\302\253(dy),
lell)
Thus we obtainimmediatelythat
L L,(lP, J f.PD.(Y)
inf
\037
D.) '\"a)
Q\".(\302\243.
To provethe reversed inequality choose > 0 and let (X\/lt'N") be a sequence \342\202\254
which isdensein Da.Forevery y e Sa define(It(Y)E D in sucha way that (l,(y) = Xk if keN is the smallest integersatisfying (xt,y) f.PD'
\037
Then(!,:8v. ..... function asis easily seenfrom) Da. is a measurable {Ya
e Sa:(le(Y)= Xk}
=
1-1
n {ye 53:(Xi,y)> V'u..(Y)+ t}
i=
I)
f\"'I
+ t}. {ye SQ:(Xk'Y) 1pD.(y) \037
we have) Moreover, inf
=J
L '.v.L,(}p, lev.) \037
IlE9l(\302\243,D.)
\302\253(le(Y),
\037
Y) t1a(dy)
L,(at.\302\260
\037
TJP'
0)))
52.Concave function
52.8Corollary.Let f. 0 and let \037
critcria
(O',')UA(TJand (ra:).:!EA(T)be the t
system of
standardmeasures of experimentsE and f: 71,ellE F holdsiff for E A (T) andfor every convexpolyhedron/)(1: \037
rx
\037
269)
every
IRa)
+\302\24311'1-'0./1.) Jtpo.dO'('( Jtpo..d'r('( \037
Proof CombineTheorem51.4and Theorem52.7.
U)
t
It is easy to formulate a similarcriterionfor the relationE 2 F. Related in Section 54.) will be considered questions \037
52.9Lemma, Assumethat M s; ST iscompactandconvex.Thenfor every upper
concavefunction f M -+ IR there is a closed,convex set semicontinuous. J)
\037
([RT)* such that
f(y)=inf{<x,y):xED},YEM.) Proof SinceM is compactthere exists E A (T) such that M s; Sa:.We will is true for D == {XE(IJf/\*:") <x,y)\037f(y) if yE M}. show that the assertion = Let W {(y,{J)E M x IR: f(y) {J}.Then W s; IRT X IR isclosedand convex. Chooset> 0 and Yo E M. Then (Yo,f(yo)+ r.),Wand hence there exist rx
\037
Xo E
(IRT)*,Y E
such that)
IR,
<Xo.Yo) + ,(f(yo)+
whenever}'E M, P We
\037
\302\243)
\037
C < c + <5
<xo,y) + yfJ
f( y). Putting y = Yo and P = f( Yo) it followsthat}'< O.
normalize Xo in such a way that y = (Xo,Yo)
\037
-f()'o)- t
\037
-
-1and obtain)
<xo,y) f(y)
if
ye M.
T SinceYo E Sa:and M S; Sa:there is a translation XI E (R )* of Xo such that)
(XI,yo)-f(Yo)-C=O \302\253
We
obtain
assertion.
Xt
E
x 1,y)-f(y) if YEAf.
D and f(yo) <XI'Yo) =f(yo)+ t. This proves the \037
0)
52.10Corollary(Blackwell[1951J). Let (0'a:)a:(IA(T) and (ta)(I:(IA(T) he the system,; ofstandardmeasuresofexperimentsEandF. Then E => F iff for every andfor every concave,upper semicontinuous functionf Sa -+ R) JfdC1('(
\037
rx
EA
(T)
Jfdta:.)
issufficientfor E ::>F ProofFromCorollary52.8it followsthat the condition sinceevery supportfunction 'I'D.with D(I R(I: isconcaveand uppersemicontinuouson Sa. Conversely,the conditionis necessary in view of Lemma \302\243
52.9.
0)))
Chaplcr9:Comparisonof Expcriments)
270
53.Hellingertransformsandstandardmeasures) Recall that if E E SeT) is an experiment then E A (T), denotesthe restriction of E to the parameter set s; T. In the followingwe define = 1.) E\037,
(X
\037
0\302\260
53.1Definition.Let E = (D,d, {p,:IEI'})be an experimentand let 91 .91 be a finite subfieldof d. If BI , 82 \"\",B\", are the atomsof 91 then the \037
conditional tran:,formgiven 91is) Ilellinger m
19I)(z):=LI tn 'p'='(J\037)
H(E\302\253
if
(X
E
A
j=
E\302\253)
(T),Z E S-z..)
53.2Remark. Recall Holder'sinequality for nonnegativenumbers atJ , lET, 1 j $ m:) \037
m
L
j= I
n 0:;
m
\037
rECI
fI (L atl' J=
rECI
if
E Sex'
Z
IJ.
EA
(1').
I)
53.3Lemma. Let E E 8 (T)bean experimentand!Ms;.f/I a finitesubfield of.f/I. Then 0
\037
H(Ea.I 91) 1 on S(lJorevery \037
ProofIt is obviousthat
(X
EA
(T).)
for every H(Ea.IfJ4) O. Holder's inequality implies \037
ZE SCI)
H(E I9l)(z)= cz
= L te-z. fl p,:'(\037) tea. fl (L P'(\037)y' 1. 0 j-I j=1) m
m
\037
53.4Lemma. Lei E E S(T) be an subfields of ,rtI.Then) fJll\037fJl2
for every
(X
EA
experimentand let 911 and
!M2
be finite
on Sa implies 1l(EaIJll)\037H(Ea.I!M2)
(1').)
ProofThe relation911 912 impliesthat the atomsof ,tjl2 are a finer partition of than the atomsof 96..This meansthat every atom of 8ft 1 is the unionof followsfrom the fact that for pairwisedisjointatomsof912 , Now the assertion unionC C of sets E v E C C disjoint 1 any I 2 2 sI, Holder's inequalityimplies)
!l
\037
\037r4,
fl'p'r'(CI U C2) = fl ('p'(C I ) + .P,(C 2)Y') n p,r,(c1) + fI p'r,(c2). /(
If'CI)
I\"a.)
\037
a.
0
tEer)
53.5Definition.If E E 8 (T) then for every
(X
E
A
(T) the Hellingertransform)))
53.Hellingertransfonns and
slandard measurcs
271)
H(EJ:S2-+ [0,1]is definedby (z) = inf
H(E\302\253)
I
{//(E\302\253
&I) (z):fJI
\302\243
d a finite subfield} .)
53.6Theorem.LetE = (D,d, {p,:t E T})bean experiment. IfrxE A (T) andif vld isa a-finitemeasuredominating{p,:t E ('j,} then)
-
dP. ' n H(Ea)(z)=JlEa: ( d )
%1
ZESa.'
dv,
V)
...,
for probability measuresvlsl. Proof It is sufficientto provethe assertion (1) Let 11c d be a finite subfieldwith atoms(81,82 , 8m). Then we
have for every j E
J
Bj
..., m}, :'
{1,2,
d [1 'Ea: (d )
dv
\037
\037
E
Z
\\
Silo')
dP' = [1 (J d dvy' II P':'(\037) 'Ea: V
BJ
'Ea:)
and this implies)
dP, J ha. dV ( )
%'
[1
-
dv
\037
f.
n P'Z'(\037).
)\"'II(a:)
Thus we obtain)
I n (dP' J 'E d )
Z'
dv
\037
V)
a.
H(Ea:)(z),ZE Sa:.)
(2) To provethe reversed inequality let I; > 0 and z E Sa be arbitrary, and in such a way that the functions choosea finite subfield \037
m
\037
d
p, (\037)
h =.\037 V (B) 1s}, t E
)-1
satisfy)
dP.' dv
Z,
(j\"
> 0,
J)
-[,
t
%. \037
1
\037'
tE
rL,
Z,
> O.)
Sincevld isa probability measurethe possibility ofsucha choice followsfrom Let = martingaletheorems. ('j,
{'1't 2 ...., 'II}'Thenwe obtain)
- }\"1 i: II dP, f1 :,>0( )
n (dd.P,V ) J lea
%.
dv
z'
\037
S I\037II,
'p'z,\037)
I\037a:
dv
n
jz.
lEa:,:,>0)))
dv
Chaptcr 9:Comparisonof Experiments)
272
%'1
\037'!.
'Ie\302\253.:,;>o
\037
dP. J --!! ft'l n n . f/'J. dv ( ) k
L
\037
dv)
dP'I
L \"t!a.:r,>oJ
[( (
%'1
%t
dV
) :' Lz, dv =. dP' ---!. = L /,z, It!a,=,>0(S (dv ) )
-
j, J ( )<. (Usethe fact that Ix - < Ix dP. \037
L
\037
f
A
dV
%tl(
n (f ) k>i
dp'\"dV dv
%II(
)
]
Z'
E;
d\037'
dv)
'..\302\253.%,>0
z
the assertion.
yZI
yl%
if
x
\037
0, y 0, 0 < z < 1.)This proves \037
0)
53.7Remarks. (1) If 'P%:Sa.
dv
-/,:':t ) ' n ( /,1( 1
dv
%11t
-.
IR:
Z
E
Sa. then we denote)
x...... n x:'. lEa)
The function 'P:iscontinuous and concaveon Sa.However, for x
\342\202\254
8Sa the
functionsz 1-+'P:(x),Z E S(%, are notcontinuous. Fromthe preceding theoremit followsthat H(E,J(z) = J 'P%d(fz, Z E Sz,where (fa is the standardmeasure of
s.
Ea, ex E A (T). 53.6it followsimmediatelythat H(Ea.)(z) (2) Fromthe fonnulaofTheorcm = H(E\037) (z) if Z E fJ S;; (1. the terminology of Section 5 it isclearthat H(Ea.)is the Hellinger Recalling transfonnof the standardmeasure (J(%, (1E A (T). This yields the following S\037,
assertion.) uniqueness
53.8Corollary.Let E E SeT),Fe SeT).For
ex
E
A
(T) let (fa be the standard
measure of and let t be the standardmeasure of F(%. 71zenthe following assertions are equivalent.') E\302\253
(1) (2) (3) (4)
02
E-F, E02 \037
-
=
F\",
f02
for every
for every
fJ.
ex
(T), E A (T). \342\202\254
A
H(4J= H(F;.)for every
ex
EA
(T).)
we ProofIt followsfrom Corollary52.8that (1) (2).FromCorollary52.10 obtain(2) => (4), Theorem5.19yields (4) => (3) and (3) => (2) is another of Corollary52.8. 0) consequence \037
53.9Corollary.Assume that T is a finite set and let
(J, t
be the
.\037tandard)))
54. Comparisonof experimentsby testing problcms
273)
are equivalent:) measuresof E andF. Then the fol/owinKassertions
F (1) E...., (2) (3)
(1=
t,
H(E)= H(F).)
Proof It isclearthat (1) (2) \037
I/(E) = lI(F)implies
I/(E\302\253)
followsfrom the fact that (3).The assertion = ll(\037) for every E A (T). 0) \037
ex
The folJowing is a useful lemma which alJows to reduceequivalence of
ratio processes. of to equality of likelihood It is an extension experiments Corollary25.9.)
53.10Theorem.Let E and Fbeexperime1ltsin I(T)andlet P/ = {p,:t E T}and !l= {Q,:t E T} the corresponding setsofprobability measures. Then E F iff)
-
dP. = !I'((dl's )leT p.) !I'((dQ:sleT ) Q) \037QI
$
5)
for every SET.)
of the precedingresultsand becausethe distribution of a isdetermined stochastic process by itsfinite dimensional marginaldistributions it issufficientto prove the assertion for finite setsT * 0.But this has beendone in 25.6. n) already Corollary Proof In
view
54.Comparisonof experimentsby testingproblems) us explorethe relationbetween comparison of experiments by general and decisionproblems by testingproblems. As indicatedpreviously both for coincide To concepts binary experiments. provethis we needa formula to Iransronn i ntegr.Isfor (J.intointegr.lsror Jl. !R where PI) E = (PI'Pl) is a binary experiment. Then we arc in a positionto apply the resultsof Chapter3.) Let
\037
(\037\037
54.1Lemma. Let E be a
binary experimentand 'PD the lower envelopeof a !}(2. 71,en)
convex,compactsetD J 'POd<1E= 'Po(O,1)+ J ['Po(l,t) \302\243
- 'Po(O, t\302\273
\037lt(dl).)
Proof Let E = (PI'P2). Elementarycomputations yield)))
Chapter 9:Comparison of Expcrimcnts)
274
= J 'PO('1h d1 d\"2) + J 'PD('11, '12)ud '12)UE(d'11, d'12) J'I'DduE \". =0 \". >0 = J 'PD(O,1)'12Uf,(d\"\"d'12)+ J !J.'D 1,'12 '11 d'12) '1.=0 '1.>0 ( '11) uE(d'1t, dPl = 0 + dP2 = 'Po(O, 1. 1)P2 'Po f ( dPI ) dP. {d(Pl + Pz ). } = 'PD(O.l)(l $ I + $ 'PD(1. I) J.lE(c/I). 0) 1\"
-
J.l,.;Cdl\302\273
54.2Theorem(Torgersen[1970]).LeIE = (PI'P2) and F = (Ql'Q2) bebinary Thenfor every 0 Ihe relationE F is valid iff E ;2F. experiments. \302\243
\037
\037
of the assertion isnot trivial. Assumethat E F. ProofOnly onedirection \037 2)
Then we obtainfrom Theorem17.1that) JfdJ.lE; JfdJ.lF+ \037
;(f'(0)
+
f\302\253(YJ\302\273)
functionf [0,oc)-+ [0.co).f(O) = O. Let D for every concave, increasing Then) be an arbitrary convexpolyedron. 'PlJ(Y)= min {<x\037,y>: t
if D = co{Xl'Xz. J 1pD
d(]\302\243
...,
\037
\037
\037
n},
R2
ye\037,)
X,,}.We have to show that
J 'PDd(]F+ e II 'Poll.)
Let us denoteXi = (\037i' '1i)'1 + max I\"d.We denoteu = min
\037
1
i
\037
1
\037i\037n
i n. Then \037
\037I
and v
1
\037i\037\"
n_ J UYI J 'I'DdUE: = J 'I'DduF.
we have II'PDII= max I 1 :;iiS\" = min '11' Since
d(T\302\243(Y)
\037d
;li\037n)
+ Ii =
S'I'D(u. O)d(TE+ u
and similarly
+U J 'PDduF = J 'PD-(u.o)duF we obtainfrom Lemma 54.1that) J
-
= ['PD-I\".0)(1, I) - vl]J.lF(dl) t) - vt]J.lE(dl)- f['Po-(u,O)(t, -u) + 1('11- J.lE(dl)- J min - u) + I(\"i-. = J min
'PDd(1\302\243
= J 'PD-(u.o,duE S'fJJD-(u,o)duF
$ 'PDduF
$
1
\302\253\037i
\037i\037n
the function))) Considering
\302\253\037i
v\302\273
1 \037i::':\
v\302\273
J.lF(dt)
54. Comparisonof ex.perimentsby ltslingproblcms
J(/) = 1mm
- u) + I(\"j-
((\037j
I 0, \037
v\302\273,
:!i\037\
we observethat it is concave.It u = \037II' V = \"j2'then we have f(O) = min (C:j u) = 1
\037I\037n)
1'(0)= min (\"i ,:,=u
-
v)
is increasingsincetli
-
\037il
\037
U
275)
=
-
v
\037
0, 1
\037
i
\037
n. If
0,
2 max I\"d. 1
\037j\0371I)
J(oo)= min(\037I-u);;!2 1max '1,=11
I\037d.
\037j\0371I)
Thus,we obtainthat)
-
-
J 'PodrJf. J 'PndrJF= JJdJ.lf. JJdJ.l,.. f.
\037
\037
2 (I'(0) \302\243(
max 1
+I(Cf:)) I\037d
+ max I
\037j\037\"
1,,;1)
\037i\037\
which provesthe assertion. 0)
..
with an arbitrary parameter spaceT 0. Let us return to experiments '\" and\" \" are identical. that t he For at least equivalencerelations\" prove thisis a consequence of Theorem25.8,Corollary25.9 dominatedexperiments and Theorem53.10. Let us consider E = (D I' \037 = {P,: experiments lET})and = {Q,: F= (Q2td2,f2 IE T}).Let (CTa)al'A(T)and (!a)2I\".4(T) be the systems of ofE and F.The followingisan extension of Theorem22.5.) standardmeasures We
=t=
l'
d l'
54.3Lemma(TorgersenrI970]).Let II
L a,\03711 'eT
Jor every a E (IRT)*
\037
II
\302\243
O. Then
\037
E? F iff
L la, leTa,Q,II- ,cT) L
e
I
.)
F iff for every Proof We obtain from Theorems51.3and 52.7 that E;2 2 x <XE A(T) and every pair (X 1,X2)E ;;!J min (XI> y) '01(dy) + L max (xl),I. J min (Xj,y) RIZ
j-1.2
CT\302\253(dy)
i-I.2
R\302\253
E:
I
'E\302\253
i-= 1.2)
It issufficientfor E ;2F to requirethe condition for all pairs(xl'Xz) E W x IRa 2 which are centeredat zero,i.e.x I + X2 = O. For such pairswe have))) &
276
Chapter 9:Comparisonof E'l:periments)
.
(Xi> y)
-2 1
=
i\037:\0372
and)
1
1 max I (Xi),I = - lxI,' 2
i'\"
),2)
.-(xz.y)
(X 1>y)
I
- xz.,I.
F iff for every a. E A (T) and every pair (XI.Xz) E RII x W It followsthat E -2 2 such that Xl + X2 = 0)
-
fl(xl,y) (x2,y)la(l(dy) \037
- (Xz.y)I'tIl(dy) - t L - x2.,I.
JI(xt,y)
Ix)\"
'CCI)
Thisinequalityis equivalentwith)
L (Xl\"
J
'\037CI
-xz,,)d'L,dp'p. d L I's s
s\037\"
.sEll)
\037
-Xl.,)d dQ, LQ
L (XI., J 'cer
S
-
d L Qs e L Ix)\" 'F(I SFCI
-x2.,I.
Hill)
Putting a = x I
-
Xz
we obtainthe assertion. 0)
Let e O. Then E F iff for every pair 54.4Corollary(Torgersen[1970]). ml EST.m2 EST the binary experimentsEo = (J P,mt(dt).J and f'o = <JQ,m)(dt).fQ,mz(dt\302\273 satisfy Eo;2EO.) \037
\037
p,m2(dt\302\273
t
ProofAssumethat E 2 F. Let ml EST.m2E ST' We
have
to show that
z)
\"aJ .p,m)(dl)+ b J 'p'mz (dl) \"
\037
for all a E
IR.
lIaJ Q,m.(dt)+ b J Q,m2(dt)lI8(lal+ Ibl))
bE IR. Fromthe preceding theoremwe know that
lIaJ p'm)(dt) + b J p,m2(dt)
II
L lamI {t}+bmz{t}I. \037lIaJQ,mI(dt)+bJQ,mz(dt)lI-e ,,,T) The assertion followsfrom the fact that)
L lam){t}+ bm2 {t} lal+ Ibl. ,e I
T)))
\037
54.Comparisonof experimentsby testing problems
Assumeconverselythat the condition issatisfiedand let (R T)*.If a, every lET or if a, 0 for every lET then it is easy to obtain) a\342\202\254
\037
277)
0 for
\037
II
L a,\03711 'fT
\037
t
L a,Q,II- '(1') L la,l. ,eT
II
Otherwiselet ml {t}= a,+ /
r a,+,m2{t}= a,-/scL a,-, IE T.
IE T
It followsthat) II
:E at rGT
\037
;;?:II
II
=
T)
rr.T Q,II- t (I :E a at
$lOT
f
s
I
+
I
r as I)
$lOT)
LT arQrll-erELT) larl.
II
rE
Thisprovesthe assertion.
0)
54.5Theorem(Torgersen[1970]).E 2' F iff E
-
-
F.)
E '2 F.Assumeconverselythat E 2' F. Proof It isobviousthat E F implies Then for E A (T) and mESawe put 1 L P, and QI = L m {t} P\" PI = oc_ 'ea ex
_
P2
rCHI
I
I
=. 1-- LQr loci
and Q2=
h.G!)
L m{t}Qr. '\037a)
ThenCorollary54.4implies (PI'Qd ...,(P2,Q2)'Hencewe obtainfrom Corollary
16.5)
!I!(
\037;,,'
p,)
\037
!I!(\037;,:Q2)
which implies
!I!
C\037
m
{I}d \037 p, s&a
i:1
.\037.
\037
p,)
!I!
m{/}d
C\037.
\037Q \037:
s.a)
I:'
.\037.
Q.).
Now an easy argument usinguniqueness of Fouriertransfonnyields)
_!..-L P = !f' dQ, !i'((\037 d :E )tia 1<xI.lu) ((d L Qs) S
\037
sea
sea)
\037
L QS .
'Ell loci sea
)
Thismeans that the systemsof standardmeasures ofE and F coincide.
0)))
Chapter 9:Comparisonof Expc:rimenls)
278
54.6CoroUary(LeCam[1964]and Torgersen[1970]).111e followingassertionsareequivalent:) (1) E 2'F. (2) E F. (3) There is a linear isometryM: span# span!!lsuch that M (P,) = Q, if
-
-
te T.)
ProofApply Theorems54.3and 54.5.
0)
55.The randomization criterion) Let E and F be experiments in G (T).The aim of thissection is to provea generalversion of Theorem25.8.)
55.1Definition.A R: !l'b(Q2' .912) X
randomization from E to F is a bilinearfunction L(E)-+ IR satisfyingthe followingconditions:
(1) IfI\037 0, a 0, then R(f, CT) 0. \037
\037
(2) (3)
IRU:CT)I 1I/11\"IICTIi. \037
R(1,CT)=t1(Ql)' The setof all randomizations from E to F isdenoted by !B(E, F).)
-
55.2Definition.A stochastic operatoror transitionfrom L(E) to L(E) is a
positivelinearmapM: L(E) L(F)suchthatllMCTl1 = IICTII ifCTE L(E),CT O. Every stochastic operatorM: L(E) -+ L(F) definesa randomization RM t 1 E to RM(f, (1)= (M(1)(f),fE (Q2\"w' from E toF according 2)' L(E).E.g. condition (3) followsfrom) \037
\037b
RM
-
(1.a) = (MCT)(1)= (Mq)(!22) -:-:M(q')(!22) M(a-)(!22) = lIa+1I-!la\"\"11 = IIM(a+)II-IIM(a-)1I = a(O.).) = CT+(O.) q-(O.)
-
-
The converseis not true in general.If R is a randomization then there is a
mappingMR : L(E) ba(02'd 2) whieh is defined by MR(a) (f) = R(f, CT), d2), aE L(E).The mapMR is a positivelinearmap suchthat fE 9'b(Q2' = (1E L(E),a O. Thisisan easy consequence of the properties MR (111. of R,) II
\302\253(1)
II
II
\037
55.3Definition.An experimentFE &(T) is a ralldom;ZQt;ml of E jf there is a R E \037(E, F) suchthat Q,= MRP, for every t E T.))) randomization
55.The randomizationcriterion
279)
of dominatedexperiments. In Section24 we have defined randomizations
two assertions are compatible.) The following imply that bothdefinitions
55.4Lemma. For every randomizationR from E to F there ;s a stochastic L(F)such that operatorM: L(E)...... lET.) IIM\037-Q,II\037IIMRP'-Q,II, ProofBy Theorem41.7there existsa positivelinear operatorT:ba(Q2'.912) ...... L(F)satisfying TII= 1,(TC1)(1)= (1)if C1 0 and TIL(F)= idL(F)'Define II
\037
M:==ToMR' Then
-
-
-
liMp, Q,II== TMRP, Q,II= IITMRP, TQ,II = IIT(MRP' Q,)II IIMRP, Q,II.) tE
II
T, which provesthe assertion.
\037
0)
55.5Theorem.An
exper;mentFE S(T);sa randomization0/ EE SeT)iff there i'ia .'itocha.'itic operatorM: L(E) -+ [.(F)such thaI Q, = M P, for every t E T.)
55.6Remarks. (1)The set14(E,F) can be topologized by the weak topology which is the coarsest for every topologysuchthat R \037 R(f,J.l) iscontinuous E \302\243l:'b(Q2,.td 2), J.l E L(E).A similar argumentasin Theorem42.3for .rJI(E,D)
/showsthat 91(E,F) iscompactfor the weak topology. (2)
A
subsetof \037(E, F) is the setfR(E, f) of all stochastickernelsK: If KE 3t (E,F) then [0,1].
QI x .912 ......
R,,:(f, /1)
\037
J I/(wz ) K(fl) , , d(J)2)/1(dwt ),
.912), J.l E L(E),is an element of fM(E, F). A copyof the proofof Theorem42.5shows that {R\"E \037(E, F):K E fit(E,F)}isdensein fj(E,F) for the weak topology.The only differencein the argument is as follows:The we may choose functions/I'''' ,I\",have to bechosenin 2, .91 2) and w.l.g. function. IfC C is an .9I -measurable of , step t z partition Qz suchthat each p h, 1 i m, isconstanton eachC , 1 k p, then we definePic 1=lck, 1 k p. Now the proofis literallythe same as for Theorem42.5. that E isa dominated t E T}) (3) Suppose experimentand F = (Q2,.91 2 , {Q,: is such that Q2 is a locallycompactspacewith countablebaseand .r42 = 9I(f1 M:L(E) ...... L(F) there existsa kernel 2). Then for every transition K 9t(E,F) such that RM = R\". The proofis easy.Indeed,RM E \037(E. F) is such that R M (.,P,) is a Daniell integralon \037b(Q2) for every tE T. Then the existence of the kernel K followssimilarlyto Theorem42.7. we criterion. We followLeCam Now, approachthe importantrandomization
/E
\302\243l:'b(Q2'
...,
\037
\037
\037
\342\202\254
[1964].)))
\302\243l:'b(D
tc
\037
\037
\037
Chaptcr9:Comparisonof Experimcnts)
280
55.7Lemma. Let
e
+ 1]T,If [-1t there isan R
0 and D =
\037
t
E;2F
then (E,F) such that
for
every
E 9t(F,D) and every mE ST (MRp')m(dt) J L,(!z Q,m(dt)+ (;,) J L'(!2 decision function Q2 e d'/(F t D) such that ProofThereis a nonrandomized
(12E
\037
\037
= J L'(!2 Q,m(dt) J L,Q2Q,m(dt) ,)
t
> 0 be arbitrary. SinceE => F there existsa nonrandomized simple decision function el e 91(E,D) such that + t + {). J L,Qt'p'm(d/) J L,Q2Q,m(dt) It may beassumedthat QI takesitsvaluesin the convexhull of the imageof Q2' Let
{)
\037
Let) '\"
01= i=L xi 1Aj 1)
\"
\"
L L (1,lj = 1tYjE im'?zt 1 j nt 1 i m. j=1(1,lj}j, j=1) Forevery j e {1t 2t t n} let wj e O2 be such that Q2(wj) = Y,;. Thenwe define the randomization R by) where Xi =
\037
\037
\037
\037
...
d2), J.l L(E), L L (1,iJf(wj) J.l(Ai)tf E !L'b(Q2t i:;:1 j:;:1) It is obviousthat R E (E,F). Moreover,for lET we have) R(j,J.l) =
In
\"
\342\202\254
\037
= L,Q2(M = R(L,Q2' L,Q2(M R P') R P,) P,)) =
In
\"
L L
,Pr(A i) (1,lj(h.,(Wj)
i\037lj\037l)
which provesthat
'\"
n
=
L L (1,ijYj\"P'(Aj) i=1 j=1
=
L xj.,P'(A ) = L'(!1 i:;:
'\"
j
p\"
1)
+ e + {). J L'(!2(Ml(p')m(dt) J L,(12Q,m(dt) Since{) > 0 was arbitrary we obtain) \037
-
inf
R P')m(dt) J L'(hQ,m(dt\302\273 (sL,Q2(M Now the assertionfollowssincethe infimum is attainedon the compact set(fI(E,F). RE\037(E,
F)
0)))
\037
\302\243.
55.The randomizationcriterion c
55.8Lemma. If E;?F then for every mE ST
281)
there exi.<;tsa randomization
R E f!4(E,F) such that
+ 1] Proof Let D = [-1,
J IIMRJ: Q,lIm(dt) e.) T.
4) (MR
\037
we denote) Forconvenience
-
,(!,m) = J L,(!(A1 R J:)m(dt) J L,\037Q,m(dt)
if R E f!4(E,F),(} E
9t(F,D) and mE ST' Then the preceding assertion implies inf
sup
R.IiI(E,F>)
\037E9t(F,D)
tP(MR ,(!,m)
\037
t.
Fromthe MinimaxTheorem(Theorem45.8)we obtain sup cP(MR' (!,m)
inf
\037
C
Q.\037(F.D')
Re9t(\302\243.I',
for every mE ST' SinceR \037
sup
(F,D)
QE!Jf
4) (MR'
on (!,m) is lower semcontinuous
the compact setf!4(E,F) the infimum isattained.It remains to show that for every R E 91(E,F) and every mE ST) = JIIMRJ:-Q,lIm(dt). sup clJ(MR,(!,m)
QEStCF.D))
Firstwe notethat obviously) clJ(MR , (},m)
-
JIIMRJ: Q,lIm(dt) for every {!E 9t(F,D).On the otherhand,for e > 0 and for every t E T there isa E function.f. b (Q2',$/2),lI.f.ilu 1,suchthat
!e
\037
-
\037
-
J/,d(MRJ:) JlrdQ, IIMRJ: Q,II-t.) Let {!,E 9t(F,D) be the nonrandomized decision function which is definedby \037
{!e= (f,)'ET then we obtain)
\037
J II MR P,
This provesthe assertion.
- Q,lIm(dt)-
1;.)
0)
In the light of Sections The foJ1owingtheoremis the randomization criterion.
22-25it solvesthe problemof \"approximate sufficiency\".)
55.9Theorem.(LcCam[1964]).Let e
t
\037
O. Then E;2F holdsiff there exists
R E f!4(E,F) such thaI II
-
MRJ: Q,II e for every lET.))) \037
Chapter 9:Comparisonof Experiments)
282
t
Proof(1) Assumethat E;2F. Then we obtainfrom the precedingassertion
-
that)
sup
inf
We
f..
J II MR J: Q,lIm(dt)
\"'ESTRE9I(E.F))
\037
apply Corollary45.7with J{\037o(T)= S1\" Let M 1 = M2 = {(IIMRP, Q,II>'ET: RE a(E,F)}.)
-
{e}and
The assumption Clearly,Mz issubconvex. implies 1pM2(m) 1pMI(m)for every mE ST'Theproofisfinishedif (M2) isclosed in IRT. But this isa consequence of Remark 45.4if we put) \037
(X
f(t, R) = IIMRJ:.-Q,II, t E T,
RE
\037
(E,F).
-
(2) Assume that there is R E (E,F) such that II MR;: Q,II t E T. Let D be an arbitrary decision space.Thenfor every (12E .<M
\037 \037
l;
for every
(F,D))
x L(E) R: (j:J-l)\037fe2(M2Jl)) p:rcb(D) is an element of 91(E,D) and it satisfies \037
P(W;, J:)
\037
W;e2Q,+ ell W;II)
for every boundedcontinuous lossfunction W: D x T
.... R
0)
to L(F),thus obtainingthe following The range of MR can be restricted of Theorem25.6.For finite T, the secondpart is dueto Blackwell extension
[1953].)
55.10CoroUary. (LeCam[1964]). (1) Let l;
stochastic operatorM: L(E) II
(2)
-
M.p, Q,/I
\037
\037
tift
\037
L(F)such that
O. Then E
t
2 F iff there existsa
E T.)
E 2 F iff there existsa stochastic operatorM: L(E)
M;:= Q\" t E T.)
ProofTheorem55.9and Lemma 55.4.
\037
L(F)such that
0)
and relatedto the concept Thefollowingversionisin the spiritof Heyer [1982]
of exhaustivity(seeSection 23).)
55.11CoroUary.Supposethat E isa dominated experimentand
F= .\"2,{Q,:t E T}) i.r .mch that Q2 is a locallycompact spacewith baseand.J11 countable 2 = \302\253(12'
14\302\253(12)')))
55.The randomi7.ation
(1) Let E
c
\037
O. Then E;2F iff there existsa
[0,1]such that II
(2)
K P,
-Q,
II
\037
t,
t
E
criterion
kernel K:QI x .flI2 stochastic
283)
\037
T.
kernel K:Q s12 --.[0,1]such that E;2F iff there existsa stochastic I
X
KP,= Q\" tE T.) and Remark 55.6(3). ProofCorollary55.10
[])
we provea generalization of the convolution theorem.) application 55.12Theorem.Supposethat G isa locallycompactgroupwith countablebase As an
.
E=(G,.\037(G),{t:g*P:geG})and F=(G,\037(G), are valid: {8g Q:g E G}).If G is amenablethe the fo/lo\037ing assertions > Let t O. Then E;2F iff there existsa probability (1) (Torgersen[1972])
and P\037lr' Let
-
measureKI\302\243B(G) such that
liP* K QII E F iff there existsa probability measure KI\037(G)such (2) (Boll[1955J)
that
P.K = Q.)
\037
\302\243.
\037
measureK such that ProofIt issufficientto prove (1).If there isa probability K.P Q E then it isclearthat II
-
.
II
\037
P* K
-
-
Q = \"P*K Q
g E G,) c which implies E =2 FsinceIlH Il*t K, IlE L(E),defines a randomization from E to F.Assumeconversely, that E F and let R bea transition from E to F such II
E,
F., *
II
II
\037
E,
\037
that)
IIMR(Sg*
P)
-
(Sg*
Q)II
\037
t,
gE G.
In view of Remark 55.6(3),R can be supposed to be a kernel. Let (KII)IIEN be a summing sequence of compactsubsetsof
randomizations RII from E to F, n E
--
-1 R II (f,Il)I=-:;A.
_
J R(jog (K\"Kn) )
G. Define
by)
\037,
1,C *Il)l,(dg), Q
fE Yb(G,\037(G\302\273, IlE[,(E).Let RoE f!J(E,F) be any weak accumulation point = of (RII)IIEN' Standard arguments yield Ro(fog-l, J.l) Ro(j,Il) if and Jl E L(E).Moreover, sincethe variational norm is lower fE Y,,(G, semicontinuous for the 9'b-topology it followsthat) of ba(G,
1;,.
\037(G\302\273
II
MRo P
-Q
II
\037
lim supII MR\" P
.n....
-Q\
<X>)
\037
supIIMR(tg P) ,eG)))
* (\302\243g
Q)II
\037
t.
\302\243M(G\302\273,
284
Chapter 9:Comparisonof Experiments)
M Ro by a stochastic Next, we replace kernell!E 9t(E,G).Let l!o:G x 11(G) a b e kernel such that [0,1]
-
Ro(f,J1)=f(!oJ1, fEt'joo(G),J1EL(E).) Define)
-eo(x,
l!(X,B):=eo(x,B) + (1
G\302\273
IB(x), x E G, BE 9I(G).)
It isthen clearthat l!isstrictlyequivariantand hencethere existsa probability measure KlaI(G)suchthat (lP= P * K (useTheorem48.9and cf. Remark 38.11 Moreover,we have) liP* K Q = Ill!P Q = Il!P Q (1) ll!oP QI(1)+ (1 I l!o(., G)dP).) (1\302\273.
-
-
to (loP- Q.This Notethat MRoPand that MRoP- (loPis orthogonal that implies 1 - J eo(',G)dP= (MRoP)(I)- (eoP)(1)= MRoP- eoPI(1)) II
II
I
\037
\037oP
\037
I
and hence)
-
liP.K - QII
-
lOoP QI(1)+ IMRoP eoP I(1) = eoP Q + MRo P eoPI!= MRu P Q
\037
II
This provesthe assertion.
II
II
II
-
II
\037
E.)
0)
55.13Remark. The assertionof Theorem38.15is a particularcaseof the
precedingresult.Let E be a full shift on an amenablegroupG then every a full shift experimentF = (G,31,(G), equivariantestimate (} E (Jt (E,G) defines = * Q:=g E G})with Q l!P. Obviously) E F and hence the assertion by {E\" of Theorem 55.12. part (2) t E T})and F = (!12' Let E = (QhSIth {.P,: SIt2' {Q,: t E T})beexperiments in that E F.Thenthere isa transition M:L(E) L(F)such cr(T)and assume = that M P, Q\" lET.It isclearthat the transition on M isuniquely determined span{p,:t E T}.The followingimprovesthis fact.We followLeCam [1974]. of (RT)* and let Let.YfoCT) be the setof all lowerenvelopes of finite subsets Jf'(T)be the vector latticewhich isgeneratedby {p,:t E T},wherep,:fiT R \037
-
\037
ontothe /'11coordinate, / E T.) denotesthe projection
-
55.14Lemma. Jf'(T)= Jf'oCT)-.1'f oCT).) Proof It isclearthat .JIfoCT)-.reoCT) .Jr(T).We have to show that JYo(T) JYo(T) is a vector lattice.SinceJYo(T) is a convex cone
-
\037
the)))
55.The randomizationcriterion
285)
-- .1tJo (T) is a vector space.From difference.1I'o(T)
-
-
(11' cp)+ = tp tpf1cp, (tp cp)- = tpucp cp,)
-
it
followsthat
it
-
iseven a vector lattice.
0)
In the followingwe denoteby Lat(E)the vector latticewhich is generatedby
{p':teT}.)
55.15Lemma. Let
E::: (Q,.91, {p,:
T}) be an experiment.There exists exactly one vector latlicehomomorphismJf(T) --t Lat (E):cp 1-+Pip satisfying tE
Pr 1-+\037, t E T.)
Proof Uniquenessis clearsince{p,:t E T} generates Jf(T). For provmg existence define(seeDefinition 52.5) Pip:=:'(cp c TJ' L
,ea)
l\037
whenever cpe.tf(T)dependsonly on the cp 0 Pa = cpo
coordinates of aEA(T), i.e.
it followsthat Sinceevery cp e f(T) is homogeneous
q>, tp E
K(T)and
E
rJ.
A
(T)issuchthat q>\" = q>, tp\" PI7.
= p.+ '1 = Pip f1 Pip\" If p\"v..' = PlpuP..\ are obvious. Thisprovesthat P\037Ip-trI'\"
=
P\302\253
Pip
tp,
is well defined.If then the relations)
P\"\"
\037
P..\"
q> 1-+Pip
is a latticehomomorphism.0)
Let E and F be equivalent experimentsin 55.16Theorem.(LeCam[1974]).
cr(T).If M:L(E) -.L (F) is a transitionsuch that M p, = Q\" t E T, then M is determinedon Lat (E) and it isan i.wmetricBanachlatticei.wmorphism uniquely \037--
----
from Lat (E) ontoLat (F).)
e .te0(1')then there are {ar .;:1 i m, t E ex} 1R such that = min L a,.i'p'. It followsthat M Prp Q_ for every cp E Jf oCT)sinceMis Pip 1 re\" a positiveoperator.Moreover,we observethat for every cp E .Yf oCT), Proof If
cp
\037
\037
\037
\037
\037i\037,\"
cp Q
p\302\253
=
cp,)
p.(1)= J cpdq\" = J cpd'tl1.= QIp(l).) Thisequation holdseven forevery cP E .K(T).Hencewe obtainfor cp E .KoCT))))
Chaptcr 9:Comparisonof Expcrimcnls)
286
II
-
-
Q., MP.,II= (Q., MP.,)(1) = Q.,(l) MP.,(l)= Q.,(l) P.,(l)= 0)
-
-
thereforealsoif qJ E Jff(T). Q. It isobviousthat Lat (E) = {P.,: E qJ K(T)}and Lat (F) = {Q.,: qJ E Jf'(T)}. HenceM is uniquely determined on Lat (E) and for reasonsof continuityon Lat (E).It isclearthat it is a Banach latticehomomorphism. The proof is finished if we show that M is isometric on Lat (E). Since = Q.,= MP.,it issufficientto showthat Q., P., for every cp e Jft (T).Since we have p..= (P.,)\"\"',etc. It lp...... p.and lp...... Q.,are latticehomomorphisms which implies = MP.,if qJ E \302\243'0(1')and
II
II
II
...
II
followsthat)
+ P., = (P.,) (1)+ (p.,r(1)= P.\" (1)+ P\" (1) = Q.,.(1)+ Q.-(1)= (Q.,)+ (1)+ (Q.,l(1)= Q.,II for every qJ E Jff(T). 0) II
II
II
If E is an arbitrary experimentthen L(E);2Lat (E). It will be provedin the followingsectionsth at for every EE 8(1')there is an experimentF --E such that L(F) = Lat (F). A property of an experimentE isheredilaryif alsoany experimentF E has \037
thisproperty.)
55.17CoroUary.Dominationis an heredilaryproperly,i.e.if E is dominated and F
\037
too.) E, then F isdominated,
-- ---
Then there exists Proof Let E = (Ql'.911, fJ = {p,:t E T}) be dominated. v E L(E) such that p, that we may V, lET. From Lemma 20.3it follows, that for every t E T) assumev E spanfJ Lat E. Domination implies d dP' f1k dv f1(kv) = lim J dP' lim P, V dV dp, k dv = O. = lim \037
\037
II
-
-
II
\037
\"-+
\"-+
J
dv
sIL... -)
A;-+w
4y
-
\037\"
-
such that Let F= (Q2,.91 tE 1'})and M:L(E) L(F)be a transition 2 , {Qt: M P, = Qt. te T. From Theorem 55.16we obtain M(P' p'f1(kv\302\273 = M P, M P, nk Mv = Q, Qt nk Mv, lET,and therefore)
-
lim II Qt
j,;...oo
This implies Qt
-
\037
-
Qt f1kMvll = lim II p, \"--00)
Mv,
t
E
T.
0)))
- P'f1kvll= 0,
-
tE
T.
56. Conical measurcs
287)
56.Conicalmeasures) The conceptof conicalmeasures is the technicalkey for a satisfactory
we consider representation theory of experimenttypes.In the presentsection from a general pointof view. They are discussed from the conicalmeasures view in Section 5 7. o f statistical point the set of all lower Let T * 0 bean arbitrary set.Recallthat jf'0 (T)denotes T envelopesof subsetsof CIR )* and let .K(T) be the vectorlatticewhich is {p,:t E T}.FromLemma 55.14we obtainthat generatedby the projections .K(T)= .Yt oCT) .Yt oCT).)
-
56.1Definition.A positivelinearfonnM on .tf(T)iscalleda conical measure. iscalledthe resultant of the conicalmeasure. We will provethe basicfact that every conical measurecan berepresented by somemeasure on the O'-field Q9 \037(IR) which is generatedby {p,:t E T}.
The point(MP,),eT
\037
IRT
IfiT)
56.2Lemma. Let M be a conicalmeasure, = O} is a vectorlatticeand C = {q;E .tf(T)j%:q>
=
Then.;\302\245'
-
\037
{q> E
.Jff(T):M(Iq>1)
O} ;sa propercone,i.e.a
convexconewith vertex0 satisfying C(I( C) = {O}.)
Proof It isclear (llI't +
is a vector space.The relations)
that.;\302\245'
-
I
I
q>21)
\037
lI't
n q>2
\037
lI't
U q>z
\037
I
q>d
+
I
q>2
1)
iseven a vector lattice.Thefact that C isa convexconeof vertex B followsbecause the mappinglp ijJ is 1inear.We have to show that = Let Cn( C) {O}. ipE C(I( C).Then there are q>1 E ip, q>1 0,and q>2 E cP, O. Sinceq>1 (fJ2 E.AI' we have M( q>2 CP21)= O. But 2 E E which the assertion. 0) + .AI', AI\", CPt 2 implies CP2 proves imply
that.;\302\245'
-
-
\037
I
I
-
I CP
1
\037
I
CPI
-
\037
I CPI
-
CP
1
\037
CPa
56.3Lemma. Let M bea conical measure. Then
\037
q;z
iff
q;z
-
q;1E
C
definesan orderrelationsuch that vectorlatticehomomorphism,)
\302\243'
(T)j.Al'isa vectorlauiceand cP t-.ip is a
is an orderrelation.For Proof It isclearthat the relation..\037\" on .1f(T)j.K the second (CPI U C(2)'\" isthe-leastupperboundof q;1and cPz. part we showthat It isclearthat (CPI U CP2) ipl and (CPI U q>2) ip 2' Let converselyip be any E (: upperboundof {iPl'iPz}'Then ip iPl and ip (fi2 and thereforeiP E ,Ai- and '1'2 E .;Vsuch that and cP CP2E C.In otherwordsthere are '1'1 + '1'1' cP CP2 + '1'2'))) \037
-
(fJ
\037
\037
\037
CPI
\037
\037
-
(fil
Chapter9:Comparisonof Experiments)
288
It followsthat
(cp
-
>
CPl)(')(cp CP2) 1/)1(')1/)2 and A ip (CPt U CP2) \037
-.
thereforecp
-
(CPl U CP2)
> 'PIn 'P2E.tV. Hence similarargument shows that lower boundof CPl and ({>2' 0) (CPt n CP2) is the greatest
56.4Theorem(Choquet).For every conicalmeasure M there is a measure JlI <8'>g(R)such that Mcp = J cpdJl if cp E .Yl(T). 'f!T) Proof The function cp t-+ M (IcpI) is a seminonnon :K(T) which satisfies M(CPl+ cpz) = M CPt + M CP2 if CPl 0, cpz O. Let H(T):=.Yr(T)/AI'be the nonnedspacewith norm ciJlIM 1=M(Icpl).Thecompletion neT)of H(T)with \037
\037
1\\
Hence,by Theorem6.9,there existsa respectto II.IIMis an Abstract L-space. measure space (D,d, and a vector lattice isomorphism Let us t/J: B(T)-+ L 1 (a,.91, define A)o S:J'f'(T)-+ Lt (0,.c(,A.): cp H CP(ip).) For every cPE .1l'. (T)we have Mcp = lIipliM= J CP(ip)dA.= J Scpd)'. We notethe relation Scp = cp 0 (SP,)reT for every cp E Yf) (1').Indeed,the setof isa vector latticewhich contains{P,:t E T}. functions which satisfy the relation = If we defineJ.l Z\302\253Sp')'fT!)') then we obtain \037)
Mcp = J Scpd\037 = J cp 0 (Sp,)u TdA = J cpdfl
for every cp E )f(T).
0)
56.5CoroUary.If III@ tI(R)is constructed as in
densein L 1 (Jl).
'E T)
Theorem56.4then Jft'(T)is
to ProofLet t > O. SinceH(T)isdensein !leT)and neT)is latticeisomorphic L 1().), for every gEL1 ().) there is cpeJf(T)such that IIScp-glll
-gilt<
Scp
II
cP
Eo
Then
-fill = J -fldJl= JI -gldA < E. = !IScp I
cP
q>
-fo(S,q),eTld).
0 (S,q),eT
0)
57.Representationof experiments) Let usconsider Ee 8(T)where T* 0 is an arbitrary parameter experiments of an experimentsatisfies a consistency))) set.The system of standardmeasures
57.Representation of experimenls
289)
tc relation.We notethat a function qJ E: .')F(T)dependson the coordinates 0 = .-. A if R Q denotes the natural only, (T), fP Pa fP. Thereby PQ: ex
ex
\037T
\302\243
projection.)
57.1Lemma. is the systemof standardmeasuresof an experiment E E S(T) then it is consistent, i.e.it satisfiesany of the followingequivalent If\302\253(JQ)QEA(T)
conditions:)
(1) H(<Ja.)(z)=H(<Jp)(z) if ZESar.Sp,ex,fJEA(T). (2) f fPdu = J fPdup iff ex, fJ E A (T),fP E Jf(T) and fP CPo:= fP 0 Pp = fP. Q
(3)
(Ja
Ps) ((-:,
= !R
( L:
Up
p\037)
HIJ)
SE<1
\037
$\037Q)
if
)
ex
\037
p, ex, PEA(T).
satisfies condition Proof First,we show that (2).We may A(T) necessarily assume If in that we mind that is then it p. keep fP positivelyhomogeneous \302\253(J\Q(:")
ex
\302\243;
foJlowsthat
J cpdu(l s.
= J cp ()
((d
i
SEa)
p.S)lEa. )
d r. Ps $(12
d L. dP. r - d L: P.S d d L. (( SFII )ru) Sf'L: Ps 3EfJ \037
= J qJ ()
.
u\037
\037
P)
d r. Ps dP. \037 = J qJ d L P.S () ((d L Ps)ru d L: Ps) U'fI _\03741\037__
S(P)
Sf'1I
=
f fP
h
dp'
L Ps ((d r. Ps) ) dse' rEQ
ufJ)
=
r
fP
tl
k, p.s)r.p) ((dHP) \037
dL
\037
U'fJ
-
=
r ill
Next, letus provethe equivalenceof (1) (3). (1)=- (3):Let ex p, ex, pEA(T),Z E SQ' Then))) \302\243
fPd(Jp.
290
Chapter 9:Comparisonof E'l:pcriments) H\302\253(1(J)
(z) = H(up) (z) = J
f1X:'(1p(dx)
S,IE(J)
=
=
::1
n ( L ) (Ua: L (lp(dx) ) .sc:r) X,
J s, r
t.
x.\037
XS
r\037a:
f1 /Ell
. /;'((L..Ps) ( L p,,) (lp) (dx).
\037
x\037c
SEa:
tEa:
s\037a)
to the proofof necessityof (2). (3) => (2):Thisissimilar => (2) (1):UseLemma 52.9. 0) 57.2Corollary.Let(\037)H A(T) be the systemof standardmeasuresof an experiment Ee 8(T).Then MfP:=JfP d(l(J if
fP
:if(T)
E
and
fP
=
fP
0
p\",)
isa conical measureofresultant MA = 1,t e T.)
57.3Definition.Let Ee G(T).Then the conicalmcasur\037
which
measureof E and isdenoted Corollary57.2is the conical by
is defined in
ME')
57.4Example.If E = (fl,.9I, t e T}) is a dominatedexperiment then its conicalmeasurecan be given in a simplerform.Let Po 1.91be a probability measuredominating\037.Let a. E A (T).Forevery cp E .K(T),cp 0P7. = fP, we have) {\037:
Mt;(cp)= S fP
--dIL_ d L Ps d (( L P\" )I!HI) sea: SEll)
-J
dL UCI
dP. I
fP
((d L ) ) P\"
IF
CI
P\"
dPo
dP.0
SEll)
= J fP
dP. d L 1',, 1
((d L
Po\"
S\037(J)
= J cp
\037\037
0
)
tE(J
dPo
!! (( ) ) dPo. \037
dPo
tEIX)
in terms of their of dominatedexperiments We obtaina characterization conicalmeasures.)))
57.Representation of experimcnts
57.5Theorem.An measureJ.l1 @
291)
experimentE E SeT)isdominated iff there eXL<;ts a bounded such that
\302\243f(R..)
I6T)
::::S rpdJ1, q> E Jt(T).)
M\302\243(q\302\273
take a probability measurePol.sfsuchthat 9 Po Proof(1) IfE isdominated Then Example(57.4)provesthe assertion.) anddefineJt = $I' \037
((
\037\037
(2) Let J.l1 @
t.
\302\243f(\037)
).r Po).
be a boundedmeasuresuch that
T)
= J q>dJ.l,
M\302\243(q\302\273
q> E
.Yf(T).)
tE DefineF:::: (IR\037, (8) \037(IR+), {Qt:
uT
T})by Qt = PtP,t E T.ThenQ\"
t
E
T, arc
since probabilitymeasures MEP, :::: 1, lET. J p,dp :::: we have ME.= MF'sincefor It isclearthat J.l dominates {Q,:t e T}.Moreover, q>
<>
Pa. = q>,
IX
e A (T),
((:rPs)1Ia.) LPsdJl=MFq>.
MEq>::::Jq>dJl=Jcp
st!c:z)
\037
Sf:
a.)
This impliesE\"\" F (Corollary52.8) and
dominated.
by
Corollary55.17,E
IS
0)
57.6Corollary.If E E SeT)isdominated then thereex;sts measure a probability = t e T});sequivalentto E. @ \037(R+)such that F (8) \037(IR +), {PrQo: QollET reT Is every conicalmeasure the conicalmeasure of an experiment? The settlesthis question.) followingassertion (IR\037,
57.7Theorem(LeCam[1974]). Forevery conical measure M un .Jf'(T)with resultant Mp, :::: 1,t E T, there isan experimentE E .t(T)such that M = ME')
Proof(Siebert [1979]).Accordingto Theort:m56.4thert: is a measure J.l on (8) \037(R) such that Mrp = J rpdJ1if q> E Jt(T).Let E = (a,.91, t E T})with {\037:
T
a:::: .!II:::: (8) 1fT rE
IRT,
measure for every
followsthat)))
\302\243f(IR)
tE
and
p,:::: Pt/l,
T. Sinceevery
T. Obviously, is a probability it q> E .Yf(T)is positivelyhomogeneous t
E
\037
Chapter 9:Comparisonof Experiments)
292
Mtp = J ({Jdp= J lp
:.'
((L.Ps)'Ell) L Psdp lEiJ
SCII)
if lp =
0 PII' This
({J
provesM = ME'
0)
The experimentE justconstructed neednotbedominatedsincep neednotbe O'-finite.)
57.8CoroUary.Let(O'JuAITIbea systemofprobability measureson \037 &B(R) 'ET such that O'II(SJ= 1 if E A(T). Then (O'II)2EA(T) is the system of standard in the sense measuresof an experimentE e K(T)iff it isconsistent of Lemma IX
57.1.)
arenecessary. To prove ProofIt followsfrom Lemma57.1that the conditions 0 H> sufficiency notethat M:({J J lpdO'II,lp E Jf'(T),lp ({J PQ, definesa conical measureM of resultant Mp,= 1.t E T.Thereis an experimentE E 8(T)such the systemofstandardmeasures that M = Mf;- If (1 A(T) denotes ofE then it followsby 52.8that =-=
11)11\"
J cpd(J{J = M lp = Mtlp = J lpdt = if tp e .JF(T).({J0 PII = cpo Thisimplies 0'12 Q
1 for every e A (T). IX
12
0)
57.9Remark. Thepreceding theorem isof an elementarynature if T isa finite bea probability measuresuch that J p,dO'= 1 if t E T.Then set.Let (J on t E T})isan experimentwhosestandardmeasureon ST Eu = (ST'\037(ST)'{p,O': with (J. The experimentEu iscalledthe standardexperimentof 0'.) coincides \302\243feST)
of experiments measures.) may be phrasedin tennsof conical Comparison
57.101beorem.Let8 0 andlet E and F beexperimentsin SeT)Then E F \037
\037
iff)
ME qJ
\037
MF lp + 6
if
Proof.Apply Corollary52.8.
({JE
JI{'0 (T),
II
qJ
II
= 1.)
0)
57.11CoroUary.Let E.Fe G(T).Then E\",F iff ME = MF .
of the importantTheorem57.7 is the Another consequence limitsof experiments.) projective
57.12Definition.A IX
E
existence of
system of experiments(E((I)u A(T) where E(iJ) E 8(1X), An experiment))) A (T), is a projective P implies system if (E(/1)iJ\"\"\" EiJ' IX
\037
58.Transfonnation groupsand invariance \302\243E
limit of the projective &(T)isa projective system
(E(CZ\302\273CU,A(1')
every
(X
EA
293)
if Ecz '\" E,cz) for
(T).)
57.13Corollary(LeCam[1972]).For every projective system of experiments there existsa projective limit Ee G(T))
(E(II\302\273CZCA(T)
E A (T),then the system ProofIf isthe standardmeasureof satisfies conditions (1)and (2) of Corollary57.8.Hencethere isan experiment E E 8(T)such that isalsothe standardmeasure of Ecz for every E A (T). (X
O\"(Z
\302\253(7CZ)IIEA(T)
\302\243(7.)'
(X
O\"(Z
Thisimplies Ell '\" E(II) for every e A (T).
u)
(X
Let us return to the problemconsidered below Theorem55.16.) in Theorem57.7satisfies 57.14Corollary.The experimentE constructed L(E)
= Lat (E) = L 1(p).)
Proof Clearly,Lat E L(E).Let us identify LI(Jl) and {fJl:fE L 1(Jt)}.In (J E L(E).(J O. Similarlyas for orderto show that L(E) L 1 (p) consider = (J Lemma41.6we may decompose 0, (J2 0,where e L1 (p) (71 + (72' L L B ut t E and (J2.l 1(Jl). (J2.l 1 (Jl) implies T,and the definingproperty (J2.l (J = (1) E L) (J.l). if (]2 = O. Hence, of L(E)yields(]2.L(] which is only possible Thus it remains to show that Lat (E) is densein {fwIE /.1(Jl)}.With the to notations of Lemma55.15 for every q> E .K(T)we have P\" = q>Jt. According is 0) 5 6.5 the set dense This t he assertion. i n Corollary .tf(T) L) (J.l). proves \302\243
\037
\037
O\"t
\037
\037
0\"1
p\"
57.15Corollary.Forevery experimentEe 8(1')there is an experimentF\",E such that
L(F)= Lat (F).)
of Theorem57.7 Proof Applying to the conicalmeasure ME the construction there is an experimentF such that MI:; = M,..which satisfies L(F)= Lat (F) accordingto Corollary57.14. 0)
58.Transformationgroupsand invariance) Let E =
(0,.91,{p,:t E T})be an experimentand letG be a group.)
58.1Remark. Assumethat there isa group(Mg)gEGof stochastic on operators such that E is invariant under It follows from L(E) Corollary55.10 (Mg)geGo that the experiments)))
Chapter9:Comparisonof Experiments)
294
IE T}), gE G, MgE,=(!l,.9I, {MgP,: are equivalentwith E. If an P, for s * t, then (Mg)geGdefines = Mgp\" ge G,se T. \037
=t=
of G on T by operation
\037s
With
thisnotationthe experiments) geG,) (fl,d,{\037,:teT)),
are equivalentto E. Now we start with a groupG operatingon T.)
58.2Definition.Assumethat G operates on T.ThenE iscalledG-invarianlif E isequivalentto eachof the experiments(fl,d, {Pg,:lET}), g E G.
glanceit seemsthat the notionof G-invarianceis moregeneral than the notionof invariance under the operationof a (Mg)gcGof stochastic At first
However,thisisnotthe case.) operators.
on T and Ihat E is G-invarianl.Then 58.3Theorem.Assume IhalG operales
a groupofstochaslic IhereexislS (Mg)gt: G on L (E)such thaI Pgt = M 9 P, operators
for allg e G, leT.)
Prouf.Let EoE 8(1')be such that Eo'\"E and L(Eo)= Lat(Eo).Such an 57.15. Firstwe showthat Eo isalsoG-invariant. experimentexistsby Corollary = DenoteEo (flo,.910 , {Q,:tE T}).Thereare stochastic operatorsM.:L(E) -+ L(Eo) and M2: L(Eo) -+ L(E)such that M.P'= Q,and M2 Q,= {E T. This impliesthat M.Pgt = Qg, and M2 Qg, = Pg, for all t e T and g E G. It followsthat p\"
(fl.d, {\037,: t E T})'\"(flo.do.{Q,,:t E T})) for every g E G which implies G-invarianccof Eo.Now, since)
.
Eo'\"(flo..910 {Qg,:lET})) there isa stochastic operatorM,:L(Eo) -+ L(Eo) suchthat M,Q,= Q,/t lET. It isclearthat M'IO(J2= Mg,oMg2 on the set {Q,:lET}.But from Theorem isevenvalid on the wholeof L(Eo) and it followsthat the groupproperty 55.16 Now define therefore(Mg)gtGis a groupof stochastic operators.
M,:=M2oM,oM.,geG.) It isclearthat (Mg)gf'Gisa family of stochastic satisfying MgP,= P\". operators g e G,t E T. It remainsto show that (Mg)g,c; isa groupof stochastic operators.)))
58.Transformation groupsand invariancc
For this we notethat M,oM2 = idIL(ED)'This followsfrom (M = M.(.P,)= Q\" t E T, and from Theorem55.16. Hencewe obtain)
1
Mg, ()92
-
- M g'1 9: M - g =
0 0 1 l\\t!. 2 0 91 (J Mg:0 2a M2 0 M 0 M1 0 M2 0 M () M1 = M
This provesthe assertion.
(Q,)
Mt--
9. M\"1
9:
fll
0 M 2)
295)
<,
.)
0)
of G 58.4Remark. Let \037(G) bea a-fieldon G and assume that the operation for in the sense that g Pg,(A),g E G, is,tj(G)-measurable on T ismeasurable Then the precedingproofshows that (MfI)(JEG is a measurable every A E Hencethe situation of G-invariantexperimentis groupof stochastic operators. in Section 48and the Hunt-Stein reducedthe caseconsidered theory completely can be applied. G-invariance it isconvenient to have a criterion Forchecking in teonsof the likelihood processes.) \037
\037c1.
011T. Then E is G-invariantiff) 58.5Coronary.Assume that G operates
!i' , dd (( l's) P,
IF T
!i'(( ) -,
p.f _
dPg, dPgJ ),
E
p. T
flf)
)
for allg E G andSET.) Proof Apply Theorem53.10.
0)
thissectionshowing that every equivalence classof G-invariant an experimentwherethe groupof stochastic is contains experiments operators inducedby a groupactionon the sample space.However,we cannotshow that in the operation is measurable the senseof Example48.3.) We finish
58.6Theorem.Supposethat
E is G-invariant.Theil there existsan equivalent on andQg, (8)&j(!R+),{Q,:(E T})where G operates
,.
experimentF = (R\037, T = IE T, gE G.)
Q,'g-l,
R\037
be the measure which isconstructed in the proofof Proof Let III'E(8)\037(R...) T Theorem57.7.LelM be the conical measureofE and Mg the conical measure of (Q,.51, {P,,:t E T}),g E G.SinceE is G-invariantit followsthat M = Mg,
gEG.
of G on RI by Q,:=P,Il,t E 1'.We definethe operation -+ It is obviousthat g: is (8)\037(IR+)gE G, XE gx:=(x(J-I')'ET' 'ET))) Let F be definedby
\037\037.
IR\037
R\037
296
Chapter9:Comparisonof Experiments)
measurable for every g E G.For every
E
CfJ
1 .11 (1'))
gdp = Mg(cp)= M(cp) = J cpdp,cp E .tf(T),g E G,) and sinceJ'I'(T)is densein L 1 (11)it followsthat p is invariant under the of G. We obtain) operation f cp
0
x)p(dx)= Qg,(B)= J IB (x)pg,(x)p(dx)= J IB(x)p,(g-l = J 1B(gX)P,(x) 11 (dx) = Q,(g-l B))
ifge G, IE 1',Be I(T)
@\302\243B(lR
t
). Thisprovesthe assertion. 0
59.Topological spacesof experiments) Let T + 0 be an arbitrary set. Then S(T) denotesthe collection of all f or the T. The elements o f the experiments parameterspace quotient are called I t be should recalled t hat set8(T)/experimenttypes.
-
59.1Definition.Let E, FE SeT)and let beE,F) = inf {c> 0:E S F}. Then A (E.F) = max {b(E.F),1>(F,E)}iscalledthe deficiencybetweenE and F.)
on G(T).) 59.2Lemma. The deficiency is a pseudodistance L1
have to prove that B inequality. ut this inequality
Proof We
E,F,GE8(T)) CI
C1
0: 8(1')x 8(1')-+ R satisfiesthe triangle is the consequence of the fact that for
E '2 F, F'2 G impliesE
h + Cl \037
0
G.
Ee 8(1')/-be the equivalenceclass with EEE. We denote is a metric space EES(T),FES(T).Then (S(T)/-,L1) L1(E,F):=L1(E,F). = since.d (E,F) 0 iff E F, the deficiencybetweenexperiments.) Thereare severalways to express 59.3Lemma. If E,Fe SeT)then .:1 1, E Jf o(T)}.) (E,F) = sup(lMECfJ MFCfJI : Let
-
-
IICfJII
\037
CfJ
ProofThisfollowsimmediatelyfrom Theorem57.10.
0)))
59. TopologiC
297)
59.4Corollary.If E,Fe cf(1')then
.:1 (E,F) = sup{.1(Ell'f;): e A (T)}.) (X
lemma. ProofNotethat Jfo(T)= U Jro(x)and apply the preceding A(T))
0
te T}) mId F= (D2 ,.9I2 , {Q,:te T}). 59.5Lemma. Let E= (Ql,.9I h {p,: Then)
-
beE,F) = inf {sup MP, Q,II:J'4:L(E) -t L(F)isa stochastic operator}. rflT) II
Proof Apply Corollary55.10.
0)
59.6Corollary.Let E = (O,.sI, lET})and F= (O,.sI.{Qr:IE T})beex{\037:
perimentswith A
the same samplespace(D,.r:#).Then
(E,F) supd1 (P\" Q,). ,eT) \037
from E to F it followsfrom Proof Sincethe identity definesa randomization Lemma 59.5and Lemma 55.4that
beE,F) sup ,eT) \037
II
P,
- Q,II
which implies for reasonsof symmetry
-
.1(E,F) sup liP, Qrll. 0 ,eT) \037
To obtain the topological of experiment spaceswe beginwith properties
the simplest case. considering If T isa finite setthen each E E 8 (1')ischaracterized by itsstandardmeasure 0'E E !fT'FromRemark 57.9(1),we obtainthat the mappingE.,....., 0'E is even SinceE'\"F impliesO'E = O'F the mappingt.,....., from (JF. isa bijection surjective. ontog'T'From Lemma5.20we knowthat ''/Tiscompactfor the weak 8(1');\"\"'
topology.)
59.7Tbeorem.If T is a finiteset then (r1(T)/-, .1)and//T are topologically equivalent.)
ProofThe mappinga H-i:;gisa bijectionfrom the compactspace/I'Tontothe Hausdorffspace(G(T)/ \"',.1).The assertion is provedif we show that the iscontinuous. mapping Let O'n
-
0' weakly where (O'n)IIE'I,I
J q>du\"
-t J q>du
if
q>
E
\302\243;
.efT'Then)
:It'0(1').)))
Chapler9:Comparisonof Experiments)
298
Sinceeachfunctionq> E .Yf oCT) satisfies a Lipschitz with Lipschitz condition constant1Iq>1I it followsthat {qJEJf'o(T):IIcpli I} is an equicontinuous and uniformlyboundedsetand therefore) \037
lim sup(IJ q>d(1\" ,,\037\037)
Thisimplies J
(E\037\",
Ell)
- J q>dql:
-0
cp
E
Jf'o(T), cp II
II
\037
I} = O.
in view of Corollary52.8or Lemma 59.3.
0)
59.8Corollary.If T isafinitesetthen (8(T)! \"',.1)isa compactmetricspace.) Let us return to the casewhen T is an arbitrary set.)
-,
Themetric space(4(T)1 .1)is complete.) 59.9Theorem(LeCam[1972]),
J. for the pseudodistancc ProofLet (E\,,tN")S; 4(T) be a Cauchy sequence E A (T) the sequence is a 8(rx) Cauchy sequence and Corollary59,8impliesthat there is a limit It is easy to seethat limitE E &(T). systemand hencethere isa projective A(T) isa projective We have to show that E is a limit of (E,,)uN' For every t > 0 there is N(t) E such that .1(E\"\"S' Em,lI) E A (T),jf n Forfixedn E and a; E A (T)the functionFt-+A (E\",II' is N(r.),m N(I'.). on 8(ex)which implies .1(E\".II' continuous EJ t if n N(t), E A (T).Now followsfrom Corollary59.4. 0) the assertion Then for every
ex
\037
(E\",\302\253)\"'N
\302\243(11)'
(\302\243(11\302\273111\"
1\\1
\037
\037
rx
\302\243,
\037
\037
\037)
\037
\037
ex
Forarbitrary parameterspaces the .1-topology It oftf(T)neednotbecompact.
will sometimes becalledthe strongtopology of 8(T)iscontrastto the topology of the followingdefinition.)
59.10Definition.Thetopologyon 8(T)j-which isgeneratedby the family of
(E,F)t-+ J (Ell'\037), E A (T),iscalledthe weak topologyof pseudodistances . 8(T)/\"'\" If T is a finite set then obviouslythe strongand weak topologyof SeT) rx
coincide.)
59.11Theorem (LcCam[1972]).The spaceof experimenttypes
compactfor the weak topology.)
&(T)fis
Prnof Let
1l1l(\037)
.\037
be an ultrafilter
-+ 8(rx)/- be the natural
an ultrafilterfor every
ex.
E
-. For projection.
in
S(T)I
A(T).If M
every
As a first
\037
rx
EA
(1) let 1l : 1l
stepwe prove that cf(rx) is an arbitrary setthen
E 1lCl(\037) or M' (M')E $i. Hence M = 11 (1l;1 7t: (M)E.'F or 1l;1 = 1l (1l;l(M'\302\273E 1l(l(ff). This means that 1l (ff) is an ultrafilter. Since 8(ex)f is quasicompact the ultrafilter 1l (..'F)converges.Let))) 1
11
1l
-
(M\302\273
cJ
1l
59.Topologicalspacesof cxpc:rimcnls \302\243(\037)
299)
e 8(a.)/-bea limit of 1t\037(.F). As secondstepwe prove that (E(\037)aeArl') isa
systcm. projectivc
isfiner than the filterof neighborhoods of p.Thefilter1t6(!F) The projection S(p)/- S(a.)/ is 2 t:/1in S(P)/ i.c.1t/1(':;;) Let ex
-, and thereforewe continuous
UU
\302\243;
\037
(\037\037(1t/l\037(E(/l))
1ti\302\2531
-
6(\302\2436)
UU
\037
1tJlIJ:
/1(\302\2436)'
have)
\037/l(E(/l)
\302\243;
1t/l(\037)'
Thisimplies \0371%(1t6\037(E(ft))
c:
=
1t/l\037(1t.8(ji'\302\273
whencewe obtain1t{1\037(E({1)
'\"
1t\037(Y)
E(\037).
Accordingto Corollary57.13there is an experiment Ee SeT) such that '\" 7t\037(E), a.eA(T).Since1ta(\037);2 a.eA(T).we obtain UUI%(7t\037(\302\243\302\273,
\302\243(IJ)
.'F
\037
U
'1
7t a
a (1tA
E\302\273
\037\037A(T))
which provesthat
E is a limit of
\037
for the weak topology.
0)))
Chapter10:AsymptoticDecisionTheory)
statistics relieseitheron weak converPractically, every result of asymptotic on a of or Besidesthe gence experiments strongertype of convergence. randomization criterionit was the secondgreat successof the theory of when LeCam [1972and 1979]was ableto show that the main experiments, the resultsof localasymptotictheory (LeCam[1953]and Hajck [1972and of weak convergence. 1973])can be provedexclusivelyunder the assumption Thisis the reasonwhy we discuss in the presentchapterweak convergence of and itsconsequences from a general pointof view. The resultsof experiments thischapterare appliedin Chapter13to obtainthe classical assertions of statistics. asymptotic The basicideaof asymptotic decision and easily explained. theory is simple Considera weak convergentsequenceof experiments. The objectof the are of decision functions. of investigation sequences asymptotic Every sequence decision functionspossesses accumulation functions pointswhich are decision for the limit experiment. Now, it can be shown that the asymptotic properties of a sequence of decision functionsare completely described by the properties I n of its accumulation a of decision functionsis points. particular, sequence asymptoticallyoptimalin somesenseiff the accumulation pointsare optimalin thissensefor the limit experiment. In Section60 we discussweak convergence of experiments. This type of of the likelihood can be expressed in terms of convergence convergence insteadof nets isjustifiedundermildcontinuprocesses. Considering sequences In case of likelihood ity conditions. contiguityit is even sufficientto consider base. with a fixed T his is in Section 6 1. processes proved in distribution of decision functionsisthe subjectof Section 62. Convergence We show that every sequence of decisionfunction possesses accumulation in that bounds risksof can be distribution and for the the points sequence in tennsof the risksof the accumulation expressed points.This leadsto the on asymptotic importantasymptoticminimax boundand rendersassertions these main o f LeCam are the results Essentially, [1972].Howadmissibility. of LeCam [1979]. Moreover,the ever, we followthe improvedpresentation versionspresented hereare sufficientlygenerallo coveralsosomeclassical local to Bahadur resultsdue and Pfanzagl [1970]. asymptotic [1964] the obtainedin Section62 Concerning asymptotic admissibility, assertions are expressed in terms of convergence in distribution. In this respectthey are which are contained much weaker than the corresponding classical assertions, There it is provedthat in the one-))) LeCam [1953]and Hajek[1972].
in
Chaplcr10:Asymplotic DecisionTheory
301)
of decisionfunction dimensional asymptotically normal caseall sequences minimax boundare stochastically which attain the asymptotic equivalent.In thisphenomenon is put intocompletely LcCam[1979] general terms.LeCam of the fact that the shows that stochastic equivalenceis a consequence for limit admissible decision f unction the experiment is noncorresponding We presentthese randomizedand uniquely determinedhy its distrihution. resultsof LeCam in Section63. Somepracticallyimportantdecision functionsare definedfor each sample a minimum path of a point by propertydependingon the corresponding Thisisthe casewith maximum stochastic e.g.of the like1ihood process. process, From the aslikelihood estimators and with generalized Bayes estimators. o f view the a minimum ariseswhethersuch ymptotic point question property remainsvalid after passingto the limit.Let us put the problemmoreprecisely. a sequence of experiments Consider weakly to a limit experiment. converging For each experiment of the sequence considera decisionfunction having a certainminimumpropertyasdescribed before.Then the questionis whether of decisionfunctionssharethis the accumulation pointsof the sequence minimum property.E.g.the question is whether the accumulation pointsof estimates are maximumlikelihood estimates for the limit maximumlikelihood orwhetherthe accumulation experiment, pointsof generalized Bayesestimates are generalized for the limit experiment. Bayesestimates in a rather general framework.It is In Section 64this problemisconsidered shown that an invariance principlefor the underlyingsequence of stochastic is the technical answer. For the caseof maximum processes key for a positive likelihood estimates such an approachis due to LeCam [1970]. It has been elaborated and extendedto generali7.edBayes estimates and by Ibragimov Has'minskii In bothcasesthe authorsrequire an invariance [1972and 1973]. likelihood themselves. for the sequence However,there may principle processes be somedoubt whether such an invariance principlefor the likelihood isof any statistical of processes meaningsinceit isnota necessaryconsequence of experiments. weak convergence In the caseof generalized the definingminimum property Bayes estimates doesnot dependdirectly on the values of the likelihood but on processes stochasticprocesses which are integralsover the paths of the likelihood In Sections 6567we show that weak convergence of experiments processes. aloneis sufficientfor an invariance principleholdingfor thesestochastic in Section 67 we arrive at a positiveanswer to our propblem Hence, processes. of for the caseof generali7.edBayesestimates requiringonly weak convergence the underlyingexperiments. The resultsare taken from Strasser [1982].)))
302
Chapter
10:Asymplotic
DecisionTheory)
60.Weakly convergentsequencesof experiments) a sequence of subsets Let T * 0 bean arbitrary setand (1;.) such that 1;.T T. In the followingwe consider of experiments sequences (E\,,eN")where E\" E \037(7\037)\037
ne N.)
-
60.1Definition.A sequence ofexperiments En e 4(1\037), n e N, c:onverge\037'
weakly
to E E SeT)if J (E\".II' 0 for every e A (T). It isclearthat in case1;.= T, n E N, this definitionis compatible with the The reasonwhy we notionof weak topologyin the senseof Definition 59.10. admit also T,.:I=T will turn out when we are dealingwith problemsof localization 82and 83).Specifyingthe followingto the (seeSection asymptotic of If (T)/ . case1',.= T, n E N yieldstopological properties forsomeC(e A (T) andfor such Forconvenience which make sense assertions c n E N that which is a matter of T,. are statedwithout the lattercondition E\302\253)
(X
\"'-'
\037
(X
course.)
60.2Theorem.A sequence (E,,)u convergesweakly iff (E\",II)nE N convergesfor '\" lim E\".IIfor every E A (T). every e A (T).In this case(Iim \".oco n o:t:;) P'V
(X
(X
En)\302\253
of of Proof It is clearthat weak convergence N implies convergence for every e A (T) and that in this casethe assertedrelationholds. assume that (En,(I)rre N converges for every E A (T).Thenit isclear Conversely, that the limitslim E\",(I'a E A (T)\037 form a projective system.The projective) \"... limit is a weak limit of (E\\"EN'") 0) (\302\243,,)nli
(X
(E\".\302\253)\"EN
(X
co)
of the precedingassertion is that every weak Cauchysimpleconsequence This of 8(ex)for followsfrom the completeness sequence weakly. converges every e A (T). Another formulation of Theorem60.2is that converges weakly iff E A This (T). leadsto pointfor every (En. N has at mostoneaccumulation Let (U\",II)
(X
(E\\"EPIO")
ex
II)\" Ii
60.3Theorem.A sequence of experimentsE\" E tf(1\037), n e N, convergesweakly
to E E SeT)iff any of Ihefollowingconditions is.tjaliified: (1) convergesweakly to every E A (T). (2) pointwiseon SII 10H(E,)for every E A (T). N converges 10 converges ME cPfor every cpE Jf'o(T).lIcpli1. (3) (ME Ps for (4) !R converges weakly to !R p\",s O'\037for
(O'\",\302\253)\"o:PIO
(X
(H(E\",a\302\273\"E
\037
..cp)\"fii'9;J
( (( 1>,..s)I.cz ))\". \037P\".t
every
(X
EA
(T)andevery S E
(X
N
\037.)))
(( ) \037P,
\037
Il:cz)
)
60.Weakly
convergent sequencesof experimcnts
303)
is equivalenttocondition (1).This is Proof It isclearthat weak convergence to the dueto the fact that (S(a)/ \"',..1)is topologically equivalent space//(1of on S(I (conferTheorem59.7). standardmeasures The converseisvalid since (1)=> (2):The implication (1)=> (2) isobvious. transforms determine standardmeasures. Hellinger a of (1)to (2). Use similar (1) (3): argument as for the equivalence of the (2) => (4):As in the proofof Corollary25.4we seethat convergence Ilellingertransformsimpliesconvergenceof the Mellin transformsof , SEa, 1',.'$' n E N, to the Mellintransformof \037;,, S)IFII IFII \".S (( ) ) (( ) of Theorem5.16proves(4). 0: E A (T).An application 25.4showshow the standardmeasures can (4) => (t): TheproofofCorollary be recoveredfrom the finite dimensional of the likelimarginal distributions hoodprocesses. Hence,the assertion. 0) \037
!i'
!i'
\037
\037\037
n E N, convergesweakly 60.4CoroUary.A sequence ofexperiments E 8(1;.), issatisfied: iff any of the followingconditions (1) (u\".J\"c; N convergesweakly for every a E A (T). (2) N converges pointwiseon Safor every 0:E A (T). 1. (3) (ME\" for every
(H(En.a\302\273\"E
(4)
II
dP,.,r !I' ( ((dp,.,$ )lEa p. )) \".S
II
\037
cunvergesweakly fur every
SEa ami every
\"EI\\I
aE A(T).)
of R(a),a E A (1'),with Theorem60.2and the ProofCombinethe compactness assertions of 53.8,53.10 and 57.11. 0) uniqueness
60.5Corollary.Assume En E
8(1',.), 8(1',.), I F,. E
n E
E FE E,
that
N,
E\"
weakly and
\037(T).Let r.
\037
F,.
O. If E\"
-:SFf',.for
weakly where every n E N
then E;2F. Although 8(T)/-- is a compact spacefor the weak topologyit neednotbe To obtain we need additional compact. sequentially sequential compactness
on the parameter set 1'. assumptions topological Let us beginwith a particularcase.)
60.6Lemma. If T is a countableset then En E
every
sequenceof experiments
8(7;.),n E N, containsa weakly convergentsubsequence.)
Proof Apply that S(o:)is sequentially compactfor every 0:E A (T) and use a standarddiagonalargument. 0) Assume that (T,d) is a metric space,The experiment E 60.7Definition.
is)))
304
Chapter10:Asymptotic DccisionTheory)
continuous if t 1-+;:is continuous. The sequence of experiments E\" e 8 n e N, is continuousin the limit if for every > 0 and every t e T there is 0(8,t) > 0 such that lim supd1 (P.w.\" P.w.,)
\302\243
\"CO
!\\j)
Notethat every equicontinuous iscontinuous in the limit.) sequence
60.8Lemma. Let (1',d)bea metricspaceandassumethat En E is continuousiff (En)\"cN is continuousin the limit.)
\037
E weakly.Then
ProofThis isdueto the fact that lim d2 (\037.s, \037.I) = d2 (\037, .P,))
\".. a\">)
for every pair(s,t) E T x T.
0)
60.9Theorem.Let (1',d) bea separable metricspaceanda.fi.fiume that (En)nf! N is a weakly conver\037enl subsequence.) continuousin the limit.Then (E\,,")4;N contains and dense. Thenthere isa subsequence ProofLet 10s; T becountable No s; N such that En,Tor'\\T\" -+ Ero weakly for nE No.We prove that (En)\"f'NO is weakly convergent. By Coronary60.4(3) it issufficientto show that for = {51' SN} E A (T) and every E Jr.'(T)with
if>
..., homogeneous {t ..., (fJ
if>
-
P\302\253
the form cp = f 0 A wheref RIa.l iR. Sincecp is positively satisfies a Lipschitz the same istrue off Let P = b condition and 'P = f 0 PIS'Then 'P E ff(T) and 'P0 PIS = 11'. We have)
and
IN} E A
- J'Pdun.,1
IScpdun.a.
-
- Sf ((d dL.
Ps
- J f ((d dL
dP, d r. PS J f {.I P Ps)IEI2) ((d L. l's)'ell) a.ufJL. .1
P,
s\037cr
P,
d L. P. J f lea.) HZ
)
oS
-
-
\302\253ufJ
crufJ)
- dp,
< LIp(f) L. J = i,;q d \037
-
\037
L P,
dp', d d L. P, L P,
I2v{J)
= 2Lip(f)
N
i
r. -=
1)))
/I p\"
- p',I/.
\302\253ufJ
aup)
d L PS JEll
s,-p)
\302\253ufJ
.
\037
((d L Ps)scfJ)
(To)
61.Contiguous sequencesof c'Iperiments
l05)
Sincep can be chosen The sequence No is a Cauchy sequence. <J tpdo;..p)\"f! close to sequence. 0) arbitrarily
ex
we
obtain that also <J cpdU\".a)\"E
N
is a Cauchy
60.10Corollary.ut (T, d) be a separablemetric space.Any equiconlinuous subsetof8(T)is weakly sequentiallycompact.)
61.Contiguoussequencesof experiments) We
of the preceding keepthe notations paragraph.)
of experiments 61.1Definition.A sequence n E N, is contiguous if E\" E &(T,.), for every pair(s,t) E T x T the sequences of probability measures Nand (P,..S)\"E (P\"..)\"4iN are mutualJy contiguous.)
if the probability 61.2Definition.The experiment E E S(T)is homogeneous
measures t E T, are mutualJy equivalent.) p\"
-E
61.3Theorem.Assume thai E\" iv
weakly.Then E ishomogeneous ifJ(E\,,a\037
contiguous.)
ProofIf E\"
-+ E weaklythen for eachpair(s,I) E
T x T the binary experiments P,.,,),n E convergeto the binary experiment(\037, P,).This provesthe (P,..s' assertion in view of Theorem18.11. 0) \037,
It shouldbe noted that the weak limit of a sequence of homogeneous need not be experiments homogeneous. For contiguous of experiments weak convergence sequences may be describedin terms of oneparticularlikelihood For the followingwe take process. an arbitrary elementof T and keepit fixed.Forconvenience we denote it by O.)
61.4Lemma. If E E 8(T)thenfor every
ex
EA
Po(N) = 0 and
- J n (dP'') dPo rr
H(Ea) (z)
lEa dR0
_
\037
(T)Ihereisa setNEd such Ihat
n [P,(N)]%',E Sa.
lea)
-
Z
1 dQ decomL P, and let dPo ,N bea Lebesgue Proof For E A (T)let Q = ) ( positionof Q with respectto Po'Then,for Z E ex
I ex
I
I.\037
Sa)))
306
10:Asymptotic
Cbapt\037r
H(E.)(z) = f J2
DecisionTheory)
(
\037ci
r dQ
dP, =J lea dPo
%'
%'
dPo+ NJ lEa n dP, dQ.
n( )
(dQ))
The assertion now followsfrom)
dP' :r dQ Jn N tu (dQ)
\037
dp, :r. 0 n dQ s \",a ( dQ) ) .y
61.5Lemma. If the sequenceof experimentsEll E 8(1',.), nE then for every E A (1')andevery Z E Sa) :r . hm H(E\",a)(z)J fl dPn.t d1',..0= O. \"-CI) I(a(dp11.0))
f\\)J,
is cOn/iguou.(j
ex
-
d L P,..tand let .d Q .,Nil be a Lebesgu\037 Proof For (; A (T) letQ\" = \037. tu lexl ( P,..o= )n E to of with Then decompositionQII respect 1',.,0' P,..o(N\") 0, N, implies II
ex
lim /I
'CI)
=0 n P...I(NIIY'
if
ZE
Sa.
I\037a)
followsfrom Lemma 61.4. Now, the assertion
-E
0)
61.6Theorem.If the seqt\037ence of experimentsE\" E 8(1;.),n E N, is contiguous then En
weakly iff for every
!fJ
P..,o ((:plI.r )/(a
-.
ex
EA
(T))
p...o_ .!t'((:p, Po)rea)
)
Po weakly.
)
issatisfiedin view of Theorem60.3 ProofIf E\" E weakly then the condition assumethat for every E A (T)) (4), Conversely, ex
dP,..r p. _ !i' dP, !i'((dp\",o ) ((dPo) )IU 11.0 _
Po weakly.
IE;Q)
)
Let ex E A (T).For simplicitydenote Jln
' )
dP,.,, !/!((d' 1>,..0 )rea
1=
P.s.o
n
EN, and
lim J
n
x\037'
dJlII
=J
n x;'dJl, tEll)))
I=!f'
dP,
((d-Po)
tEa)
Let us show that) II-.CI) lEa
Jl
Z
E
SIJ'
Po).)
61.Contiguous sequencesof expcriments Firstwe notethat for a > 0 and n E J
\037
N)
n x;'dJl,. It'a: L r n x;'dJ1,. L na ( r> xsdJl,.Y' L2 ( r> X,dJln):'. I x,1 x.1 \037
maxlxrl>/I I\"a:
307)
1%,1>/1 $1\"2
I\037a:
\037
t' II SF
I
III
I
t'
I
III)
that Now, Theorem18.9 implies Jim
sup
J
11I.'00 ,.ENm;axlx,j>1II
n x:'dJl,.= O. I\037a
11:a:)
show that) The same inequalities
nX:'dJl= O.
lim /I
-0co mu IJx, I > Q I E II lEa)
Thus,we obtainfor every Z E SII and every continuitypointa > 0
of)
\037(maxlx,1fIx:'Jl) IEII
I Ell)
n
J lEa: X;'dJl
\037
n
x:'dJl+ RI (a) J mulx,I:i/llEa: 11'11)
= lim
J
,.-owmulx.I\037Q
n x:'dJl,.+ R 1(a) '\037a
IFII)
\037
\037
\037
lim inf J 11X;'dJl,.+ \"-':10
R 1 (a)
\"11)
lim supJ fI X;'dJl,. + Rda)
,.-ro
lEa:)
Jim J ,,--romu'x,l\037o
n X:dJln+ R 1(a) + R 2 (a)
rea:
lEa:)
= \037
n
X:'dJl+ RJ (a) + R 2 (a) J lEa: mulxrl;io IEII)
n
J IE2) x:'dJl+ R 1 (a) +
R 2 (a)
where)
= O.) Jim + /1- (IRI(a)! IR2(a)1) tXJ)
This provesthe assertion.
0)
61.7CoroUary.A contiguous n E N, con.fiequenceof experimentsE,.E 8(1;.), vergesweakly iff for every
IX
EA
::., '
!l'((
,.,0 )
lEa:)))
1;,,0 )
(T)the di.'itrihutinns n E
\037,
convergeweakly.
308
Chaptcr 10: Asymplotic DecisionTheory)
Proof If (E\u")N convergesweakly then the finite dimensional marginal of the likelihood distibutions to Theorem processes weakly according converge 60.3.On the otherhand, if thesedistributions convergethen Lemma 61.5 that the Hellinger transforms implies convergefor every E A (T)which proves fJ.
the converse.
0)
62. Convergencein distributionof decisionfunctions) Let nE
of experiments 1;.t T and assumethat (E\nE") is a sequence E\" E 8(1;.), N
N, converging weakly to E E SeT).)
62.1Definition.Let p\" E 1I(E\",D),n E N. The sequence (p\\"O+l") distribution to P E 11(E,D) if) Jim \"-tQ)
converge.fiin
p\"(/,P.w,,) = P(/' P'), IE f{?b(f),t E T.)
It is easy to seethat this definitionis an extension of the corresponding notions. I f is a o f estimates elementary sequence ,,-measurable (\"\\",,") N decisionfunctionsare Q,,(., B) = 1B K\", K,,:Q\" D then the corresponding ne N, Be \037o(D),and hence for converges weakly to \037(\"IP,) t E Tiff in to) every (e\\"e") N converges distribution \037CI
t)
\037
(\037(\"\"IP.w,'\302\273\"\037N
(}:(lll,B) 1-.1 8
,,(m),(m,B) E Q x 9Bo (D). =:lp\", n E N, then in {1}) Similarly,if D = {O,1}and (l\"(., N converges iff n to the functions t E distribution (! N, converge power p\" ,
\302\253(l\\"e")
\037
62.2Definition.An element P e 1I(E,D) is an accumulationpoint(in distriof generalized decision functionsp\" e 1I(E\", D),n eN, if bution)of a sequence for every 7 E A (T) and every finite set G \037b(D) there is a subsequence \037
N
(a.,G)
\302\243;
N
such that
p\"(/, P\".,)= P(/' P,), fe G, tea.. \"'N(<<.G)) Jim
62.3Theorem.(LeCam[1979]).Every sequenceofgeneralized decision functionsp\" E \037(EfI' D),n E N, has accumulationpointsP e \037(E, D).) Proof For every)'e A (T)and n E N let K\", y: Q x
kernelsuch that
IIRK..,\037P,
-
P\",rll
<
L1
(/::\"..\"
E.,)+\037. t e
y.)))
.\037\"
\037
[0,1]be a stochastic
62.Convergencein
distribution
of decisionfunclions
309)
For this,conferRemark 55.6(2),and Theorem55.9.Then) P\",y:(I,Jl) \037
P,,(f,R\"\",yJl), IE \037b(D), Jl E L(E), are elements of rM(E, D).Let /1y be a weak accumulation pointof (/1\".'I)\"\" N and let p be a weak accumulation pointof (P'l)Y4iA(TI (cr.Theorem42.3). Fix 0:E A (T)and a finite setG fib (D).We show that for every 8 > 0 and every no E N there existsnc no suchthat) p\".(/,P...) P(/' P,) < 1:,f E G, t EO:.) \302\243
\037
-
I
I
Firstwe notethat there exists E A (T),r ;20:,such that) \"I
IPy(f,P,)
- P(f,
<
P,)\\
3' IE G, tE ex.) \302\243
Let a:=max {lI/lIu: lEG}and n 1 E N be such that
1 11(E\".'I'Ey ) + < n
Thenwe choosenc
\037
'f
f;
n
I
3a
-
\037
I P\"e
2' IE G, tE
that)
\302\243
ex.)
- P\".,(I,P,) (I, p\"e,')- (f, R,,\"..P,)
1/\037,,'
=
.)
max {no,nd in sucha way
P,) Py(f, P')I< IP\"..'1(f, Since)
nt
\037
I
Y
-
P\"e
y
I
e
< IE G, t E ex,) 11/1111 1'\"\", R\"\"'.'1P'1I \"3'
the assertion is proved.
0)
It is now clearthat a sequence (P\,,e")N which has exactly oneaccumulation in distribution to this accumulation pointconverges point.)
62.4Corollary.Supposethat (T, D.W) is a decisionproblemwith a lower sem;cont;nuous losslunction.Then every accumulationpointP E 14(E,D) of a E P\" rM(E\", D),n E N, satisfies sequence P (J.Y\" P,) lim supP\" (J.Y\" \037
Proof Let V
\037
W
,,-co)
1'\",,),t E T.
be a boundedcontinuous lossfunction,Thenfor fixed(E T
there existsa subsequence N, N such that))) \302\243
310
Chapter
10:Asymptotic
DecisionTheory)
P,) = lim fJn(V\"
P(V\"
nEN
P,.,,) limsupfJ,,(H-;,P,.,,). ,,-co) \037
r
Passingto the supremum over a)) V
W
\037
the assertion follows.
0)
lower boundsof Now, we arrive at the result which is the basisof asymptotic risk functionsfor a large variety ofsituations.)
62.5Theorem.Supposethat (T, D, W) is a decision problemwith a lower lossfunction.Let !M !M(E,D) be the set of allaccumulation semicontinuous \302\243
Pn E pointsof the sequence hull. Thenfor every subset
\302\243I(E\",
\037
\037
lim inf sup
,,-a)
,
E
To
(\"\\
D),n E N, and co itsweakly closedconvex \302\243I
T
1;'p\" ( H-;, P,.,,)
inf
\037
II
(!co91
sup , P ( H-;, P'). CI
To)
a subsequence Proof Let mE STo and = supp(m). Choose fX
\037m
lim infJ p\" (H-;,\037,,)m (dt) = lim J PIle .... :n E \"
\"
J.\302\245\"
\302\243
N such that
P\",,)m (dt).
FIlm)
Let P E !M(E,D)bean accumulation pointof the sequence (P,,)nEN.... Obviously, thisisalsoan accumulation of the It that for follows point originalsequence. lossfunction V W there exisL\037 a subsequence every boundedcontinuous \037
Ny
\037
satisfying)
N\".
= J P(V\" lim J P\"(V,,p,.,,)m(dt)
p')m(dt).
\"fNv)
Thisimplies that = lim J P,,(H-;, lim infJP,,(H-;, p\",,)m(dt) p\",,)m(dt) n-a) \"EN...) \037
Jim
> lim J P,,(v,. p\",,)m(dt) supJ PII(V\" p\".,)m(dt)
,,\037N...
,,\037Nv)
= J P(V\" p')m(dt).)
FromLemma47.2we obtain) lim inf J n-a))
P\"(J.\302\245,,
p\".,)m(dt)J P(\037,p')m(dt). \037
It followsthat)
-inf ,csup P,,(H-;,p,.,,) = Jim inf sup J P,,(\037,p\".,)m(dt) ,,- S1.0
Jim \"
00
To
co
f'\\
'I;')
\".1;
n T\)
62.Convergencein \037
sup inf J IJ(
J.Y\"
meSTo6e.1l) \037
/It;
An A
co.)
of decisionfunctions
31
1)
p')m(d/)
SUp inf J {J(J.Y\" ,\"\037STO
distribution
p')m(dl).
the proof. of Theorem46.3completes application
0)
minimaxboundfOT risk functions.) particularcaseisthe famous asymptotic
62.6Corollary(LeCam[1972; of Theorem 1979]).Assume Ihat Ihe condilion:,' Thenfor every sequence D),11EN, (62.5)are satisfied. PIt E !!J(E\", Jim inf supPIt /:,) inf supfJ P'). ,,-\037 (J.Y\"
\037
t\037T\"
(J.Y\"
fJc9l(E,D)t(T)
Another application dealswith admissibility. Recall,that a function r: T -+ 00,+ 00]is an admissible risk function for E and (T, D, W)
(-14
every /3 E
if
for
(E,D)
{J(U-;,P,)
\037
r(t), tE T, implies {J(Ut;,P,) = r(/), IE T.)
62.7Theorem(LeCam[1972]). SupposethaI (T, D, W) is a decision problem with a lowersemicontinuous lossfunction.Let r: T -+ (-00,+ 00]he an admi'isibleriskfunctionfor E. If a sequence of 13\" E 14(E\",D),n E N, satisfies r(t), tE T,) limsup{J,,(U-;, P\",r) \"-00) \037
then)
lim {J\"(J.Y,, p\",,)= \"-.0())
r(l), lET.)
point Proof From Corollary62.4 it followsthat every accumulation fJ E a (E,D) of N and even every {J E co31satisfies J;):5; r(/), tE T, {3(U-;, which implies that by admissibility = r(t), t E T,) fJ(J.Y\" for every fJ E co91.Now Theorem62.5provesthe assertion. 0) (fJ\1Ifi")
\037)
Another version of thisresultis)
62.8CoroUary.Supposethat (T, D, W) is a decision problemwith a lower semiconlinuous lossfunclion.Let r: T -+ ( 00,+ 00] be an admissiblerisk funclionfor E. If a sequence ofPIt E &i(E\",D),n E N, satisfies)))
-
312
Chapter 10:Asymptotic DecisionTheory)
lim inf
,,-QO)
P\",to)
P,,(J.\302\245,ot
< r(to))
for someto E Tt then there existst JET such that P\".t.>> r(lI)'
limsup ,,-QO)
P,,(J.\302\245,.,
63.Stochastic convergenceof decisionfunctions) isa sequence of experiments E\" E & (T,.)tn E Nt decision weakly to E E tf (T).Let D be a topological space.) converging Let T,.r T, Assumethat (E\\"E") N
63.1DefinitioD.Let a\" E [;it (E\",D)and t\" E [;it (E\",D),n E N, be two sequences
of decisionfunctions. The sequences (a\\"E") Nand (t\,,t,") N are stochastically
equivalentif) lim If
0\"\"
,,.....It'J)
-fr,,1P...t= 0, IE
c'jf,,(D),
t
E
T.
of decision It is immediatethat stochastically functions equivalent sequences
have the same setsof accumulation points.)
63.2Remark. The conceptof stochasticequivalenceis closelyrelatedto in measureof decision functions. convergence be Let a metric (1) (Dt d) compact space.Let A 5:r.tb(D)be the setof all functionssatisfying)
-I(
1 and I/(x) 11/1111
d(x,y), Xt Y D.) it is easy toseethat A is compact and) Then by the Arl:ela-Ascoli-theorem \037
y) I
\037
\302\243
-
sup{If(x) f(y)l:fEA} = d(x.y), x.Y ED.) that) Stochastic equivalence implies
lim
-/,c\"
Ifa\" ,,-coJ sup IFA) which is equivalentto
l
dP\".t=
0,
tE
T,
= Ot tE T. lim JH d(x,Y)l1\"(w,dx)t,,(w,dY)P,,.r(dw)
\"-00)
(2) Let (D,d)
it
followsthat lim
JJ \"-00J \"\)
be a metric spacewhich isseparable From(1) and complete. = 0, d(x,y) a,,(w,dx)t,,(w,dy) P...t(dw)
t
E
T,
63.Stochasticconvergenceof decisionfunctions
313)
that (U )IIEN and (T\IIEN") converge to some K 5:D.Suppose for every compact we find for Since is on every e E [;it (E,D) in distribution. qp' tight !?I(D) may c > 0 and t ETa compact setK D suchthat II
\302\243
< lim supJ 0\",,(., D\\K)dP\",I
,,-< E. lim supJ T\"(,,D\\K)dPII,r \"-00) \302\243,)
co)
Thenstochastic equivalence implies) lim (u\" \"-CO)
@ t,,) P.s,r {(x,y) E
D2:d(x,y) > c} = 0, c > o.
= (3) Let D = {O,t} and denote q>1I followsthat
0\" 11
(
.,{l}),
11'\"
=
TII
(.,{t}),n e N. Thenit
= 0, nE N. lim JIq>II-1p\"'dP\",r
\"-cO)
be a metric spacewhich is separable and complete. Assume -+ + that there are sequences \"II: !lll D, nil:!lll D, n E N, satisfying) B) = 1B o nil' ne N, BE!M(D). ell(\" B) = 1B o \"II' t,,(., (4) Let (D,d)
of (2) Then,under the additional assuptions lim
P\"\"
II-cO)
{de,,\", n\") > t} = 0, t > O.
Supposethat two sequencesof decisionfunctionsU e 91(E\",D) and in distribution to the same limit e E (it (E,D). TII e (it (Ell'D),n e N, converge When is it possible to conclude that the sequences and ('II)IIEI'II are which is stochastically equivalent?The followingis devotedto this question II
(O\"II)\"EN
answeredby Theorem63.6. Let p E (E,D). A distributional of fJ is a neighborhood anothersetof the fonn) \037
set containing
-
< t,fE G, SE a.}) F= {\"IE 91(E,D):Iy(f,Ps) P(f,!'s)1 rc,,(D)is a finite subsetand E > O. The system of distributional of p is a filter which iscoarserthan the filter of neighborhoods weak neighborhoods of p. Let uscallan element if P E (E,D) uniquely determinedby itsdistribution = = p.) y(j,!'s) P(f,Ps) for aUfe\037b(D) and.fET implies}' 63.3Lemma. Supposethat PEal(E,D) is uniquely determinedby its distribution.Then thefilterofdistributional with thefilter neighborhoods offJ coincides where a. E A (T) and G
\302\243
\037
of weakneighborhoods of
p.)))
314
Chapter
10:Asymptotic
DecisionTheory)
of P and denoteby:Fthe filterof ProofLet U bea weakly openneighborhood distributional of Since neighborhoods {3. p is uniquely determinedby its we have) distribution
n
F = {fJ},
F@\037)
where F denotes the weak closure of F\\ U = '/),
}\037
FE !F.It followsthat
n F,')
and
since (E,D) EM
U.
F\037F\037
IS weakly
compact,there existsFE.rF such that
0)
63.4Remark. Let E = (0,.91,{p,:t E T})bea dominatedexperimentwhich is and let D be a locallycompact decision base. complete spacewith countable Thenevery q E 9t(E,D) is uniquely determined To seethis, by itsdistribution. = let P E (E,D) be suchthat p (f, P,) If}P, for aU E beD),t E T. If f}o is a substochastic kernel such that f3(f,Jl)=f(}oJlif fEtGoo(D),JlE L(E),then tE T. It followsthat (}o is a stochastic kernel and f(}oP'=f(}P',fE\037oo(D). = that P (}.) hence,P E fit (E,D).Now, completeness implies
I
\037
63.5Lemma (LeCam [1979]).Supposethat
\037
E weakly and that If (} is non(}\" E 9t(E\",D), n E N, convergesto (} E [it (E,D) in distribution. randomizedanduniquely determinedby itsdistribution, then
- (fe,,)2P,,)= 0, IE
lim (f2(!\"p\"\"
\"-'00)
Proof.Fixfe beD),f\"4= such that) \037
if
P\".r
(fe). U\302\253(},
\037
-+
\037b(D), t E
T.
a number N(e) 0, t E T, and t> O. We win construct
f 2 e\"P\".,(fell)2 < e Let Jl =
E\"
n
\037
N(t).
and define)
e):={PE \037(E, D): 13(f2,.P,) I
t -1 P,I< 4' {J(f, -ff}JlI< 8}') 2
C
(}
Jl)
I
of fl, and in view of Lemma 63.3 U(q,e) is a weak neighborhood Obviously, there are E A (T),a finite subsetG
{pE
\302\243iCE,
-
\037
fJ
D): P(g,\037) g(}\0371< fJ, g E G, I
,<;
E
(X}
Let}'= Gtu{t}and chooseN(f.) such that II(E..,> E,)< min
, g E G} {2mox {II:II,:
411\0371I;
})))
\037
U(f},E).)
63.Stochasticconvergenceof decisionfunctions
315)
and)
if Igell\037.s-ge\037I<\037 sE\037,gEG,) whenevern
> N(r.). For every n
that IIP.\"
\037
N(E) there is a kernel KtI
- RK.P,II< min {2ma.{II:II.:g G}
SE Y.)
II
(2\302\273.
- ge\0371< b,
and thereforeUti E
411\037IIJ
Yt(E, Ell) such
DenotingU :=(]II RKn E {il(E,D) we obtain
(cf.Remark 55.6 Igu,,\037
.
\302\267
E
E
U\302\253(],
E), n
-
SE ex, n
N(E),)
\037
N(E). This implies by
\037
definitionof
V\302\253(],
E))
t
If2 UIIP' f 2 eP'1< 4' Ifu,.(fe'P,)
-
e
fe\302\253(fe>'
P')I< 8')
Sincee is non-randomized it follows from Theorem42.9that
-fe)2P, f2 - (JU)2P, -f2 - 2(full -f (})(f . .P,) = f2 c-c
o Sf 2 u P' ([U)2 P, lI
II
ulIP,
\037
U\"
+ (full
II
P,
fl
P,
(}
n
4
followsfrom Now, the assertion
-
f2 (lll P,..I (frlll)2
-(f -[ +2
\037.I
E
2
f2
UII
RKn
= f 2 UIIP,
P,
(
flIt
RK,,)2P, +
2
UII)
\037
E;
\037
\037
t,
,,\037N(t).)
o)
to give a complete solutionof the question Now, we are in a position posed. ConferTheorem23.10.)
63.6Theorem(LeCam [1979]).Let
EE tf(T) be an experiment and D a are equivalent:))) topological space.Forany e E 9t(E,D) the followingassertions
316
Chapter10: Asymptotic DecisionTheory)
(1) Whenevera sequence of experimentsE\" E &(T,.),n E N, convergesweakly to E and 0;,E Yt (E\",D), n E N, and E [;it (E\",D), n E N, converge to e in then (u\,,") Nand distribution, N are stochastically equivalent. anduniquely determined (2) e isnon-randomized by itsdistribution.) 'fIt
\"
(t,,),,\342\202\254
(2):We define an experiment F= (D x D, \037o(D)@ d, Proof (1) {Q,:t e T})by Q,:(f,g) 1-+P'(fe.g), fE g e !R,,(d),leT.) Let us show that E --F. The relation E S Fisclearsincefor every r e .(jf (E,D) there is u: c.o),B).-. -r(m,B), (x,m) E D x D, BEfBo(D),in {il(F, D) \037
!R,,\302\253(fIo(D\302\273,
\302\253x,
satisfying
fuQ,=ftP', te T,fe\037b(D).
-
the mapping L(E) L(F)beingdefinedby Conversely, \037:
,) p(fe'g), fe :l',,(fJo(D\302\273,ge :l',,(.9I) 7iP:(f,g).-. is a transition. This implies E 2 F. Now we definea non-randomized function) decision K: D x D D:(x, m) 1-+x for F. For every fe \037b(D) we obtain
-
fKQ,= J/o KdQ,= fIfo K(X,m)e(m,dx) p'(dm)= feP\" t e T. On the otherhand,(1: m), B) 1-+ e(m,B), (x,m) ED x D, Be &10 (D),is in Yt (F,D) and satisfies \302\253x,
f(1Q,=.f(}p',tET. It followsthat .fuQ/= fKQ\"
tE
T,
-
impliesby condition(1) that Ifu fKIQ,= 0, te1:fe \037'b(D). A standardargument yieldsIfa fKlp == 0 for all p E L(F)and therefore f(1J1 = fKJ1, f ErebeD),J1 E L(F).)
-
which
Putting Jl =
f. Q\" t E T, it followsthat
(fq)2p, = f2qp', t E T,fe \037b(D),) that is non-randomized which implies by Theorem42.9. It remains to show that (! is uniquely determinedby its distribution. If \037
te1:then a directappli-))) e1 e91(E,D)is such thatfelP'=feP',fercb(D),
64. Convergenceof minimum cstimatcs
! !!
317)
-
cationof condition (1)impliesthat Ifl! fl!11 P, = 0,f E \037b(D), t E T, which provesthe assertion. (1):Let U\" = (0'11+ tIt)' n E N. Since(Qn)neN convergesto e in (2) that it foJIowsfrom Lemma (63.5) distribution = 0, fE \037b(D), t E T.) lim (f2(1\"P,.,r (f(1,,)2 P\",r) ,,-IX))
?
Denoting = t (0'\" tIt)' n E N, we have (fcp,J2+ (fe,,)2= [(fC1,,)2+ (ft,,)2] U2 + f2\"(,,]= f 2 e\",)
\037
and therefore)
G\"
.
2 n EN.) f2 - (f(1,,) This provesthe assertion. 0)
(I
2
\037
(1\"
Thereisan interesting of the preceding the assertions consequence concerning
IL should determination of decision functionsby their distribution. be recalled that completeness isnota propertyofequivalenceclasses ofexperiments ascan in general, beseenfrom the caseof exponential it may Therefore, experiments. determines the underlying decision function for one happenthat a distribution However,if equivalentexperiment. experimentwhereasit doesnot for another the underlyingdecision function is non-randomized then uniqueness carries class.ConferalsoTheorem23.9.) over to the whole equivalence
63.7CoroUary.Let (1E 9t(E,D).If there existsan experiment F = (D\"silt{Q,:t E T})'\"E and G E \037(F, D) such that f(1p, = fGQ\" fE 'Cb(D),t E T,) and uniquelydeterminedby itsdistribution, where G isnon-randomized then also(1
isnon-randomizedand uniquely determinedby itsdistribution.)
64. Convergenceof minimumestimates)
Let (T,d) bea metricspaceand letT,. !;;T beopensubsets satisfyingT,. r T. Let sI n be a of a nd stochastic E (0\", p\,") sequence probability spaces consider n EN, whereZ,,(t): 00,+ 00]are sI,,-measurable Q\" processes (Z,,(t\302\273UT\"' functions,n E N. Assume that the stochastic processes (Zn(t\302\273re T\"' n E N, are let a be Moreover, (0,.fII,P) spaceand let T bea probability separable. stochastic where Z(t): (-\037,+ 00]is ..c1-measurable.))) separable process '\"
\037,
- (!l-
(Z(t\302\273r\037
318
Chapler10:Asymplolic DccisionTheory)
64.1Examples.(1) The resultsof the presentsectionmay be appliedto the Let En = (On, followingsituation.
d,
\037r;/\",
{\037.I: t E \037}), n E
E = (0, {.P,: t e T})be experiments. Then we fix n E N, p:;;;; Ps, and define
\037,
and
somesET,put p\":=p\".s'
.
dP,..rtE\037, liE N, and Z,,(t) = -logdP.
-, \",s)
Z(t)= -Iogddp' te T. Ps)
In sections 65-67we shallassumethat Tisa Euclideanspaceand apply the results of the presentsection to the samesituation asunder (1),butdefining (2)
-
Z,,(I)= f W(t s) f;ds), t E 7;, n E N, and Z(t)= J Wet .v)F(ds), t E T,)
-
where (W (.
-
lossfunctionand F\" are posterior distributions for E\" with respect tosomepriormeasures n E 1111' .v\302\273u
T is a
\037.)
For notational convenience we introduce) 1={tE 1;.: B:(w,,) Z,,(t)(w,,) seinf Zn(s) (ro,,)+ e} \037
'f\
and)
Br.(w) := {te T:Z(t)(w)
\037
inf Z(s)(w)+
IE 'f)
c},
which isintroduced in the followingdefinitions is e N, t: O.The tenninology to decision motivated by itsapplications theory.)
n
\037
kernel (1:Q x BI(T)..... 64.2Definition.A stochastic [0,1]is a minimum estiif g(.,EO) = 1 P-a.e.) mate for (Z(t\302\273'ET
-
64.3Definition.A sequenceof stochastickernels(},,:U\" x !J4(\037) [0,1], if for every > 0) of asymptoticminimum estimates n E N, is a sequence \302\243
= lim J l!n(.' T\" \\ B;)dP\" O.
\"-00)
64.4Examples.(1) Considerthe caseof Example 64.1(1).An estimate for every Ps.sET, is a maximum (l E [Jt (E.T) which is a minimum estimate A sequence of estimates likelihood estimate. ell e gp (E,.,7;), n e N, which is a minimumestimates forevery sequence of asymptotic sequence (\037.s)nE N' sET. of asymptoticmaximumlikelihood estimates. is a sequence 64.1 T hen the respective estimates are Consider the case of (2). (2) Example of asymptoticBayesestimates.))) and sequences calledBayesestimates
64. CoO\\'crgcnccof minimum estimates
319)
Now we statethoseconditions which are usually neededfor a satisfactory theory of minimumestimates.)
64.5Conditions for T' setK T such that) (1) For every E>0 there is a compact P{ inf Z(x) inf L(y) +\037} < t. (Z(t\302\273'E
\037
\037
'5T)
zf;T'K
pathsP-a.e.) (2) The process (Z(t\302\273,c T hascontinuous
64.6Conditions for (Zn(t\302\273,t.T..' nE\037. (1) For every e> 0 there is a compactsetK T such that lim supp\" { inf ZII(X) inf 2,.(y)+ I;}< 1;. n--co XE T..\\K ,eT..) and are (2) The processes (Z,.(t\302\273,a.., n E N, have continuous pathsp,,-a.e. and boundedon compacts in (\037)-probability.) equicontinuous of existence of minimumestimates.) the question First,we consider \037
\037
Then 64.5(1) and (2).are satisfied. 64.7Theorem.Suppose,that conditions
there existminimum estimates for (Z(t\302\273'F
T')
followsfrom Proof.If we can prove that P{Bo(w) 0}= 1 then the assertion For the existence theoremfor measurable Theorem 6.10. selections, every n E N setsuch that let Kn c T be a compact =t=
P Since)
inf
Z(x)
\037
{xeT,K\"
inf Z(y) +
)leT
inf Z(X) > inf Z(y) +
{xfT\\K.. it
)lI'T
\037
n
\037
n
< . \037
}
}
\037
n)
{BO =t=
0} J:r-a.e.
followsthat P{BO 0} 1-\037, n EN, which provesthe assertion. =t=
\037
o)
n)
64.8Theorem.Supposethat conditions Then 64.6(1) and (2) are satisfied. there existsequences whicheven satisfy ofasymptoticminimumestimates lim
,.-ro)J
en(.'Bno)dP\" = 1.
Proof For every k E
let KJc
\037
setsuch that T be a compact
Z,.(x):$inf {x( Tn'KIc
lim sup1>,. /I-\302\253)
\037
inf
)IE
Tn)))
- < -1 . l
ZII(y) + k
}
k
Chapter 10: Asymptotic DecisionTheory)
320
Let N be suchthat n Ic
P\"
inf
{xeT\"\\Kk
\037
N\"
Z,,(x)
\037
implies1',. KIc and \037
inf YET\"
-
l Z,,(y)+ k < k)
\037.
}
Since) inf
{.:c.T\"\\K,,
Z,,(X)> inf
Z,,(y)+ k.l
}
y.T\"
\037
=t=
{B\"O
-1
0} \037-a.e.,
follows that P,.{B\"O * 0} 1 if n N\", which impliesthat Jim p\" of measurable * 0}= 1.FromTheorem6.10followsthe existence \"....00 such that \",,(w,,)E B\"o(w,,). Define\"\" arbitrary on {!to maps\",,:{B\"O * 0} 1',. = 0}and e\"(w,,,B):=18 BE14(1',.). 0) WISE !l\", it
\037
{B\037
\037
-.
(\",,(w,,\302\273,
64.9Remark. The readerwill notethat the proofof Theorem64.7is a literal copyof the secondpart of the proof of Theorem37.8.which provesthe of generalized In a similarway we shallprovethe existence Bayesestimates. For of maximum existence ofasymptotic estimates. Bayes provingthe existence estimates ourTheorems likelihood 64.7and 64.8are lessuseful sincein thiscase the stochastic processes Tn' n E N, dependon SET whereas the of sET.) maximumlikelihood estimates must be constructed independently (2,.(I\302\273'e
64,10Lemma. Suppose that condition 64.6( J), issatisfied. Thenforevery e > 0 setK T such that) there existsa compact lim \037{!tc K} > 1 t, ne N.) \037
-
\302\243;
\"
.00)
Proof Let E > O. ChooseK lim supP,,{ inf
,,-co
XFT,,\\\"
\037
T such that
Z,,(x)
\037
inf )'fiT\
Z,,(y)+ e} < 6.
Since)
{ inf ;rl:T,,\\\"
inf Z,,(x)> YfiT\
the assertion is proved.
ZII(Y)+
\302\243;
\302\243}
{\037
\302\243;
K}
0)
64.11Definition.A sequenceof stochastickernels ell:!1x .'M(1\037)-.[0,1], 11
nE
N, isuniformly tight if for every I:> 0 there isa compactsetK
\302\243;
Tsuchthat)
Iimsupp\"{en(.,K)< l-e}
64.12Theorem.Supposethat condition64.6 (J). is satisfied.Then
every)))
64. Convcrgenceof minimum cslimalcs
sequence of asymptoticminimum estimates for
(ZII(t\302\273,e
:nt)
T..'n e N, is uniformly
tight.)
Proof Let t; > O. ChooseK
ell(.'T,,\\K) n
\037
T accordingto Lemma64.10. Then)
\037
ell(\" T,.\\B:)+ ell(\" B;,\\K),)
e N. This implies that) by definition lim supJ QII(\"
11-00
T,.\\K)dp\"
lim supJ QII(\" B:\\K)dP\" <
\037
\342\202\254
11-00)
smce) =F 0}< p,.{Q,,(., \037\\K) > O} \037{B;\\K \037
for sufficiently large n E N.
\342\202\254)
0)
This tightness assertion hasimportantconsequences.)
64.13CoroUary.Assume that T is a locallycompact space.Supposethat conditions kernels 64.6(1) and (2),are satisfied. ,f a sequenceof stochastic 0',,: (211x \037(T,,)
\037
n EN, is stochastically [0,1]. equivalentto a sequence of
asymptoticminimumestimatesfor (ZII(t\302\273UT\"'
of asymptoticminimumestimates for
(ZII(t\302\273,C
nE
N, then (0'11)isalsoa sequence
T,,'n E N.)
of asymptotic minimumestimates for ProofLet (Qn) bea sequence T..' n e N. which isstochastically a compact equivalentto (un).Let > 0 and choose (Z,,(t\302\273re
setK c T such that for n
\302\243
\037
N(e)) f.
J QII(\" T,.\\K)dP\"< 4') e
-.
T,.\\K)dp\"< 4) JO',,(.,
64.6(2),there exists > 0 such that the sets By condition \037
A II
:=
-
IZn(x) Zn(y)1 {4(x.)')<6.x.Yf!\
satisfy P\"(A\037) < X,YE K,WE (2\",)
SUP
\037 \037
}
as soonas K c 7;.. Hence,PII(An)
\037
1
\037
xeB:/2(w),d(x,y)<\037 imply yEB:(w), neN. It fo))ows that for n
\037
N(E))))
-\037,
and for an
Chaptcr 10: Asymptolic DecisionTheory)
322
J (!n(',8:12) a;,(
t
.,
T,.\\ B:)d\037
K n B:/2) Un (.,K\\B:)dP + 2 J QII(\"
\037
n)
t + J Qn(.'Kn B:,/2)U (., K\\B:)dp\" II
\037
An)
tI(ellx.)')II6. x,)'(; (In(''dx)u(., n
\037
dy\302\273dP\".
K)
\037
Stochastic equivalence implies Jim II
.supI QII(\" B:/2)U (.,T,. B:)dP\" t:. \037
isa sequence of asymptotic minimumestimates it followsthat
Since
\302\253(111)
lim supJ un n'\"
(.,
T,,\\Jr,.)d\037
8.
\037
<;0)
This provesthat (Zn(t\302\273,c
\\
II
<;0)
Tn' n
of asymptotic minimumestimates isalsoa sequence for
(0'11)
EN.
0)
64.14CoroUary.Suppose that condition 64.5and64.6aresatisfied. Assumethat for every rl E A (T) =2 lim P), weakly.) 9'\302\253ZIl(t\302\273'EcII\037)
\302\253Z(t\302\273,u
I
n\"'QC)
Thenfor every compactK Jim
\302\243;
T
supp,.{B:n K =*' 0} supp,. n K =*' 0} lim lim n.'<;O P {BOn K * 0}
n-<;o
{BIIO
\037
C
\302\260
\037
.)
\037
Proof Let K c T be compact.The first inequality is trivial. Moreover,we observethat) = P{BOnK:f:0} JimP{JrnK=t=0} C
\302\260) \037
sincecompactness of K implies) n {BCnK* 0}= {Bor\",K * 0} P-a.e. c> 0) lienee,it sufficesto show that) Jimsupp\"{B:nK 0} P{BCnK =*' 0}+ 26) 11'<;0) =t=
\037
for sufficiently small > O. For every sufficiently small f. > 0 and n setKc c: T, Kc ;2K, such that))) have a compact F.
\037
N(t) we
64. Convcrgcnccof minimum estimates
-
s; Kc}> t \037{a:
and P{RCc:Kc}> 1
[;
-
323)
r..
This followsfrom Lemma 64.10. Since)
B:n K =*' 0 wheneverB:
\037
BE
\037
K
Kc and Kc
n K 4= 0
inf2\" +
inf Z,.
iff
\037
and since)
1;\"
inf Z
iff
F.
Kc)
inf Z + t
\037
K
Kr.)
jf B s; Kc, and sincefrom condition 64.6(2) jt followsthat) weakly on \037(Kc) C
-.
9'\302\253(Z(t\302\273'f,K.IP)
\037\302\253Z,.(t\302\273hK
we obtain)
lim supp,.{B;n K =F 0}) \"....
00)
\037
\037
t + lim supP,. {infZ\" \"....00
\342\202\254
\037
K
K.)
+ P{infZ infZ+ \037
K
Hencethe assertion.
inf Z\"
\342\202\254}
\037
2\302\243
K.)
+ t} + P{Bt(\"\\K* 0}.
0)
of resultconcerning in distribution Now, we arrive at the desired convergence minimumestimates.) that conditions 64.15Theorem.Suppose 64.5and 64.6are satisfied. Assumethat
for every (l E A (T)
lim 9'\302\253Z,,(t\302\273'E::IIP,,) =
\"-00)
weakly.)
9'\302\253Z(t\302\273'<.\302\253IP)
itsinfimum in exactlyonepoilltP-a.e.. process If (he stochastic T attains then every sequence of asymptoticminimum estimates for T\", fl E to the minimum estimate convergesin distribution for (Z(t\302\273,c
(Z,.(t\302\273,.,
\037,
(Z(t\302\273'\037T')
of asymptoticminimum estimates for (Q,.)is a sequence n EN. Let K If > 0 and K T be compact. then we have)
ProofAssume that (Z,,(t\302\273f\037T\"'
\037
\302\243
\302\243;
7;\"
T..\\B:)dP\". SQ,.(.,K)dp\" \037SQ,,(.,KnB:)dp,.+ SQ,.(., Thisimplies that limsupJ (},,(., K n B:>> O} K)dp\" lim supPIt {e(., \037
1I\037c()
II\037(X))
\037
Jim \"
supp,.{Kn B,.L + 0}. .00)))
324
Chapter
to:Asymptotic
UecisionTheory)
Corollary64.14yields limsupJ (lll(\" K)dp\" P{BOn K II -4 \037
=*=
0}.
<XI)
Let (l be the minimum estimate for (Z(t\302\273tE T' Sincecard BO(w) = 1 P-a.e.it followsthat (!is non-randomized and) K) = 1}= J e(.,K)dP.) P{BOnK+ 0}= P{(l(., This implies limsupJ (!II(\" K)dP\" J (!(., K)dP. 11\037
\302\253>)
the proofof the assertion. Now, uniform tightness completes
0)
the previousresults In the subsequent sections are appliedto asymptotic Baycs estimation sincetherebyit ispossible toguaranteeconditions 64.5and 64.6in a estimation is way. The problemof maximum likelihood statistically plausible
discussed below.)
64.5and 64.6are satisfiedwith the notations 64.16Remark. If conditions of 64.1 for then the of the to r esults Example (1) every sET, application previous
maximum likelihood in theory is a matter of routine.The problemconsists the validity of thoseconditions. of experiments Weak convergence establishing doesnotimply the secondcondition and thus,for an invarianccprinciple for the likelihood have additional to be made. W e processes usually assumptions shallreturn to thisin Section84.)
65.Uniformlyintegrableexperiments) A Borel Let (T, d) be a metric spacewhich is locallycompact and separable. K 5;; T. measure is a measureJ1 E aCT)such that J1(K)< 00for every compact a Let 1',. of opensubsets such that 1',. T. Consider 5;; T, n e N, be a sequence = of experiments I E \037}), n E N. Moreover, let(1l1I) (1JII \"s;f\"' sequence Ell on .tM(T). of Borelmeasures be a fixedsequence of thissectionis concerned The main assumption with the separation of of their neighborhoods by tests.Recall Definition pointsfrom complements
{l;'.,:
i
36.1.)
65.1Definition.Suppose that the experiments Then Ell'n E N, are measurable. the sequence if for every SETand every (; > 0 (En) is uniformly (JllI)-integrable there are a compactset K T and a sequenceof criticalfunctions))) \302\243;
65.Unifonnly q>\"
E
\037
integrableexperiments
325)
(D..,.!:II ,,), n E N, such that P,.,;rq>\"
-
>1
and
t:
J
Tn'\
P\"\"
q>,,/-t,,(dt)< e
assoonasSET\"and K T,.. \037
84,1n. The is satisfiedseeDiscussion For an example where this condition isjustifiedby the followingtheorem.) tenninology
65.2Theorem.Supposethat the experiments E\", n E N, are separableand Then the sequence measurable. (E,,) isuniformly (/-t,,)-integrable iff for every sET and > 0 there isa compactsetK s T such that) F.
dP,..,
/-t,,(dt)> e } < e J T\"\\K dP.
p\",s {
II,S)
assoonassET\"and K
\037
T\".)
Let sET and Proof (1)Assume that (E,,) is unifonnty (\037,,)-integrable. a compactset K T and a sequence of tests E :F(D\",dII)' o < t: < 1.Choose
\302\243;
nE
N, in sucha way P.\".s(ft\"\" > 1
that)
82 ---
and
2)
J
\037,,(dt)
P\",,\302\253(P,,)
Tn\\K)
< (.2
2
assoonasS E 1;.and K 1;..Oenote) \037
.dP\"., - /-t..(dl)> l:}, nE N.
B\"I={ J
_
.I'\"\\K
dp\",s)
Thenit followsassoonas sET,.and K
-
) = (1 P..,s(B 1.. II
\037
+J q>,,)dP\",s
P\",Al
\037
q>\"dp\",s
B\
q>,,)
+
]
II
q>\"
\302\243.Tn\\K
\302\2432
\037\"2
T,. that)
:
dp\", .
j1,,(dt)dP\".., d1',..$)
r.
+ < e.) 2
set K (2) Assumeconverselythat there is a compact
\302\243;
T such that)
dp\"\"
J:,.s{ J -) T..\\K dI\".s)
\037,,(dt\302\273\302\243}<\302\243
> 0, assoonassET\"and K Note, that there are sets M\" E
for a given
\302\243
\302\243
\037r#\"
T\",
such that
P\",s(M,,)=
1
and)))
326
Chapter
10:Asymptotic
DecisionTheory)
tE T\", n EN. (Thepointisthat Mil isindependent \037.II\037..(lM.. \037.sl\037\"..(lM..t of t E 1'\" which is possible in view of separability.) Let) \037
< t}, Cn :=M..(I { J dP..., Il..(dt) Tn\\K dP...s) Then we obtainthat P,..s(C..) >1 J Tn\\K
-
t;
EN.
11
and)
dP\"., p..(dt)= Tn\\\".n P\".,(C..) J ), dP. d\037.$p..(dt) u
\".s)
-
= J J dP\".11l..(dl)dP\".s < Cn Tn\\K dP. \".s)
assoonassET,.and K
\302\243;
E;
0)
1\037.
of uniform integrability for posterior considerthe consequences distributions. w e need a lemma.) First, Next, we
lIs:
65,3Lemma. Suppose thaI (Ell)iseqlliconlinuous and Ihalp,,(U) > 0 ifU Tis open,n E N. Thenfor every
P,..s{ J B(r,6) a.\037
dp,.
of
E
T and eL'ery t > 0 there is
\302\243;
<5
0 > 0 such thut)
p..(dt)< 2 p..(B(St } < t 0\302\273
...s)
dP.\037-
soonasSET,.andif 0 < < f10') f1
36.12.ChooseSET
and let N (s) it follows N: S E 1',,}. Fromequicontinuity that for every t > 0 there > exists(jo 0 such that)
Proof Let
us apply Lemma
-= rnin {tlE
II
P,..s-P,..,<
if
II
tE
1\037
and
des,I) < ho.)
\037
choose0 > 0 suflicientlysmall to guarantee that B(s.f1o) provesthe assertion. 0) We may
<'5
\037
Tf'f(s)' This
A sequence of estimates 65.4Definition. Qn E .\037(E..,T)t n E\037, is uniformly setK T such that tight if for every sETand every 6 > 0 there is a compact < t K) < 1 P,.,s{e..(.
-
assoonas SE and K S 1\037
\037
r.)
\302\243}
1\037.
of Borelmeasures Throughoutthe followingwe assumethat the sequence K T.))) (Il..)converges strong1yto a BorelmeasurepifM (T) on every compact \037
65.Uniformly
327)
integrablecxperimcnts
andthat J.L(U) > 0 if U T 65.5Theorem.Supposethat (EJ isequic:onlinuous i!;open.Then (E,.)is uniformly (J.L,.)-inregrable of posterior iff the sequence \037
to distributions F,. with respect
J.L\",
n E
is uniformly
\037,
tight.)
Let sETand > O. By Proof(1) Assume that (E,.)is uniformly integrable. we may find 11> 0 such that Lemma (64.3) \302\243
\037.s{J
dP,.\"
/1n(dt)<,,}< 2 \302\243
dP.
\",s)
for sufficiently large n E N. By Theorem(65.2)choosea compactset K <;;;.T
such that
dP,..r P...s{ J dP. J.L,.(dt\302\273\302\243,,}<2) T,,\\K n.s \302\243
that) assoonassET,.and K 1',.. Then it followswith Theorem36.14 > c} P\".${F,.(T,.\\K) dP...1 = P...s{ dP...r J.L,,(dt)> J.L,,(dt)} \037
J dP. ,..s
\302\243
T,,\\\"
J
d
dP...rJ.L,.(dt\302\273c < 11 =P...s{S T,,\\\"
P...s)
e
}+2
dP.,..s)
forsufficiently large n E N. Thisprovesonepart of the assertion sinceuniform for a finite subset o f N is trivial. tightness distributions are uniformly tight.It is suffi(2) Assumethat the posterior cientto provethe condition of Theorem65.2for sufficiently large n E N. Let subsetsatisfying) Ko T be a compact \037
<1 \037.s{F,.(Ko)
-
\302\243}
:3)
and)
Jl,.(Ko) a> 0,) \037
for sufficiently large n E N. tebysev'sinequalityyields) 3 p.\".s. J dP,.,rJ.L,.(dt ) > Jln(Ko> < . c } 3) {\"0 dP.,.,oS \302\243
Choosea compact setK S T, Ko S K, such that) p.\",5 E,.( T,.\\K) >
{
\302\2432(1-
t)
3a }
<
\302\243
3)))
328
Chapter 10:Asymplotic DecisionTheory)
assoonass E
J\037,
K
Then it followsthat)
T\".
\037
Pn(dt) > t P...sTn\\K J . \037;./ } n,s
{
-
\302\243
\037
3
+ p.II.S
{
J
Ko
f.
F.t
+ P,..s
{
\"'\"'\"\"3
E \037
23
dP. \037 Pn(dt)
J Tn\\K
>
d\037\037
dP\"
.,Pn(dt)
2 f.
}
3 PII(Ko)
dp,.,$)
(1',.K) >
g
\\
f\037Ko)
2
3p..(K o) }
+ ?t.s F,.(T,. \\ K) >
{
r.
\03723+P,.,s {
F,.(T,.\\K\302\273
E2
-
(1 r.) 3J1(Ko) } [.2
o)
and that J1(U)> 0 if U T 65.6Corollary.Supposethat (En) isequicontinuous ;s a .'M(T)wheref T -+ is open.If (En) is uniformly (Iflp..)-inregrable measurahlefunction,thenfor every SETand t; > 0 there isa compactsetK s T \037
\037m
such that
P .S { J T II
If(t)!F.:(dt\302\273\302\243}<\302\243
..\\K)
assoonassET,.,K
\037
T,..)
ProofCopypart (1) of the proofof Theorem65.5.
0)
in the sequel.) importance The followingtwo lemmasare of technical
and uniformly (tln)-integrabl that (En) isequ;cont;,molls 65.7Lemma. Suppose Assumefurther that p(U) > 0 if U T isopen.Thenfor every sET,t > 0, and > 0 such thut) U ofs there exists\" every neighborhood lim P,.,.s{f\037(U) >,,}> 1 t.) \302\243
n--
-
\302\253J)
Proof Let sETand > O. Chooseb > 0 in such a \302\243
that for sufficiently large n
p..., s
B(s,b)
\037
U and
N)
dP,..,J.t..(dt ) < Pn(B(s,[))} < . 3)
J 8(s,6)
{
E
way that
dp,.,s
\037
2
This is possible by Lemma 65.3.In view of Theorem65.2we
may find
a)))
65.Uniformly
integrableexperimenls
329)
compactsetK s; T such that) { P,..$
e
dP\".,
f
T\"\\K
f:
> < dP. Jl.,.(dt) j } 3
SE
if
1'\",
K
\037
1'\".)
11..1)
Moreover, tebysev'sinequalityimplies) P,..$kr
dp\"
r
Jl,,(dt)
{ dP,.:s
We
- e-
> 3 Jl.,.(K)< B
obtainfor sufficiently large11E P.,..sF. ,.( U) < Jl.II(B(!i,
2
{
3'
}
\"
11
N)
/(
O\302\273
.
SE 1 , K S; III')
\037
3
+
3
Jl.n(K\302\273
)}
\037
dP,..t (dt) < Jl.,.(B(s, + - p. {f dP. 2 } <5\302\273
\037
JIll
\"..<
U
\"..1)
Choosing . . f J,l,,(B(s, < 1mm O < t1 = 2 <5\302\273
\\
\"\"'00
provesthe seertion.
_
/(3 \302\243
+
3
2\037
3
< 1:.
Jl.,,(K\302\273
\302\243)
)
0)
and uniformly (Jltt)-integrable. 65.8Lemma. Supposethat (E,,) isequicontinuous Assumefurther that p.(U) > 0 if U S;T is open.Thenfor every sETand every t > 0 there is C(r.)< 00such that
>
of posteriordistributions is Proof Let sET and > O. Sincethe sequence uniformlytight it issufficienttoshow that for every compactK T there exists C(e)< 00such that F.
\302\243
:::
p..,{F.(K\"{ > C(e)})>.}
P.\".$
{f
f. dP,.r ---=dt) < <( dP.\" 3 Jl.
11..1)))
<5
}
330
10:Asymptotic
Chapter
DecisionTheory)
for sufficiently largen EN. Let Bit..., of K and Ii e B. 1 i B,. be a partition such that k, \037
\037
-
IE B} < ' 1 i k.) sup{IIP,.\" p\"\".II: 6J111(K) \037f,2
\037
j
\037
The existence of sucha partitionfollowsfrom equicontinuity, from compact-
nessof K and from J111(I()J1(K)> O. Let C < 00be an arbitrary constant. \037
Since)
dF
J1\"
-\037.
{d
1 >C < = -.
nE N
} C'
J111)
,
it followsthat for sufficiently largen E N p,.,$
dP\".. J111(d/) > \037
f
dP,..s {Kn {\037;.c} ,,\"
< P\",s =
dP\".t
J
{Kn {d,,\"
}
4\302\243..>C
dP.\",S
dP,..I , + C 1=1dP,..s 2 } !\037
\037
.\037
> J111(dt)
dP 1-1dP.II,S J1 ( \"
II .\"
.\037
-1=1
p\".s
{
>c}dp,.,$
K\"{\037\037
- { -1 J :$:r.L. D II,S
t 2
\037
\302\2430
8'''{\037\037.>C}
dP\".t,
dp\",t
1-1
r
-
IIP\",t
\037
\037
t_ 6
Eo)
\037.
This implies)
(d )
f,\037
2} dP\".s\" 2 . oe2 = _e ' P\".,.IIJ1,,(dt)
dp\",$ 8.,,{\037:->C}
Ie
dp,.,$
---/11>-
\037
i
3
::
> K() P...{F.( { c})>.} d > $ 3t + P,.,s J dP. J1,,(dl) P,.\"
{Kn{ >c}
f,\037
II,S
}
\037\037:
\037
_23c +
p\",s
.. \037
\037e
3
+
dp,.r I:O >_ dP. C 2} 11 \",S { 1
A:
\"
.\037
i Ceo. \037_
i= I
J
J1\"
\037P\",'j
- 3 + l:!!_.
dP.\",S <
dP\".s
2\302\243
C\302\243o)
.
d t f we Ch oose Th e assert10n IS prove
C(\302\243)
> 6k 2.t' F.
0)
})
}
dP. d) .---!!:!. J1,,(dl)>-
dP. \037 J1,,(dl) L Ie
\037
d
{
I;O
>C + 2\"
df\037
\037('\\
o)))
2}
66.Uniform
of generalizedBayescstimalCS
tightness
331)
66. Uniformtightnessof generalizedBayesestimates) the casewhere(T, . I) isa finite dimensional In the followingwe consider linear = measureon T. In addition, the situation isthe spaceHnd 11. A.T, the Lebesgue the in section. sameas the preceding Call (E,.)unifonnly (Jl,.)sequence if for g:11-+ o f order it is ItIP. Let p uniformly (g Jl,.)-integrable integrable W (0) = 0, be a lossfunction.) W: T -+ [0,00), I
66.1Remark. In the followingwe denote
-
2,.(1):= J Wet x)f;.(dx),t E T,..Il EN. We
64and 65.) shallpermanentlyrefer to Sections
66.2Definition.A sequence ofestimates of (!n E fJi (En' T,.),n E N, isa sequence if it is a of estimates Wand (for asymptoticBayes sequence asymptotic n EN, for every sequence for minimumestimates (P,..s)' T.) (Jln\302\273
.\\\"E
(Zn(t\302\273IE:T\",
66.3Lemma. Assume that (E,,) is equicontinuous and uniformly (Jl,.)-inregrabl W 1. is order and order level of p of separatingthen for every If p, compact h < supW such that (; > 0 andevery SETthere exists \037
P,.,s{Zn(s) > b}< f: assoonas SE
1\037.)
Proof Let sET and > O. First,we assume that W is not bounded.From Corollary65.6it followsthat there isa compactset K STand > 0 such that) E;
P,..s{J
W(s
-
\037
> c5} < I; I) F;.(dt)
T\"\\K)
assoonass E 1',.. K f.;; 1;..Now the assertion followssinceW(s on K.
- .)isbounded
that W is boundedand separating. Next assume Let c < supW be such that = U {sE T: W(s t) c} isa neighborhood ofs.From Lemma65.7we obtain > 0 such that)
-
\037
Yl
< 0i}< I; Pn.s{F,;(U) assoonas U S W.l.g,'1> 0 can bechosenin such a way c < supW V;;,From
-
J W(s
7\037.
that
- t) F,.(dt) supW. F,,(1',.U) + (sup - V;;) = supW(lsupW. F;,(U)- 0iF;,(U) \037
W
\\
- V;;.
F;,(U\302\273
= supW
+
f\037(U))))
\037(U)
332 it
Chapter 10: Asymptotic DecisionTheory)
followsthat) \037.s{JW(s
- t)F;.(dt)> sup W
t1}
\037
< 01}< t. p\".s{F;.(U)
0)
66.4Theorem.Assumethat (E,,) isequicontinuous and uniformly (jJ,,)-integrab/e > W 1. that order is order of p As.\\\"Umefurther of p,levelcompactand separating. Thenfor every SETand every e > 0 there existsa compactsetKt T such Ihal) { inf Z,,(x) inf Z,,(y)+ e} < e P...s \037
\037
:;crT\"'''\"
assoonasS E 1;.,Kc
yrT\
1;..)
\037
Proof Let SET,t: > O. By Lemma 66.3there existsb o E (0,supW) such that c t) F,,(dt) fb} 1 P,..s (J W(.\037
-
\037
\037
-
2)
assoonasS E Reducinge> 0, if necessary, we achieve and such that (bo + t:)/(1 t:) < supW. Let C s;; T be compact e \037.s {J\037(C) < 1 e} <
-
1\037.
-
2)
assoonass E 1',.,C 1',.. This is possible by Theorem65.5. \037
Let
-
-
> 1 e}.) {F;.(C) > 1 e. Now we choosea compactsetKc s;; T in such a way Then P\".s(A\".c) A\".t:={IW(s
-
that)
W(x
-
t) >
t)
_
bo +e 1
-t:)
F;.(d/)
\037
bo} (\\
whenevertEe and x Kc. \037
sinceW is levelcompact. If x f Kt then we obtainon AII.t This is possible J W(x \037
As a
-
t)}\037(dl)
J W(s
-
\037
cJ W(x
t).Fn(dl)+ e
-
\037
t)}\037(dt)
> bo \037 F;.(C) bo + e
inf J W(y
)lET\
I-t:
-
t)
f\037(d/)
\037
+ e.
0
of asymptoticBayes we obtainexistence and tightness consequence
estimates.)
66.5Corollary.A\037'sume that the conditions of Theorem66.4are satisfied. for Wand (jJ,,). (1) Thereexi.'It.\037equencesof asymptoticBayesestimates (2) Every sequence of asymptoticBayese\037.timatesfor Wand (Jl,,)isuniformly tight.)))
67.Convcrgenceof generalizedBayesestimates
333)
Proof (1) Apply Theorem64.8. (2) Apply Theorem64.t 2.
0)
67. Convergenceof generalizedBayesestimates) the generalcasewhere(T,d)isa metricspace.Let usconsider First,we consider the situationof Section65 but assumeadditionallythat the sequence of experiments (E,.)converges weakly to an experimentE c cr (T). a is basicapproximation lemma.) Our first assertion
67.1(.emma.Supposethat
-.
Let K <; T be compactand (Ell) is equicontinuous.
and bounded.I1Jenlor every t > 0 there are a (T)-measurable measurablepartitionB\\, B2\"'\"BAI of K, poi1J/SIj E B, 1 $;i M, and E IR, 1 i M, such Ihat)
f
K
IR'\"
f?4
\037
j
\037
<Xj
\037
P,..sicf(l)dP p,.(dr). d\037.r
r
II
wheneverSET\"and K
s
\037
M rJ.
dP\".r j
j\037l
J\037,.(Bj)
dP,..s
<
t;)
T\".)
Let t > 0 and K T ProofThe proofisalmostthe same asfor Lemma36.16. =. = It isclear t hat n <00. I fC E C 0 orIIfll..= 0 then compact. sup{p\"(K): N} the assertion is trivial. Assumethat C>0 and 11/11.. > O. Fromequicontinuity of (E,.)it followsthat there existsb > 0 such that) \302\243
\037,s
dP.,..s ,..s dP'-
dP\".I' dP\".I:<
t .- I.f d(t..( 2) < a, 1\\,1 2 E K, 2CII/I'..) \037
whenever sET\"and K T\". Sincef is boundedthere M = and g L 1B. such thatllgll.. 11/11.. \302\243;
(Xj
i-I)
is a step function
\037
-
M
L sup l(t) j\"'l rEI(
rJ.
j
18
_.' r.
;(1) < 2C)
i M, may be chosenin sucha way that diam B; < b, 1 i followsby M,and that {Bj : 1 i M}isa partitionofK.Nowthe assertion the chain of inequalities in the proofof 36.16. 0) already considered
The setsB;,1 \037
\037
\037
\037
\037
-
\037
67.2l..emma. Supposethat
f
K
Rm
-
Let K (E,,) is equicontinuous. fJI (T)-measurable andboullded. If E,. E the\)
\037
T be compactand
Chapter 10:Asymptotic DccisionTheory)
334
dp\"
f
,
JlII(dl)IP\",\037)
ff'(t
-+
dp'
f
ff'(i\\f(l)dl'sJl(dl)I\037))
weakly,)
for every SET.) I.el) Proof By Lemma60.7the limit experimentE iscontinuous. lfJII
dP,..t (dI), (I) If d P. I(
=
-
JlII
n E
N,
and)
\".S
CPo
= J f(l) dp'Jl(dt). d Ie Ps)
for a given e > 0 choose a partition81, 82 \"\",8M of K, pointsIi E B Moreover, 1 i M, accordingto Lemmas67.1and 36.16. and Denote) \037iE\037,
\"1'\"
\037
\037
M
=
dP,..t, \"Jll! (\037),
.L dP 1-1 \"
CXj
nE
t\\J,
and
\".S)
dp,j = 11'0 ,-1 dP. J1(Bi)' \037
CX
.\037
j
oS)
Thenwe have) -+ !I'(lpol\037)weakly, !I'(Ip,,/P\".s) < assoonas s E 1',.,K S 1',.,) lip\" P,..\037
-
lfJ\"
I
E:
and) \03711PO
-lfJol< e.)
Since > 0 is arbitrary it followsby easy arguments that) 1-;
\037(cpIIIJ\037.s)
-+ \037(CPoll\037) weakly.
0)
are the basicfacts for limit theoremson Bayes The followingtwo assertions
estimates.)
that (E,,) isequicon/inuous anduniformly (JlII)-integrable 67.3Lemma. Suppose Then E is Jl-integrable.)
Proof By Corollary36.4it is sufficientto show that = 0, SE T.) p. fJ(dt) = {J :;()} Let sET,c > O. Choose a compact setKt S T satisfying))) \037\037
67.Convergenceof generalizedBayescstimates
-
dP,..,
335)
f.
J dP. p,,(dt)> & < 2 n.s) } {T\"'K
p\".s
assoonasS E Tn, it
Kt
\037
1\037.
Thisispossible FromLemma67.2 by Theorem65.2.
followsthat for every compactK ;?Kr) dp'
f>s
&
p(dt)> e < 2 {K\\K.J dP. } 5)
that) which implies
dp'
= . p(dt)> & < J dP. } 2 {T\\Ke s)
\037
\302\243.
Hencewe obtain)
!,(dl)> C + 2
P,
{I
-
\037\037
\037
Ps
!'(:.) }
dp'
J d p(dt)> T\\K. Ps {
e
e
<2+2<&'
\302\243
-
2--
+ Ps J tip, p(dt)> Jl(KrJ } {K. dPs } f;)
0)
-
that (En) i.'i equicontinuous and uniformly (Pn)-integrahle 67.4Lemma. Suppose oforderp 1.Assumefurther that Jt (U)> 0 if U s; T isopen.Iff T !Rill isof orderp then) \037
-+ .sf<Jf(t)F(dt)I\037) weakly) if<Jf(t)\037(dt)IP\".s)
for every sET.) Proof Let seT. It followsfrom Lemma 67.2that for every compactK with SE /()
-
weakly.) if<JJ(/)\037(dllK)I'p\",s) !l'(Sf(t)F(dtlK)I\037) Since)
-
IJf(t)\037(dtlK) Jf(t)\037(dt)1 + J r;,(\037\\K)JJ(t)\037(dtlK) \037
-
and)
Iff(t)F(dtlK) Jf(t)F(dt) + F(T\\K)ff(t)F(d/lK) \037
f(/)F,.(d/)
T\"\\K)
I
f T\\K)))
f(t)F(d/)
\037
T
Chapter 10:Asymplotic DecisionTheory)
336
followsfrom Theorem65.5and Corollary65.6that for every t > 0 there isa compactsetKe S T suchthat) it
-
Prt,sjJf(t)f;.(dtIKt)J/(t)\037(dt)1< t)
assoonasS E 7;.,K( S T,., and)
- J/(I)F(dt) < 1:.)
\037IJ/(t)F(dtIK\302\243)
I
Now) a similarargument as in the proof of Corollary67.2 provesthe assertion. 0)
-
Fromnow, we assumethat (T) I. I) is a finite dimensional linearspaceand = Let W: b e a T [0,00) lossfunction.) Jl AT' and uniformly (p,,)(E,,) ;s equ;conl;nuous 1.Let W beof orderp. Then Ihe slochastic processes)
67.5Theorem.Supposethut integrableof orderp
\037
tE\037,nEN,) Z,,(t):=JW(t-x)\037(dx), in (P\".s)-probability areequ;continuous for every seT.)
ProofWe have to show that for every sET,x E T, and every > 0 there exists E:
0> 0 such that)
-
> t} sup IZn(Y) Z,,(x)1 P....s{ B(.'(.6))
<\037:
YE
assoonassET\",B(x)\037) c ChoosesET,x E T and t> 0 arbitrarily.Let .5) > 0 be such that W(y t) ;:;;C3 tiP + C4 ifY E B(x,01) and t E T.Thenwe may find a compactset K 5 T
-
1\037.
1
such that)
P\".s
sup
J W(y T..\\IC
{YEB()I;.61)
as soon as SET\" and M21=sup{W(y - t):
Y
E
K
s
- I) F,,(dt)> 4e
}
T.w.
E
<3 .)
This is clear by Corollary65.6.Let
B(x)\037)) t E K}. By Lemma 65.8there is M) < 00
satisfying) E;
f.
> Mt > 4M < 3) 2} { {dJl\" } assoonassET\".Let O2 > 0 be suchthat) P...sF\"
sup )'\037
df\037
f B(x.(h)k
I
W(y
_
-I)- W(x - t)\\AT(dt) < 8 e
M))))
67.Convergenceof generalizedBayesestimates which implies
sup kf W(y ,..B(x.62) I
- - - t)IJl,,(dt)< t)
W(x
1;
4 J\\1
I)
b = min {<5h 2} then for sufficiently large n E N. If we choose
it
<5
P\"., sup
{
Yf'
\037
}
sup kf IW(y-t)-W(X-t)lf\037(dt\302\273\037 P,..syeB(x,6) -
}
{
{
W(Y-/)\037(d/\302\273\037
}
>
F. P...{M2' {:\037 M,} + Mt' sup YF
\037
followsthat)
IZ\"(Y)-Z\"(X)I>E:
B(x.6)
sup J +2P\".J)'eB(x.6) T..\\K \037
337)
B(x.6)k
P...sM2 . F;,
{
f
I
W(y
- /) - W(x - t)IJl,,(dt)> 2t-
}
dF,. > M
i;
> < } 4 } +\"4
_2t2
2\037
1
{dll\"
+
E;)
for every for sufficiently large n EN. Since(Z,,(X\302\273xeT..iscontinuous P\".s.a.e. follows. 0) fixedn c by Lemma 37.7,the assertion \037
and uniformly (p,,)(E,,) is equicontilluous 1.Assumefurther that W is\037'eparating, level-compactand
67.6Corollary.Supposethat integrable oforderp of orderp.
\037
11EN, isstochastically (1) If a sequence ofestimatesE !it(E\",1;,), equivalent isalso with a sequence then a estimates of asymptoticBayes (0',,) sequence of (f\"
asymptotic:BayesesLimates.
-
If the stochastic processZ(t):=J W (t x) F(dx),t E T, attaillsits insET,then every sequence fimum in at mosta single of asymptotic point\037-a.e.. to the generalized Bayesestimates convergesin distribution of E.) Bayesestimate (2)
Proof(1) Apply Corollary64.13. (2) Apply Theorem64.15. 0)
67.7 Corollary.Assumethat the conditions 67.6(2),are satisfied. orCorollary
Then every sequence which convergesin distribution to the geneof estimates ralizedBayes estimate of E is a sequence All of asymptolicBayes estimates. estimates are of asymptoticBayes sequellces stochastically equivalent.)))
338
Chapter
10:Asymptolic
DocisionTheory)
it Moreover, Bayesestimate ofE isnon-randomized. ProofThe generalized itsdistribution since the distribution is uniquely detcrmined determines) by \037
M:A
1-+J II W(x
-
t)e\302\253(JJ,
dx) P'(d(JJ)p(dt),
A
E.G',
A)
and)
-
dM = JJ W(x t)Q(.,dx) F(dt).) dP(\037j
lienee,Theorem 63.6impliesthat all estimateswhich convergeto e in are stochastically In particular, distribution equivalent. they arc stochastically of asymptoticBayesestimates which existsby equivalent to a given sequence 67.6 66.5. the assertion f ollows. Corollary By Corollary 0)))
11:GaussianShiftson HilbertSpaces)
Chapter
In the precedingchapterswe have met Gaussianshifts several times.The rolewhich theseexperiments distinguished play in the classical theory isdueto
the fact that they admit a particularly simplestatistical analysis.Roughly However,in speaking,their theory can be presentedin terms of linear algebra. and came the that m ore more statisticians to conclusion the lastdecades data almostnever originatefrom Gaussianshift experipracticalstatistical ments.Nevertheless, of this type are stillof centraltheoretical experiments The reasonare the resultsof asymptotic statistics. significance. The asymptotictheory of statisticsdealswith localapproximations of for large samplesizesby simplerexperiments. Therebyit turns out experiments conditions limit experiments are usually Gaussian that under mild smoothness
shifts.
Forthe application of the asymptotic decision theory of Chapter10tocases where the limit is a Gaussianshift we need a complete statistical theory of In caseof finite dimensional Gaussianshift experiments. parameter spacesthe are results scattered o ver the pertaining precedingparts of this book (cf. sections28.30,34, 38).The asymptotictheory of non-parametric methods which are Gaussianshiftswith infinite leads,however,to limit experiments is dimensional This the reasonwhy we take up the theory of parameter spaces. Gaussianshiftsagain and presentit general enoughto coveralsothe infinite
dimensional case. The caseof infinite dimensional parameter spacesrequiressometechnical in Section with linear processes toolswhich are collected 68.It is concerned and Section of a Gaussian 69dealswith the generalconcept cylindersetmeasures. shift experimenton a Hilbertspace. We characterize the likelihood of a process Gaussianshift and show that linearprocesses define generalized decision functionsin a canonical way. Gaussianshift experiments are invariant under the translation groupof the I n is of Hilbert case the Hilbert finite dimension the space. underlying space Gaussianshift can be represented asfull shift on this finite dimensional Hilbert But if the Hilbertspaceis space,generatedby the standardnormal distribution. notof finite dimension then sucha representation isimpossible sincethere isno standardnormal distribution on such a Hilbertspace.If the Hilbertspacecan becompleted in a suitable it becomes to possible way (Abstract Wiener spaces), the Gaussian shift an lion of the Hilbert generate experimentthrough opera on the T hese space samplespace. questions, togetherwith someimportant are discussed in Section 70.))) examples,
340
68.Linear stochastic processesand cylindcr set measures)
In Sections 71 and 72 we presentthe theory of testingand estimation for with Gaussian shifts. S ection 7 3 deals the c ase of Abstract general particular Wiener spacerepresentation which admitssomesimplification. The contentof this chapteris well-known.The first presentation in the from in can literature our viewpoint be found the thesisof Moussatat[1976] which waswrittenunderthe guidance ofLeCam.Fromthere we take the proof of Theorem71.14. The thesisof Moussatatalsocontains a particularcaseof the minimaxboundgiven in Theorem73.6. Thistheoremisformulatedfor the to first time by Millar L1979] resultsof LeCam.The referring unpublished shouldbe general minimax boundof Theorem72.7, althoughunpublished, welJ-known,too. A
of the subjectwith a similarintentionas oursis Millar presentation
[1983] .)
and cylinderset measures) 68.Linearstochastic processes
...
Let (H,< ., .
H
be a separableHilbert space.Any linear functionZ: 1i'(0,d),where (0,d) is a samplespace,iscalledlinearprocess.) \302\273
H be a measurable 68,1Example.Let K: Q... mapping.Then
Jz E H, defines a linear process, Z (h):=(h,K To exhibitanotherimportantexamplewe needsomebasicfacts onGaussian (.\302\273,
processes.) Let T:f= 0 be an arbitrary set.A stochastic 68.2Discussion. process X = (0,d, P,(X')'ET) is a Gaussianprocess if all finite dimensional marginal of X arc Gaussiandistributions. The function) distributions
/).-J (Xs - P(Xs
K:(s,
\302\273
(X,
-
P(X,\302\273dP,
s,t E T,)
i.e.) the covariance isthe covariance of X. Obviously, K ispositivesemidefinite, It
It
i-Ij-l) \037
\037
(Xj(Xj
K(tj,t j) 0 \037
...,
..,
for every choice (t 1> /,JE T', . (X,,) E R'. As a matter of fact, every positive semidefinite symmetricfunctionK on T2 is the covariance of a Gaussianprocess. To seethis,let Q = \037T, = (R)T and definefor every (Xc A(T) a probability measure \037I\037(\037)Q! by) \302\253XI'
d
J exp
; (L ) xrJ\037
'\037Q!
\037(dy)
= exp
(
.-1 L 2
S\037Q!
L x(x,K(s,n , ItHI)))
)
\037
68.Linear slochaslicprQCe$SC$and cylindcr set measurcs
341)
(x,),uE W. Then it is easy to see that (\037)a.A(T) is a projectivesystem of limitand measures. DefinePI\037(Rlto be the projective Gaussianprobability ta ke X,
= PH
tE
T.
in ariseswhen T itselfis a Hilbertspace The casewe are interested
In this case<.,. ) is positivedefiniteand thereforethe covariance ., function of a Gaussian (H,< .
\302\273.
process.)
68.3Definition.A GaussianprocessX = (0,d.P.
(X(h\302\273,,<;
Gaussianprocess (for H) if
H)
is a standard
(1) JX(h)dP=O, hEH, (2) J X(h l ) X(h 2)dP =
...
68.4Theorem.Let X: H L2 (0,,f;/, P).ThenX isa standardGaw;sianprocess hE H.) \037(X(h)1P) = iff X islinear andsatisfies \\'o,lIhll\037'
Let (al Proof (1) Supposethat X is a standardGaussianprocess.
and (hi'\"h,,)E H\",n E N. Then f
(.t.a,
dP =
X (h,\302\273)'
,t,
i,ala
Jf
\"
\"
\037
\037
...a,,)
E
R\"
X(h,) X(hJ)dP)
..
= L... L...a.a.
=
II
2: C(j h i=
i Il
2
.
I)
\"
\"
Hence,L ajh = 0 implicsL \037jX(hj) = 0 P-a.c. 1=1 i=l that X islinearand (X(h) I P) = (2) Assumeconversely, C(,,) E Ifi\" and (hi'\"hI!)E H\",11E N. Then we have) (C(t j
9'
...
=
fexP(i.t
a.X(h.\302\273)dP
1\" - 2: -. 2 (
=exp
11
\037JchJc112
Jc=1
\302\253h\",
hE I/. Let .I,,\"1,
fexP(iX(.t,a.h.))dP)
)
=exp
... h,)}
Thisimplies that the vector (X(h l ) and covariance matrix
vo
I \037a.'
X(hll
\037\
(
\302\273
-.21\"\" L L C1.JcC1.,
)
isjointlyGaussianwith mean zero o)
68.5Example.IfH isof finite dimension then any standardGaussian process 2 X: H... L (Q,d, P) can bewritten asX(II) (h.K).h E H, whereK: 0 His))) \037
-
342
Chapter
11:GaussianShifts on Hilbcrl Spaces)
a random variable.E.g.take an orthonormal base(el II K 1= X(ej)ej .
'\"
ell)
of H and define
\037
i= 1
-
HilbertspaceH a It will turn outbelowthat in caseofan infinite dimensional be derivedfrom a can never standardGaussianprocess mappingK: Q H.)
68.6Definition.Let!fbe the system of finite dimensional linear subspaces L s:H. A family ofBorelmeasures \037LI\037(L), L E 2',isa cylindersetmeasureif it isprojective, i.e.if Ll L2 impliesJILl = !f(PL\\1ilL).) \037
68.7Examples.(1)The system of standardGaussianmeasures N,.,L E :.e, It a iscalled the standard set measure Gaussian defines cylinder Nil:(N')'.f!:/.\"
-
distribution on H.
linearprocessZ:II 9'(0,d) definesa cylinder set measure to the followingconstruction: Forevery L E !f'let ZL: Q L besuch according that 2(h)=
\037
\037L
Z under P(=df(ZIP\302\273.
isthe ofa standardGaussianprocess It iseasy to seethat the distribution standardGaussiandistribution.) (3)
68,8Discussion, In a separable Hilbertspacethe Borel-cJ-fields of the weak and the strongtopologycoincide sincethe closedballsare alsoweakly closed. of H (=dl(lf\302\273. Therefore,it is not ambiguousto speakof the Borel-a-field A Borel set B II is a cylinder set if there existsL E!I!such that B = (BnL)
\037
\037,
1
\037
\037L
Borelmeasure.)
68.9Theorem.If H is of infinite dimensionthen Nil cannot be ex/ended/0 a Borelmeasureall .91(11).)
ProofWe shallshow that the outermeasureofevery balliszero.If Nil couldbe to extendedto a Borelmeasurethen the measure of the ballsshouldincrease radii tend to infinity. oneas their baseof /I. Define Let r> 0 and (ej)je an orthonormal n 8 = {IIE H: L <\",ej>2 r 2},n E N. Then BII!B(O,r) as\" 00.It will turn j= out that Nil(B ) t 0 asn 00.))) 1\\1
11
\037
I
II
\037
-
69.Gaussianshift
cxpcrimcnls
...
The set Bn is a cylinder setwhose baseis the ballof radiusr dimensional subspace Ln = span{e1 e,,}.Therefore,) N lI (Bn) =
the finite
n) NLJB\"r.L 1)
J
n
(2n)2) \037
in
343)
\037
... J)
x\037
\037
exp
\
(-\037 it,X?)dX , ...dx o)
,-\\)
1 , A.,,(B(O, =. - ----,.20\"r (2,,)2 r\302\273
--.-.
r\"
(1+
\037))
which tendsto zeroas n -+ 00.The assertion follows.
0)
68.10Corollary.If H isofinfinite dimensionthen a standardGaussianprocess cannOIberepresented asX(h) = Horelmeasurablefunction.)
(X(h\302\273hH
(h,K(. hE H, whereK: Q -+ H isa \302\273,
a standardGaussianprocess, Proof.If there existeda mappingK generating m easure itsdistribution a Borel on 9f(H)with the would be coinciding standardGaussiandistribution. By Theorem68.9,such a Borelmeasure does not exist. 0) then
69.Gaussianshift experiments)
(.,
is a Euclideanspace.The standardGaussianshift on Supposethat (H, is the experiment(; = (H,rJI(H),{NH * Ch : h E H}).) (H,
(.,
.\302\273
.\302\273
is an (., experiment t!(H)which is equivalent to the standardGaussianshift on (H . . a separable Hilbertspace. extendthe In the following(II,(., denotes definition.)
69.1Definition.A Gaussianshift on a Euclideanspace(H,
.\302\273
in
t
< ,
\302\273.
We
.\302\273
preceding
A Gaussianshift on (H,< . , . is an experiment E E G (H) 69.2Definition. such that for every finite dimensional L subspace H the experimentEL is a \302\273
\302\243
Gaussianshift.)
69.3Lemma. (1) Any equivalent.)))
Iwo Gau.5sianshifts on the same
Hilbertspaceare
344
Chapter 1I:GaussianShifts on Hilbert Spaces)
(2) Every experimentwhichisequivalent toa Gaussian shift isitselfa Gaussian shift.)
are true for Euclidean Proo}:It is obviousby definitionthat the assertions the followsby considering the restrictions to spaces.Hence, general assertion finite dimensional 0) subspaces.
otherwords,the propertyof beinga Gaussianshift is a propertyof and there isexactlyoneequivalence ofexperiments classfor equivalenceclasses eachHilbertspacewhich containsGaussianshifts. Thereis a strongrelation betweenGaussianshift experiments and standard processes.) In
69.4Theorem.An experimentE = (D,d, {Ph:hE H})isa Gaussianshift iff the stochastic (X process \" which isdefinedby) (h\302\273hI!
-
dP = 1 hE H,) exp(X(h) 211h1l2), dP; under Po.) isa standllrdGaussianprocess and thereforeits equivalence classis Proo}:In any caseE is homogeneous determined of show that for a by the distribution H'Easycomputations c finite dimensional L H subspacc (X(h\302\273,,(
d(NL * f.h) = dNL
.
exp\302\253h,
- 2\"hll), he L. 1
IdL.)
2
observethat the distributions of I.underNL and (X iff isa standardGaussianprocess. 0) Po coincide
We
\302\253h,
idL\302\273he
(h\302\273he
I.under
(X(h\302\273hell
It shouldbe notedthat on a given probabilityspace(D,.rd,P) there exist which are differentfrom eachother.On several standardGaussianprocesses with a particular the otherhand the standardprocesswhich is associated P-a.e.) Gaussianshift is uniquely determined
69.5Definition.Let E be a Gaussianshift.The standardGaussianprocess of E.) associated with E iscalledthe central process
69.6Corollary.For every standardprocess(X(h)ho;H there existsa Gaussian is central.) shift Efor which (X(h\302\273h4iH
of (D, Proof If (X(h\302\273hH is a standardprocess and Phld',hE H,
\0379/,
by)))
P) then we define Po1=P
69.Gaussian shift
experiments
345)
1
dP\"
2 hEH. dRo) =exp(X(h)-2I1hll),
It iseasy to seethat (D,.>I,{P,,:hE H})is an experiment.
U)
can bewritten If(H,(.,.\302\273isa Euclidean spacethen a linear process asX(h) = (h,X),hE H, where X: Q 1/isa random variable.If H is central for a Gaussian shift E then letus callthe randomvariableX central for
-.
(X(h\302\2731IE1I
(X(h\302\273'H:
E, too.)
(.,
69.7Examples.(1) Let (H, . be a Euclideanspaceand E = (H,fB(H), {P,,:Jz E H})the standardGaussianshift on H. Then X = id H iscentralfor E. (2) Let H = IRk and (x,y):=x'fy where f is a positivedefinite (k x k)matrix. Then Nil = r I. Since) dv\" .I'= dvO.r-1)(x) exp(h' I x 2 h' f h), h E H, x E H, \302\273
--
I
the experimentE = (IR,\\ X = id H iscentral. (3) Let l/ = Ilk and
Thenwe have)
-d n.,.
dV
-
,1 -
\\10,
{vlt
\037\",
.r
,:hEIR\"})is a standardGaussianshift and
(.,.as before
but let
>
x- 21hI r h),
(x) = exp (hI
\\10,1')
E = (IR\\ 14k, {\\In.r: hE R k }).
hE H, x E H,
-,
which implies that E is a Gaussianshift and X = r idll iscentral. Note,that E is not a standardGaussianshift on (H,( . (4) Let (H,( . be an arbitrary Hilbertspaceand L H a finite dimensionalspace.Let E = (Q,.>I, {P,,:hE H})be a Gaussianshift on Hand EL its If ofE and if k\" restriction to L. H is the central process e,.}isan
.,
.,
\302\273.
\037
\302\273
...,
(X(h\302\273\"E
orthonormal basisof L then) XL =
,. j-L,) X(ej)e j
isthe centralrandom variableof E,..Thisisdueto the fact that for each hE L) X(h) = X(
,.
L, (h,ei)ej)
i\"\"
=
/I
,.
L (h, = (h,1:=') L X(el)e/). ;=1X(el) e)
If Lit L2 are finite dimensional and L, L2 then it isclearfrom subspaces abovethat XLI = PLIoXL:' then))) (5) If (/I,( . is a EucJideanspaceand L H is a ]inearsubspace, \302\243
.,
\302\243
\302\273
346
Chapter
11:GaussianShifts on Hilbert Spaces)
E = (H,91(H), {Nil.e,,:hE L})isa Gaussianshift and PL isitscentralrandom variable.This is the caseconsidered in Section 30.)
...,
69.8Remarks, (1) Let us computethe Hellingertransforms of a Gaussian shift.For (ZI, z,,)E S\" and h E H, 1 i n. we have) \037
j
\037
.; = 1 L(hj) - 211hdl2 dPo) dPo J exp J j1)1 )) ( ( ( ) 1 1\" 2 = cxp - - L zill h dl 2 + -- L zjh 21=1 ) ( 2 j =1 2 1 . J cxp L _ zjh j ,r. :i:. ) 2 1-1) ) dPo ( (,-I dP
\"
\"
Zj
i\037l
d\037
\"
j
Zjhj
= exp
1 ( 2
\"
2 zjllhl 1l +
t\037l
1
i
n
zlh
l)
I\037l
})
Conferthe result of Example5.11. betweentwo measures of a Gaussianshift is) (2) The Hdlingerdistance 1 1 2 2 h dl2 +lIh2 2 d2(PItI,Pltl)=I-exp -4(lI )+Sllhl+h211
(
=
t-exp
)
11
(-\037lIhl-h,II')'h,.h,EH.)
It followsthat any Gaussianshift is uniformlycontinuous. invariant. This followsby easy (3) Every Gaussianshift is translation transfonnsgiven in (1).) from the Hellinger computations and 69.9Lemma. Supposethat process 11 is a stochastic = E (Q,d, {P,,:hE H})an experiment.Then E isa Gaussianshift and (X itscentralprocess iff (X(h\302\273,,\037
(h\302\2731tE
II
(1) the distribution logdP\037 , X(h 2) Po is Gau.'isianwith expectation !\302\243
(
-
II \037
h
(
dPo
)
,II', andcovariance(h\" h,),hi E H, i = 1,2,andeither
0)
(2) E isa Gaussianshift or under Po.) isa standardprocess (3) (X(h\302\273IaEH
Assumeconversely, ProofIt isclearthat conditions (1)and (2) are necessary. that (1)issatisfied. Then it followsthat the variance of)))
69.Gaussianshift
dP;
dP log
X(h) +
347)
experiments
1
2 11hll2
under Po iszerofor every hE H. Hence, (2) and (3) imply each other.
69.10Theorem.A stocha.'ttic process
(X(h\302\273\"\"
E = (D,SII,{P,,:hE H})iff)
0)
i'icentral for the Gaussianshift
II
2'(X(h1)IP\"2)= V(I...II2>.II\"IW for a/l
hi
E
H,112 E H.)
is central.Let hi E H, h 2 E H. Then we Proof (1)Supposethat invariancethat) obtainfrom translation (X(h\302\273\",,\"
= !t' 2'(X(h,JlPo)
(lOg\037:
+ IIh,lI' Po) \037
= 2' + 11\",11' (lOgd\037\037::, PA') 1 2 = Y X(h 1 1 + 11hz + +h 2 2 2\" 111 PA')) 211h1 ( \037
)-
\"
11
21h2
)-(h ,h )IP ).)
= Y(X(h1 h2 E
1
2
It2
that .5f(X(h1)IP\"2)= \\'<\"I.II2).1I1t1l12 for all hi (2) Converselyassume H. A standardargument showsthat for every hElland C E R)
-1
1( \037
1:\037\037>uP(c-illltl'2)}
(X(I.\302\273c)
-dP\"
)(dPo
-21 (
exp c
11
E
H,
> -a e )) = 0 0\
h II 2
Po
the Po-expectations of theseexpressions vanish. It followsthat Moreover, -l{x(h\302\273cd Poll{los +illllIl2>c}
= 0)
\037;\037
for every h E
Handc E R. Thisimplies that)
1 dP\" _ 2 10gdP -X(h)-21Ihll Po-a.e.,hEH.) o
o)
(.,
69.11Corollary.SupposeIhal (H, is a Euclideanspace.A random variableX: Q H ;s central Ihe Gaussian for shiflE = (Q,d, {Pit:hE H})iff
-.
\037(XIPh)= eh *
-
Nil'hE H.
.\302\273
If H is a finite dimensional Hilbertspacethen the centralrandomvariable H is a non-randomized decisionfunction in :?t(E,I/).In general, docsnot definea decision function in at(E,II).))) however,the centralprocess X: Q
Chapter
34M
11:GaussianShifts on Hilbert Spaces)
functionf H Iij iscalleda cylinderfunction if there is L E !Rsuch that Such a process ( H be an arbitrary linearprocess. fOPL = f Let Z = (Z definesa systemof randomvariables(ZL)Le\037which are connected by) A
\037
(h\302\273\"
ZLI = PLI oZLz if
LI S L2
.)
Z isof the form Z(I1)= (h,K), 69.12Remark. Supposethat the linearprocess hE 1/,where K: Q -+ 1/is a Borelmeasurable function.Then for every L E !f 0 = we have Z,. P,. K. If P E :lI(E, functionwhich isdefinedby H) is the decision K
then)
P(f,Jl) = ffo Kd}l= ffo PL 0 KdJl = ffo Z\"dJl)
iff E f/ b (If) is a cylinder function satisfying f 0 P,.= f for someL E !R and if Jl E L(E).For every L E !f' the random variableP,.C K =:Z\" isalsoa decision functionand since)
-P,.(x)= 0, x
lim !Ix
L-H) it
!I
E
H,
followsthat) lim ffo ZLdJl, fE rcb(H),Jl E L(E). P(f,Jl) = L--II)
In otherwords,P is the weak limit of (ZL)Le\037in
\037
(E,H).)
shallshow that for every linear processZ the limit of (Z,),.e2existsin to Gaussian dI(E,1/).It shouldbe notedthat the followingis not restricted We
shift experiments.)
69.13Theorem.Supposethat Z =
(Z(h\302\273\"EII
random variables (ZL)LCY converges in
pE
91(\302\243,
isa linearproceS.f. Then the net of
!?l(E,I/)to
a decisionfunction
H) sati.(jfying
P(f,Jl)= JfoZLdJl if f'\"PL=f, fE \037b(l/) and L E If, and Jl E L(E).) for cylinderfunctions Proof For every L E
!i'let
PL E \037(E,
H) be definedby
PL(f,Jl) = J/(ZL)dJl, IE fib(H),Jl E L(E).) Let P E pointof (PL)LEY' ThenP has the H) be any weak accumulation asserted property.To show this let LI E!f'andfef(lb(H) such thatfoPL,=1 L ;2L I , it isclearthat))) Then for every L E 14(\302\243,
!i',
69.Gau$sianshift
349)
experiments
= Jf(ZL)d\037= Jf(ZL)d\037 P,.(f,\037) = P'd(f, e L(E),) \037),
\037
that) which implies
P(f, = PL, (f, \037)
\037),
\037
e L(E).)
we obtainthat all weak accumulation pointsof (PL)LeY consequence To prove convergence we have to show that coincide on the cylinderfunctions. is determined weak accumulation by its value on the every point uniquely cylinder functions. Let fe f(fb(H), e L(E) and f; > o. If P is a weak accumulation pointof a n s uch that) then there exists L e (PL)L
!i'
\037
-
\302\243.)
Since)
PL(f,p) = Jf(Z,'>d\037
= J (fopJ (ZJd\037=
thisimplies
P(/\"PL'
\037),)
-
IP(f,p) P(fcPL, p)1 < E.) Sincef 0 PL is a cylinderfunction,the assertion follows.
69.14Definidon.If Z =
0)
is a linear processthen the limit iscalledthe decision pz E (E,H) of (ZJL\037!I' functiondefinedby Z. If Z = is a of fonn linearprocess the H Z(h) = (h,K), II e H, where K: Q -+ H is a Borelmeasurable mappingthen every linearfunctionf H -+ defines an image f\" Z: Q -+ by fo Z: W f(K(W)}, we Q. The random 0 variable f Z is characterizedby the propertyx.(f 0 Z) = Z (f. if x*e (RA:)*.Sucha construction can even becarriedthrough if Z isan arbitrary (Z(h\302\273hI;
H
\037
(Z(h\302\273\"c
IRA:
IRA:
\037
(x.\302\273
linear process.)
69.15Lemma. Let Z =
(Z(h\302\273ItEH
bean arbitrary linearprocess andf H -+ RA:
a continuous linearfunction.Then there exb:ts a randomvariablef 0 Z:Q -+ which isuniquelyde/erminedby the propertyx*(foZ) = Z(f*(x*\302\273,x*E (lR k)*.) IRA:
isclear.To proveexistence let H = L 61kerfand {e1 ProofUniqueness m an orthonormal baseof L. Definef 0 Z:= L Z (ej)f(ej)and notethat 1-1)))
...
em}
350
Chapter
11:GaussianShifts on Hilbert Spaces) \"'
x.(fn
Z) --,
;L=
Z(ej)x.(f(ej\302\273
I
=
\"'
L Z(e)
;'\"'1)
m
= Z C\037: i
= An
=
=
(f* (x*),e > e;) j
1)
Z(/*(x*\302\273.
0)
interestingparticularcaseariseswhen f H -+ R. Thenf E Hand /0 Z
Z(f).
the notation of the preceding lemmathe randomvariablesZL, L E of Example 68.7(2),definedby a linear processare nothingelsethan ZL = PL 0 Z, L E Moreover,it isalmostobviousthat (fog) 0 Z = f 0 (g 0 Z) if hothsidesare well defined.) With
!\302\243,
!\302\243.
69.16Remark. (1) Let L E
and let EdLbe the restriction of the Gaussian = = PL 0 X is shift E to the subspace L.If X (X H is central for E then XL centralfor ElL. casearisesiff H -+ islinear continuous andof rank k.If (2) A particular -1 L == (kerf)l. then P,.==f* 0 (f 0f*) 0f and hencethe central randomvariable 0 (foX). of ElL is/*0 (/0/*)-1 the second casewe reparametrizeL by flL:L -4 IRk which isa (3) Continuing In orderto get an isometry we endow with the inner linearisomorphism. productdefinedby the matrix of (fof*)-Iand obtainthatfoXisthe central variable of thisexperiment. To seethisfact,notethat) !\302\243
(Jr\302\273\"
,
[\037It
r
IIh 11
2
rf(h)
= f(h)'
\037Ic
if hE L,)
and)
X(h) =f(h)'ffo X if hE L.)
The underlyingexperimenthas representations given in Examples69.7(2) and (3).)
70.Banachsamplespaces)
(.,
If (H, . isa finite dimensional Hilbertspacethen there existsa Gaussian isof shift experimentonH whosesamplespaceis H and whosecentralprocess the form X(h) = (id,h), hE H. If H is not of finite dimension then such a is not possible.The reasonis that the standardGaussian representation distribution Nil cannotbeextendedto a Borelmeasure on !Jl(H).))) \302\273
70.Banach sample spaces
351)
likethe standardGaussian Intuitively, the reasonwhy cylindersetmeasures on H, is that H is too distribution cannotbe extendedto Borelmeasures in a suitable \"smalJ\" in somesense. Theextension works ifII iscompleted way. In many casescompletion isdoneby embedding H intoa Banach spaceB by
,:
means of a continuous, injectivemap H -+ B.)
70.1Remark. Let (B,II.liB) bea separable Banachspaceand -r: H -+ B a linear
to definethe image .!:f('rIN continuous mapping.Then it is possible II) of Nil under,in the followingway: A cylinderfunctionon B isa functionof the form l\037i\037k,kEN. If KE\037b(B)is a whereflRkand x!eB*, fo(x\037,...,x:) cylinder function on B then .!:f(,IN,,) II , (g) 1=Jgo-rdN
which iswelldefinedsincego E fGb(H) isa cylinderfunction on H.It isclear that \037(tINII)neednot be a Borelmeasure on \037(B).Analytically, we have Q = 2'(,INH ) iff 1\"
x.0 ,)dN
J exp (i for all
H
= J cxp(ix*)dQ
x.E B*.)
Banach spaceand T: H -+ B a 70.2Definition.Let (B,II.II B) be a separable linearcontinuous mappingwhich isinjectiveand such thatf(H)isdensein B.If can beextended to a BorelmeasurePI\037(B)then (H,D, t) iscalledan .!.f(tlNil) Abstract Wiener space. In the fo))owing we exhibitsomeimportantexamplesof Abstract Wiener in Theorems 70.3-70.5,it folJowsfrom spaces.In any of the casesconsidered the Stone-Weierstrass theorem that T(H)isdensein B.)
70.3Theorem.Let H = L2([0,1], \302\2431([0,1]),i.)and B= {xc:\037([O,1]): x(O) = OJ. r
Considerthe mapping H -+ B defined by ,(h):t J hdJ..,0 t 1. Then o with the Wienermeasurew: Z(tINH ) isa Borelmeasureon {fI(B)andcoincides Thus, (H,B,,) isan Abstract Wiener space.) f--+
1\":
\037
\037
ProofWe needonly show that W= Z(TINH ). To verify this we have to show that)
J exp(ix*0 T)dNH = J exp (ix*)dW
for all x.e B*.The dual spaceB consistsof all signedBorelmeasures on It issufficientto provethe equation of \037([O,1]). only for linear combinations since are in B*. Let) measure these dense one-point weakly \"
x* = L Ie
\"\"\"' 1)))
(lkgr\", (rx le
) ERn, (lk)E
[0,1]\".
352
Chapter
11:Gau$sian Shifts on llilbert Spaces)
We obtain) \"
J exp (ix*)dW= J exp(i
\037
W(dx)
ex\037Xt..)
1e=1)
1 L =exp - 2 ,::; ( L '''' \"
\"
_
t,} .
ex,ex,min {t\"
1
)
\\)
On the otherhand we have) '\"
\"
ex, r hd)')NH(dh) J exp(ix. t)dNH = J exp(iL 'po'1 6) 0
=
n
L ex..1[o.tr.)hdi..)N H (dh) Jexp(iJ(''''I) n
= J exp(i(L ex..' h\302\273NH(dh) 1[0.h,)' \"=1)
- 21\" . 1[O.h,)
2
= exp
o)
( ) 70.4Theorem.Let JI = {hE L2 ([0,1], .:M([O,1]),i.): J hdi.= O} and B = {xE 1J):x,(O) = x(1)= O}. Consideragain the mapping t: H -+ B C(\"
.)
'\037I
f\342\202\254([O,
N H ) isa Borelmeasureon !M(B) definedby T(h): t t-+ J hdl,0 t 1.Then !I!(TI o and coincides with the distrihution u.o of the Brownian hridge on B. Thus. (H,B,T) is an Abstract Wiener space.) \037
\037
ProofThe proofissimilarto the proofof Theorem70.3.Again taking)
x.=
\"
L
1=\\)
ex\"E,,,,
(ex/c)ER\",
we have o = exp J exp (ix*)dW and)
(tJE [0,1J\",
-, 1
-
\"
\"
(min {t\", I,} L L ( -1-11-1) CX\"iX,
n
. I,)
)
I\"
'\"
J exp(ix*0 T)dNH = J exp(iIe=1 J0) hd)') NH (dh) c(\"
\037
= Jexp(iJ(
,,=\\)
= exp
-
n \037
ex,(1(O.tlc) t 1\302\273hdi.)N
( .t.
-I.)2).
\037.(I,o.,.)
\037
H
(dh) 0)
distribution 70.5Theorem.SupposethaI F is a continuous function on Rand the mapping))) let H = {hE L2(R,\037,F):J hdF= O}.Let B = q,}o(R)and consider
70.Banach sample spaces
t: H -
353)
r
B definedby t(ll):tt-+ S hdF.tE IR. Then \037.:=\037(tINH);s a -/C
measureon 1I(B).Thus, (H,B,t)
Borel
is an Abstract Wiener space.)
Proof Let F*:{XE\037([O,1]):x(O) = x(1)= O} .....(Co(\037) be such that F*(x) = x 0 F, x E \037([O,1]). Define w;..:=\037(F.I\037). In other words, if is the distribution of the (D,d, P, [0,1)is a Brownian bridge,then = stochastic (D, d, P,(XF(I)'t'R)'To show.that WF \037(tINII)'we take) process (Xr)r\037
\037.
II
x* = L 11=1)
\037i
E
(\037i)
\302\243'1<'
1R\",
(Ii)E [RII,
and observethat) F J exp(ix*)dW
= exp
- .2 \"\"1'-1) L L ;.(/t;.(,(min{FU/t),F(r,)}- FU/t), 1
(
II
II
and)
n
J exp(ix*t)dNII = J exp(iL ,,=I
)
F(t,\302\273
I\"
0
J0) hdF)NH(dh)
\037Jc
-
n
= J cxp(iJ ( L \037,,(t( - <x:.',,) ,,-
Nil (dh)
F(t,,\302\273hdF)
I)
=exp - 2 (
2
II
1
L :IIe(te
X:\"I<)-F(t,,\302\273
Ie
-=
1)
)
.
o)
the existence of particularstandardGaussianprocesses for Next, we establish Abstract Wiener spaces.)
70.6Theorem.Supposethat (H,D, t) is an Abstract Wiener space.Then Ihere exi:itsa uniquely determined standard Gaussian proce.\\'s on 0 = (B, P) wilh X(h) t (h,.),hE H.) (X(h\302\273ltell
\302\243B(B),
im t* isdensein H. Otherwise,there exists ProofSincet isinjective, ho .iim t*, = = 0 (t*(x*),ho) x* for all x*E 8*.Hence ho E H\\{OJ,which implies t (110) = 0 which is a contradiction. im t is densein B, t* is Similarly,since injective. For every hE im't*let X(h) = x* if t*(x*)= h. Then X(h) is well-defined sincet* is injective.Moreover,if t* (x*)= h we have) (t(llo\302\273
2'(X(h)IP)= 2'(x*IP)= 2'(x*otIN II ) =
-
2'\302\253h,
Thisimplies that X: im t*
.)Nil) = I
VO.l/hI/2.
L2 (D,(jI(D),P) isa linearisometric mapping,and)))
354
Chapter I I:GaussianShifts on Uilbert Spaces)
thereforecan beextendedto the wholeofH. By Theorem68.4,X isa standard Gaussianprocess, and by construction it satisfies)
- (11,.))
X(h)OT -= X*Of -, (T*(X*),.)
if h = f*(x*).Sinceim 1'*isdensein H, this iseventrue for every he JI. the caseof Theorem70.3.Let 70.7 Examples.(1) Consider standardGaussianprocess on B which satisfies X(h) t = (h,
be the
hEll.This .), the Wiener (X(h\302\273\"EH
\037
0)
isusually calledthe Wienerintegral.Let us compute process integral for particularfunctions.First. letx E im t, i.e. there existsa derivative In this case,we have) x' E L2 ([0,1]). X(h) (x) =
1
1
(h,x') = Jo h(t)x'(t)dt= J0) h(t)dx(t),
where the latter integral is a
Stieltjesintegral.If x E B\\imf then such an of X(h) isnomorepossible for every hEll.But,if hE im t* then interpretation we have by construction X(h) = x.if h = t* (x.).E.g.if h = L IX/I10\"i) then, i .- 1
\"\"
Ie
() l' = f* ( L (lil;,,)and hence (lit,) 1 j:::
obviously, = ( L h
j
\037
1)
Ii:
X(h) (x) = i
=
Ii:
L IXjf.t;(X) =1
!
= i
L =1
IX
j X(I;)
1
h(l)dX(I),X E B.) 1
Theseproperties motivate the notationX(h) (x) =:J h(t)dx(t)for arbitrary o in hE H, x E B.It shouldbekept mindthat in spiteof this notation the Wiener if x B is a E is notofbounded not variation. integral Stieltjcs integral the caseof Theorem70.4.Let (2) Consider H be the standard 0 = on D, which satisfies Gaussianprocess X(h) f (h, he H. Thisis nothing to H and to the samplespaceB.Indeed, elsethan the Wiener integralrestricted denotethe Wiener-Integraland) let X(h), h e L 2 [0,1]. fe \037[O,1],t e [0,1].) I(f)(t):=f(O) + l(f(1) Notingthat Wo = .sf(id /1W) and X(h) 0 / = 0, It e H, it can easily be in checkedthat (B,[;I(B),Wo , (X(h)IB)hl') satisfies the characteristic properties Theorem70.6. the caseofTheorem70.5. Alsoin this casethe standard consider (3) Finally, It e H, can be X(h) l' = (It, H on
.),
(X(h\302\273\"6
-
(X(I1\302\273hc
-
f(O\302\273,
\"
.), by)))
70.Banach sample spaces
355)
F-1(S)= inf{xE\037: F(x) S}, Sf:(0,1).) The right inverseneednotbe continuous, but satisfies \037
SI
/,.-1(sd F-I
iff
\037.\\\"2
\037
(.\\\"2),
SI'.\\\"2
E
(0,1).)
It can thereforebeusedfor a transfonnation of Stieltjes intoLebesgue integrals
integrals, namely) +
\302\253;,
)a)
hdF=
1
!hoF-1d).1
for every h e L2 (F). in Now, we notethat the Borelmeasure WF on \037o(!R) which isconstructed 1 to the subset) the proofof Theorem70.5,gives probability {x0 F:x E 1f
yE
\037
0 (\037)
f\342\202\254([0,
1])}.)
is of thistype, then y F-1 is in '6'([0,1])and we define) \037
1
1 X(h)(y) 1=J h 0 F- (t)d(yv F-I)(t), hE L2(F). o)
shallprovethat thisisthe standardGaussianprocess of the desired nature. Let y E reo(\037) be such that)
We
t
y(t)= J gdF, leR, -\037)
for somegEL2 (F).Then) F-I(S)
s
(yoF-I)(s)= -00 J gdF=JgoF-ldA.I' 0) Hence,by definitionof the Wiener integral we obtain) 1
I 1 = J hgdF. X(h) (y) = J (h 0 F- ) (g 0 F- )d}'1 o)
This provesthe assertion sincey = 'C(g).
admit a representation AbstractWienerspaces ofGaussianshift experiments in the usual sense.) as shift experiments
70.8Theorem.Supposethat (II,B,t) is
an Abstract Wiener spaceand 1= Let let Po \037('t I Nil)' H be the .fjtandard Gaussianprocesson , = t (D,EiI(D),Po) satisfyinK X(II) <\", II e H. Tllen))) (X(h\302\273,,\037
.),
356
Chapter 1I;GaussianShifts on Hilbert Spaces)
E = (D,rA(D), {Po\"8(h):hE H}))
is a Gau.\\....ian shift on H and
itscentralprocess.)
(X(h\302\273Iu;H
ProofDenotePh :=P * tr(h)' hE H. For every hE H let Q\"I\037(B)be the Borel measure which is definedby) dQ\"
dPo
(x) =
-
t
2 XE B.) exp(X(h)(x) 2I1hll),
hE H} is a Gaussianshift whose standard Then, by Theorem 69.4{Q,,: is X. It followsfrom Theorem69.10that Gaussianprocess = V<\"\"\"2).II\",lIl, \037(X(hl)IQ\"2)
hiE H, i =
t, 2.
Thesame istrue with Q replacedby P sincewe shallshow that the distribution
of 2(X(hl )IP\"2)is by definitionthe distribution xt-+X(h 1) (x + T(h2
B, if hi = t* (x*)forsomex*e D*then under the probability measurePo.Indeed. \302\273,
XE
X(h 1) (x + T(h2 = \302\273
x*(x + T(h2 = x*(x)+ x*('r(h 2 = X(hl)(x)+ (hi'h2>') \302\273
\302\273
Thus,we obtain) \037(X(hl)I P\"2) =
V<\"I.hl).lIhdI2
if hi E im T*, and by continuity even for every hi e H. It followsthat) \037\302\253X(hl\302\273\"'EHIP,,)
SinceX('r.
(x.\302\273
=
=
\037\302\253X(hd)\"leHIQhl)'
e H.)
x.,x.e B.,we obtain)
Y(x*/P,,)= Y(x*'Q,,),hE H, which yieldsP\"
h2
= Qh, hE H.
x*\342\202\254
fl.,
0)
The Gaussianshift experi70.9 Example.Consider the caseof Theorem70.3. with thisexampleisfamiliarin the signaldetection ment associated theory and differentialequation) usually written asstochastic dyer) =
whereh E
dx(r)+ h(t)dt, 0
\037
r
\037
1,
/}([0.1])isan unknown signal.Theobservation of the dy(t) consists
the equation should signalh(l)dland white noisedX(I).Inour terminology
be)))
71.Testingfor Gaussianshifts
357)
written as) yet) =
,
x(t)+ Jo) Il(s)ds, 0
\037
t
\037
1,
wherex,y E f{f ([0,1])and x isa randomelementbeinggoverned by the Wiener measure. The centralprocess of this experimentis the Wiener integral. ThecasesofTheorems 70.4and 70.5are obtainedaslimitexperimentin non80,82and 83).) problems(seeSections parametricstatistical
71.Testingfor Gaussianshifts) be an Hilbertspaceand E = (fl,.5II,{P,.: /I}) a Gaussian (., Forthe caseofa finite dimensional shift experiment. Hilbertspacethe problem Let (H,
Il E
.\302\273
of testinghasbeentreatedin Sections 28and 30. a continuous, We beginwith the analysis of testingproblems concerning linearfunctionf H -+ R First,we considerone-sided Let testingproblems. = = HI {IlE H:f(ll) O},KI {hE H:f(ll)> O}.Recall,that a criticalfunction is unbiasedof level e [0,1]for the testingproblem(HI'KI) jf
ex
\037
\037
P,.cp\037a. if
l(h\302\273O.)
It issimilarof level for the testingproblem(HitK ) if P,.(/)= 0 whenever f(ll)= O. Every criticalfunction which is unbiasedof levela isalsosimilarof Ct.
I
levela. Let e E H be a unit vector suchthat kerf and fee)> O. Then f(ll) = (e,h) IIfll,hE H. Moreover, L S H such that eEL for every linearsubspace the norms IIfldl and IIfllare equal.)
e.l
71.1Lemma. If
P,.cp
\037
P,.cp
\037
\037
cP
+
f(h\302\273
NrJ
+
f(h\302\273
Nfl,
( (
11/11 ) IIfll)
)
if
the testing problem
f(h\302\273O,)
if 1(1l) O.) \037
linear subspace Proof Let hE /I and let L S II be a finite dimensional hand Then e. is a finite dimensional Gaussian shift and the containing EI,. assertion followsfrom Corollary28.2or Theorem28,6. 0)))
Chapter 1I:GaussianShifts on Ililbert Spaces)
358
71.2Definition,Suppose that q> E f7 (!l, d) issimilarof level Thenq> is optimalof level for (HI'KI ) at 11E H rx
Pllq>
=
N +
(
IZ
fX
for (HitKJ).
if)
f(It\302\273
IIfll)
.)
It is uniformly optimaloflevel for (HI'K1) if this is true for every hE H.) fX
71.3Theorem.Let X = II be the centralprocess of E. A critical function q>* E F(D,.9I)is uniformly optimalof level(J.for(HI'KI ) iff) . foX N 1 if I II/If> =) q>* Po-a.e.) . foX o if < NI IIff! (X(h\302\2731tE
cr
'
cr)
we obtainthatfoX = X(e) '1Ifli.Since ProofFrom the proofof Lemma69.15
= V(c.,,).1 = v/(II)/IIfIl.I' he H,) !e(X(e)IP,,) we notethat it followsthat q>* is uniformly optimal. To prove uniqueness forE implies for EI(ker Thus,uniqueness followsfrom optimality optimality
Corollary26.5.
f)\037'
0)
It isclearthat q>* isadmissible for (HI_ K1).)
71.4Corollary.If a criticalfunction lp* E :F(.0,.$I)is optimalof level for fX
(HI'K.)at any hE H with f(lt) 0, then (fJ* is uniformly =+=
optimal.)
linearsubspace hande. ProofLet L H be a finite dimensional containing ThenCorollary 28.7implies on that q>* isuniformlyoptimalon L,in particular kerfl. Thisprovesthe assertion. 0) \037
Fromnow on let Now, we turn to two-sided testingproblems. H2 = {hE H:f(h) = O},K2 = {hE H:f(h) O}.Recall,that a criticalfunction (D,.91)is unbiasedof level E [0,1]for (1/2 , K2) if q> E =+=
F
\037
Pltlp
\037
Pltq>
2:
\037
\037
if if
f(h) = 0, and f(h) * O.)
71.5Lemma. If lp E :F(!J,s1)is rmbia\037'ed oflevel for (H2,K2) then) fX
\037
-
4' N,/2+ + 4> N / 2 ' he H.))) IIfll) IIfll) ( ( f(I1\302\273
PIICP
cr
f(h\302\273
7J.'Ieslingfor Gaussianshifts
359)
Proof Let hE H. By Theorem 28.8the assertionis valid for any finite L If containing hande. 0) dimensional subspace 71.6Definition.Supposethat q> e (D,.91)isunbiasedoflevel for (//2'K2). Then q> is optimalof level for (Hz,Kl) at hE H \037
\037
\037
rx
P,.cp=
Na/2 +
if)
+ 4J
f(h\302\273
(
IIfil)
(
N a/2
- IIIII f(h\302\273
)
.)
It is uniformly optimalof level for (Hz,Kz) if this is true for every hE H.) (X
71.7Theorem.Let X = be the centralprocessof E. A critical function cp* E.F(D,,Jl/)is uniformly optimalof level for (Hz,Kz) iff) (X(h\302\273IIeH
(X
q>*
1 if .IfoXI > N 1 ,,-
=
II
0 if
j
13./2
IfoXI < N
-11111- 1
-a/
,
Po-a.e.
2)
ProofThe proofissimilarto that of Theorem71.3. 0 for (H2'K2). It is clearthat cp.isadmissible
71.8Corollary.If a criticalfunction q>* E :F(D,.r:I)is optimalof lel.Jel for (1/2 , K2) at any hE H withf(h) 0, then cp* isuniformly optimal.) Proof The proofissimilarto the proofof CoroJlary71.4,replacing28.7by 28.9. 0) (X
=t=
lineartestingproblems. Let Lo H be a linearsubspace of Next, we consider finite codimension. This means,that is a finite dimensionalsubspace. = Examplesof such casesariseif Lo kerffor somelinear functionf H -+ IRk, We consider the testing kEN, or in a trivial way if H is of finite dimension. = = problemH3 Lo.K3 H\\Lo.) \037
L\037
71.9Definition.A criticalfunction cp E .'F(D,d) isunbiasedof level
(X
for (H3'K3) if P\"q>
\037
(X
P\"q>
\037
(X
E
[0,1]
he Lo. if hE H\\Lo. if
Define4 = {hE H:IIh
- pl.o(h)= c}, c> O.) !I
71.10Theorem.Let X = processof E. LeI E [0,1] H be the central 0 X > ka}= (x. Then the test))) and choose ka. C [0,00 J such that Po{ (id-P1-o) (X
(X(h\302\273\"E
II
II
360
Chapter
q>
.-
11:GaussianShifts on Hilbert Spaces)
_ 1 if
{0 if
(id-PLo)0 xII>ka, 0 II (id-PLo) XII
is unbiasedof levela for (HJ,KJ).Moreover. if qJ 17aCHJ,KJ) then for every c>O) \342\202\254
inf Phq> \"(;8.
inf Phq>.. :s\"E8.)
the experiment(D,.r#,{P,,: hE Ltn and the testing ProofLet c > O. Consider = = /l S ince is unbiased of level for this problem J {OJ, KJ Li\\ {O}. qJ restricted experiment and for the testingproblem(H3'K3) it followsfrom Theorem30.2that \037
inf \"
..B.P\"q>
inf
\037
\"
6 B.\" I-t,)
P\"q>
:S
CJ;,,\037(C,,\037\037(a\302\273
function of th\037 non-c\037ntrall2-distrihution with where Ck.clis the distrihution 2 k 1=dim(L\037) degreesof freedomand non-centrality parameter c > O. It is 2 c. function C\",clwith clearthat 9\"(II(id-pz.JX P,,) has the distribution 2 2 = c Ilh pi.o(h) , hE H. This provesthe assertion. 0)
-
11
1
11
71.11Corollary.Keep the notationof the precedingtheorem. If a critical
function q> E 17(U,,t\"J/) PoqJ
\037
satisfies for at leastonec > 0
a,
P\"qJ\037P\"cp.if he\037,
hl.Lo ,
then cp = cp.Po-a.e.Hence,cp* fj' admissible for every testing problem({O}, Bcr\"lLt)and is uniquely determinedby its powerfunction on {O}v(B,r\"lLt)
c> O.)
Proof Considerthe experiment ({l,,w, {P,,:h.1LoD and apply Corollary
30.3.
0)
71.12Corollary,Let C
\037
convexsubset. Then) Lo be a closed
1 if (id-Pl-o)0 X C, 0 X E C, {o if (id-pi.o) is an admissibletestfor the testingprohlem (Lo,H\\ Lo) and is uniquely cp* =
determinedby
f$
itspowerfunction.)
and to the testing Proof Apply Theorem30.4to the subcxperiment ElL! o problem({O}, \\ {On. 0) L\037
71.13Remark. Considerthe problemof testinga linearfunction.fH -+ W.l.g.,assume that im(j)= R\". Considerthe testingproblem({f=
IR\". O},)))
71.Tcstingfor Gaussianshirts
361)
the projection (30.8)is literallyvalid if we replace {f O}).Thenthe Discussion =t=
PL by
X= the centralprocess
(X(h\302\273/u;
II
of E.
SinceLo = {OJisnotof It remainstodiscuss the testingproblem({O}, H\\ {O}). 71.9-71.13 the discussion doesnot apply to this case.But finite codimension we may extend the assertion of Corollary71.12obtainingat leasta consideThe prooffollowsthat of testsfor ({O}, H\\ {O}). rably large classof admissible Theorem30.4.)
71.14Theorem(Moussatat [1976]).Let (h\,,")
to
\037
\037
II and (r,,)\" I;
\"\"
\037
\037
bearbit-
Then rary sequences.
I if X(h,,) > r\" for somen EN, {o if X(h,.) r\" for every n E isan admissible testfor the testingproblem({O},{hE H:h cp* =
\037,
\037
determinedby
=t=
itspowerfunction.)
O})and is uniquely
Proof If any hIt = 0 then eithercp* is a trivial testor it can beomittedin the definitionof cp..Thereforewe assumethat h\" * 0, n E N. Suppose that there existsa criticalfunction cP E (H, such that PoCP Pocp.and Phcp Phcp.if h 4= O. We shallprovethat in this casecp = cp.Po-a.e. Assumingthe contrary,i.e.Po{cp4= cp*}> 0, it followsby PoCP PoCP* that Po{cp*> cp} > O. It isdearthat \037
\037
f!l(H\302\273
\037
\037
{cp.> cp} = U
{X(h\")> r\"} () {cp <
I}.
\"e\037)
Hence,there existssomen E
\037
such that)
> r\"} n {cp< I})> O.) Po({X(h..) Fix this particularn E N and let).> 0 be arbitrary. Since) P;'h\"CP.
\037
we obtain
P;'h\"CP,)
-
-
o J (cp. cp)dP;'hn= J (cp. \037
-
cp) exp(lX(h\")
lI,ih 2) dPo Il 1l
\037
=
-
1
exp( 2
= exp
+
(-
11
2 )'h\"11
II
)'h\"
2 11
\037
J Xlh\\037,\"")
-
(cp.
-
-
+ ).r\")J (cp.
cp) exp(J.(X(h,,) r,,\302\273dP o
+ A.r\")
(cp.
(
J
Xlh\"\302\273,\"
-
-
cp) exp()'(X(h ll)
-
cp) exp(I.(X(h\")
r\"\302\273dP
o\\. \037)))
r\"\302\273
dPo
362
As
Chapter A.
11:GaussianShifts on IlilberlSpaces)
-+ 00 the secondintegral remains bounded. The first
integral,however,
tendsto infinity since) {X(h\">> r\"} {cp.== 1}.) Thereforewe arrive at a contradiction for sufficiently large J. > O. \037
0)
isan extension in the 71.15Remark. Theassertion of Theorem71.14 of71.12
in a finite dimensional iscontained L 1/then the criticalfunction cp.is of the form) subspace
followingsense:If the sequence (h\\"")
\"
rI:I
\037
l
cp*= o { where C =
n
II\\;
if
PLoX;C,
if
PLoXEC,
convexsubsetof H. {hE H:(h,h,,) r,,}is a closed, \037
\\:)
72.Estimationfor Gaussianshifts)
(.,.)
Hilbertspaceand E = (D,S:/.{Ph:hE H}) Again, let(H, > be a separable a Gaussianshift experiment. linearfunction.Recall,that an estimate Let f H -+ IR be a continuous, for f if Q E 9t(E,111)is median unbiased 1
P,,(e(.,(-x>,j(h)]) 2' \037
1
P,,(e(.,U(h),
\037
00\302\273
2')
e.l
for every hE H. Let e E H be a unit vector such that kerfand fee)> O. Define the loss function by W,,(x)=t(lx-f(h)I), xe\037, heH, where = t O.) is (0) [0,00] [0,00] non-decreasing,
(:
-
Then 72.1Lemma. Supposethat t is lower semicon/inuous.
unbiased estimatee E fit (E, satisfies
every median
\037)
WhUP\"
\037
hE H.) Jt(I.I)dvo.llfIlJ,
Proof Let liE /l and let L
\037
H be a finite dimensional linearsubspace
hand e. Then Ell.is a finite dimensional Gaussianshift and the containing assertion followsfrom Theorem34.3sinceIIfll= IIfILI!.
0)
and (!E ,(jf (E, 72.2Definition.Suppose that W is lower semicontinuous median unbiased. Then e is optimalmedianunbiased (for W) at hE H
\037)
if)))
is
72.Estimation for Gaussianshifts W\"UP\"
363)
= J t(I.l)dvo.II/lI l,
if this holdsfor every hE H.) and uniformly optimalmedianunbiased
be the central process ofE. Then K = f o X is a uniformly optimal medianunbia.redestimate offfor every lowersemicontinuous
72.3Theorem.Let X =
(X(h\302\2731IE
II
t: [0,00]-+ [0,00].)
= V/Chl,II/lIl, followsfrom the obviousfact that g'(KIP,,) Proof The assertion
he H.
0)
72.4Theorem.Let X = bethe ce1ltral that t is process of E.Suppose lowersemicontinuou.t, offiniteorderand separating. If eoE (E,A) satisfies (X(h\302\273\"\"H
.\037
l if J t(l.l)dvO.II/li {foX}) = 0 Po-a.e.)
W\"UOPh
then eo(.,A\\
\037
h
1.kerf,
ProofCopythe proofof Theorem40.11 replacing PL by
L = (kerf).l.
XL where
0)
solutionfor the estimation The preceding assertions offera complete problem of a linearfunctionf H -)0 lit Beforewe enter the problemof estimatingan of the arbitrary linear mappingf on H, we shaH considerthe estimation identity. We beginwith the casewhere H is a finite dimensional Hilbertspace.This in casehas beentreatedalready in Section38,but only considering estimates at (E,H). In view of the general decision theory we have alsoto take into
consideration the estimates in Define the loss function
H)\\9t(E,H). Wh(x) = I(x -Ir), x E If. hE H, where -+ I:H [0, is a rneasurablefunction.The followingassertionisan immediate consequence ofTheorems38.22and 43.5in caseI is level-compact.) 14(\302\243,
by
OC\302\273
72.5Theorem.Assumethat dim H < 00.LeIt: H -+ [0.oc)belowersemico1ltinuous andsubconvex.Then) inf
supP(W h , Ph) = J to X dPo= J I dNII .
fJF!iI(E.H)\".11)
we assume that E is a standardGaussianshift.The inequality ..ProofisW.l.g. obvious.The otheris provedwithin severalsteps. \037\"
(1)Assumethat I = I
- Ie
where)))
364
Chapter
11:GaussianShinson Hilbert Spaces) It
C == n {xeH:I<X'Yi)l Pi}. Yi e H, Pi O. i'\" \037
\037
t)
Let L = span{Yt, ..., \037} ThesetsC and C(\\ L are closed, convexand centrally symmetric. Toseethis,let {Yil Yi m } {Yt bea base Moreover, L iscompact. Then for every x e of L and Land alii} its orthononnalization. _
...
ell {at... s=1,....m)
..,ell
\037
\302\273:}
k I
<x,as)
\037
I
L PillyEll
j -=
1)
which implies) 2
Ilxll =
k
III
L l<x,a,,>12m(L PiIlYill)2. j.:1) \037
$\"'1
we have toPL ;:: t. Moreover,
To prove the assertion of t we notefirst for thisparticularchoice inf
.
W inf supP(Wh sup hIllP( h Ph) JliSl(l::,1I} hlL) H) we definep.E 14(E,., L) by
/lE\037(I::.11)
For every P E
\037
.
Ph)
that)
.
&'1(\302\243,
p:(g.Jl) 1-+P(g0 PL'Jl), g E \037b(L), Jl e L(E). the risksof P and P for h e L, we obtain Comparing P(W\",Ph) =
sup{PC/'P,,):fe'(jb(H),f\037W h } sup{P(g0 PL,Ph):g E (jb(L),goPI. = sup{P(g,Ph):ge 'Cb(L), g W\"ld
\037
\037
Wh}
\037
= p(Whl,.. Ph),
sinceW h
\"
PL
=
Wh
he L,)
. Thisimplies that
inf
inf
supP(WhIL,P,.).
h, L p,!II(E.H) supfJ(W\",P,,)\037 PEt
L)
and we arrive at) Now, W h I L is level-compact inf
II f'
'.)supP(W\"I/.,Ph)
SJ(EL. he I.
-
inf QE\037(\302\243,.,L)
== o == JtdPo, h sup(Whl,JQP Jt(X,,)dP hEL)
Theorems43.5and 38.22. convexand centrallysymmetricset. (2) Let C 5 H be an arbitrary closed, = Let the lossfunction t 1 1c. Consider l be the classof lossfunctions from aboveand))) in part (1)of the proof.Then \"YI isdirected considered by
-
f
72.Eslimalion for Gaussianshifts
t = sup{
365)
: V t}. there To seethe latter,consider theorems, any x C.Then,by basicseparation are y E H, fJ ;::; 0, such that (y.x)> fJ. but l(y,z)1 fJ if ZE C.) Let D= {ZE H:I(y,z)j P} and define V= 1 l D. Then C\037 D implies V t, and x f D impliesVex)= {(x). E
V
\"1/ 1
\037
\037
\037
-
\037
-
\037
If V E 1/.1let V,. 1= V(. h). Now, the assertion followsfor t inf /l
f:
ii1(E.II)
by
supfJ (Wh , Ph) hf: H)
> sup
inf supP(Vh , Ph) y,,'t\"\\:ystfj,,!j(E.H)\"\"H)
sup J VdPo = JtdPo.
Ye't\"I:Y:i1)
(3) Next, let t then)
bea simplefunction. If im { =
t = L (ctj -
{\037..
...,}, Ci ll
Ci l
< ct2 <
.,.<
Ci\",
II
}\037
Ci
j 1) J-j
t)
n-1. ...,
where Ci o = 0, Vo = 1, J.j= 1{(:>crJai' 1 \037j n We note,that for every j = 1, 1 the function J-j is of the type in part (2) of the proof. considered functions. Let 81j .'!I (E,II) be the set of all strictly equivariant decision 48.13. Thus,we get) Now, we apply the general Hunt-Stein-Theorem \037
\037
inf
/I e SI(E, H)
fJ( Wit, sup hr, H \"
= inf /l E 91,
L- (Cij
}
=
L
/I '\"
- j- ) Ci
pet,Po)
{JIj)
1 P(\037,
-
(a.) a.j I) inf
j=1 \"
L (Cij j= 1
=
= inf
Po)
1)
n \037
Ph)
- jCi
P(J-j, Po)
/10:.91,)
inf
1)
fJE9I(\302\243.1I)
sup POj.h'Po) hell)
\"
= ldPo. L I) j - (a.j a.j J \037dPo J 1)
considered (4) Finally,let t bearbitrary. Let \"1/) bethe classoflossfunctions in part (3) of the proof.Then isdirected from aboveand
t = sup{
V
E
l'J:
V
\037
f) (} .)))
366
Chapter
11:GaussianShifts on IlilbertSpaces)
To seethe latter\"we have to show that for every x E Handr. > 0 there exists such that I(x) Vex) + c. For this,let < I(x) I, V E x, {I <x} there are Y E H, P 0,such that (y,x)> p, but l(y,z)1 P if fez) ex. V
\037
\"1\"\"3'
<X
\037
\037
Let C = {ZE H:l(y,z)1
fJ}
\037
\037
<x}
<X
+ c.Since
\037
\037
{I
\037
\037
and define V=
S C which implies V = a(1- 1d od1t>a/ I,) \037
and Vex) =
ex
\037
lex)
-
f;,
<x(1-IdE1'3'Then
\037
sincex C. \037
0)
-
72.6Corollary.Assumethat the conditions Then of Theorem72.5aresatisfied. the central randomvariableX: Q H isa minimaxestimate.) Proof UseCorollary69.11.
0)
resultto the generalcasewherethe dimension of Next, we extendthe preceding H is not necessarily finite.)
72.7 1beorem.Supposethat convex.Then)
inf h\037(E.1l)
Wit, supP( hiH
I:H -+ [0,(0) is lower semicontinuous and sub.
Pit)
= sup f t dNI.' LEY i)
..
Proof(1) In the first part we prove that the inequality
\037\"
is valid.For this
we apply severaltimesthe MinimaxTheorem46.3:) inf 6E9I(\302\243,HI
supP(J-Y,., Ph) IIEH)
= sup
inf
mlSHP.,AI(E./I))
= sup sup
J Pcw\",Ph)m(dh) inf
JfJ(w\",Ph)m(dh) Lc!/'m(SL 11([\"(I:.H)) = sup inf supP (Ut,.,P,J LEY PE\037(E.H)
\037
/lEL)
supJ {oXLdP o
LEY)
sinceXl. E (E,H), L E 2. (2) For the proofof the reversed inequality we introducethe class\"Y of and subconvex, and functionsV: H -+ [0,00), which are lowersemicontinuous 0 = V for L H. This in addition V some f inite dimensional satisfy PL subspaceS classsatisfies))) \037
72.Estimation
for
Gaussianshifts
367)
t = sup{VE 1/:V t} \037
which followsfrom parts(2) and (4) of the proofof Theorem72.5. Moreover,
1/ is directedfrom above.To seethis,let ;PI..= L=span{L,L2} then JIloPL= i= 1,2, and VI
=
VI
V;,
I
(V; U V2). (3) Let us prove that for every
inf
E
V
1/ such that
V0
and V2 P\" l = V2 . If hence (VI UV2)oPI.
P,.=
<>
V
supP(\037,Plt)\037rVdNL' i)
6E\037(E.H) ItEII
First,it isdearthat) inf
sup P(\037, Pit) #tEll
PE\037(\302\243,/l)
inf
\037
'..<\302\273(\302\243,11)
supP( v\", Pit). hI.)
Secondly, similarlyto the argument in part (1)of the proofof Theorem72.5it
is shown that)
inf
sup hI.P(JI\", Pit)
6.91(\302\243,11)
\037
inf
supfJ (v\"I/.,Pit).
hI.) 6e9l(E,..L)
followsfrom Theorem72.5. Now, the assertion (4) With part (3) of thisproofwe arrive at inf
{J( WI., Ph) sup Itflll
PE9I(\302\243,lI)
\037
sup J f:(
VE't'\": V
V
dN L(V)
L(V))
where L(V) H is a finite dimensional such that subspace remains to show that) \037
sup
V
0
PL(V)
=
V. It
L. VdNLW),-supJtdN L\"YL)
J
VE't\":V;Zt'I.(V)
This is proved,showingthat for eachV E ..r
sup J VdNL . J VdN,.(V)= LEY
L(V)
L)
It isclearthat L ( V) J
I.(V)
L implies
\037
V dN,.(V)=
sinceV u PL =
J V dN,. L)
V, in
J VdNLI
L.
\037
thiscase.Thus,we need only show that LI <;: L2 implies) J VdNL2
L2)
.
or equivalently,)
!
eo
N L. {V
>
!
eo IX}
dct
\037
N L2{V
> ct} dct.)))
368
In
Chapter
11:Gaussian Shifts on Hilbert Spaces)
otherwords,we have to provethat for every centrally symmetric,closed
convex setC s; H)
NL.(CnLI ) NL2(CnL2). If we put D:;; enL2 then thismeans \037
NLz\302\253DnL 1) <11 Lt)
NLz(D)
\037
where Lt denotesthe orthogonal complementof LI In L2 . But the follows from inequality r dNL2 = J 1D-X2(XI)NL,(dxt)Nu(dx2)
'
q },
'b
\037
last
ID(x 1)NLI (dX l) NL; (dX 2) lJ.l, ,
= N L2
\302\253Dn
Lt) Lt), \037
where we have usedLemma 38.20.
-
0)
and sub. 72.8Remark. Supposethat t: H [0,00)is lower semicontinuous convex.If Lt S L2 are finite dimensional of H then) subspaces I tdNL, J tdNLz . \037
LI
L2)
This has beenprovedin part (4) of the preceding proof. Now, it
iseasy to find minimaxestimates of the identity.)
72.9Theorem.Let X = II bethe central of/::and fJx e !!J(/::,H) process the decision 10Definition(69.14)). Suppose functiondefinedby X (according that t: H [0,00) islower.liemicontinuow; andsuhconvex.Then fJx isa minimax estimate of the identity.)
-
(X(h\302\273he
of Theorem62.5is valid alsofor the net Proof Noting,that the assertion (XL)LC,i\"in 1I(E,II), we obtainfor every Loe!f1
-
inf t dP = lim inf t dNL suppx(w\",Ph) $ lim I.-II hFsupJ (XL h) h 1..11.f.
life
1.0
10()
which implie.\037 that
sup/fxUY,,,Ph) hell
\037
sup J tdN1.. 1..2 L)
Equality followsfrom Theorem72.7.
0)
in In many importantapplications the decision problemis of finite dimension the sensethat the lossfunctionis a cylinder function.)))
n.Estimation
I:H - [0,(0)is t It
Gaussianshifts
for
369)
be the centralprocess of E. Supposethat and subconvex. lowersemicontinuous If LoE is a subspace
72.10Corollary.Let X = satisfying
Q
PLo =
sup JL L.
(X(h\302\273\".;
11
!e
then
.
tdN, = J tdNLo
\037
Lo)
and the randomvariableXLo is a minimaxestimate.)
Proof Since\"\037\" is trivial we needonly prove \"\037\". Let L H be any finite dimensional and let L1 = span(L u Lo).Thenby Corollary72.8 subspace JI.IdN, J IdNL1 \037
.
\037
1.1)
and)
J tdN\"1= LaJ t\037Pl.odNLI= Lo)J Id!i'(pLoINL'>
LI
0
= J tdNLo' Lo)
-
of linearfunctions is the estimation f. II B with importantapplication B. to finite dimensional The has two range problem aspectsleadingessentially
An
in definingD = Handt = t 1 ofwheret 1: the same solution. The first consists := lead B [0,(0). DenotingL (f) (kerf)J.Theorem72.7and Corollary72.10 to the minimaxrisk)
-
sup P(w,.tPh) = J 11 ofdNLII) ,..11 L(/))
inf \037. 91(\302\243.11)
is morenatural.) and to the minimaxestimate PX' The second possibility
72.11Theorem.Let B bea finitedimensional spaceandlll B a continuous, linear functionsuch thar imf = B. Supposethat t 1: B [0,00) is lower and subconvex.DenoteLen:=(kerf)l, semicontinuous
-
\037
-
If D = BandW h 1=t 1 (.
hE H, then
f(h\302\273,
)= supPO.J',.,P J 11 ofdNL(j)' hcH fJcBl(E.H) L(n) inf
h
The estimate foX isa minimaxestimate.) \"
\"
is obvioussincefoXL(f)EPA(E,B). To provethe otherinequalitywe show first that)
Proof The inequality /I e
inf SI(E.B)
\037
supP( w,.,Ph) h F II
inf
\037
If F
.\"(EL(f).I.
(f\302\273
sup
For this,let fJ E (E,B) and definelIE:?l(EL(/), L \037
P\302\253/
h F '.(f))
(f\302\273
f),.,Ph)'
10
by)))
370
Chapter
11:GaussianShifts Qn Hilbert Spaces)
p:(g,JL) {l(g0j-I,JL), g e ((jb(L(f),Ji e L(E), t\037
-.
wheref-I:B L(f) is the inverseoffl'o(f)'Then oj)\",P,J= sup{fJ(V,P,,):V (II<>j)\" on L(j), VE lfS'h(L(j)} =sup{p(Vorl,p,,): Vorl \037(I('-f(h\302\273 on B, VefCb(L(j)} on B, Vi fCb(B)} sup{P(VI' Ph): VI (a<. \037
P\302\253(tl
\037
-
\037
\342\202\254
f(h\302\273
=P(w\",P,,),heL(j).) followssince) Now. the assertion inf If d'(\302\243LIII'
sup P
L(j)) lit! L(j)
\302\260
\302\253((I
f)\",Ph) = J (I ofdNL(f) '.(f))
ofTheorem72.5.It isclearthatf\037' isan immediateconsequence estimate off 0)
X isa minimax
72.12Remark. Keepthe assumptions of the precedingtheorem.As to the of the minimaxboundlet B = computation
\037k.
Then for t e
= IVLo([*(I)) = exp J exp(it'f(x))dNLo(x)
... is ... respect
If (al with
ale)
(
-\037
IRk)
> <[*(t).f*(t) )
an orthonormal baseof La and if r is the matrix of (fcf*)-1 to (a I at) then
ffl(fINLo) =
Vv ./\"
and hence J I. ofdNr-o= R\J
Lo A
I. dv oJ '
is possible assertion if we only consider uniqueness equivariant estimates.)
72.13Coronary.Supposethat the assumptions of Theorem72.// are satisfied. Assumefurther that I. is level-compactand .<;eparating. he /I. If P e (E,B) is equivariantand Let D = Band w,,:=I. (.
-
f(h\302\273,
satisfies)
P(Wo, Po) = J II t(f))
\037
'1
dNL(f)
with the non-randomized then P coincides estimatefoX.)
Proof SinceL(E)= L(EI,.(/)we may restrictour attentionto the finite casewhere H = L (j).Then the only differenceto the assertion dimensional of Theorem38.28is that now we do not assume that {lE (jt (E,B).)))
72.Estimation (or Gaussianshifts
371)
First,we show that there existsan equivariantestimate fl E {it (E,B) suchthat J.v,.qP/J
\037
fJ(UI,.,P/J), hE H.)
of Lemma43.4.Thekernell!o obtainedthere Forthis,we copythe construction satisfies)
g(.- h)floP/J= (3(g(.- h), Ph) = (3(g,Po) = g(loPo, hE II,
for all g e
'i (B).It followsthat 00
Ph(flo(., = Po B\302\273
\302\253(lo(.
,B\302\273,
hE H.)
Let X be the central random variable of E. SinceX is \302\243.sufticient we may of assumethat eo dependson (I) E Q through X. Hence,by completeness = we that N The \"e obtain const case H} qo (., B) wa.e. eo B) {\037(XI Pit): = 0 cannothappensinceotherwise
(.,
fJ ( w\",
P ) = 1.( (0)> J IIdPo, Ir
hE
H,)
the assumption. contradicts Thus,having established eo(.,B) > 0 we of Lemma43.4and observethat the decision function (l copythe construction we obtainedthere isequivariant. Theorem 38.28 a nd conclude that Now, apply with o X. This implies) (} coincides which
l
fJ(Wo
-II(oc).
Po) = (Wo
On the otherhand it isclearthat) (3(Wo by
Po) = (Wo
- II
(oc\302\273(!Po.)
(00\302\273
qo Po)
definition of flo.Since)
-II
(l Po =
1
(\037eoPo eo( \ B) followsthat (10( ., B) = 1.HencefJ = (10= (!= loX. (Wo
it
-II(00),
-II
(00\302\273
t.(oo\302\273
0)
As a consequence we obtainthat the non-randomized estimate/o X isuniquely determined by itsdistribution. We finish this section with a general versionof Boll's convolution theorem.)
72.14Remark. Let us calla linearprocessZ = decision function {3z is equivariant, i.e.
-
(Z(h\302\273,,{
if)
{3z(f,Po}= (3z(/(. h), Ph)' hE H.Iet:tib(H).) In terms of the linear process Z this means that)))
H
equivariant if the
372
Chapter
11:GaussianShifts on IIiIt)ert Spaces)
-
!i'(Z(h) (htt l
h z )IP\"2)=
\037(Z(hl)!PO)' hiE N, i =
E.g.every standardprocessisequivariant in 72.15Theorem.Supposethat Z =
1,2.)
this sense.)
is an equivariantprocess.Then there existsa cylindersetprobability measure(QdL.:,g' such that 9'(Z,.IPo) = N,. Qu L E !I!.)
.
(Z(h\302\273hc
H
Proof SinceZ is equivariant, the random variables ZL are equivariant estimatesfor the restrictedGaussianshifts Ell.,L E !f'. The existence of m easures that such (L) QL1.'11 probability
.
!i'(ZLIP o) = N L QL'
Sf, followsimmediatelyfrom Theorem38.15. It remains to show that (QL),.dr is LE
Projcctivityof (ZJLE:L'impliesthat for Ll projective. If'(PLIINL1* QL1)= NLI * QLI . However,it
\037
Lz
isclearthat)
!1' (PLII
NL1 *
Q,)= N LI * !1' (PI.IIQLJ)
which proves
Q,.I=\037(p\"IIQ,.).0)
73.Testingandestimationfor Banachsamplespaces) 71and 72 can begiven in a particular of Sections Theresults simpleform if the
by an Abstract Wiener space. underlyingGaussianshift is represented hE H})the Let (l/,B,t) bean Abstract Wiener spaceand E = (B,gd (B),{P,,: = ofE isdenoted Gaussianshift where Ph Po tr(h)' hEll.Thecentralprocess that the centralprocess satisfies X(h t ) (t(h2 = (hi'h 2 ), H' Recall, by
.
\302\273
(X(\"\302\273h\342\202\254
,h
h1
2E
H.
The testsofTheorems donot simplifyvery much when 71.3,71.7and 71.10 in the Abstract Wiener spacerepresentation. Thereforewe confine considered
to an example.) ourselves
73.1Example.Considerthe caseof Theorem70.3and Example 70.9.Then onH isof the form!= <.,\"0),\"0 E H. W.l.g. function! any linear continuous = assumethat 11\"0111.Sincefo X = X(ho) is the Wiener integral the optimal testof Theorem71.3 is)))
73.Testingand estimation for Banach sample spaces
cp.(x)=
1
{0
J lIodx> N . a, :f If I
1
X
E
373)
(i([O,1]),
Jhodx
subset.Then Let C s B bea convex,closed 73.2Theorem(Moussatat[1976]).
.
'
if x C, X E B, {0 Iif X E ( ,) is an admissibletestfor the testingproblem({O},H\\ {O})and is uniquely cp
-(x) _
determinedby
1
\037
itsdistribution.)
in Theorem71.14. From ProofWe show that cp.is of the form considered there exists a theoremsit foHows that basicseparation sequence (x:)uN B* \037
such that
C= {XE B:x:(x) r , nE N}. \037
n
Let us define11,.:= isprovedsincefrom the proofof t*(x,.),n E N. Theassertion = Il E N. 0) Theorem70.6we know that X(hn )
x:,
73.3CoroUary.Any testof the form if IlxlI8>c, XE S, cp*(x)= o { if IIxliB
is an admissibletestfor the testingproblem({O}.H\\ {On and is uniquely determinedby itsdistribution. Next, we attemptto computethe minimax boundin the caseof estimating the linear mappingT.)
73.4Discussion. Similarto the remarks precedingTheorem72.11we may two versions of this estimation Let t: B -+ [0,(0)belower distinguish problem. and subconvex. semicontinuous (1) Define D = H and \037: x (t c t) (x h), x E II, Ii E H. Then the minimaxrisk of this estimation problemis \037
inf supP(w;.,Pit) = 6eJl(E,H)itcH
accordingto Theorem72.7. (2)
-
sup f tot dNL ,
L.\037 i)
Sometimes, however, oneis dealingwith estimation problemswhere -. X. (x t x E hE H. Since the B, \"'I.: experiment E has)))
D = B and
-
(h\302\273,
374
Chaptcr
11:Gaussian Shifts on Hilbert Spaces)
samplespaceB, the identity of B is in
.\037
(E,B) and therefore)
supP(w,.,Ph) J t dPo.
inf
\037
pE\037IE.8) itcH)
shallprove that even equality holds. dealswith part (1)of the preceding Our first assertion It wil\\ be discussion.
We
in this case.) shown that the minimaxrisk can be simplified
-
73.5Theorem.Supposethat t: B [0,(0)is lowersemicomimwusandsubconvex.Then)
sup
r L ,ut' t)
t t dNL = J t dPo. u
r be 1: - [0,
.. r
the classof all lower scmicontinuous subconvcx functions B on x E B througha finite subset{xt,.. x:} B* XJ) which depend only.It shouldbe clearfrom the precedingproofsof thissectionthat is directedfrom aboveand t = sup E 1/: t}. It is thereforesufficientto for E \"r. provethe assertion If c; 1/, then there existsa finite dimensional subspaceLo c::;H and a 0 0 1t. linear 1t: B that 1t t = P'.oand = continuous, mapping Lo such For the proofof this assertion, let xr, E B*be such that I: x YEn ker implies(x) = (y). W.l.g. that xf, are we assume
Proof Let
\302\243
1
1
-
1=1
\037
-
1 lot
.... x: 1
1
x\037
1
{l
..., x:
and f* (xt),...,f*(x:>are an orthonormal system.Delinearly independent notee :=-:f* (xt), 1 i k, and define Lo = span{e ..., e,J.Then) \037
j
-
l'
\037
I:
1t:B Lo:Z 1-+ L xt(z)el = 1
1)
is of the desirednature. For the determination of the left sideof the asserted equationwe needonly
notethat)
(lot) PLo = (1 t)\" (1t0 t) = lot CI
\037
and apply Lemma (72.to) to obtain r sup I.< co j,
dim
1t dN,. = 1.1..t dN <.
Lo
'
0)
of the right sidewe rcmemberthat Po = 9'(rINIl ) and Forthe detcnnination hence)
1
J dP = J lo'f>ndP = J 1\"t\"n 0 tdNH = I'otoPLodNH = 1otNLo'
J
..0)))
73.Testingand cstimation for Banach sample
Thiscompletes the proof. Now, we provethat
in
spacc\037
375)
0)
73.4even equality holds.) part (2) of Discussion
73.6Theorem.Let D = B and \037: x.-.t(x-
t(Il\302\273,
XE
t: B -+ [0,(0).Theil) semicolltinuous subcollvexfUllctio1l
B, hE H,fora lower
inf supP(u-;.,Ph) = J t dPo. tJ,,\"(E.BIIIlH)
ProofLet -y bedefinedasin the proofofTheorem73.5.It is sufficientto show that for every e -y
1
inf
supP(a\",Ph)
6E91fE.B) he II)
.-.l(x-
\037
S
1dPo
x e B, he H. where JY,.: x Let E \"f/'. Then, in view of the proofof Theorem73.5,there is a finite linear mapping7l: B -+ Lo, dimensional subspace Lo H, and a continuous, such that 1t u t ,..,PLo and = 1\"t u 1t. We shallprovethat for every fJ E PJ (E,B) there exists a /30 E fA (E,Lo)such
1
t(h\302\273,
1 \037
that)
fJ(\037, Ph)
\037
fJo\302\2531..
t)h' Ph),
he Lo.)
For this,let fJ e 11(/::,B) and definefJo E f:i(1::, Lo)by fJo
(g,J1)= fJ(go7t,J1),gE\037b(Lo),J1EL(E).)
If h e Lo,we have Po t)\", Ph) = sup{Po(V, Ph): Ve'C,,(Lo). v\037l\037t(.-h)on Lo} \302\25310)
u =sup{fJ(V 1t,P,,): Ve(Cb(L o), Vo1t\037lot(1t(.)-h)on B} \037
=
sup{fJ (V, Ph): V E \037b(B),
V
\037
1(.--
t(ll\302\273
on B}
fJ(\037, P,,),)
Hence,Po has the requiredproperty. We obtainthat) inf
/I e 81(F..H) \037
sup fJ (w,.. Pit) ItF H
inf supP\302\253tot)\", hr. Lo fleSl(/';.'-o)
Pit)
= J 1.0)
1\037
tdN,.o'
From the endof the proofof Theorem73.5we seethat)))
376
Chapter
11:GaussianShifts on Hilbert Spaces)
1
10f dN1.0= J dP, Lo J
which provesthe assertion. 0)
73.7 Corollary.Supposethat the assumptions of Theorem73.6are satisfied. Then the identity on B is a minimaxestimate.)))
and Chapter12:Differentiability AsymptoticExpansions)
foundationfor a In the presentchapterit is our aim to providethe technical of the asymptotic of large classof applications theory.The actual application isconsidered in Chapter13. decision asymptotic theory to theseexamples of a statistical Theideaof independent replications experimentisformalized Consider a of triangular array of experiments by multiplicationexperiments. and form the products of every row.The resultingsequence ofexperiments has accumulation pointsfor the weak topology.The questionariseshow to all possible describe accumulation pointsand to give criteriafor the convergenceagainstcertain types of accumulationpoints.For this problema isknown. The particularcaseof binary experiments complete, generalsolution is treatedin LeCam [1969].The caseof arbitrary parameterspacesis in LeCam[1974], Milbrodtand Strasser and in Janssen, considered [1985]. the problemin full generality. We are only In thisbookwe do not consider in the weak convergence of productexperimentsto Gaussian shifts. interested Weak convergence of productexperiments to Gaussianshiftsis provedmost of the likelihood asit of a means stochastic expansion convenientlyby process, has beendonebeforealreadyin the early papersofWald.Thepresentchapter is exclusivelydevotedto the proofof this expansion. a stochastic of the To be moreprecise,we have to establish expansion logarithmof the likelihood process.The classical Taylor argument for this 7 4. which are almost i n We use is Section expansion presented assumptions which are necessaryand sufficientcan be found Relatedconditions necessary. van Zwet [1979] and in Janssen, Milbrodt in LeCamL1969J, in Oosterhoffand and Strasser, [1985].
Of basicimportancefor the followingare the concepts of a differentiable curve in a family of probability and of a tangent vector of a family of measures broaderrange of applications measures. probability They have a considerably than we can presentin thisbook.The readeris referred to Pfanzagl and We confineourselves to exhibitthe relationbetweenthe Wefelmeyer [1982]. curve and the stochastic tangent vectorof a differentiable expansionof the Section 7 5. isdone in likelihood This process. In Section 76 we consider differentiableexperiments with finite dimensional is the This main parameter space. subjectof the classical asymptotictheory. Theessential is the of an construction inner o point product n the tangent space)))
378
Chaptcr
12:Differentiability
and Asymptotic Expansions)
of the parametersetsuchthat it becomes to the tangent spaceorthe isomorphic
experiment. For sometime there was considerable as weak as effort to find conditions which implydifferentiabiHty ofan experiment. Our proposal in Section possible 77 istaken from LeCam[1970] Theexamples of Section 78 and Hajek[1972]. the usefullness shaH illustrate ofour conditions. The preceding can be appliedimmediatelyto obtainthe stochastic sections for differentiableexperiments in caseof identical and independent expansion thiscase In the i s stochastic replications. expansion parametrized by the we have to replications tangent space.However.in caseof non-identical are parametrize by triangular arrays of tangent vectors.Such expansions in Section 79.They occurin the literature considered for the first time in Hajek and Sidak[1967], in connection with rank tests.In view of the importance of in full detail.Further information this special caseit iselaborated concerning iscontained in Moussatat[1976] and in the caseof non-identical replications Becker[1983].)
74. Stochastic expansionof likelihoodratios) Consider a triangular array E\"j = (D\"\" sI\"\" (P\"\" Assumethat ken) i 00.Denote) binary experiments.
Q\",\302\273,
E\"
\"(II)
= (D\",sI\",(P\".
Q,,\302\273:=
n
E\"I>
1-=
n
1 i k(n),n E N, of \037
\037
EN.
t)
For 1 j k(n),n E N, let be a Lebesgue of Q./ , decomposition ( N.) with respect behaviour of the to 1>,.j.We are goingto investigatethe asymptotic ratios) Hkelihood \037
\037
\037<J,:'I
\037Q,!
d1>,.
=
ii dq\"i, d
,=1
n
\302\243:
N.
1>,./)
74.1Definition.A triangular array of functionsgill E L2 (D\"\" .r#\"I' P\"I)' 1 i k(n),n E is a Lindebergarray \037
if)
f\\j,
\037
\"(II)
(1) lim sup \"-.00 / (2)
Jim \"\"'00
OUf first
P\"i(g;i)< 00, and
\037 '\"'
1)
,,(,,)
LI I::
f
= 0, g;,dp\",
\302\243
> O.
/11\"tI>t)
is basic.For simplicitywe usethe followingnotation: assertion A)))
74.Stochasticexpansion of likelihoodratios
379)
of randomvariablesr,,:a\" -+- R is op\" (1) sequence lim p,.{I'\" > f.} = 0, I:> o.)
if)
\"
--
I
1X))
74.2Theorem.Supposethat
g.,'\0372
V\037\037:i
.-1
dQ = dP. exp ( .L (g\"i
- 2 .=1 .L
II
_
P\"i(gll/\302\273
\037
i
\037
k
- .-1) .L
1 k(,,)
k(\")
\"
-I).
1
( Lindebergarray. Thenfor every n E N
k(,,)
P\"/(g;i)
dQ\"i =
dP\"i
A
1 2 log g\"i + 1 , 1 (2 )
\037
i
\037
ken), nE No)
simpleTaylor expansion yields
X-210g where)
if (i+I)=;\"(X) x>-2,)
-
I
2(1 s) ds, x> -2,) r(x):=Jo sx 2 1
(
+ 2)
r (0) = 1 and satisfies
-
-
Ir(xl ) r(x2)1 Clxl x21 We
\037
if
Ixd < 1,IX21< 1.)
obtainthat)
--
dQ = 1:(,,) logdP. 1=1g\",
1 1:(11)
\"
II
=
. ,(g\",)
4 ;=1)g;,
\037
I:(It)
gIll
-1 4
i\0371
\037
1(11) 1\0371
I 1(11)
g;l+ .4
For every > 0 we have) \342\202\254
lim sup1',. { max 11-+(0
1
\037/
Ig\"i
I
> 6}
\0371(II)
1:(,,)
\037
IimsupL P,.;{lg\"il>6} ,,-+0;) ;-=1) k(II)
\037
limsup L \"-+\302\2531
i\"
1
1 2 \342\202\254
f
X;jdP,.i= 0
Ig..d>E)))
-
g;/(1 r(g\"J).
i\0371
nE
N,
is a)
Qlli(N,,/)+ 0,.,,(1)
Proof Firstwe notethat) ]og
(n),
)
380
by
Chapler12: Differentiability
and Asymptolic Expansions)
(2).We will provethat assumption lim p\"
n.....oo
L g;i L { /-1 kIn)
kIn)
P\",(g;i) > f; =
0, f. > o.)
}
i=1)
followsfrom (1)aswe shallseebelow.Choose t: > 0,b > 0, Thenthe assertion = X X' For random variable let X Then either arbitrarily. any luxi<6}.
-
kIn) p\"
kIn)
g;1
P\"I
i \0371
\037
2 > p,. .= gni .L g;j-.L 3} 1 .= 1 {
1(n)
k(n)_
2 +p,. Lgni { = j
or) kIn)
..
I;
- 2: kIn)
i=
1)
-
2 P\"i(gni)
}
i \0371
1(n)
p\"j
_ . (gni 2)
1)
3.)
- _ p,. max p.{ } { ;! _ - _) > L L P,.i(gni gll/ j. kIn)
kIn)
r.
L g;l L Kni 2 > 3 j'\" 1 j '\"
1(n)
p.{)
l(n)
-
1
-
i'\"
1(n)
1)
.'\037
and)
-
kIn)
L J>,.j(g;j) 1\"'1 I
\037
1
1)
9 '\" P.. < = t 2 L.. i-I nl (gn.
it
>H)
-2)1>t
P\"i(gni
i\037l
Since)
> f.
(g;/)
{
-
j
\037
Iglld > O
}
kIn)
.
2
;}) 9b kIn) 2.) = 2 L.. P..(gnl' )< 2
\037
t i-
_
nl
1)
kIn)
Pnj(gnj2)1 \037
L
g;,dPn\"
S i\"'119..11>6)
followsthat) lim supp,.
{
11-+0()
kIn)
L g;j i'\"L1 P\"i(g;i) 1=1
9 1msup '\" '-r e 1-1) l(n)
\0372
;S;
-
k(lI)
U
\"
I
n-ow
p\"j
>t})
2
) (gn/'
This provesthe assertion sinceb > 0 can be chosenarbitrarily small and we arrive at the expansion)
1.-1 4
1(n) dQ . n = dp'n exp ( gnl .\037
kIn)
P\"i(g;i)+'n
.\037 .-1)))
)
,
n
EN,
74.Stochaslicexpansion of likelihoodratios
381)
where)
> = 0, e > O.)
\"... p,.{lr,,1e} Jim
co)
Now, we useRemark 2.13and Lemma 2.10notingthat = 2d\037(J)\"i' Q\"J, P\"i(g\"i) = 8di(P\"h Q\"i) 4Q\"I(N\"I)') P,,/(g;l)
-
1
\037
i
\037
-
ken), \"eN. This leadsto the asserted expansion.
We want
0)
to replace the Lindeberg array ofTheorem74.2by an equivalentone.)
74.3Lemma. Supposethat
(g\"i)t
n c\037,
\037i\037l(,,)'
and (h\"i)t
\037i\037II(,,)'
nE
N,
are
triangulararrays satisfying) lim \"\"'00
(1) If
l:J -
1(,,) i=
(g\"i\302\273)
(g\"i
h\"Y dP,.i=
O.
t)
\037i\037k(,,)'
nE
N, is a Lindebergarray then (h\"i)1 ;;ii;;ik(,,)' n E
Lindebergarray. too. k(,,)
(2)
J \037\037\037
(3)
lim
(
(g\"i
i\0371
\"(\
-
-
P\"i(g\"i\302\273
i-L (P\"i(g;i)
It(,,) (h\"i
-
\037,
is a
2
)
P,.i(h\"i\302\273
i\0371
dp,.= O.)
= O.
P,.i(h;i\302\273
1)
\"\"'\302\253J
Proof (1)We have I h\037idP\"i
\037
\037
-
(VIg\037idP\"i + VI(h\"i g\"i)2dP,.i)2 2<J g;idP\"i+ J (h\"i g\"i)2dP,.i)'
-
Thisprovesproperty74.1(1).For the second propertywe obtainsimilarlythat) \"(It)
lim sup L \"\"'(0
1=I
It(n)
J
h;jdp,,1 lim supi=L \"'(0 \037
1\"\",1>1:
I
we have for > 0 and n E Moreover, \302\243
1(,,)
1(,,)
L
J i= 1 I\"\"d>c
L g;idp\"j i-I \037
and) k(,,)
L i=)
\037)
,(,,)
+L J g;idP,.i i= t
\"\"II>f Ic(,,)
J
g;tdp,.t\037L
'''\"'-9\"''>t
J g;ldp\"j'e > O.
I\"\";!>')
i=)
J
I\"\".
4
J 19\"d\037
I
g;idP\"j
'\"d>f) Ic(,,)
2 g;idP,.i+ dP\"i' 2 L J(h\"i-g\"J e i;;..1)))
382
12:Differentiability
Chapter
and Asymptotic Expansions)
74.1. Putting tennstogetherwe arrive at property(2) of Definition (2) We have 1:(11)
f
(glli
(
-
-
P\"i(glli\302\273
i\0371
1:(11) (hili
ll
i\037t
1:(11)
=
\037
/= I
I (gni
1:(11) \037
j
I (glli
\037 \037
- hili
-
hili)
2
-
P\"i(glli
-p,.,(h )
2
dp,.)
/\302\273
h\"J)2dP,.i
dP\";. n EN.
I)
(3) We have)
-
k(lI) \037
i=I
(P\"i(g;i)
P,.i(h;i\302\273
k(lI)
L (f (glli i= 1
\037
1:(11)
-
k(lI)
L 1=1
P\"i\302\253glli
h lli )2dp\";)ll2($ (glli
( L f (glli /=1
\037
-
=
'
h lli )2dP\"Y 2 (
hili)
. (glli +
hili\302\273
+ hili) 2 dP\"Y/2
k(lI)
J2 L $ (glli + h lli )2dP\"Y . i=1)
Since) J(gni + h ll J it
2
dP\"j
\037
2(Jg;ldP\"i + J h;jdp,.,))
followsthat) lim sup /1\"'00
I:(n) \037
2
I (glli + h ll,)
i=1)
which provesthe assertion.
dp\",< 00 0)
74.4Coronary.Suppose that (hlll)1
\0371;5\"(II)'
Jim
k(/I)
-
1 Q. 2!-1- .-h
.L J (\037 11-00 1=1 dP,.i
2)
ll,
nE
N, isa Lindeberg array satisfying)
2
)
dP,.i= O.
Then the expansion of Theorem74.2is valid (replacing (glli)hy
Proof.CombineTheorem74.2and Lemma 74.3.
0)))
(hili\302\273)')
75.Differentiablecurves
383)
75.Differentiablecurves) Supposethat (a,d, v) is a C1'-finite measure space.Let 9 be the set of all on (a,d) which satisfy P }'.We identify fJJ with a probabilitymeasures dP 1/2 2 p\037 subsetof L (Q,.9I, v) by dv- , Pe \037
( )
\302\243P.)
75.1Definition.Let > O. A curve t H P, from (
-
\302\243
I.....
-
\302\243,
t:) to
9 isdifferentiableat
1E ( E, E), isdifferenliableat 1= 1= 0 (01at Po) if the map 'f.: ( )\"2, ,0. at t = 0 then the derivativeis of the form t If I 1:is differentiable h, IE IR, where he L2(Q,d,v).) \037
\037
\"2
75.2Theorem.Let hE L2 (D,sI,v). PoE
Then
9 of a differentiablecurve t 1-+P, in
t
fJJ,
t
\037
1.h,
t
E\037,
;s the derivative at
iff
d!
(1)h = g' If(iP;for somegE L2 (a.sI,Po),and V
(2) J gdPo= O.
dP.
= 0 v-a.e,Let BJ = Proof(1) Let us show that h = 0 where \037 dv dP.o = h < 0 . Then for every D > 0) dv 0,
}
{
\037
fdpo 1( - dv _ 2 ) cJ' = 2\" dP, +1( VdP,
{)2
I
1Id\037
1
.dv
V
V
- -'
t
(j
2
dv
I
\037
lJ
h
\037
h
dv
dv
4
h 2 dv
)
ForcJ ->0 we obtain h'dv= O. DefiningBz =
1,
2 h dv
dP. = O. Defineg = h \037
1 h 2 dv . .J 4
8.
= 0,h > we obtain d;'o {- 1/2 o}
where dPo/dv> 0 and dv ) = 0 elsewhere. g condition Next, we show that g satisfies (2).It isclearthat) in a
similarway
\037
j
(
\037
.
t
11m2\" J
''''0t
1- . g 2 dPo . .P, -
t
(V;
2
= O.
) dPo
SinceL2-convergence L1-convergence we obtainfrom Remark 2.13) implies
-
.2 . 1 .2I sgdPo= hm = - hm -d dPo dPo ) ,-0 (P ' P,). '....0I J(V5f P,
2 2
t
t)))
O
384
Chapter 12:Uifferentiability
and Asymptotic Expansions)
But differentiabilityimplies
lim \037
'--0
(Po..P,) < 00.
d\037
I)
which provesthat J gdPo= o. We have to (2) Let g E L'1.(Q. Po) be such that condition (2) issatisfied. definea curve 11-+ at I = 0 and has derivative(1-+ h .p, which is differentiable
d.
with h
= g'
f!;.
\037
For this,letfo ,=
-
\037o
and define
1/2
12 I g,:= 1 \"4llhllty ./01/ + 2h.I E ( ) '1.
\037.)
Then for every IE IR, f, :=g,2 isa probability density since)
-
,2 (2 Jf,dv= 1 \"4l1hlltyJ fodv+ 4 J h'1.dv= 1.) ( ) at ( = 0 in f,1/2= Ig,l,/ E IR. Let us provethat 11-+g,lisdifferentiable L'1.(Q,.9I, v) with derivativet 1-+ h. For this.we have to show that) We have
I
\037
I 2 dv=O. lim J g,I-lgol-h 2 ) ,'0 ( 1
I
t)
First.we observethat) lim
1
'.0 t)
'
_ IKr(W)I-lgo(W)I2 (
h(W\302\273
v-a.e.
=0
)
This is trivial wherefo(w) = 0, and wherefo(w)> 0 there existsan t(w) > 0 suchthat Igr(w)1= grew) if III< e(w).Moreover,for sufficiently sma)) /)
1
1
-
t (lg,(w)I-lgo(w)1)itiIg,(w) go(w)1 \037
- 1:1(V;--
-\037
-
f\037 .-
\037fJ/2
- I)f':'2+ ll,,.'- I I
.1--IIh
\037
h
II\037IG'-!
\037
I
+ 1hl \037fJ/2 + 1hl 2 2
.)
Hence. the assertion follows from the theorem on dominated convergence.
0)))
75.Differentiablecurves
elementgE J.2(!1, ,r;/,Po> is a tangent vector curve t...-. at PoE 9'0 if there existsa differentiable J{ in &'0 whose
75.3Definition.Let &>0 of
\037o
385)
\302\243
&>. An
g' Vd;'o. E R. The setof all tangent vectorsof9\".at
derivativeat Po is t 1-+
t
\037
PoE 91'0is the tangent space1Po(BPo).)
d.
9.
= {gE L2 (D, Po):J gdPo = O}.for every PoE 75.4Corollary.1',.0(9) If E = (D,d, {Ps:9 E en is an experiment then we write 1P\037(E>
:!)e}).
1=1P8Q({P s E
of Tpo (3P)which iseasierto It issometimes useful to work with a subspace := handle.Define 0 {gE 1Po(9):g is bounded}
4
.)
(&\302\273
4
75.5Lemma. Thelinear subspace 0 (9) is L2 (Po)-dense in 1;.0(9).)
-
nE in is a sequence Proof Let g\" = g' l\\1gl <,,).n E Then (g\" Po g since D,o(9) and approximates 0 J (g\" \037(g,,) g) 2dPO S J (g\" g) 2dPO = J g2dPo. n E Igl>\ \037.
-
(g,,\302\273,
-
-
\037,
\037.
75.6Remark. Thereexistsparticularly simplecurves t t--+ P, which have tangent vectorsin
dp' = dv
4o(\302\243\037)'
dPo < It (1+ tg)
-j;'
It is easy to seethat Indeed,we have)
P,
1 ')
IIgILx:
at t = 0 with tangent vector g. is differentiable
(V
dv
.- dPo..) = 1 V1 tg - 1). dPo \037 dv
- < t
+
\037 dv)
':0)'''.L'-convergencefollowssince
convergespointwiseto g ( Vi +tg 11 C.t for sufficiently small t.)
which I
t t--+
I
--
dp, .
1 t
Let g beboundedand such that Po(g)= O. Thendefine
-
\037
\037
74 we needsome Beforewe are in a positionto apply the resultsof Section
infonnation the singularpartsof p, with respect to Po.If t...-. concerning P, isa curvelet be a Lebesgue of p, with respectto Po.) decomposition (:\037 N.)
.
75.7 Lemma. If t t--+ J{ is differentiablethen = o. lim ';'J{(N,) '....0 t)))
386
Chaptcr 12:Differenliability
and Asymptolic Expansions)
ProofLet g be the tangent vector of 1 dPo --dv dv
dP, -
f:(N,)= J
(V
N,
N,
t
\037
dv
f}, dv
, We
have
dv
p. - 2J (\037 \037 2 ) P. --.!..
\037
Now, the
2
)
V
and h = g'
P,
\037
-
dP.o
2
h
d\\'
dv
+2J
N,
1
2 h
(2) )
dv.
assertionfollowsfrom differentiabilityand from condition (1)of
Theorem75.2.
0)
for differentiable curves.) Thus,we arrive at the fundamentalexpansion
75.8Theorem.Supposethat I g
E
(\037). Then 1;.0
dp,\",Vi
dPo n
ProofWe
\037
P,
is differenliable wilh langent vector
t\".L g(m,)- _1 IIglipo+ opo(1), (!J Q\", EN. 2 ) (1;: 2
(Q,J) = exp
V
n
I
2
E
=1
n
have)
!i\037
nJ
-1-1 g] dPo
dP.\" r- 1/2
[(
;\037:\"
)
2
=
O.)
2\037
It is clearthat h.,'= g, 1 i n, n E N, is a Lindebergarray. Hence, Vn Corollary74.4implies \037
\037
dp,1t, Vi
(m- ) = dP.\" o
(g(w,) .t. (Vn
exp
Po(g\302\273
\037.
- i P./v,(N,/v,)+
Po(g')
O,'
FromTheorem75.2and Lemma75.7 we obtainthe desiredexpansion.
0)
The final assertion makes clearthat expansions may alsobeobtainedwithout differentiabiJityconditions.)
75.9Corollary.Letg E 1\037o(9).A sequence (Qn) 9 satisfies) 1 dQ\" + opo(1), (cg)= exp g(w,) 2 dP8 (V;; ) \037
1\" i\0371
(!JE Q\", n E N,
iff)))
\"g\"\037o
. hm ,,\037'X)
J
-
dQ , r: n (V dv
n
[
V
-
76.Oifferenliablecxperimcnls
Po - \037 --Po) - -21g . \037 J dv
2
=
d\\'
dv
387)
o.
\037)
Proof (1) Assume that the expansionof the likelihoodratiosis valid. curve with tangent vectorg. Then Moreover,let 11-+ p, be a differentiable dP1\", v\" _ dQ:-+ 0 (P,\"0) as n -+ 00. dPo dPO that the sequences of likelihood it follows From the expansion by Lemma6.12 t hat ratiosare unifonnly (P\037)-integrablewhich implies by contiguity lim d1 (P1\"'Vrit Q:)= O.) \"\037cn)
Hence,) lim vn dz (P1/I/ntQ,,) = 0
11-en)
which provesonepart of the assertion. followsfrom Corollary74.4providedthat we show that (2) The converse
= 0) Jim /1- nQ\"(N\") \302\253))
of Q.with respect to po.n EN. . areLebesgue decompositions \037;; N.) ( the same as in the of Lemma 75.7. 0) But this followsin
where
quite
way
proof
of the proofthat the precedingassertion It is obviousby a literal repetition remains valid if g E 1;.o(\037)is replacedby a convergent sequence (g,,)c:Tpo(\037)')
76. Differentiableexperiments) that M isan opensubsetof a finite dimensional linearmanifold.The Suppose isany norm of T We consider underlyinglinearspaceisdenoted by T and 1.1 an experimentE = (0,d, {P : x E we always assumethat E MD.Subsequently, x isseparable and that densities arc @ \037(M)-measurable.) .\037
A:
= 1,0 < Xi < 1,1 i k} and conM = {XE I-LI Xi siderthe experimentE = (D,.\037, {Px:X E M})where !2= {1,2, k},.r4 = 2f1 and Px = L X EM. It is clearthat M is an open subsetof a linear
76.1.:xample. Let
IRA::
...,
\037
\037
\"
Xi\342\202\254jt
I\"\"
I
\"
manifoldand the underlyinglinear spaceis T = {IE IRk: j;
Ii
\037
1)))
= O}.
388
Chapter
12:Differenliability
and Asymplotic Expansions)
and letv 1.91 be a d-finitedominating that E is dominated measure. Suppose dP.x 1/2 ThenX 1-+ XE into .9l In the followingwe assume dv ) t Mt maps L2(Qt t v). ( Firstt we notethat thispropertydoesnotdepend that thismap isdifferentiable.
on the dominating measure vl.9l.)
76.2Lemma. Supposethat E is dominated. Let Xo EM. Then the following are equivalent: assertions (1) Thereexistsa dominatinga1initemeasurevld such that XH' x E M, differentiableat Xo.
( ) 112t
i\037
(2)
i.\037
Forevery dominatingd1initemeasurevld the map x 1-+
v \037x
( ) 1/2,
differentiableat Xo.
ProofWe leavethe easy proofto the reader.
X
E
v) \037x
Mt
0)
From now on. we fix a dominatinga.finite measure vld. Let F: 2 factor2 is to avoid confusingfactorsin (\037)1/Z, X E MIf F(The fonnulas.) is differentiableat Xo E M, then the derivative subsequent \\') which satisfies DF(xo)is a linearmap from T to L2(Q,.9I,
n....
_
as t-O.) J(F(xo+t)-F(xo)-DF(xo).t)2dv=o(ltI2) 76.3Definition. The experimentE is differentiableat Xo EM if the map F is differentiable at xo.)
76.4Theorem.If E is differentiahleat Xo c M and DF(xo) is injective.then d 2 = L E .of. ( E) (Q, g' {g p ..): T,..o d E im DF(xo)}. PlCO
V
V)
ProofDenote D = {gEL2 n (\037e,
. D xo Elm d,Pxo)'. g ,!dP F(xo)} -dv.
.)
V
curve 1-+Px + (1) Ifg e D then g isthe tangent vector of the differentiable 'E ,). where t E T is such that g' v = DF(xoHt).Hence Vd:.: D Tpxo (E). 'E ( 8,8), be a differentiablecurve in (2) Let conversely 1-+ x EM} such that Qo= Pxo'SinceDF(xo)is injective there is an open))) {P,,: \037
\037\"
(-'.
\037
,
Q\037,
-
76.Differentiableexperiments
-.
389)
x E M} isa difleomorphism U ofXo E M suchthat F:U {P,,: neighbourhood 1 Thenfor somelJ > 0 the mapping1p: H F- (Q\037), E lJ, lJ), is a differentiabJecurve in M. Let t.= Dtp(O)E T and g be the tangent vector of Q( at O. It is then clearthat \037
(-
\037
\037
\037
1= Dtp(O)= DF(xo) which
I.
(g.va:;. )
that DF(xo)(I) = g
-
im:lies D. 1P\"o(E)
0)
va:;.. Hencege D.Thus,we have proved
As a consequence we obtainthat dim 1P (E) = dim TifDF(xo) isone-to-one. JCo
Sinceon l',.xo (E) there is a natural inner product,let us introduce an inner producton T such that 1',. xo(E) T as inner productspaces.) \037
76.5Definition. at x E M. Then the bilinear Supposethat E is differentiable function)
Bx:(s,t) H J (DF(x). s) (DF(x). t)dv,
.'i,t c T,)
covarianceof Eat x. is caJJedcanonical at x E M is positivedefiniteiff DF(x)is injective.) covariance The canonical
76.6CoroUary.Suppose that E isdifferentiableat x E M andDF(x)isinjective. Then 1PJ((E)
:!! (T,
BJC)')
76.7 Remark. If E is differentiable at x E M then 1
x + u ). 8 Bx(s-t,s-t)=1im .0t);di(Px+wP \037
Thisfollowsimmediatelyfrom
-
-
d22 (P\" t u' Hr) _1 J (F(x+ es) F(x+ 8 \037
et\302\273
2 d\\'.)
76.8Definition. at x EM and DF(x) IS Supposethat E is differentiable -+ The d-measurable Q T functiongx: which is definedby) injective. 1!dPJ( IE T,) DF(x).I = BAI,gJ<)' \"d-;' V
iscalledthe canonical derivative of E at x E M. Fromthisdefinitionit isclearthat the tangent vectorof the differentiable curve
\037.......\037+\037\"
\037E
XE M, IE (-c,c),isBAt,g,J,
T.)))
390
12:Differentiability
Chapter
and Asymplotic Expansions)
76.9Coronary.Under the assumptions derivativeg)C at of 76.8the canonical x E M satisfie.<;) (1) PAg:c)= 0, (2)
PABis,g)C) . Bx(t.
=
g)C\302\273
Bis,t), s,t E T, or briefly P:c(g:c@ g)C) =
76.5. ProofApply Theorem75.2(2),andDefinition
B\".)
0)
77.Conditionsfor differentiability) keep the notationof the precedingsection.In particular,recaJ] that x E M.) F:x 1-+2 We
)'''.
(\037
77.1Definition. A linear function DF(x):T -+ L2 (Q,.91, v) isa derivativeofF at x E M in P)C-measure if) lim \037 (F(x+ t)
'-001/1)
-F(x)- DF(x).
t)
= 0 in P,,-measure.
In many casesit is easy to find a derivativein Px-measure.The problemis in any case then to showthat it isalsoa derivativein /}(D,.r;/,v). Nevertheless, we may definethe positivesemidefinite bilinearfunction) BAs, t):=J (DF(x). s) (DF(x). t)dv,
s,lET.)
77.2 Lemma. Assume thaifor every x E M there existsa derivative of F at x in Px-measure.If)
-
! ! 1
F(x+ I) F(x)= DF(x+ ct).Ide, then)
1 (Px,PU') 8 BxH..,(t, t)d\037,
t E T,)
I
d\037
Proof We d\037
\037
t E T.)
have)
1 Px Px +,) = J (F(x+ t) 8
'
-
F(X\302\2732dv
f(i DF(xHI) 'Id\037)(t DF(x
+ ql)'Idq
= \037
)dV)))
77.Conditionsfor
=
differentiabilily
391)
1 1 1 .
J <J DF(x+ \037I)' 1DF(x+ '11)'Idv)d\037 d'1 8 J00) 1 1
\037
\037
.1- J J (B,,+\037,(/,
(B\" +
8 00 11 8 B,,+\037,(t, I)d\037.)
1\302\2731/2
!
'I'(I,
l\302\273l/2d\037d'1
o)
fordifferentiability. It is in The followingtheoremrevealssufficientconditions the spiritof Hajek[1972].)
77.3 Theorem.Assume Ihat DF(x)is a derivativeof F at x
= 0 wheredPx =
DF(x) every x E M and satisfies
dv
!
-
in
P\".measure for
0, v-a.e.If
1
v-a.e.lET,x E M, and (1)F(x+ I) F(x)= DF(x+ \037t). (2) (x,I) 1--+Bx(I,I), lET,x E M, is continuous, Ihen E is differenliableon M.) Id\037,
Proof Let en -+ 0 and In -+ 1E T. We have to show that 2 .11m F(X + 6\"-. tIt) F(x) (3) DF(x)'t dv = O. /1-00 J ( en ) In view of Lemma 77.2and by assumption (2) we have 1 dv s;J (DF(x)1)2dv. (4) lim sup 2 J (F(x+
--- - --
\302\243,,1,,)
\"-00
Put D =
-
-
F(X\302\2732
elf
>
Thenit followsthat
{\037 o}. lim sup /1-00
.\037
C\"
r 'b
(F(x+ E;nt,,)
-
F(X\302\2732dv
\037
J (DF(x)/)2dv.
D)
SinceDF(x)isa derivativeofF at x in Px.measureit followsby the Lemmaof Schef1e that (5)
-
.
2 + . 11mJ F(X 8n I,,)-- F(x) ---DF(x)'f dv=O, D
c,,-O
(
which implies that
lim \037
r
,,-QOf.\" 'b
)
en
(F(x+
\302\243,,1,,)
-
F(X\302\2732dv
= (DF(x)'t)2dv = f(DF(x)'t)2dv.))) L
392
Chaptcr
12:Diffcrentiability
and Asymptotic Expansions)
Togetherwith (4) thisentails F(X + tll t ll ) F(x)_ lim J n\"'co D'
-
(
t
DF(x). 1
dv
)
ll)
and by (5) we obtain(3).
2
=0
0)
theoremthat DF(x)isthe derivative It followsfrom the proofof the preceding
in L2 (Q,.9I, v) and
B)(
covariance.) the canonical
77.4 Example.Denote (w, x):=log
v \037
(w), WE
are satisfied: ing conditions (1) For every (J) E Q the function x \037
1\302\253(1),
on M. differentiable
M.Assumethat the follow-
x)
is real valued and twice
is P.x-integrablcand P.x(t'(.,x\302\273 = 0, XE M. (2) The function w......('(w.x) {\"(w, x) arc P (3) The functionsco......(w, x) @ (w, x) and w......
t'
t'
integrableand
-
ro
x eM. Px(t'(.,x) @ t'(., = PA(\"(., onM. PAt\"(., iscontinuous (4) The function x...... is usually calledthe Conditions This setof conditions of Cramer and Waldo Somecomments might be illuminating. and finite onQ for that the densities are strictlypositive Condition (1)implies can be eachx E M. Conditions (2) and (3) are satisfiedif differentiation well-known with Px-integrationin a suitable way. In fact, the interchanged function) of the log-likelihood property x\302\273
x\302\273,
x\302\273
P.x({(., x\302\273
= sup
\037\302\253((.,
y\302\273
y\037M)
al x then the derivativeiszero. isdifferentiable that if y...... implies of regularitycondition This is the background (2) is satisfied (2).If condition derivationand integrationare neglected, with interchanging and if difficulties \037(t(.,y\302\273
then we obtain)
0= D(P.x(t'(.,
x\302\273)
= \037(t\"(., + PAt'(.,x) @ x\302\273
t'(.,
x\302\273.)
Thisservesas a motivationof condition (3).
Let us show that conditions (1)--(4) imply differentiabilityofE. Thisisdone by means of Theorem77.3.Obviously,
DF(x):= F(x). t'(.,x)) \037
is a derivative of F at x
in
x E M. Moreover,it \037-measure,
is clear
that)))
77.Conditionsfor
393)
differentiability
For the proofof 77.3(2),we notethat condition 77.3(1),issatisfied. t) 2dPx S(t'(.,x).
Bx(t,t) =
xEM,tET. =-Jt\"(.,x).(t,t)dPx, Now, continuityfolJowsfrom (4).)
77.3 (1),sincethe In the precedingexample it was easy to derive condition densitiesare everywhere positive.If the densitiesare zero with positive v-measurethen things are morecomplicated.) 77.5 Disctmion mt:asureand define (Hajek[1972J).Let vl,1af bea dominating
h(.,x) 1=
M. We consider the casewhere M R is an openinterval. The mappingF is given by x 2h 1/2(.,x),X E M. Let us establishthe of Theorem77.3under the followingconditions: assumptions (1) The mappingx h(.,x),X EM, has a derivativeh ' (.,x) ELI(fl,.9I,v), x E M, in v-measure,which is .91@ [j'(M)-measurable.) X
E
\037
\037x..,
\037
\037
.11:2
(2) h(w,x2)-h(w,x1) = J h'(w,y)dy if wEfl,X1 <X2 . .1'1)
(3)
For every x E M) 2
0< l(x)1=J hl(.,
< (h\037x)) h(.,x)dv 00) X\302\273
and x \037
l(x),x EM, is continuous.)
Let us denote
1 h'( ., x)
S(.,X)I= 1
where
h(.,X)1/2)
h(.,x) > 0,)
otherwise.)
\037
First,we have to showthat DF(x)= s(.,x),X EM,isa derivativeof Fin For this,we notethat) Por-measure. \037
\037
h(.,x+c)-h(.,x) - -)
----\302\243)
=
..
h1/2(.,x+e)-hl/2(.,x) 1/2 -- (h112 (.,x+c)+h t)
e > 0, x E M. Then it
isclearthat on D =
(.,x\302\273,
{h\302\253.,
x) >
O})))
Chapter 12: Differentiability
394
and Asymptotic bpansions)
(., + e) -h l/2(.,
lim (h 1/2 \037
t....O t:)
X
X\302\273
s(.,X) in v-measure.
=
Moreover,we observethat DF(x)= 0 where h(.,x) = 0 by definition,and 77.3 (2),is satisfiedby assumption condition (3).Thus,it remains to verify
77.3(1). condition Beforewe do so,we showthat thereare a setNEd, v(N) = 0,and for every x E M a number 11(x) > 0 satisfying)
...
x ,,(x)
J
x -If(x))
Is(.,y)ldy
For every x EM letheX) > 0 be such that) \" + cJ(x)
-J l(y)dy<00.
x -'(x))
Since) x t -'(x)
Xt
f Is(.,y)ldy x -6(x) \037
it
(2c5(x\302\2731/2
1/2
6(x)
J s(.,y)2dy x -6(x))
)
(
followsthat) x + 6(x) x
-J Is(.,y)ldy
-
setof v-measurezero.Since{(x c5(x), Let Nx be the pertainingexceptional subcover.Let (Xj)jE be x+ x EM} coversM there existsa countable i E N} coversM.If we put N = U Nx, then such that {(Xj c5(Xj),Xi + i 1 is proved. the auxiliaryassertion 77.3(1).Let WE Q\\ N. We shall Now, we beginwith the proofof condition = provethat for every pairXl < X2, XjE M, i 1,2,)
-
b(x\302\273:
\037
QO
c5(x,\302\273:
\037
= J sew,y)dy. hew,X2)1/2- hew,X.)1/2 X2
XI)
We have
two cases. to distinguish
Case1:Assumethat y.-..hew,y) ispositiveon (xl'x 2). Thenit isclearthat
for every e> 0)
hew,x 2
-e)1/2-
X2
hew, Xl
l
= J sew,y)dy + e)112 XI+l)
sinceYl-+h(w,y)is boundedaway from zeroon [Xl +e,x2-e].Now, the followsfrom the choiceof N. assertion in Case2:If Xl' X 2 are arbitrary the intervall(Xl' X 2) can be represented the)))
78.Examplesof differentiableexperiments
395)
form) (\302\243)
(XI'X2) = U (aj,Pi) u {ye (XI'X2):hew, y) = O}, i:= I)
intervalson which y........ where (ai'Pi),i E N, are disjoint h(w, y) is positiveand = = hew,ai) 0 if XI' h(w, Pi) 0 if Pi X2' It is then clearthat) <Xi
\037
\037
-
hew,X2)1/2 h(w, XI)1/2=
w
L (h(w, fJi)1/ 2 i= I C()
- hew, a;)1/2)
p,
X2
Ct,
XI)
= L J s(oo,y)dy = f s(w,y)dy i= t IS)
sincey....... s(w,y) vanishesoutsideof U (ai'Pi)' i=
I)
78. Examplesof differentiableexperiments) First,letuscontinueExample76.1.)
78.1Example.Recall the notationof Example 76.1.If v 2VX;, 1 j countingmeasure then F(x):j....... \037
\037
I
d denotesthe
k, X E M, and
tE T, XE M, DF(x):1.......(tj/\037)lo!j\03711' is the derivative in v-measure.It isclearthat condition 77.3(1),issatisfied. we have) Moreover,
A:
BX<s, I) = L Sjlj/Xj' S, tE T, XE M, j= t)
which iscontinuous and therefore Theorem77.3implies that E isdifferentiable
on M. Sincethe canonical covarianceis even positivedcfinitcthcre existsa derivativegJC at every X E M; it is of the form) canonical ( - Xl' gx:j.--.
- l' ..., Xj_
1
Xj'
-
Xj+ It
...,
XA:),
1 \037j
\037
k.)
For this,we assumethat M Next, we turn to exponential experiments. = isopen,henceT
\037
IRA:
IRA:.)
78.2Theorem.Supposethat Ee 8(J\\.f)isan exponential experimentof rank k with
densities)
dP.x =
dv
A:
C(x)exp(i=LXiI;), I)))
XE
M.
396
Chapter
12:Differentiability
and Asymptotic Expansions)
for somedominatingu-finitemeasure v I.
-
P,,\302\2531;
-
P,,(1;\302\273
PA1j\302\273),
\037
\037
Proof For the followingrecallTheorem77.3.We observethat there is a
derivativein v-measureat x E M)
' t,(li-
1 1 C (x) . t + DF(x). t = 2 F(x) C(x) ( =
\"
1
2 F(x)
\"
ti 1\0371
1;
)
PxCli\302\273,)
i\0371
by Lemma 5.6.The formula for Ox and
77.3(1)are immediate. of Lemma 5.6showsthat Condition 77.3(2) issatisfied Another application
which provesthe assertion. 0)
.,....,
The . .functions
\037,
Imp1lesa 1 =
1;\"
k
are calledlinearlyindependentif L
= a\" = 0
I\"\"
1
a,Ii= constv-a.e.
.)
78.3Corollary.Assume that
.,
the
conditions of Theorem78.2are satisfied. If
.. 1; are linearly independentthen the canonical covarianceat derivative at x E M i'i positivedefiniteand the canonical g\" = r,,-l(Ii PA1J)l \037,
-
xE M
is
\037i\037k')
in Section 76. Let us return to the generalcaseconsidered If H = (D,.c1,{\037: x E M})and II':U --.M, U \037(, then we may definethe that U is experiment E = (D,.91, y E U}).In the followingwe assume \037
{P\037h\302\273:
open.)
78.4Theorem.Supposethat E = (D,.9I, {Px: x E M}) is differentiableon M then covariances Bx. x E M. 1/ tp: U --.M ir differellliable with canonical on U and the canonical covarianceat ;s E = (U,.9I, E Y U}) differentiable Y
E
U ;s
By
{P\302\245'b):
(s,t) = It(y) (tp'(y) . S,
tp' (y) . t), S, t E IR(.)
and dcnotc ProofLet v sf be any dominatingq-finitemeasure 1/2 1/2 dP dP F:x r-+ 2 dvx , X EM, and F:Y 2 . d:
\037
..
78.E'l:amplesof differentiableexperiments
=
DF(tp(y\302\273
. tp' (y),
immediate.
Y
E
397)
U. The formula for the canonicalcovarianceis
0)
78.SCoroUary.Assume that the conditions of Theorem78.4aresatisfied. (1) If B;;,; ispositivedefinite.x E M, and IJ' isan immersionthen ii), ispositive y E V. definite. derivative ofE at x E M. then the canonical derivative (2) Ifgx isthe canonical K). of at Y E V satisfies \302\243
s, tp'(y)'i,,)= B..,(,,)(tp'(y)'s, B\037()I)(tp'(y)'
glf(Y\302\273'
sE Rt
.)
For (2).we notethat (1)isobvious. ProofAssertion
. d;!! f/2
= DF(y)'\" iJ,(.',K,) (
. (tp'(y) . s)
= DF(
tp(y\302\273
(
d\037\037\"
= Bep(y) (tp'(y) . s,&,(y),
SE
f/2
R(.
0)
78.6Remark. Under the assumption (1) of Corollary78.5 the canonical derivative of E at y V is uniquely determined by 78.5(2).It can be given explicitelyin the followingway. Fix y E 11(1and let A: -+ T be a linearmap. \342\202\254
\037I
Definethe adjointA* for Bw(y) by = A* t) if BIp(y) (As. t)
s E R(, t E T. rank the map tp/(y).:tp/(y) is invertible. Then \302\273,(s,
Sincetp/(y) is of full gy
= (tp'(y)* 0 tp'
(y\302\273
-10 ,p'(y)*g
\037()I)'
Let us appJy the formulasobtained sofar to a simpleparticularcase.)
is ..., (y): {1,....k}. IfO
78.7 Example.Suppose that U = {(PI
\302\243;
IR
is an openintervaland
a family of probabilitymeasureson k, then '1':Y\037(PI(Y)\"\"'PIc(Y\302\273' 1\037i::;' the ]inear E V into M m anifold of y V, maps Usingthe Examples76.1and 78.1. =: If notations introducedthere we have (D,.9I, &) = (D,.9I, {PIp(y):Y E V}) then is differentiable on V. Let us compute the canonical 1p is differentiable covariance at y E V. FromTheorem78.4and Example78.1we obtainthat By(s, t) = st/(y), s,t E iR,) \037
y E V}
Pic
(y\302\273,
l.
\302\243
where) Ie
I(y) =
L i;:;; I
(1p;(y\302\2732
tp,
() Y)))
=
logfJ)2).
PIp()') {(Dy
398
12:Diffcrcntiability
Chaptcr
and Asymptotic Expansions)
In caseI(y) > 0, the canonical derivativegy satisfies
i
tp; . sl(y) gy = s t--ItpiY) (K.,(Y)i, s y\302\273
\342\202\254
IR,
\302\253
hencewe obtain) 1 . Ipj(y) 1 \037j k. gy:JI-+I(y) Ipj(Y) In otherwords,gy = I(y)- Dy log'P.)
'
\037.
\037
I
are given as\"smooth\" Someimportantexamples of exponent. subexperiments Theseare sometimes calledcurvedexponential ial experiments. experiments.)
78.8CoroUary.Assumethat the conditions If of Theorem78.2are satisfied. = is then E U M is on E (D, .91, tp: differentiable {PI;(Y):Y U}) differentiable U andthe canonical covarianceat ye U is)
-.
-
By(s, t) =
s'I-:yt,
S, t E ( , \037
In caserlj)()') is positivedefinite and 1p;''1an \037=1p/(y)'r'l'()')1p'(Y). immersionthen ispositive derivative at Y E U is too,andthe canonical definite. where
-
J\037
gy
=
r, 1 tp' (y)'(1;
PIf(Y)
1JI
\037i
fA')
covarianceare conseProofDifferentiabilityand the form of the canonical Theorem the of 78.4. F or canonical derivative 78.3 quences apply Corollaries and 78.5.
0)
78.9 Example.Consider the experimentE = (IR, 14,{va,nl : a E !R, q2> O}).Let = U {(a,(12): a E R, (12)O} and M = {(Xl'X 2): XI E IIi, X 2 < OJ. If we define p;,J\0371
by dPJC
2
l w+X2 W), d)' (w)=C(x)exp(x
w\342\202\254\037,xEM,
:;
= p.,...., where 'I'(a.a') .(a.a') U.The v.... ( ). of rank 2 and we may apply experimentE = (R, {P: M})isexponential then we have
fJI,
x
X
E
E
2\037'
oi.
(I) E IR. We have Corollary78.8.Let Ii (w) = w, 12(w) = = 4a2 u 2 + 2q4, Cova'1J1(J;,1;)= Varo. 1(J;)= q2,Var a,01(1;) 1J
which yields
= f.,...., (2::'
2a.)))) 4a'\037\037:'
2au2
78.Examplesof differentiableexperiments
399)
Since) 1)
tl\"
(a,0'2)=)
t \302\260)
2(7'4)
is of full rank it followsthat) 1)
o)
=) i:.(1'% \302\260)
The canonical derivativeis given by) \037
Kg,
(1'%
(W ) =
' ( Y )' f.-111' )I
w
((.02_ a2 _ 0'2)
= (T.\"', (YW' '1\"
=
-a
(W'
\037 \037
\037
\",) W
=
V2a
\037
\037)(w'
a' \",) (W _ a)' \",), WE
Ul)
\037
\037
78.10Example.Consider the experiment E = (\0372, {V(! '[2,K) E U}) ) : (a,b,(12, b.) \302\24312,
Ie
\302\253(1'%
\037
t)
-
where U == {(a,h, (12,t 2, ,,)E R 2 X (0,00)x (0,<X) x R: (121'2,,2> O}. Then the Lebesgue of this experimentare) densities 5
C'exp( L1 'P}(a,b,(7'2, t 2, ,,).Ij) where Tt:
\302\253(.0..
J; T2: (W1> W 2)'-(.02, T3:(W1'(.02)\037wf, (.02)'-(.Ot,
T4: (W.,W2)\037\037W\037, and)
'1'1= tP4
=
at2 -b\" \0372
'
Ts:(W1,W2)\037W1W2')
_'[2 = '1'3 2 t 2 ,(2)' ,,2'
b(12_aK
= t2 _ ,,2 '1'2 (12t 2 _
\302\253(7'2
_0'2 \" = '1'5 _ _ ,,2 0'2'[2 2(0'2'[2\0372)' .)
If we define P:c \0372, X E M = 'I'(U), I
by)))
-
400
Chapter
12:Differentiability
and Asymptotic Expansions)
S dP. = C(x) exp(L Xj 1j) dA J= 1\037
...,
1)
then we have v(:)
= (\0372
2
fl.,,)wheretp =
P\037(CI,b.(J1.
t\037)
('Ill'tp2'
tp5)'The experi-
ment E= (R2,tM , {P,,: XE M}) is exponential and we may apply Corollary 78.8.Elementarycomputations matrix r'#!(CI,b.tz2,r2,x), yield the covariance given by)
= (12 Var(7;)= '[2 Cov(\037,7;)= K Cov(7;,1;)= = 2aK Cov(\037,13)= 2a(12 Cov(7;, '4) = 2b,2 = 2bK COy (\037, COy (7;.\037) = b\"-t ar2 Cov(\037,15)= aK + b(12 Var(\037)
1\037)
Var(1;)= 4a2 q2 + 2(14
Covel;, = 4abK+ 2K2 Coy(1),1;)= 20-2 K + 2a2 K + 2abq2 \037)
Var(r.)= 4b2 t 2 + 2'[4 Cov(14, 15)= 2'[2K + 2b2 \"
2ab'[2 + K 2 + 2abK+ a 2 ,2+ b 2 q2. Var(Ts)= (12'[2 The Jacobiantp' (a,b, 0-2, t 2, K) is (with Ii 1=(12t 2 K2)) +
-
t2 J
-K
-K
q2
J
J
0
0
J
0
0
0
0
- at J2 aK1 -
4
bKt2 2
bK 2
L1
2
'[4
b Kq 2_ aK2 ,d2
-
aKq2 L1
K
b0-4
2
2
2Kat2
-
2
b(K ,d2
-
2Kbq2 a(K2 + (12,2)
-
L1
2
K't 2
2112
2L12
\037
2
(14
K (12
--
2/.1 - Kq 2
K
2 L1
2
-Kt2 L1
2
2
J2)))
+ (12t2)
-
/.12
(12t2 + K 2 L1
2
78.Examplesof diffcrentiableexperiments
401
Thisgivesat (a,b,(12,t 2, K)
rip
. tp' =
1 0 2a 0 b
0 0 0 0 1 0 0 0 0 1 0 0 2b 0 1 0 a 0 0 1
and ascanonical covariance
'[2
-K
A
A
-K (;2 A
f
== (tp/)'
A
. r\", . tp' = 0
0
0
0
0
2
_ Kt 1
K
2A 2 K
0
0
0
'[4
0
0
0
2A 2
2
JZ
-
(14 2A 2
2A 2
2 ..1
-Kt2 _- Kq2 2 2
0
A
The canonical derivativeisgiven by)
K(12
A
q2\"(2+K2
._\037-)
-a -b wi - a - - '[2 WI
ga.b, (11,fl./((w I'(2) =
W2
(rill'tp/) -I
Z
W\037
b2
WI W z
-
1
0
0
1
-2a
0
0
-2b
-b -a
0 0 1 0 0
0 0 0 1 0
0 0 0 0 1
WI
-K
-a
- a - (12 - _ '(2 (:)1 - - ab w\037
2
b2
W2
ab)
w1-a
wz-b wi
=
(12
K
(J)z
=
.- b
(W2
- 0)2 -(12 - b)2 - t
(WI
-a)(w2 -b)-K)))
(WI
2
402
12;Differentiability
Chapter
and Asymptotic
\037xpansions)
79. The stochastic expansionof a differentiableexperiment) there.We are introduced 76 andkeepthe notation the caseof Section Consider ratios of the likelihood in the stochastic interested expansion
i
(II)
n( 1\"_
dPJI,. +/\"/
w dP. (
t
)()
i'
ro -
\342\202\254
,
Dklll) n
\342\202\254
N,
\302\273)
n E N, is a triangular array in T satisfying the following) where (/'1i)15:1\0371c1\'")
79.1Conditions. lim Max Itllil=O, (1) 11-
l:i I
IX)
\037
1(11))
1(11)
(2)
lim sup L I/IId2 < 00,
I-I)
II-IX)
the validity of the nonnon T. SinceT is of finite dimension doesnotdependon the particularnonn. conditions is) The main resultof thissection that E isdifferentiableat x EMandDF(x)isinjective. 79.2Theorem.Suppose Let(tlli)1\037if1(1I)' ne N, bea triangulararray satisfying 79.1(1)and (2).Then) where 1.1is any
kIll)
lJ cy E
D\",
n
dP.
(cy) = exp
;;,,'\"'
kIll)
BA/llj,
gx(Wj\302\273
-1
11\"'
\"2 i\0371
(i\0371
BAt llj , 'ilt)
+ oP:
(1\302\273,
EN.)
rojED, 1 i ken), nE N. We shall n E N, is a apply Corollary74.4.For this,we have to show that (hll Jl Lindebergarray. SincePx(h;i)= BAt lli , t lli ) by 76.9(2),Condition74.1(1) 79.1(2). followsfrom Assumption 74.1(2) we notethat) For the proofof Condition
Proof Define hlli(w/)t=
\037
B:.;(t\"i,KAcl)j\302\273,
\037
\037/\0371(1I)'
1(11)
L
1=1
J
gx)2dPx BxUII;'
B\",('\"j.gx\302\273c
1(11) \037
L
;=
Bx(tlli,t lli
,,2)
:> B.(g.,gJ mu 8.(t o,'t.,) I
which
BAgx,gx)dPx)
J)
\302\273
1)
s/\037\037.))
together with 79.1 (1) and (2) implies74.1 (2) 10 view of
d,f'x) and
IgxlE L'l(D, lim
Px BAg\",gx) 11-00 {
>
82
max BAllli,llli)}
:i
I i;'i
kIll))))
= O.
19.The stochastic expansion of a differentiableexperimcnt
403)
the assumptions n E N, is a Lindcberg Hence,(hlli)t array. To complete of Corollary74.4we have to show that) :Si\037k(II)'
lim
k(lI)
.L J ( I\" 1 11.'00
-
dP.\"+ d
I\037
Px)
- - 21 1
2
BAtni, g),)
)
dPx=
o.
of differentiabilityof E at x E M and is an obviousconsequence 79.1(2).Thus,wc conclude by Corollary74.4that the asserted assumption = is valid,sincePAg,,) 0 and P,,(gr@ gr) = Br by Corollary76.9, expansion and sincea similar by differentiabilitythat) proofas for Lemma75.7 implies But this
k(lI)
lim \"\"'00
L
1=
= o. P,,+,\", (Nx +1...>
0
I)
79.3Corollary.Suppose that Fisdifferentiableat x EM and DF(x)i<; injective. Thenlorevery bounded sequence (tll) S T)
dP;+Vn'\" dP;
(q) = exp Bx
t ll
( (
,
1
II
1 ) 2
gAwj) I\037I
V\037
BAt ll , t ll) + op;(1))
)
nE N.) (gE (111,
caseof Theorem79.2which we discussa little Thereis an importantspecial morethoroughly.)
!!l; 1(.-
dP.\" = 79.4Examples.(Hajek and Sidak[1967J). Let M = x),X E IR, d. and assumethat the conditions of Discussion 77.5 an:satisfied. It isclearthat =: of x E R. Then we have) l(x) I, x E R, is indcpcndcnt
B,,(s,t)= stI, s,t,xEIR,) and) _ g\"
j(.- x) I /(.-x)
= 1.
.1\\/(.--x\302\273O),
XE R.)
I
function F of hasa right inverse Recall,that the distribution
F-1: s...... inf {XE R: F(x) s}, SE [0,1].) 1 it isnevertheless true that [Ioc. cit.p.21,34] Although F- 0 F id R in general, , on jo F-1 = j(x) A-a.c. \037
=+=
Ii:'! 1-::
(F(x\302\273
f(x)
{I>O}.
I we may Hence,definingIf':=-(joF/loF-1)1\\/oF-'>ol
write)))
404
Differentiability Chapter 12:
..
-1 . (f)(F(.-
gx = We
I)
Px-a.e..
x\302\273
and [Ioc.cit.p.19,20]) notethat (f) E L2 ([0,1]) 1
Jo q>
2 d)'
If (t1l')1
=
nE
\037i\037k(II)'
arrive at)
k(lI)
I,
1
J0) q>d)' = O.
N, is a triangular array satisfying 79.1(1)and (2),then we
dP.
n i-1 qJ E
and Asymptotic Expansions)
k(lI)
(cy) = exp( L Ini l=t
;;'\". x
...
' q>(F(Wj
- - 2I _
x\302\273
k(lI) \037
I=t)
t;i+
Op,.,,(1\302\273,
QII, n EN.)
..-
...,
isrelatedto rank statistics. Thisform of the expansion If (!)= (w1, WII) isa let Rlli(Q) be the rank of Wi in Q), 1 i n. The ideaof rank randomsample, StatiStICSIS to repIaceq> (F(00,
general q> it
-x
\037
\302\273
- --=-- I.f ) ( n+1 to Rlli
by
q>
\037
(W\302\273
q>
is technicallymoreappropriate replaceq>(F(w,
R\" ,(01)/11
n (R\"/(OI)
J - 1 q>d)',
1
.
IS mcreasmg, b ut x\302\273
fi
or
by)
:5i < n.
)/n)
that It hasbeenshown by HajekandSidak[1967] 1(11)
IimJ iL -1 11-
[
\302\253>
t lli
2
R\"j(w)/II
n
(
(R\"I(OI)
J-
=0. q>d).-q>(F(wj-X) Px\"(d(!})
)]
1)fll)
we provethis fact at the endof this section.) (Forthe sakeof completeness Thus,we arrive at the expansion)
-
k(lI) dP. = _I L . n q>d)' n Lilli exp J i=1 ;;'\"1 x ) (i\"'1 ( (R\"I-1)/1I ) 2 i=1 + op\"(l) A:(II)
R\"/III
11(11)
t\0371
It)
is the startingpointof the theory of rank tests.) Thisexpansion 79.5Theorem.(Hajekand Sidak [1967]). Suppusethat the conditionsof n E N, be a triangular array Example 79.4 are satisfied,LeI (tlli) 1 \037i\037l(II)'
satisfying 79,/
k(lI)
(1)and (2),and in additioni=L
t lli
= 0, n EN, Then
I)
- q>(F(w )] P;(dq)= 0, 2
Jim
J
11\"'00
[1-1 ( \037 tlli
n
R\"rUII (R\"j(w)-1)/1I)))
q>d)'
j
_X\302\273
XE tR.
79.Thc stochastic expansion of a differentiablee'l:pcrimcnl
405)
ProofThe proofisdividedintoseveralsteps.
(1) Let
cp\"
i)
=
\"
II
LI n J cpdi.'1[i...=...1. i.) . n E
iJ
i\037
\"
\037.
II
\
t
-
I
-
L2 ([0,1]), J0 cp\"d)'= 0, \"E N, and J0 (cp\" cp)2d)' 0 by well-known martingaletheorems. (2) Fix x E and let V\".,(qJ)= F(w, x), 1 i n, ([) E (1\".Thenit isclear over [0.1] under that V\", I' and uniformlydistributed V\".\" are independent P;. Let (VII,IIJ' . V\", [IIJ) be the orderedsample(V\".I' U\".,,).Recall,that fromeach otherfor each RII .,,)are independent (VII.!I)' VII.!,,)and (RII,I'\"'' n E N. Define)
Then
cp\"
E
-
...,.., ..., \037
\037
...,
\"
'PI!=
\037(cp(V\",IiJ)'l [i-;'I .\037)'
i\037l
2
\037
1
nEf\\L)
-
1
-
0, n E N, and J (V'\" cp)2d). O. 0 the first two assertions are to see. The third isprovedby Scheffes Again, easy lemma.Jensen's inequalityyields)
Then'P\"
E
L
([0,1]), Jo
'P\"dA.
1
1
J'P\037dA
\037
o A
=
2
J cp
nE N.
dA,
0)
littlemorelaborious is it to show that
11'\"
-
cp
i.-a.c. To thisend notethat)
!
'P,,(s)= P:(cp(U\".[\"S)+ =
n!. 1
!
I\302\273
1
cp(t)
_ t)\"-(\"$)-1 1[lIsJ(1 dt) [ns] (n _ [nsf\037 I)!
=, J cp(l)f,,(I)dt,SE [0,1]. o)
0 :f:sand f,,(t)dt 1,\" Since.f..(t) Lebesgue pointss of 1
if t
all
(3) Let us show that f \037\037\037
(Y'.V\037't
cpo
J0
)
E
\037
-+ cp(s)for N, it followsthat 'P,,(s)
dP:= O.)
under the integral we get three terms.The Evaluatingthe quadraticexpression
first
is)))
406
-
79.Thc stochastic c'l:pansionof a differentiableexperiment) 2 J tp,.
(n
,.
i. ,. d1';= L J tp,.2 d\037 + 1) 1= R..,=1 (n + 1 )
R,.t
\\
' 1\" \037= fV';dJ., L 11'; I
=
nj-l (n+l)
is) the second
6)
1
J q>2 (U\".,)dP:= Jo) lpld;',
and the third is) R ,. -2J '1',.(n+l lp(U\",)dPJ( ) rtJ
.
i
J ,11',. =-2,\037 n + l lp(U,.,liJ)dPx\" .=1R\".=.
(
'
=
)
- -2\"L '1';(\037 = - 2 '1';dl. + 1) n
i= I
1 f
n
6)
Thus,we obtain) I
('P.(
nR\037'I
)
1
u..,))'dE';
'PC
-
1
1
= J tp;d)'+ J q>2 dJ. 2 J tp;d)' o 0 0
! 1
-!
= lp2d)'
1
tp;dJ.)
which tendsto zeroby part (2) of the proof. (4) We show that
I !\037\037
-
= O.)
dE'; ('P.V\037'I) 'P(u..,))'
Indeed,we have) f
('P.V\037
'I) 'P.(- ))'dP: /\037\\
=!ni-l i(
tp,.
which
\037 - lp,. \037 (n+1) (n+1))
2
-
= J ('1',. lp,,)2dJ. 0)
tendsto zeroby (1)and (2).Thus,(4) folJowsfrom (3).)))
79.The stochastic expansion of a difTerenliableexperimcnt
407)
to provethe assertion of the theorem. Since) (5) Now, we are in the position cp\"
R. ----!!!..... =n
(
+ 1)
n
- I)f\ cpd)',
R\"('\"
1
J
(R\",
\037
i
\037
n, n EN,)
we needonly show
\037\037':
1'; ([;\302\243:
I., n\037;, ((0.( )
-
(U..
(0
,\302\273)
= o.)
J)
under the integralwe have) Evaluatingthe quadraticexpression
1;1J ;\0371
+ =
( ;1) ({'(V\"J dP;
(
({'\"
n
J ,\037/.\".j ((0.(n\037't
(! k(\")
((
/\037I
\037
)
1
J
I\037')
+
2
R .
k(,,)
2
)
I,,;
t\037;)
- ((0.V;j,) -(O(U.,)YdP:)
dP:
(0(U.j
(O(U.,\302\273)
\302\273)
((0.(:\037\037)
2
(!
)
J
1(,,) i\037l
1;/ J
) ( (n
((0.(
:\037\\
which tcndsto zeroby
- )( 1) ({'(VIII)
R I ({'\"
({'\"
\037
) (O(U.,)Y dP'\
(
R 2
\037\037
79.1(2) and part (4) of this proof.
1
)
-- q>(V\"2) dp:
)
0)))
Chapter13:AsymptoticNormality)
12.The general framework for the exampJes considered in Chaptert 2 is the conceptof of experiments asymptoticnormality.It means that a sequence converges shift experimentthe type of which iswell-knownto weakly againsta Gaussian us from Chapter11.From the resultsof Chapter10 we obtainstatistical the sequence assertions of experiments. concerning In Section80 we explainwhat is meant by an asymptotically normal of experiments. For such a sequence the logarithms of the likelihood sequence are linear. This observation leadsto the fundamental processes approximately of a central of stochastic or randomvariables.We concept sequence processes illustrate both asymptotic normalityofsomewell-knownexamples considering the resultsof Chapters10,11und This chaptercombines
finite dimensional as well as infinite dimensional cases. Beforeasymptotic decision was in the form of Chapter10 available theory
the mostimportantinstrument for proving asymptotic optimalityof decision functionswas the so-called exponential approximation. Essentially, exponential approximation can befoundalreadyin the early papersof Wald [1943] and LeCam[1953]. The hiddenargument was isolated Let LeCam by [1960]. us mentionthe papersof Hajek[1972and 1973], and Hajekand Sidak[1967] isappliedin an explicit wherethe exponential approximation way. We present thismethodand itsrelationto an early ideaof asymptotic sufficiencyin SCl,.'lion 81.By way of examplewe obtainthe famousresult of Hajekand Sidak[I')()7] concerning asymptotic sufficiency of the vector of ranks.Finally, it is convenient to prove at thispointthe old theorem of Bernsteinand v. Miseswhich becameimportantagain in the paperof LeCam [1953]. of globalasymptotic At thispointwe have to mentionthe concept normality which is relatedto asymptotic sufficiency and is the intrinsic argument of Wald'spaper[1943]. Thispath pointedout by Wald was continued by LeCam and Micheland Pfanzagl[1970], and Pfanzagl[1972a] [1956], Hajek[1971], betweenlocaland globalasymptotic [1972b]. Recently,the relation normality and has beeninvestigated [1983], by Droste,[1985]. by Milbrodt, Refinementsof localasymptoticnormality, ratesof convergence and asof higher order are considered by Pfanzagl ymptoticoptimalityproperties
[1985].
82 and 83 are concerned Sections with testingand estimation for asymptiof experiments. At this pointwe have to introduce the cally nonnalsequences ideaof localparametrization. We explain it by examples with independent, that distributed It out decision of non-))) o bservations. turns identically problems
80.Asymptotic
normality
409)
linear,but smoothnature ean be replacedafter resealing by linear decision for the limiting Thus,the theory of Chapter11becomes applicable problems. of asymptoticallyoptimal Gaussianshifts.We obtaina complete description both for finite dimensional as well as for testingand estimation procedures, The resultsare illustrated means of classical infinite dimensional situations. by examples. In Section remarks on how to obtain 84the readercan find someconcluding which are known from the precedingto be the basisof central sequences, We show that central sequences are of functions. optimalsequences decision characterized propertyor by Bayesianoptimality.) by the maximumlikelihood
so.Asymptoticnormality)
.,
Let (H,< . be a Hilbertspace.Considera = (D\".91\". n E N, where H\" T H E\" {\037,II: hE H,,}), Recall Definition 60.1.) \302\273
sequenceof experiments as n 00,0 E H\", n E N.
-.
normal if it converges 80.1Definition.The sequence (E\,,,I'>.I") is asymptoticaJ/y weakly to a
Gaussianshift on H.)
80.2Theorem.The sequence ofexperimentsE\", n E N, isasymptoticaJ/ynormal n E N, definedby iff the stochastic processes (L,,(h\302\273ItEH..,
:\037,It \",0)
= exp L,,(h)
(
-
Ilh 112 \037
)
. hE H\".
n E
\037,
satisfy the followingconditions:
--.vO,1I1l2 weakly, hE H, (1) \037(L,,(h)I\037.o) + PL,,(hz) L,,(a.h 0 (P,..o),whenever a,fJ E iR, (2) a. L,,(h.) l + Phz )
-
hi
E
-.
1t
H, h 2 E H.)
Proof It isdearby Theorem69.4that (E\,,")N isasymptoticallynormal iff the finite dimensional of marginal distributions Hn under p\",o, n E to the finite dimensional of a standard distributions convergeweakly marginal Gaussianprocess on (H,<.,. Thereforewe have toshowthat conditions (1) and (2) are equivalentto sucha property.But thisisan immediateconsequence of Theorem68.4. 0) \"
\037,
(L\"(h\302\273\",,
\302\273,
80.3Remark. Sometimesit is possibleto splitthe sequence(L\") into a of linearprocesses (X,,) and a remainderterm such sequence
that)))
410
Chapter
13:Asymptotic
Nonnality)
LII(h)= XII(h)+ 0\"\".0(1), hE H, n EN.
The followingexampleillustrates this case.)
80.4Example.Let (D,d,Po) be a probabiJity space.Let
H = {gE L2 (D,.9I, Po):Po (g) = O} and M:;:;{gE H:po(g2) 4}. For every \037
geM)
-:=(-
-2 g + V1 1
dPg
dPo
I
4)
, Po (g )
2
)
definesa probabilitymeasure each P Po admitsa Po. Conversely, :;:; of the form P with representation \037
\037
\037
\037.
g=2
(V:\037
-a(P. Po\302\273).
Hence, the sets {PI,\037:P Po} and \037
{\037:
gE M} coincide.Considerthe
experiment)
E:;:;(D,.9I, {\037:gEM}). We
are
interested in the asymptoticbehaviour of the productexperiments
limit experiment we En :;:; (un, d'e,{I;': geM}).To obtaina nondegenerate the sequence stabilize (E\")by scale-transformations. Let H. = Ell
II:
{ge
ge \037
M}
and define
= (D\",.91\", gE II,,}),lle N. {p.\",vn.g:
Moreover,from the secondpart of the 1--+ 75.2 it that t t E ( t, e),is a curve differentiof Theorem follows proof ableat t = 0 with tangent vector g. Therefore,from Theorem75.8we obtain Now it
isclearthat H\" i H as n
-+ \037.
\037g,
the expansion)
dPtiv;,.g dPo
exp
(\302\2430):;:;
1;' -
(vn
g(\302\243O
j
)
i\0371
1 2\"
-
2 Po(g ) + 0,.\037(1
.
\302\273)
(!)ED\",n E N. With the notation)
X,,(g):
1
\"
L I;:j..t)
cg 1--+
V
g(\302\243O,),
n
(!)E
D\" ,)
which implies that (E,,)converges the assumptions ofTheorem80.2are satisfied 70.5and 70.8thisGaussian weakly to the Gaussianshift on II. By Theorems an Wiener space.))) can be Abstract shift sometimes by represented
80.Asymptotic
(E,,) is asymptoticallynormal.A
80.5Definition.Supposethat
linearprocesses (X,,) satisfying (1) .!e(X,,(h)IP.w,o)-vo,II\"\"2 weakly, hEH, (2)
= exp
:\037,'!.. \".0
(
X\"
(II)
normality
-
IIhW
+
)
op\".o(1\302\273
\037
,
n
411)
of sequence
EN, hEll.
for (E,,). iscalleda centralsequence are always available.) If H is a Euclidean spacethen central sequences
(.,
80.6Corollary.Suppuse that (H. of
experimentsE\", n E \037, randomvariablesX,,: D\"
-
(1) .P(X\"IP\".o)NH (2)
dp\"
\"
dp,,\037o
isa Euclideanspace.Then the sequence of asymptoticallynormal iff there existsa sequence
- II, ;.\\\"
nE
.\302\273
N, such that)
weakly, and
1 - exp (h,X,,) - 211hll ) 0 (P\",o),wheneverliE 2
(
H.)
...,
theoremthat the condition issufficient. To ProofIt followsfrom the preceding baseof H. Then we define provenecessitylet {elt ek}be an orthononnal n (forsufficiently large N) 1 dp\" _ ej + X,,:=:.L = ( dP,..o) 2) \342\202\254
,. log-
\"
\"
I
I
likelihood ratiosare positive.Then condition(1) IS satisfied. Condition (2) isalsovalid sincethe distributions
where the
- <\",X,,) + 211hllP,..o) ' 1
dp\" .P logdP.'\"
(
n.O)
converge weakly to
2'(108
-(h,X) +
2
nE N,)
= '0
po) where E = (D,d.{P,.: hE H})denotes a Gaussianshift and) \037\037:
\037lIhll2
d\037. X:= L log\037 + -2) e,
1
Ie
j\"'\"l
(
dPo
)
is a centralmap (cf.Example69.7
(4\302\273.
0)
SO.7Definition. that E\",n E N, isasymptoticallynonnal.Thenevery Suppose ofrandom variables sequence (1)and XII: '1\" H,n E N, satisfyingconditions for (E,,),,\037 (2) of Corollary80.6iscalleda central sequence
-.
N')))
412
Chaptcr
13:Asymptotic
Nonnality)
80.8Example.Supposethat M isan opensubsetof a finite dimensional linear manifoldwith underlyinglinearspaceH. Let E = (D,.91,{Px: x E M})be an with injectivederivatives. experimentwhich is differentiable the productexperiments Consider (1) (One-sample problem). = E\" (D\",d\", by a scaletransformation Y EM}),II EN. We stabilize aroundsomefixed pointx E M and obtain) 1/2:he II,,}),) En = (fl\",d\", {PX\"+\",\" {\037\":
where
H.
{hE H: x +
\037
hEM}, n E I'll. Sincethe triangular array
J,;
conditions79.1(1)and (2),we obtain h, 1 < i ;>n, nElli,satisfies
/.,,= \037
from Theorem79.2that) (cg)= exp
d\037\"+\",,-1/1 d\037\"
h,
( ( Bx
II
1
v;,
1
) -:2BAh,h) + op;(1) )
gAw;} i\0371
,)
cgeD\",ne N, he H. Since)
!i'(
cg \037
1
\037
V\"
1:.gAWj) i\037t)
Px\"
-. )
Vo.
P\",lg\",
\342\202\254I
g.)
weakly, and sincePAg\" (8) g,,)= B\", it followsthat VO,P\"(g\" @g,,) = N(H,B,,)and the assumptions of Corollary80.6are satisfied. Hence,(E,,) converges weakly is given by) to the Gaussianshift on (H,Bx).A centralsequence
I!:L
XII: Q.J1-+
V
1
II
\"
i=
n eN. gAwj), (!Je !l\",
I)
the productexperiments Consider (2) (Two-sample problem). 10\\ 11 = (D\", .91\" p!'2(\'") , {P!a(II) '1 \\01 12 . ( .Tl'Y2) E M2}),) where k 1 (II)+ k z (II) = II,n E N, and)
E\"
. (n) = aE hm-(0,1), lim \"... k1
co
n)
II\037CI)
k 2 (n) n)
= p E (0,1).
If we stabilize aroundsomefixed point(x,x) e M 2 by a scaletransfonnation
then we obtain)
nil J41', - (')\", ,,{P'
E
...A\"
ktl,,)
X+\"I\".
Sincethe triangular array)))
1:210\\ \\CI
k 2( ) . (hI' h 2) e H\",) 2 P.X+\"2'\" 1/2. } 11
lie N.)
80.Asymptotic 1
7h1 if
t\"j
Vn
=
1
normality
413)
\037i\037kt(n),
EN, if k.(n)+l\037i\037n,
-h
11
1
2
vn)
79.1(1)and (2),we obtainfrom Theorem79.2that conditions satisfies d_ (P1+d\") /(ri 10\\ '<::J\0371('1I +Ill/Vii ) \037
\037
I
-
(w )
\037
dP\" x)
= exp Bx(hl ,
(
1
leI (\")
,I: V
11
j
Lt
gx(w
j
\302\273
+ Bx(hz ,
k2(n) - 21 kt(/l) + Bx(h BAhl' h i) -n -;, ( d\ = exp a.Bx(h, vn lieL (n) ( 1 + P Bx(h , tin ---- L ( )
-
t
-
z
k1
k z 11
1c
L
V
11 i 'II(,,) + t
gx(wJ)
) + oP:(I)))
z , h z)
gAwi\302\273
t
i
i
1\" ,I:
\"
I (\")
+1
gAwi\302\273
(a It(h[,hlH IiB.(h2. h 2 +
--
\302\273
0,;(1\302\273).)
\037
2
QJE Q\", nE N, (ht,hz)eH
hi
\302\253
h;
) (h 2) (hz'
I \302\273
. On H2 we definean inner productby) .
= a. Bx (h t, h d + P Bx (hz, h 2)
--
.)
Then it followssimilarlyas under part (1)that (En) converges weakly to the
Gaussianshift on (HZ,
\302\253.,
1
.
\302\273).
Ie
A
centralsequence is given by)
tin)
vn k 1 (n ) .L1 gJ( (W ) X,,: 19H n 1
.
j
I\"\"'
Vn k
L
z (l1)j=Ie,(\+")
1)
WE
\"I
Q , n EN.
)
gAw ) j
80.9Lemma. Supposethat (En)\"E N isasymptoticallynormal.Then any central sequences (X\neN") and (Y,.)\"EN satisfy X,,(h) y\"(h) -+ 0 (P,.,o),hE H.) Proof Obviousby definition. 0)
-
The criterion for centrality in Theorem80.12 is preparedby two lemmas.)))
414
Chaptcr
13:Asymplolic
Nonnality)
80.10Lemma. Supposethat
(E\\"E") N is asymptoticallynormal.A sequence of iscentral h H linearprocesses all E (X,,) iff for hi' z
!R
IOg\037;
(
'-'!!, X,,(h2)
\037.O
\",0)
)
\037
VG.M
where)
a= _.1 2 /lh 1
M=
IIZ
) ')
( o
weakly,
'
dl (hI' . (hl,hz > 11hz :'>\\ )) lIh
(
II
ProofIt is obviousthat .5f(L(h l ), L,,(h2)1\037,0) VO,M weakly. \037
II
If (XII)iscentralthen this implies that
!t'(L,,(h), X\"(h 2)1P,..o) \"O.M
weakly.)
\037
t
if the latteris true, then Conversely, -+ to weakly, !t'(LII(h) X\"(h)IP...o) hE H. which implies L,,(h) X\"(h)= op\".o(1),
-
-
0)
80.11Lemma. Supposethat (E\ntiN") isasymptoticallynormal.If a sequence of ;s then) linearprocesses central. (X,,) !R log
(
where)
b
=
\"1
,
X\"
\037\037.
(h2) P,.,Itl I
11,0)
-
IIh11l2 \037
((h1,h2>) )
)
\037
Vb.
M
weakly
.
ProofLetfE(&00(IR ) and hi'h z E H. Thenwith the notationof the preceding Z
lemma we have)
dP.\",It. dP.\",0 , X\"(h 2
lim J f(1og \"....\302\253>
\302\273dP\".,,.
= J f(x,y)e1CVG,M (dx,dy) R2 =
}2f (X,y)Vb ,M(dx,dy).
0)
For the foHowing asserLionrecall Example 68.7(2),accordingLo which
the)))
80.Asymptotic
415)
normality
is a cylindersetmeasure. or Weak convergence of a linear process distribution = i n has tobe terms convolution ofcylindersetmeasures JI interpreted (jl,J,.E y L fR.) of their components Il,., \342\202\254
80.12Theorem.Supposethat
(E,,)
linearprocesses (X,,) iscentral iff
9'
(X\"
I \037.II)
is asymptoticallyIlormal.A sequellce of
-+ NH * ell weakly. h
\342\202\254
H,)
lemmait isclearthat every central has ProofInview of the preceding sequence the condition the asserted property.Conversely, implies \037
(X,,(h1) I P\".II2)-+ V(III,II2).IIII.W
weakly
for all hi' h z E H. A standardargument showsthat for every hE H, C E Rand nE N)
l dp..>c'
( {dP..o
in\"\"I
dP.\".\" -l(x..(III>cIdP. } ) ( \".0)
-ec-
I
21111112> =0
)
\037.o
-a.c.
the P\".o-expectations of theseexpressions Moreover, by contiguity convergeto -+ zeroasn 00.This implies that)
-
P\".o 1{Io, ...0)>c-11111111} l(X..(III>CI-+ 0, \037\037...\"
for every hE /I, C E R. Now it
is easy to seethat
-
1 2 dp\" + 0,....0(1)' hEll, n EN. logdP.\037 = X,,(h) 211hll 11.0) II
-1
= logdPII,II + 1/h1/2 and Z\" = X,,(h),n E N. Since the dP,..o 2 that they are distributions of (Y,,)and (Z,,) are unifonnly tight,we may assume K S R. We know that for every CE concentrated on somecompact
Toshow thisfact, put
y\"
IR)
lim \037.o {Y..
\037
,.\037Q)
Let
\302\243
>0 and let {c1, Pn.o{IY,.
-
c< Z,,}= 0, and lim Pn.o{Z\" c< Y,,} = o.
..., CN}
II
be an
\302\243-net
\037
of K.Then)
N
Z\"I > t} L P,..o{Y,. i\"l \037
.00)
\037
N
c( < Z,,}+ L P,..o {Z\" i\"l)
\037
Cj
< y\"},
which provesthe assertion. 0)
Sincea Gaussianshift iscontinuous any sequence (E\\"EN") which is asymptoti-)))
416
Chapter
13:Asymptotic
Nonnality)
in the limit.However,such a sequence neednotbe cally normal iscontinuous Recall that is equicona of continuous equicontinuous, sequence experiments tinuousif for every he 11and every > 0 there existsa o(c,II) > 0 such that \342\202\254
hi
-
< 6.) e H\",IIh l hI!< o(e,h) imply dt (P\",,,,P.r,,,.)
SO.13Theorem.Supposethat (E\\"EN") is an asymptoticallynormalsequence of continuous and denote) experiments dP\",,,
iP.\",0)
=exp
1
2
(X,,(h)-2I1hll
+r,.(h\302\273,
heH\".neN,
isa centralsequence. Then (E\,,e")N is equicontinuous on compact subsets K H iff any of thefollowingconditions issatisfied: >0 (1) Forevery convergentsequence h\" E H\".n e N. andevery lim p\",() {Ir,,(JI,,)> t} = O.) where(X,,),,\037
'\\J
\037
\302\243
I
\"-?(IO)
(2)
Forevery compactK
\302\243
}[andevery 6 > 0
lim supp\",0{lr,,(h)I>8} =0.
\"-00\"eK)
Assumefirst, that Proof It isclearthat conditions (1)and (2) are equivalent. = lim h. Then we have is on Let equicontinuous II\" compacts. (E\\"EN") \"-00 lim dt (P\",,,\", P\",,,)= 0 and therefore) \"-0
(I))
lim J
,,-QO
dP\".o
dp\",,,\"
dp\",,,_
dp\",o)
dP\".o= O.
ThisimpJies)
dP. dP. \",0)
\037-Iog \037-+ 0 (P. 0) log dP. dP. \",0
\",
which proves(1).
Choose h) > 0 such that (1)issatisfied. d2 (6\",. Nil'8,.. Nil) < ,t;;) if Iht hi -) 0(6,h).
Assumeconverselythat
0(\302\243,
-
2V 2
\037
then there isa compact If (E\\"EN") were notcquicontinuous on compacts subset K II and hE K such that we may find a sequence of pointsh\" E K,
- hi < o(c.h),
\037
Ih\"
nE
N, with)
d1 (P\",,,\", P\".,,) c for some c > \037
O.)))
\037O.
normality
Asymptotic
417)
that (\"ft)n{.1\\: converges we assume to\"o K.Thenit followsby Definition W.1.g. 80.1and by Theorem60.3(4), that \342\202\254
9'(lOg\037;.::P..,)- 9'(lOg
::;;it,. ). NH
\037\037::
weakly.
This implies lim supdl (\037.,,\"' P.r.,,) V2 lim d2 (P\".,,\", \037.,,) II'1CC n-'(()) \037
= V2 d2 (elto * NH , elt * NH ) <
of (hft)ft I!N' which contradicts the construction
i
U)
on compacts is equivalent to If H is a Euclideanspacethen equicontinuity In this casethe convergentsequence (\"ft) of condition (t) may equicontinuity. be replacedby a boundedone,which isnotnecessarilyconvergent.) in Example80.8(1)and (2),are 80.14Example.Thesequences (Eft) considered
if the This is clearsinceTheorem79.2remains applicable equicontinuous. are defined to be) triangulararrays t ni
1
:=
h n ,)
1
\037
i
\037
n, n E
\037,)
ifn
and)
1 \"n.1 1ft'
1=)
if 1
\037
i
k I (n),)
\037
nE
V\037
1 17:.-h\".2 if v
k
1
(n) + 1
\037
i
\037
N,)
n,)
n)
in H.) respectively,where (h\,") (h\".I)and (\"\".2)are convergentsequences
SO.15Example.Consider the caseof Example80.4.The sequence (E\")consideredthere is equicontinuous on compacts. To show thislet (g\") c:H be a Then PI/V'\"' 9\" convergentsequencc.
_.
-. -
-
dPo
is given by)
1
.
(20' V V--;j;-=(j-;' dPI/v;i.y\"
g.+VI
::- - -
-
1
2
4n
Then arguments of part (2) of the proofof Theorem75.2imply
. }1m \"-+
0')
J
n
[(
dv VdPI/V.
\037
-
-
that)
dP-o 2 . dv 0 ) 2g\" V J
Po
d'\\.'
nE N.)
,
P.(g.\302\273)
J
dv)))
_
418
Chapter 13:Asymplolic
Normalily)
It foJlowsby CoroJlary75.9(and the remark below Corollary75.9)that the expansIOn
-dPr-
dPrlVn'gn
-
(w) _ exp
1\"g\"(w) -
1
j
(vn
\"2
j\0371
2
llg\"lI po + opg(l) ,
Q)E Q\",n E N, is valid.Hencethe assertion. with the discussion We finish thissection of an
)
importantexample.)
80.16Example(Hajekand Sidak[1967] andMoussatat[1976]). Consider the casediscussed in Theorem79.2.For simplicity let M = H. Define 1 H = {hE L2 ([0,1]):hd)'= O} and let)
!
\"
c\"j(h):=n J hd)',
1
\037
L=.1 \
i n, he H. \037
\"
It is then clearthat L c\",(h)= 0, n E N, hE H. Fix x E iii and define j::q) II
11:= .n 1-
1>,.,
1
PI, x'\"
Thetriangular arrays
h E H, n EN. ;:;::rlll(\ V\
(
n \037
c.,
.i.: n
J
(h\302\273)
E
I\\J
. satisfy79.1(1)and(2) for every
hE H. ThereforeTheorem79.2yieldsthe expansion -dp\"
.
\"
dp.n. 0
\302\253(y)
= exp
\"
1
L B,,(cnj(h),gAw n 1-= (11:
-2 \037
n
= eXP
'i:.B\"(c,,j(h), 1-1
c\",(h\302\273
(BAl,1)'
1\037 n V
\302\273
n
j=
+ oP...o(l \302\273
.i
1=I
)
C\"j(h)g,,(w)
- 21B,,{1,I)!i c;,(h)+ WE
j
1
V
j
OPII.o(I\302\273
1)
),
n E N. Let X,,:Qn --.H be definedas) !2\",
.L gAw ) . 1[\037.1\ )., (!1E Q\", \"
X,,:tg t-+ V;;
.\037t
j
\"
and notethat part (I) of the proofof Theorem79.5implies)))
81.Exponentialapproximation and asymptotic sufficiency
-1 L c;i(h) II
lim
11-000n
'.1)
419)
hE II.
== Ilh112 ,
This givesthe expansion
1 ,. = exp(B.a:(l, !<XII'h) - 2BA1, l)lIhll + 1) dP. dp\"
2
oP...o(1\302\273,
nE N.)
11.0)
The central limit theorem implies
.!l'
\302\253XII'
h) I P... 0)
-
v o,
II
\"112
weakly.
of experiments Hence,by Theorem80.3the sequence .21\", Ell = (0\", {\037,,.:hE H}), n eN, is asymptoticallynonnalfor (H,Bx(1,J) <., of Fromthe expansion it isclearthat the sequence in the sense (XII)iscentral = I(. x),X E R, the discussion Definition of (80.7).In case0 = R and t ells us that also) Example(79.4) .\302\273.
-
\037\"':
-1 tin. L II
XII: Q>'-
/
R\"j(w)!/I
i-I (R\",(t.))J
qJd)\"
i) ' [1.=.1,
1
\"
1)/\"
qJE0\",
II)
for (Ell)' is a central sequence If (h,,) 1/ is a convergentsequencethen the triangular array 79.1(1)and (2),too.It followsthat (Ell) is , n E N, satisfies n ) (V on compact subsetsof 1/.) equicontinuous \037
Clli(hll\302\273
,.\037.,
I\037I\037II
81.Exponentialapproximationand asymptoticsufficiency) we provestatesthat a sequence The first assertion (EII)IIf'N which isasymptotiin a strongersenseby cally nonnalmay be approximated exponential experiments.)
a
of sequence
81.1Theorem(LeCam(19601). Let (H,<., be a Euclideanspace.Suppose .\302\273
isasymptoticallynormal.Then thereexistsa sequence ofexponential = .91 H n hE E experime1lts 1;, (QIf' , {QII.,,: }), N, satisfying that (E\lIc")N
11
II
lim d QII.\")= 0, hE H.) 11- l (P,..,., Let(XII)ne be a centralsequence of randomvariables.The exponential experi-))) CIO)
1\\01
420
13:Asymptotic
Chapter
Nonnality)
ments /0; can he chosensuch that
(1) Q\".\" P\".o, hE H\",n EN, \037
(2) \037\037\".\"
\",0)
= C,,(h)cxp\302\253h,X:)hE Hn, nE N, \037llhIl2),
where)
(3)
X: = X,,'
(4)
a>O. Jim ,,-cosup IC,,(h)-tl=O,
n E N. k\" r 00, :ii1c../.
l(lIx..l\037
11\"11
\037II)
we introduce the notations) ProofForconvenience
dP,..\" . /,\",,,(w,,)= dP.\".0
\302\253(1),,),
g\".1t
dQ\".II \037(wn
(wn ) =
\".0)
Il\"
).
W\"
E Q\",)
W\"
E
!J\".)
he H. 9'(exPlv-tll\"lIl.II\"1I2).
=
Firststcp:We show that the truncation(k\,,e")N can be chosensuch that for every a > 0
-11= O.
lim sup IC,,(h) \"--
11\"11
\":III)
Notethat the family of functions 1
x t-+/;(x,h):=
exp\302\253x,h)-2I1hIl2) IIxll\037i. if
-1
exp( 211h112)
if
II
xII > i,)
on {IIx i} for every i, is uniformly boundedand equicontinuous and Rao implies e.g.Bhattacharya (1976:Theorem2.4]) (applying
h II i E N. This II
that
\037
II
\037
lim sup IJ/;(x,h)\037(X\"I.P,..o)(dx)-J/;(x,h)NH(dx)I=O II
\"-00
il
II
\037l)
k n r 00such that) for every i EN. A standardargument yieldsa sequence lim sup I She\" (x,h)!f'(X\" I p\". 0) (dx) She..(x,h) N H (dx) = O.
\"-CO11\"11
-
DefiningX::=X\" It1IX ,,1I ;i1c..}we obtain
sup J \"-00 Jim
I
11\"11
I
\037k\"
\037II)))
exp\302\253h,
-
-
X:) -21.lIh 2)dP\".0 (NH Il
.
t,,) {lIxll$ k,,}1=0
81.Exponentialapproximation and asymptotic sufficiency
421)
this implies for every a> O. Obviously,
-
lim 11-00sup J exp\302\253h,
-11= 0, a> O. X:> 2 IlhIl2)dP\",o
CII(h)=
X:> 2\"hI2)dPII,o
I
11\"11
\037Q)
Since)
1/S
\037
-1
exp\302\253h,
is proved.) the assertion Secondstep:We notethat QII,,,)= li\037.\037pd.(P..,,,,
he 1/.) li\037.\037pjl.r...\"-gll.,,ldP,.,o'
\037
Thisis an immediateconsequence of contiguity.
-
Let us provethat lim 1/11,\" K\",,,I = 0 (P\".o)for every he H. Notethat for 11-<7) every e> 0 there existsct < 00suchthat lim P\",o{IIX\"II ,,-
\037
ct }
\037
t.)
00)
1
illhIl2).If we denote
Let d(t).=exp(ctllhll-
1 2 dP\".\" hE H, (h, XII> + 211hll, ',,(h).=logdP. \".0)
then)
-
If\",,,
K\",,,
\037
I
- X:>1 -c\",,,1 X:>- 211h1l2) 1 -2\"hIl2) 'Iexpr,,(h)
If\".\"
2
IIh1l)1
exp\302\253h,
\037
+ =
lexp\302\253h,
exp\302\253h,
+ exp implies)
J:a.o{Ij\037,,,
XII)
-1
X:> 2 > t} KII.,.I
\302\253II,
-
exp\302\253h,
11
II 112) I
-
(',,(h)
-
X:> (II,
X,,\302\273I
11)
1
\037
p\".o{exp(lIhll'IIXnlli llhIl2\302\273d(t)}
-
+ P .0 lexprll(h) II
+
exp\302\253h,
P...C.(h)- 11> 2:(cJ I
Thisprovesthe assertion.)))
-
X:> (h,
XII\302\273
\\
> .\037 2d(t)}
422
Chapter
13:Asymptotic
Nonnality)
Third step:We provethat
(/,.,\u") is uniformly N
hE H. By Lemma 6.12we have to show that lim
!I'ifll, P,..0) = 11 I
,,-c())
1111
for every o)-integrable
(\037.
vaguely, and
= fSIl,,(ds). lim J sjf(f\".\"IP\",o)(ds)
11-00)
= 1. The The first equationis obvious.Moreover,we know that J sll,,(ds) secondassertion followsfrom
.!
dP,. lim J s!f'(f\".\"I\037.o) (ds) = lim J dp dP,..o 11-00 11.0) n'-'<:O and from contiguity.
for every Forth step:We notethat (g\",\\",N") is uniformly (\037,o)-integrable rd hE H.Thisisprovedin the same way asin the 3 step.In the presentcasethis
isalmostimmediate.
0)
81.2CoroUary.Assumethat the conditions of Theorem81./aresatisfied. If the
then the assertion sequence(E\,,eN")i.'t even equicontinuous of the preceding theoremcan be improved10
a>O.
lim
,,-etj sup d.(P\".\",Q\".J=O, iscloselyrelatedto the conceptof asymptotic The exponential approximation 11\"11
\037Q)
sufficiency.)
81.3Definition.Let
\037,,\037
n EN. The sequence d\" be sub-t1-fields, (\037,,) IS
asymptoticallysufficient for (E,,) if
-
= 0) lim \"... If CPlldP,1I J P,..o(cplIl\037,,)dP,..1I1 II
c())
forevery sequence ofcritical functionslp\" e .(ji(0\",.91,,), n E N, and every h e H.)
81.4Theorem.Let H be a Hilbertspace.If (E,,) is asymptoticallynormaland is a centralsequencein
(he asymptoticallysufficientfor (E,,).) (X,,)
senseof Definition80.7then
(.QJt(X,,\302\273
is
Then we may apply the Proof (1) Assume that H is of finite dimension. of Theorem 81.1. Since exponential approximation d(X,,)issufficientfor F\" we have)
J ip\"dQ\",
\037
\"
An
All
e
\037II'
CPII
S Q\".o(cp\"ld(X\"\302\273dQ\",,,, An)
e ,g;;(D , d,,), n e N, he HII'This provesthe assertion.))) II
81.Exponentialapproximation and asymptotic sufficiency (2)
(d
423)
If H is notof finite dimension let h E H and defineL = {th:t Then This implies that for is asymptoticallysufficient for (Ell I.,..\",,). \342\202\254
\302\253h,
\037}.
I
XII\302\273)
every sequence (lpll)IIC
N)
-
lim I J lp.dP\",,,J p\".o(lplll d\302\253h,
= O.
XII\302\273)dPII.,,1
II-<XI)
to the subspace L we obtain) Applying the exponential approximation J \037.o(lpllld\302\253h, XII\302\273)dP,.,,, dQ\".h = J P,..o(cplIl.\037\302\253h, \037dP\".o+ 0(1) XII\302\273)
and since.sI\302\253h. followsthat
XII\302\273
11.0
s ..t#(X) and !!is n
\037'it 11.0)
d\302\253h.
Xn\302\273-measurable,
n
EN, it
J P\".o(lplll.Qf\302\253h, XII\302\273)dP,..\" = J P\".o(qJIII.\037(X\"\302\273dQII.h + 0(0 = f PII.o(cplIl,\037(XII\302\273dp\",,,+ 0(1).)
Thisprovesthat Jim n-<XI)
I
-
J lpndP\",,, J P\".o(lpllld(XII\302\273dP,..hl
= O.
0
similarargument as in the precedingproofshows that every sequence of q-fieJds.sI\";2d(X,,),n E N, is asymptoticallysufficientfor (Ell)') A
81.5CoroUary.!f(EII)isasymptoticallynormaland equicontinuous, and if (XII) isa centralsequence in the sense ofDefinition80.7then .sI(XII)isasymptotically sufficientfor (Ell) uniformly on compactsubsets of H, i.e.
-
lim sup J lplldP,..\" J 11-00 he\" An I
P\".o(lplll.<:f(XII\302\273dP,..,,1
=0
All)
for every compactK S II,every sequence All E d(X ), n EN, and every sequence (D , .rdn ), n E N.) lpll E .(ji II
II
81.6Example(Hajekand Sidak[1967]).Consider and especiExample80.16 ally the caseD = \302\243fll
.sI(Rill'.
IR,
= f(.
..,,,),i.e. \037JC
- x),
X
E IR.
For every n E N let
the t1-field generatedby the vector of ranks.Then is asymptoticallysufficientfor (Ell)unifonnJy on compacts. of the exponentialapproximation is concerned with Another application =
RII
(\302\243VII)
For this,we have to restrictourselvesto Euclidean posteriordistributions. . Let with respectto the distribution spaces(H,< F;. be the posterior measure AHltM(Hn ). In the followingwe assume that Ii\" T Lebesgue
.,
\302\273.
H.)))
424
Chapter
13:Asymplotic
81.7Theorem.A....\\'ume
Normality)
(H,<., .
that
a
Euclideanspace.Supposethat (En)ftE N is an asymptoticallynormal !iequence of continuolL'i,Awimegrable a and is central experiments (X/I)nE N sequence for (En). Then) LV
\302\273
=0 lim J dl(\037(.IK),(NH.exJ(.IK)dP.t.o
n -+
(;Q)
for every compactK c:H.) ProofWe keepthe notationof the proofof Theorem81.1.Thenwe have by
Theorem36.14)
F,.(B)=
J /,..\"AII(dh)
8\"H\"
--, Be
_._
\037(H),n
EN.
J.H(dh)) LJ...\"
Let us denote)
j 1-
J Brl H 1= GII(B) \
gn.\"i.,,(dh)
, BEdI(/l),n EN.
gn.\" II (dh )
then for n ;:::: If K c His compact nK)
J K
--1n.,.AH(dh) g/l,h).ll(dh)
__f/l\037._ J K
=
gn .,.__.
J
All
(dh)
K)
../\":.\" - - - _--.b\037_ + _ In.\037_ J.I 1n.,.AH(dh) g/l.,.AH(dh) g/l.,.AH(dh)) I I
It:
gn.\" A\"
J gn,,,A.,I(dh)
(dh)
K)
\037
(dh) IfIe/\",\.11")
f \"fn,,,AH(dh)
+
-
2
f g/l k
'\"AH(dh)
-
1 f 11n,,.gn,,,1II (dh) J gn,,,J.(dh)
gn.,,1 . - -- . - -
!
J I/\",It
K
I
. gn,,,A\037{dj,)-') I h.\.H(dh)") I
K)
:$;
-
A.H(dh)
-
gn,\'H(dh))))
A.
81.Exponentialapproximation and asymptotic sufficiency
425)
that) This implies
dt (I-:(.1K), G,,(.I K) 1
-2
!!
\037
-
X:>
C,,(h)
exp\302\253h,
. 1h..I.C,,(h)
cxp\302\253h,
t
2 IIh11
\302\273).H
\"2
-
(dh)
1
X:> 2\"h/l2)/)'H(dh).)
Choosinga > 0 arbitrarily and denoting
{V.(h) C.(h)cxp -
M.'= \037
we obtain)
exp
!
(
\302\253h,
II
X:>
h /I Q
II
h
II').!\" (dh)
\037
/I h
(dh) II').!\"
})
\037
IddJ,:('IK),G,,(.IK\302\273dP,..o K), Gn('1 K) P,.,o(M\037) + f d1 (f.:(.1 M..) \037
\037
p\",O
{Ix\"
1
> a}+
--
dP\".0
-- -
2 dt (P\".,., I
-- -- -
Q\".\"\302\273).Il(dh)
- - 1 I C\"(h)exp( IIhlla 21Ihll2}).u(dh))
-
This implies 1im d1(f.:(.IK),G\"(.IK)dP,..o0)
\"... J
\037
00)
for Gn , n E 1\\;, sincea> 0 is arbitrary. Thus it remains to provethe assertion
insteadof f.:,n EN. For thiswe notethat)
-1
1 -------1---=(NH*eX\037)(B), f
exp\302\253h,
x:> :iIIhll 2
exp
X:> 211hIll))'H(dh)
\302\253h.
\302\273).II(dh)
BE!N(H),n EN.) Similar reasoning as at the beginning yields)))
426
Chapter
13:Asymptolic
Nonnality)
(.1 C,,(h) llexp X:>
K), (Nil d.(G,,(.I
\302\267
2! < - -= I
I
K\302\273
\302\243x\037)
\302\253h,
-
exp\302\253h,
-
- IIhI1})'H(dh) -1 - - 2
\037
x:> 2I1hIl2)J.1I(dh)
\0372supIC,,(h)-11 Ire\ K for every compact
\037
lim J d1 (F(.IK),(NH ,,-\302\253>)
Since)
NH it
.
H and therefore tx\037)
(.IK\302\273dP,..o
.* . . . ex.. NH ex:
jff
IIX\"II
> kIt'
= O.
n E
r\\I,
followsthat)
lim supJ d1 (NH II'\"
co
\302\243X\037,
= Jim NH f.x)dP\".o n-'
which provesthe assertion. As a
P,..o{IIX\"II> kIt} = 0
co)
0)
we obtaina result which goesback to Bernsteinand von consequence
Mises.)
81.8CoroUary (LeCam,Wolfowitz [1953]). under the assumptionsof distributi011 is uniformly tight then Theorem81.7the sequence ofposterior
.
fl\"
lim J d (F,.,NH txJdP\", o = o. 1
,,-<
IX))
82.Applicationto testinghypotheses) that the sequence of experiments Let (H,<., be a Hilbertspace.Suppose h E Hit})' 11E N, is asymptoticallynormal. Let E,.= (0,.,.PI\",{P,..Ir: = h E H})be a Gaussian E (fl, {P,,: shift on H. The basicideais to handletestingproblemsfor (EIt ) by a transition to the limit experiment. For limitexperiments which are Gaussian shifts lineartesting solution. for admita complete However,the originaltestingproblems problems neednotbeof a linear transition t o the limit transfers (E,,) type.Nevertheless, linear o nes t hat the \"smooth\" i nto (E,.)is testingproblems provided sequence obtainedby a localization of the originalparameter space. We beginwith someremarkson how a suilable locali:lation of the parameter transfers smooth i nto linear ones.))) space testingproblems .\302\273
d,
82.Application
to tcsting
hypolhcscs
427)
M is an opensubset T. w ith linear Let linear manifold a of space underlying = be a of experiments. n x N, sequence Usually, En (an' n . {P,..x: M}), takesplacearounda fixedparameter pointXo E M which is testinghypotheses or the resultof an initialestimate. Statistical eitherknown by priorexperience of the true and the measurementof a deviation inferenceaims at the detection value x E M from the startingpointXo E M. of the Typically,d1 (P\".\037, p\".)') 1 as n 00if x y. Hence,the situation is very different for different samplesizesn E N and onecannot statistician with small samplesizecan be approximated well by expectthat the situation it is situationswith large samplesize.For an acceptable approximation d o f at the behaves that least necessary magnitude 1 (P,..x. qualitatively P,.,y) that 82.1DiscWtSion(Theideaof localization). Suppose
d
\342\202\254
\342\202\254
-
-
stableasn
=4=
-.00.
of thisproblemisa reparametrization In many casesthe key for the solution of the sequence at Xo E M with (En) localizable (En). Let uscallthe sequence bn
!
0,
if)
0< lim inf d1 (P,.,\037o'
P\".XQ
II'\" (f)
< 00 supd1(Pn.xo,P\".xo+6\",) ;-6\"') lim II . \037
CD)
for lET,I O. In such a casedefine Q\",,'= IE T, nE N, wheneverXo + b,.tEM. P\".xoH\"\" the sequence Consider of experiments) =4=
F,. =
1
tE (!In,dn , {Qn.,: b (M xo)}), n c; N. n)
The sequence for every / E T (F,.)satisfies
0< lim-.inf d1(Qn.o,Qn.,)
\037
n
iff /
<JO)
lim supd1 (Qn.O'Qn.,)< II
.t
1)
IX))
one.) * O. Thus,any weak limit experimentF of (F,.)is a non-degenerate
82.2Example.Let M bean opensubsetofa finite dimensional linearmanifold with underlying linearspaceT. Suppose that E == (a,.siI, {\037: x E M}) is a
differentiable experiment with injective derivative at each x EM. Assume Thenx t-+ PJ( iseven a diffeomorphfurther that x t-+ P:.;is a homeomorphism. ism (considered as mappingfrom Minto L2 (Q,.fi,v) for somedominating measure vi .sf).We shallprovethat (En) is localizable at every .t AI with bn = n - 1/2.n E We shalleven provethe considerably strongerresult that lim supndf (1) 0 < lim inf nd;(PJ(\"' pJ(n+,..!v7.) < 00 \342\202\254
\037.
PJ(\"+I\"fI/;;')
II-c()
\037
PI-:C)
iff
(2)
0 < lim infl/nl lim supIlnl < 00.) \037
11-'00
11.00)))
(PJ('\"
Chaptcr J 3:Asymptotic
428
Normality)
Differentiabilityand regularity of the derivative togetherwith Rcmark 76.7 imply that) (3))
----.-
. In . f d2 -- P:.:\"+s . d2 - -\"-) - )1m 0< hm sup (\037\",
\037
Is,,1
\"-00
-.
\"-00
(\037\". Pl/;\"+s
)
\"-
< 00
Is,,1)
is immediatethat (t,,) satisfies (1). if (t,,) satisfies Conversely, (1)then t,,/VI' 0 sincex Px is a homeomorphism.Hence,(3) may beappliedto SII = tll/V;which yieldsthat (III)satisfies (2).) whenevers\"
O. Thus,if (I,,) satisfies (2) it
\037
\037
82.3Example.Consider Thenthe sequence the caseof Example(80.4). (E\")is 1f2 = = n N. at g 0 with nE To seethis recallfrom the proofof localizable
.
b\"
Theorem75.2that
lim nd\037(Po,
11-00)
= 1 Po(g2). 8 _
\037.,,-I/\037)
that Thisimplies
0< lim infndi(P o , \037'n-1/2)
\037
,,-c()
iff
< 00 lim supnd\037(Po, \037.,,-1/2) .CO) \"
g * O.)
82.4Discussion of testingproblems). (Localization Keep the notationof at Xo E M, f(xo) = 0, and 82.1.Let f M -+ R be differentiable Discussion the problemof testingHo= {fs OJ, Ko = {f> O} near Df(xo) * O. Consider of criticalfunction (q>,,) is asymptoticallyunbiasedof level xo. A sequence IX
E
[0,1]for (Ho,Ko) lim supp\",x\"q>\"
if)
\037
ex
\037
ex
if (x,,) Ho.) \037
\"-00)
lim inf ,,-oc)
p\".:.:\"rp\"
if (x..)s Ko,
cannotbe for a sufficiently large classof sequences (x,,) such that (P,..l/;J from (P\".xo)'i.e.) separatedcompletely lim supd1 (P,.. < 1. P,.,:.:.J :':0' \"-00)
This implies that lim supp\". %0+6,,'lp\" ,,-.n)
lim inf
,,-co)
for every t E
11.)))
P\".l/;o+6\"t lp\"
\037
ex
if f(xo+ b\" t)
\037
ex
if f(xo+ t5\"t) > 0, n EN,
\037
0, n EN.)
82.Applicalion to tcsling hYPOlhcscs
429)
Then it is not difficult to show that a Now, let (En) be equicontinuous. of critical f unctions is sequence asymptoticallyunbiasedof level for (lfJn) rJ.
(H,K) iff Jim
supP\".XO+6n,lfJII
\037
0: if
Df(xo)(t) 0,
if
Df(xo) (t) > O.
\037
II-\302\253j)
lim inf P,.,xoHnllfJn n\"'oo) \037
rJ.
Thus,in tennsof the localized sequence (F;,) the criticalfunctions(lfJn) are unbiasedof level (1. for the lineartestingproblemHI = asymptotically {Df(x) O},KI= {Df(x)> O}.) \037
Thepreceding remarksindicate how the analysisof \"smooth\" testingproblems in asymptoticallynormal situations can be reducedto the analysis of linear the we In testingproblems. following restrictour interestto lineartesting
problems.
a continuous beginwith the analysis of testingproblemsconcerning -+ linearfunctionf H IR. The corresponding non-asymptotic theory hasbeen in Sections 28and 71.First,we consider one-sided discussed testingproblems. Let HI = {hE H:f(h) O},KI = {hE H:f(h) > O}.) We
\037
82,5Definition. of criticalfunctions (1) A sequence of level0:E asymptoticallyunbiased limsupp,.,,,lfJII 11-00) lim inf
n-
(2)
A
lpn 1',..,.
lfJn
E ,rF
(Qn'
d
n), n E
\037,
[0,1]for the testingproblem(HI'K.)if
\037
rJ.
if f(h)\037O, and)
\037
C(
if
is
f(lI)> O.)
co)
of criticalfunctionslpn E :!F(On,.PIn),n E N, isasymptotically sequence
similarof level0:E
[0,1]for the testingproblem(HI>KI ) if
lim Pn\",.CPII =
n-oo)
f(h) = O.
0: if
which isasymptoticallyunbiased of level isasymptoticallysimilar sequence of level0:.This is true sincethe sequence in the limit.) (EII)uN iscontinuous A
rJ.
82.6Lemma. If a sequence of testslpll E oF(011' .PIli)'n E N, is asymptotically similarof levelafar the testingproblem(HI'KI ) then) 11msupP\".,.lfJn \"
11-00
li\037\037f
+.
f(h\302\273
\037
\037
(
Nfl
P...'P.> 4> (N.+
II
f
II
)
if f(lI)> 0,) if f(lI)
\037
\037j\037
))
O.)))
430
Chapler t 3:Asymptotic
Nonnality)
ProofApply Theorem62.5with Ii = {h}.Notethat the setPA of accumulation in ,\037:(Hh pointsof(Q>/I)iscontained Kt) andsodoesco Thus,it followsthat lim sup1'\".h tpn sup Ph tp if f(h)> 0, \037.
\037
/I
\037
a)
lim inf
,,-a)
I.K
\".,
\037I(H
J\037,hQ>/I
\037
11)
inf
PhQ>
'Pe'/(III.K.))
if f(h)
Now the assertion foHows from Lemma 71.1.
\037
o.
0)
82.7DefinitioD.Suppose that (
. P\".hQ>\"
=
\037
\037\037\037
(
Nil
+
f(II\302\273
IIfli)
'
hE H.)
') (.,f)\".)
In the followingwe identify f with a pointof H, i.e. =
82.8Theorem.Let (LII)\"EN be definedas in Theorem(80.2).A sequenceof critical functions E (!1/1' .fd/l)t n E N, isasymptoticallyoptimal oflevela for (Hh K1) iff it isstochastically equivalentto the sequence) t if L,,(f)> IIfIlNt -:\" 11E tp,. = {o if L,.(f}< IIfllN( uIJ') Q>/I
.\037
\037I
I\037.
that the sequence ('1'/1)is asymptotically ProofIt isclearfrom Theorem80.12 (Q>/I) which isstochastioptimalof level for (HI'K1)'Henceevery sequence us to shares the Let optimality. provethe converseby cally equivalent ('1',,) Theorem63.6.Let X becentralfor the limit experimentE.Then(tp/l) converges in distribution to) !X
I if f\"X>lIfIiN-a , tp= o if v X '5;. f lIfIlNt { I
-\302\253.)
isnon-randomized and by Theorem(71.3) Now, 11' 'I'isuniquely determined by itsdistribution. Hencethe assertion. 0) n E Nt which 82.9Corollary.If a sequence functions E!Ii(D\"t ofcritical K isasymptoticallysimilaroflevel(1.for is (HI' l ), asymptoticallyoptimalat some hE H with f(h) '*'0, then it is asymptoticallyoptimal.) Q>/I
\037\037/I)'
of (tp,.)uissimilar of level point E :F(H, ProofEvery accumulation (1. and optimalatf(h) 9= O. FromCorollary 71.4it followsthat tp is uniformly pointof (Q>/I),..Nand optimaL By Theorem71.3, is the only accumulation to in This the assertion. distribution. proves (Q>,.)\", N converges tp Q>
ff4
(H\302\273
I'll
Q>
0)))
82.Application
-.
to testing hypotheses
431)
82.10Remark. If there existsa centralsequence for (En) then) (X ) of processes tl'n
_ I
if
{o
If
n
f\"Xn >llfIlN1 -:r' n E foX\"< f liNt-a,)
\037I
1'1),1,
II
is asymptoticallyoptimalof levelrx for (HI'K.).)
Now we turn to two-sided Fromnow on let H2 = {hE H: testingproblems. f(h) = O},K2 = {h E H:f(h)9= O}.) (!l\",.\037 ,,), n E N, is for the testingproblem(H2 , K2) if
of criticalfunctions 82.11Definition.A sequence
C{'n
of levelrx E [0.1] asymptoticallyunbiased limsupp,.,,, rx if f(lz)= 0, and)
E
.\302\245
\037
C{'\"
,,-IX))
lim inf
p\".\"C{'\"
\"-00)
\037
a
if
f(h) * O.)
82.12Lemma. If a sequenceof criticalfu'tlctio'tlS E:?(Q\",sI,,),n E N, is asymptoticallyunbia.'iedof levela for (H2 , K2) then) . hE H.) 1',..11 4> Na/2 + IIfli) + 4> ( IIfll) C{'\"
f(h\302\273
C{'\"
(
\037
hr\037_\037p
NI%/2
-
f(h\302\273
'
ProofFromTheorem62.5it followssimilarlyto the proofof Lemma82.6that limsup1',..\" C{'\"
\"-'00
sup
\037
P\"
C{', h E
H.
1p\037\"'\302\253('h,\"lI)
followsfrom Lemma 71.5. Now the assertion
82.13Definition.Suppose that
oflevela for N isasymptoticallyunbiased isasymptoticallyoptimalof level for (H2 , K2) at he H
(Hh K2). Then
. P\",,,lfJ\"
=
(C{'II)\"E
rx
(C{''')''f\"P\302\245
if)
11)
4>
\037\037\037
Na!2+
(
[(h\302\273
IIflf )
+ 4>
(
Nal2
[(h\302\273
-ILiiI) Theorem80.2.A sequence oj .)
82.14Theorem.Lei (L\\"f'N") be definedas in n E N, isasymptoticallyoptimal critical functions E !F(0\",.\",,), of levelrxfor (11 2 , K2) iff it is stochastically equivalent to the sequence) I if L\"([) > f N I - ' I n EN. ,_ N 0 < { if L\"en fll 1--a , C{'n
tj
,,-
I
I
II
I
I
II
II
2)
Theorem Proof.The proofissimilarto the:: proofof Theorem82.8employing 71.7insteadof 71.3. 0)))
Chapter 13:A$ymplotic
432
Nonnality)
82.15CoroUary.If a !;equence of criticalfunctions
cP\"
E
.'\302\245
(!1\". .r:I,,),n E N,
whichisasymptoticallyunbiased oflevel for (H2.K2), isasymptoticallyoptimal at somehE H with f(h) 0, then it is asymptoticallyoptimal.) (X
::f:::
ProofThe proofissimilarto the proofof Corollary82.9.
0)
82.16Remark. If there existsa centralsequence for(E,,) then (X,,)of processes tp\"
- {0
_ 1
if
> IIfllNI\037'
If\037 X\"I
If
if
\037
< IIfllN
x\" I
I
nE
\037
,
\037.
is asymptoticallyoptimalof levelC( for (H2 . K2).)
the two-sample 82,17Example.Let us consider problemfor the location-scale We have to combine family of the Gaussiandistribution, Examples80.8(2), = and 78.9.The parameter spaceU JR x (0.(0)has the tangent spaceH = JR2, Therefore,the sequence (E,,)convergesweakly to the Gaussianshift on (1R4,
\037
..
}\302\273.
where)
1 \037
q2
-
0
G:)
(1 0)
isgiven by) (a,(2).A centralsequence
.; L i-I k.(n) ( .-.) V
1
n
x\":Q}
ktC'\"
-
-a (w -a)-(1 ) . l!J (}\", EN.) a i: ((w,- 0)2 _ q2) Wi
i
vn k 2\037 (n) i\"'It.(\+") I Let /(al,qI,a2.q\037) = WC
G:)
(S3.S4)
+
and x =
0
}> = (SI,S2)
al
2
2
n
E
Wi
- .which yieldsDf(a,(2) (SI.S2'S3' \302\260
S4) = SI
2
-S3'
have)
Df(a.u') =
\037
\037)
\037\037}. and thus obtainthat
II
Df(a,0'2)11= V20'2. Thereforean optimalsequence
of)))
82.Application
- --- -
to testing hypotheses
433)
criticalfunctionsfor (HI'K1) isgiven by) 1
if)
YII
\"E
k
\037.
-
t
1 (n)
1;1(11) Wj
-
1
II
L L w/ > k 2 (n) j-Iq(II)+1 = .If V;;(kl(n)1:1 ) N 1 -'I' '/2=2 V.a <) jo t
on alonemay replace2al by) To eliminate the dependence I; 11\"'
L wI + k /-1
1
2 (n)
\"
L (I); /=I;IIII)+t
1 - (k1(n) -( L ) 2
I; I (II)
/=t
(I);
2
\"
t
-
k 2 (n)
2:
Wj
i\037I;II,,)+l)
)
of criticalfunctionsnot stochastically equivalent sequence = on x (a,(1'2).) depending which gives a
-
82.18Example.ConsiderExample 80.4and let cp:0 R be d-measurable and bounded.Let f P.-J ({JdP I cpdPo, P Po, and considerthe nonparametric testingproblemHt = {f\037 O},XI = if> O}.
-
\037
Since,asiseasilyseen,)
1(1'.)=
\037
q>g'dP o+
f
V
criticalfunctions)
\037
Po(g')f gq>dP o = f gq>dP o+
E
Po.
V;; o if ;;:L (cp(wr)1
t if
1p,,(cg)=)
II
(c,o(w/)
1
n
Po(
II
j\037
-
\302\260O
(po(g'\302\273)
H is the of asymptoticallyoptimalsequence
i\037l
V
(1.)E
-
g.g H, which means that J cpgdPo, at Thus,we obtainan
we have Df(Po):
derivativeoff
I
Po(cp\302\273
(cp
Po(cp\302\273
- Po(cp)
E
> IIc,o Po(cp)IIpo. Nl -a , < IIcp
II
Po
.N t
Q
,
1)
Oil, n EN.)
82.19Remark. The preceding example is
little more subtle if Thepointisthat in this casefis only defined cp e L2(U,d, Po) is not bounded. on a densesubsetof H, but has a derivativein H. Thus,the localized testing > betreated as itsrelation b ut problem({Df(Po) O},{Df(Po} O})may above, to the originaltestingproblem({f\037O}, {f> On must be treatedmore 82.4.This topicis treatedin the carefullythan it has beendonein Discussion literature under the label\"von Mises-funetiona}s\". The reader wi)} easily note))) \037
a
434
Chapter
13:Asymptotic
NonnaJity)
that Discussion 82.4can becarriedthrough when f isonly definedon a dense subsetof M, but hasa derivativein the tangent spaceof M. Let Lo c H bea linearsubspace of Now, we turn to linear testingproblems. finite codimension and consider the testingproblemHJ = Lo,KJ == H\\ Lo.The in Sections 30and 71.) corresponding non-asymptotic theory hasbeen discussed
!F(D\",sill)'n E N. is of levela E [0,1]for the testingproblem(11 asymptoticallyunbiased 3 , K3) if)
82.20Definition.A sequence of criticalfunctions
cp\"
supPn, h
Urn II'\"
CPII
\037
IX
if
h E
IX
if
he H\\Lo.
E
Lo.
IX)
lim inf P,..hcP\"
11-
\037
:X))
For the following, we denoteBe = {h H:IIh \342\202\254
-
pLoCh)\\!=
c},c > 0,)
andsupposethat (XII)is 82.21Theorem.Assumethat (E\\",,,..")isequicontinuous a central of processes for (E,,).For E [0,1]let be as in Theorem sequence are true: 71.10. Then the followingassertions <X
kf1.
(I) Thesequence of critical function.\\')
.-
lp\"
-
_ I if lI(id
{0
if lI(id-
PLo)
PLo)
n
XliII>kf1.'
0 X\"
II
< kf1.')
isa.\037ymptotically unbia.\037ed of levela for (1/3 , K3). which isasymptoticallyunbiased (2) /f (cplt) isanothersequence of level for <X
(HJ,KJ) then) limsup inf \"-00
P\".h cP\"
\037
lim inf
\"-00
Pn,h
CP:.
hEB\037)
hl:B\037
at leastonec > 0 (3) /f(cp,,);s sati.ifyingfor any sequence ofcritical functions lim supP...o <x. and) cP\" \"... \037
00)
lim inf P,.,h cP\" ,,\037\302\253>
\037
lim P,.. h CP: whenever
hE
Be.h J. Lo,
1I\037t;())
then (
of fit in the discrete 82.22Example(Goodness case).Consider Example76.1 e s; from an subset and letp:e -+ M be a differentiable open mapping IR'.)))
82.Application
to lesling hypotheses
435)
,
localizethe problemaroundsomepointpeS)E M, 9 E e. Then a similar reasoningas that of Discussion (82.4)changesour problemintothe testing problem(Lo,H\\Lo)where H = {tE L:1 t i = O} is the tangent spaceof M It
IRII:
and Lo = J;,(3)(p(e\302\273. The resultsof Examples78.t and 80.8show that the sequence tell}),nE N, convergesto the Gaussianshift on E\" = {.P,:(S)+I/V,j: (1/,.8p(S) where 1 i\037
\302\253(1\",\037cf\",
o)
pd9)
t) = 5\ B'(9)(S,
.t,s,tEH.
'.\037-
o
PIIUJ))
A
central sequence is given by)
x.
Sl
.
\037
-
n
.
P.(9\302\273
(S, n'p,(9)))
\037
,
ne\037,)
\037
where Sj is the frequencyof the event j in a sample, 1 j k. Forthe construction of the asymptoticallyoptimalsequence of testsaccordhave to we find Theorem 82.21 a o f random variables to ing sequence Y,,: Q\" 4' Lo,which isstochastically equivalentto (PLo0 XII)' For this,the usual of estimates (1\" -+ such isas follows:Let (-9'..)be a sequence -9'..: procedure iscentralfor F\".9 = that qln(-9'\" S E \037I}), n E N, d\",{1';(HS/Vii): for every 9 E The existence of such estimates isdiscussed in Section 84.It followsthat \037
-
\037
e
8.
9\302\273\"EN
\302\253(1\",
-
v;,(P(-9'II)
= pl(9}(V n (g\"
p(9\302\273
-
8\302\273
+ ()p;'i,(l).
-
FromTheorem78.4we seethat (p'(9)(Vn(g\" is a centralsequence 0 for(E\" LO)/IfN and hencestochastically to equivalent (Pl. o X,,)neN' Thesame is then true of (V;;(p(g,,) of Putting tennstogetherwe arrive at an asymptoticallyoptimalsequence testsof the fonn) 9\302\273)\"EN
-
1
p(8\302\273)neN'
1
if
CP:=
0 if
(Sj- npj(9 i. 8 j=1 npj() (Sj i. 9) )-1 11
\302\2732
npj(9,,\302\2732
> k Q;'
nEN. < k Q;'
nPJ()
The parameter 9 in the denominator can also be replacedby (-9',,),thus a Thisis the of criticalfunctions. equivalentsequence obtaining stochastically usual x2-testfor goodness of fit problemsin the discrete case.)))
436
Chaptcr
13:Asymptotic
Normality)
of Theorem73.2to testsof the Our lastexample dealswith the application type.) Kolmogoroff-Smirnoff
8%.13Example.(Goodness of fit in the continuous case).Let Po 1 bea nonatomicprobabilitymeasure with distribution function Fo.We parametrizethe P Po as we have donein Example 80.4,thus set of probabilitymeasures = a n E \0371, {\037:gEM})where obtaining experiment M = {gE H:PO(g2) 4} and H = {gE L2(PO): Po(g) = O}.) I
\302\2431
\037
(\037,
\037
toconsider the testingproblem({O}, Localization around J\\1\\ {O}). the to82.3 of yields sequence experiments) Po according E H,,}), E\" = (R\",11\", {P:tV\037'II: g where H\" = {gE H:PO(g2) 4n},n E We shallshow that the sequence of criticalfunctions(calledKolmogoroff-Smirnoff-tests) if supvnl\037 fOl > c, We want
\037.
\037
\037:
R
=
I:)
if supVIi
I
\037
-
Fo I
<;C,
H)
the empirical distribution function for the samplesizen), is (where F;, denotes To be if admissible. precise,(cp,,)isany othersequence satisfying) asymptotically lim supPolp\"
\037
II....
lim inf Polp:, and)
<10)
lim inf n\037w
\".... <10)
PffVi,.g
cP\"
\037
lim supPfwn.,CP:, g E n-w)
H,
to Theorems62.7and is stochastically equivalent to (cp:).According in distribution toa criticalfunction 63.6we needonly show that (cp:)converges which is and uniquely for the limit non-randomized admissible, experiment, cp. determined by itsdistribution. The limit experimentof (E,,) is a Gaussianshift on then
it
H = {gE L2(R,\037,Po):J gdPo= O} and representation) E = (\037o(R),
{WFo
\037(
r
where r(g):r t-+ J gdPo,r E -<10) 1)
cp*:x t-+)
IR.
.
by
\302\243t(II):
g E H}))
the critical function Consider
if supIx(t)1> c, R)
o
Theorems70.5and 70.8has the
if
xE(io(\037)')
supIx(r)1 C, \037
rrR)))
83.Application By
to estimation
437)
and criticalfunction cp* isadmissible Theorem73.2,the non-randomized
It remainsto showthat (cp:)converges by itsdistribution. uniquely detennined to cp*.But this isan immediateconsequence of the wellknown in distribution
for the empiricaldistribution function.(Seee.g.Parthinvariance principles Ch.7, Theorem9.1]).) asarathy [1967: in the preceding 82.24Remark. Replacing examplethe testscp:by if n J (F,. F(,)2dF(, > c, * _ I 'Pn
- {0
-
if
n
-
J (F,.
F(,)2dFo
\037
c,)
Thereason conclusion. (calledCramer-vonMisestests),would leadto a similar in to distribution conditions is that under suitable ('1':)converges I if J x2 dFo > c, .x tp*' {0 if f x 2 dF(, c, \037
\037
is a continuous and convex.function on
83.Applicationto estimation) Consider the same situation as in the precedingsection.) of the resultsof this sectionis again mainly 83.1Remark. The application
with suitable concerned localizations of experiments aroundsomefixedpoints of the originalparameterspace(cf.Discussion But in contrast to testing, 82.1). for estimation to treat the problemonly problemsit is usually not possible toobtainestimates which 10caIJyaroundsomefixedpoint.It israther desirable f or have optimum properties every localization around any point of the
originalparameterspace.)
83.2Discussion of estimation (Localization problems). Keepthe notationof 82.1and 82.4.Let f M --+ be differentiableat Xo EM and Discussions the problemof estimating Df(xo) O. Consider f which is to be localized IR
=t=
aroundsomepointXo e M. A sequence of estimates Kn: Qn --+ R the estimation of if 1 Jim inf {Kn I(x n )} 2, \"-IX) 1 lim inf Pn.x\" {Kn [(xn )} _ , 2 n-oo)))
I
Pn,:Ie\"
\037
\037
\037
\037
is asymptoticallymedian unbiasedfor
438
13:Asymptotic
Chapter
Normality)
for a sufficiently largeclassof sequences cannotbe (XII) s:;AI such that (P,.,xJ from This that separated completely (P,.,xo). implies 1 _, Jiminf 1:...1:0+6\"' {\"II \037f(xo + bill)} 2 \"... 1 . lim inf P,..xou..t {KII \037f(xo + bill)} ... 2 then it isnotdifficult to show that (K ) for every IEH.If (Ell)isequicontinuous is asymptotica))ymedian unbiasedfor f iff \037
IX>)
\037
II
C())
.,
II
...'(K.-f(
xo''\"' (K. \037rI:,. Ii:\". V. li\037 \037f
P.o.xo
Df(xo)
\037
f(xo\302\273
xo\302\273
\037
(I)}
\037
Df(xo)
\037,
\037
(I)}
{\037:
\037,)
for every t E 1/.Thus,in terms of the localized problemthe sequence
-
(1 (KII f(xo ) \302\273
liEN
II
must beasymptoticallymedian unbiased for the estimation
of the linearfunction Df(xo)on H.
Similarly,we shaH calJ the sequence (KII ) locallyasymptoticallyoptimalin isasymptoticallyoptimalfor somesensearoundXo E M if (K\" f(X o
1 ( \"
\"' ) sensefor the localized \302\273
of Df(xo)in the respective the estimation
To seethe justification for this,notethat) J
:. (( (( :.-
I'\\j
sequence (Ell)'
(K. f(xo + o.t)) dP'.xo.'\"'
)
(K. f(xo)) - Df(xo)(t) dP'.xo u\", 0(1)
=J
)
t is sufficiently smooth. two ofestimates FinaJly, sequences
providedthat
(K\0371)
and
(K\0372)
equivalentaroundXo E M if
;
(K\037l)
-
K\0372)
\037
are locallyasymptotically
4 0 (P,..xo+6\"t)'t E H.
II)
beginwith the problemof estimatinga linearfunctionf H IR. The in Sections 34and 72.) non-asymptotic theory hasbeendiscussed corresponding We
ofestimates 83.3Definition.A sequence ell E .\037(EII' for fif))) medianunbiased
-\037
\037),
nE
N, isasymptotically
H3. Application to cstimation 1
lim
inf\302\253()\"\037,,,) 11\"'(1()
([f(h),00]) _2, \037
-
439)
and
1 inf(l?\"\037,,,)([<X).f(h)]) _2
Jim
\037
11\"'(1())
for every hEll. -+ [0,00] function with 1(0)= O. Define be a non-decreasing Let t: [0,00] the lossfunction by W,,(x).=t (I x f(h)I),x E hE H.)
-
\037,
83.4Theorem(Pfanzagl [1970]). SupposeIhal I is lower semiconlinuous. If ()\"
E EJt(E\",
forf then R), n E N, ;sasymptoticallymedianunbiased
lim inf
\"...
W h l?IIP\",h
\037
o. n!2' hE H. J 1(1.l)dv 11
a:>)
Proof FromCorollary42.8we obtainthat !B(E,R) may be identifiedwith estimates forf It /li(E, Let fJt(E,IR) be the setof all median unbiased iseasy toseethat .'Fisweakly closedand convex,and that every accumulation is in Y. Hence,combiningLemma 72.1and Theorem62.5the pointof follows. 0) assertion .\037
\037).
\037
\302\253(),,)
83.5Theorem (LeCam[1953]and Hajek[1972]). Supposethat t ;s lower
semicontinuous, If a sequenceof estimates of finite order and separatinl{. ()II E
fft(E\", R), n E
N, satisfies
limsup W\"e\"P\".\" n-oo)
\037
hE N, JI(I.I)dvo.lifIl2,
then (U,,)isstochastically equivalentto (L,,(f\302\273u\037' BO.2.))
\302\253Ln)
isdefinedasin Theorem
pointP E 81(E, of(ell)satisfies ProofEvery weakaccumulation by Corollary \037)
62.4)
hE H.) P(W\", P,,) J I (1.1) d\\.o, 11/112, \037
FromTheorem72.4wc obtainthat f X, where X is central for E, is the only accumulationpoint of (ell)'Hence,(Un) and L,,(f) convergeto fo X in of Theorem63.6provesthe assertion. 0) distribution. Now, an application <;
83.6Remarks. (1)The precedingassertion alsoimplies that
the boundgivcn there isthe asymptotic minimaxbound.However,it will beseenlater,that this
isalsovalid without t beingof finite orderand separating. (2) If there existsa centralsequence (Xn ) for (En) then (Ln(f) replacedby (f
0 XII)')))
may
be
440
Chapter
13:Asymptotic
NormalilY)
83.7Discussion. (1) At this pointwe have to discussthe questionwhether there existsequences of estimates (e,,)E 9l(E\", 11E N, which attain the lower boundin the sensethat) = ft(I.l)dvo.1I/112, lim W\"e\"P;',,, he II. \037),
\"\"\"00)
From Theorem83.5we
must be stochastically see that such a sequence equivalentto (L\" N' If t is boundedthen (L,,(f) in fact attainsthe boundsince asymptotic (f\302\273\"t:
-+ v/<M, II/II' weakly, 2'(L,,(f)IP...,,)
hE H,
of( donotmatter sincethey are of Lebesgue and discontinuities measurezero. to achieve then we have to impose additional conditions But,if t isunbounded > i f t is oforder 0 then it is the desiredconvergence. sufficientto require p E.g. that (L,,(f\302\273 is uniformlyp-times i.e. integrable. = 0, \"E H. lim sup J I L,,(/)IP dP,..II
0....0 \"EN
1',\"(/)1>0)
possibilityof dealingwith this problemis as follows.For it isrealistic to suppose that the lossfunctionisbounded and practical purposes it is not is but realistic to assumethat the lossfunction known.One continuous, a set}/'of boundedcontinuous has to take intoconsideration lossfunctions. Let W be the upperenvelope of With this interpreation the asymptotically risksare rather) interesting (2) Another
f.
(a)
he H, sup lim supVile,,}>,..,.. \"-CO)
v(
't\"
than)
(b)
limsup Whe\" P\",,,, hE H.) II-a:>)
of Theorems83.4and 83.5remain valid It is easy to seethat the assertions attains replacing(b) by (a).But,doingso it followsthat the sequence the boundin any case.) (L,,(f\302\273
formulations of the preceding resultsare 83.8Example.The famous classical
in Discussion concerned with the caseconsidered 77.5.The local isconcerned with the experiments) estimation asymptotic problem mainly
E\"
=
(Q\",\037\037\",
IE H,,}),l1eN,)
{\037\"+r,,-I/l:
the function x 1-+x, X E M, is equivalent to estimating11-+ t, estimating -+ Q\" Then a sequence of estimates M, tEN, for the localized K,,: problems. 11E N, islocally i n the sense o fTheorems 83.4 and 83.5))) asymptoticaJlyoptimal and
83.Application
to estimation
441)
83.2)iff (cf.alsoDiscussion
- x) h'(w , x) 1 . 1 = L I(x) ,I:1=1h( )
V;;(K,,(l9)
\"
+ Op;1(1), 1\\1I(...xI>Oj(Wj)
j
V
Q}EO\",n E
Wj,x)
n
N. (Centralityfollowsfrom 79.4or 80,8).)
with Po(
-
K,,:q}\037
n
\"
L i=
l!!E !1\",n E N
(Wi)'
1)
In of estimates is a locallyasymptotically optimalsequence Theorems83.4and 83.5.)
the
senseof
In 83.10Example.Let us discussa generali7.ationof the precedingexample.
estimation\". it is treatedunder the labelof \"minimumcontrast the literature Let 11'(x.t) = (
-
J 11'
(:<,f(P\302\273
inf J 11' (x,t) P(dx), P(dx)= ,eR)
K II : of estimates and the optimalsequence
i
-!.......
j
\"
\"
L lp(x (1) i-I
it K II
(.!\302\273
= inf
L (Xi> t), JE R\". 'ERj-l)11'
is This relationbetweenthe functionalf and the optimalestimator sequence alsovalid for moregeneral functions'I': -+ R. of regularity conditions.Let For simplicity we omit any discussion \0372
be sufficiently smoothand such that tl-9J 11' (x,t)P(dx), tE R, has a uniqueminimumfor every P Po,say f(P). Let
tp:
1R2
-+ R
\037
..
.
a2 0 = = tp(x, I) at tp(x, I), tp(x,I) iJt 2 w(x, t).) the function g 1-9 Then,by the implicitfunction theorem, f(Pg),g E II, hasthe derivativeat
Po)
-
.p(x,f(Po\302\273g g 1-9 J
Pu (dx)
J 1/J(x,f(Po\302\273Po(dx)
,)
gE N,)))
442
Chapler13:Asymptotic
!
Normality)
or in otherwords II,:
f--+
f
Ii>
Po dx) (
(x'/(
(
;o\302\273
;t,
vi
\037
!
(x;,/(
E
Po\302\273).
R',)
isa sequence ofestimate!\\ which i!\\ locallya!\\ymptoticallyoptimalin the sense of Theorems83.4and 83.5.Supposethat (K,,)is definedby equation(1)and of existence and measurability. Let usshow that (K,,)and (K,,) neglectquestions i.e. are locallyasymptoticallyequivalent,
-
V\037(K\"
K,,)
\037
0 (Po).
-
For this,we have to assumethat the distributions of q/;;(K\" are unifonnly tight.This is a consistency propertydependingon 'I',the proofof in thisbook.But if tightness is fulfilled, then the which is not discussed f(PO\302\273\"CN
expansIOn) \"
0= L v, (Xj, i= ,)
=
K,,(-!\302\273
\
L1 ,p(xj,f(P o Is
\302\273
+ (K,,(\037)
-
\
f(Po\302\273
L JP(xj,f(P o + \302\273
joq)
...
yields)
. 0= ,I:L tp(xl,f(Po\302\273 + \"
1
V
nil
r
-
+ V n (K,,(J) I
f(Po\302\273'
1.
\"
L 11 I\"\"
which
..
tp(x;.f(Po\302\273
+
.,.
1)
stochastic of (K,,)and (K,,), (cf. Levit equivalence obviouslyindicates
and Pfanzagl [1982]). [1975]
a linearmappingf: H R\". The Now, we turn to the problemof estimating in sections 38and72. corresponding non-asymptotic theory has beendiscussed x be a functionand define t E Let t: R\" IR\", h E H. W,,(x):=(x [0,co) ofH. Let L(f) = (kerf).L.For subsets Denoteby % the systemof all compact \037
-
\037
f(h\302\273,
convenience we denote(in view of Corollary72.10)) J tofdNH := J
t(f))
tofdN,.(f)'
83.11Theorem.Supposethat t:
IRA:
-to
andsub[0,co)is lowersemicontinuous
convex.Then every sequence ofestimates \302\253(),,)
sup lim inf sup W\"q\"p\",,,
K{,)f'
\"....QQ
\"'K)))
\037
E
fJt(E\", IRA:)
J t\"fdNH .
satisfies
. H3. Application
10estimalion
443)
Proof For fixed K E.*,\"it followsfrom Theorem62.5that lim inf supWhl}/lP\",h heK
\"-+00
An
inf
\037
\037E\037IE,AIt)
P(W\".P,,), sup /te\
of the Minimaxtheorem46.3and of Theorem72.11 yields application sup
\" E JI'
\037
E
= ft
inf
supP(W\",Ph)
inf
P(W\",P,,) = f tofdNH . sup \"t!\
SlIE.Jilt) hE\
t! SlIE,RIe)
0
83.12Remarks. (1) Perhapsthe readerdid expectinsteadof Theorem83.11 the weakerassertion)
liminfsup Whq/l\037.,. /1\"'00
\037
H. Jt<>fdN
\"\037\"\
there is no hopeof obtainingestimates But, under generalassumptions (0/1)
such that
limsupsup W\"{!/lP\",,. = JtcfdNH . /1-+<10
\"F.H..)
that (X/I) iscentralfor and X iscentral forE.Thenwe have Suppose -+ ff'(joXIP,,)weakly, he H.) !e(foX/lIP\",h) If (E,,) isequicontinuous on compacts and if t isboundedand continuous then thisimplies) (2)
(t.\037)
lim supfW,.(foX\dP\",,,")=
/I'\" 00
\"f!\
JtofdNH , foraB
Kef.
If t isnotboundedand continuous then the remarks of Discussion 83.7apply. (3) A particularcaseariseswhen dim H < and [= idH . Thenany central is asymptoticallyminimaxin the sensediscussed above. sequence Accordingto Jamesand Stein[1960]the identity is not an admissible estimate for a finite dimensional Gaussianshift and quadraticlossif dim H>2. an of the assertion 83.5cannotbeexpected, in Therefore, extension uniqueness a reduction under of the class of estimates consideration general.However, leadsto an admissibility assertion.) \037
k 83.13Definition.A sequence of estimates eft e fJ/(E\",R ), n E N, is asymptotioff) callyequivariant(for the estimation if)
lim n
-..
for all g E
Ig\302\243.1/1
-
P...1tg(.+
CI))
\037
b (LR\")
and II E /I.)))
[(h\302\273
\302\243.1/1
P\",,.I
=
0)
444
Chapter
13:Asymptolic
Nonnalil)')
83.14Remark. Suppose that ( islowersemicontinuous If (en) and subconvex. isasymptoticallyequivariantthen it isclearfrom Theorems 62.5and 72.11 tbat lim inf W\"(>nP\",,, S(01 dNH , hE H, /I.... \037
00)
sinceevery accumulation has constantrisk.If in additionthe pointof a has limit i.e.if there existsa probability distribution, sequence probability \302\253(>,,)
\302\253(>,,)
measure
RI\302\243lk
such that
= JgdR, gE'Cb(\037')' lim g(>/lP\".o
,,-\037)
then)
R=
!f(/oXIN
H
).Q
is anotherprobabilitymeasure. This is the famous asymptotic The reasonis that all accumulation convolutiontheorem,dueto Hajek[1970]. in are equivariant the senseof 38.26and have the same pointsof which can be represented as a convolutionin view of the distribution where QI ffr
\302\253(>\")
theorem 38.26. convolution assertion. Now, we arrive at the announced admissibility Although
it
a centralsequence reallynecessarywe assumethat (E\")possesses (X\.)
is not
83.15Theorem.Supposethat ( issubconvex,levelcompact,separatingand of finiteorder.If a sequence 01estimates E !Jt(E\",IRk), n E N, is asymptotically (>\"
equivariantand satisfies lim supW,,(>\"p\".1t n....QO)
then
\302\253(>\,,c")
rill
\037
I
J to dNH , hE H,
isstochastically equivalentto (10 X\\"E")
N
where (X\\"E")
01(E/I)\"EN') sequence
pointP E fJI(E,RA:) of Proof Every weak accumulation and satisfies by 62.4and 72.11
\302\253(>/1)\",
N
N
isany central
is equivariant
H , heH. P(W\",P,,)=J(cfdN with the non-randomized FromCorollary72.13we obtainthat p coincides Hence estimate pointof loX.Therefore I\"X is the only accumulation 0 X in of to distribution. Now, an application (U/I)\",N and (foX\/l4;") converge Theorem63.6provesthe assertion. 0)
1
\037
.
\302\253(>\\"e")
rill'
Let (H,D, t) be an Abstract Wiener spaceand E = (D, (D),{P,,: hE H})the Gaussianshift where P\" = Po E \'") hE H.) \302\243I
fC
(:B -
83.16Theorem.Let
D::Band
(x -
x \037 xED, hE H, where subconvex. i s lower semicontinuous and Then every sequence of))) [0,00) W,,:
t(h\302\273,
83.Application
to cslimalion
44S)
estimates Q\" E fJt(E\", B),n E N, satisfies lim inf sup WlIll\" \"-00
P\",II
J t dPo.
\037
\"CoHn)
Proof CombineCorollary62.6and Theorem73.6.
0)
it can be shown that even Similarlyas in Theorem83.11
supliminf sup <1>0 \"-00 11\"11
W\"C!\"P.t.,,\037JtdPo'
\037
83.17Example (Minimax propertyof the empiricaldistribution function. Millar [1979]). Consider the caseof Example80.4.Assumethat Q == and Po \037
is non-atomic. For every P Po let Fp be the distribution function of P. = the experimentE Consider in the fM, {PI\037:P Po}).We are interested of the distribution estimation function Fp. Let be the setof all continuous distribution functionson IR and define lossfunctions) \037
\037
(\037,
\037
L,,:(F,P).-. supVn (eR)
-
I F(\037)
FE .\037, P
FP(\037)I,
\037
Po,
for every sample sizen e N. We parametrizethe sequence aroundPo (E\")locally in the way of Example80.4.Thus,we obtainthe experiments E\"
= (R\",6f\", {\037iv,,: g ell,,}),n EN.)
n EN. The risk functionof (1\" for IR\" -+ !Fbenon-randomized Let (1,,: estimates, E\" and L\" is
gt-+J supVnIO\",,(\037)(\037)
-
f;..IV-;;(\037)I\037iVli(d\037),
ge H\".
\037EiII)
!
It is easy toseethat F,.,/v.;(\037)
uniformly for
\037
=
\302\243\"0(<:)
+
1 gdPo+ 0 ( ) ,
-00 v n
.\037-
n)
e lit This impliesthat for every FEff
L,,(F,
\037!V'ii)
-
= supI V;;(F(<:)
FPo\302\253n
(f:[oC
\037
\037
\302\253(1,,(\037)
-
\037
FPo)'
\037
E
R\",
1Fi\"
)
.
-+
\"E N.
xEfGo(R), with -r(g): D=l6'o(lR)and \037:xl-+llx--r(g)IIII' 1-+ 1 gdPo, E R, g E H, we obtai\037 that -00)))
Letting \037
VIi
f
Vn)
estimates Now, definenon-randomi7..ed K\": K,,:
- -00gdPol+ 0 (.
\037
446
Chapter
13:Asymptotic
Nonnality
L.(.,p\"v.)\".P,';V.W,K.P'iv.+ o (
JJ, ge 11.,ne N.
\037
It followsfrom the remark belowTheorem83.16) lim inf sup L\" (., P)a\" P\"
\"... ct';
\037
P 'II Po)
lim inf sup sup 0>0
\037
\037 K\" \037jVil
PO(gl)\037o)
\"--\302\253>
J 1I.lIudWpo
'4'0(11I))
since(H,\0370(1R),-r) is an Abstract Wiener space,and where Wpo is the measureconstructed in Theorem70.5. probability of empiricaldistribution functionswhich are Now, let (\037) be the sequence made continuous of estimates Thus,(F,.)is a sequence by linearinrerpolation. f fF. fact for the decision It is a well-known that (cf.Parthasarathy [1967]) space
!I!(IIv;,(F,:-
Fj,)
P\") 11..1
\037
!I!(II . 11..1) Wp
weakly,
that PI&tJ. This implies uniformly for all non-atomic probabilitymeasures attainsthe asymptotic lower bound.)
(1,:)
of centralsequences) 84.Characterization can becharacterized In thissection we show that central assolutions sequences of wellThis will leadto formal statements of certainoptimization problems. of maximumlikelihood and Bayesestimates. known properties We beginwith the discussion of maximumlikelihood estimation.)
84.1Discussion. Suppose that e c R k is an open subset. Let ge en,neN, be a sequence of dominatedexperiments and (D\",.9I\", {P\".,g: let {p\".s: .9ER}
estimators g\": D\"
-e
\037
of nE N. In the usual terminologya sequence if) of maximumlikelihood estimates is a sequence
\\I\"'.Q(\",
dP. dP.3 = \037 sup\037
n
eN.
9(8 d\\l\ \\I\"-a.e., It is commonreasoning, that sucha sequence is a rather goodchoiceof an and basedon estimator alsotests this arc generallyoptimal sequence sequence in somesense.In the followingwe supportthis opinion by a localasymptotic the maximum method. To fix 9 E e and likelihood bemoreprecise, propertyof consider the localized experiments))) d\\l\"
84. Characterizationof central sequences En.\037
447)
= (Dn,.\"'n, t E 1;, (8)}), {P,..8+c5\",:
= {tE T:9 + \037nt E 8},n E N. Assume that theseexperiments where 1;,(9) are that the asymptotically nonnal.We shallprove under certain conditions of maximumlikelihood is of the property (\037n) equivalentto centrality sequence 1 \037
(gn-9), nE\037.
n)
in established In view of the various optimality ofcentralsequences properties this assertion is a strongargument in favour of the the precedingsections under which it can method. the conditions maximumlikelihood Unfortunately,
be provedare rather restrictive. 64. of Section We beginwith a generalassertion which isbasedon the results such that Tisa Euclidean vectorspaceand n E N, are opensubsets Suppose = nonnal se. T. t E be an that 1;.i Let En (Dn,.5#\", {\037,,: T,.}) asymptotically quenceof experiments. 84.2Theorem.Supposethat the likelihood have con. processesdP,.,/ 1'\".0 ) ( tinuouspathsp\".o.a.e., n E N. Assume that the following conditions are satisfied: (1) Forevery t E T and r. > 0 there existsa compactsetK T such that . dp,..S dP\".s . I1msup $; sup (1 e) < e. sup \"-a) Sf!T\" dP,.., {$(; T,,\\K d } 1;\"
\037
--
P,.\"
I
f!
T\"
\302\243
-
\037.r
.nElli,are equicontinuous in (P..,).
(2) The likelihood processes P..,
( ) \037
1;.\"
for every t E T. probability Then a sequence of random variables(Y,.) is ce1ltra/iff il is a sequenceof maximum likelihood i.e.) estimates, asymptotic
.
I1m P,.,r
dP,.'\" y
{ dRn.r
\"....IX;
\037
-
dp\"\037. s
sup d
s,=T\"
P,.,t)
&fiT\"
-
(1
\302\243)
}
=
1
for allt E T, c > O.) Proof Considering Example 64.1,conditions (1) and (2) are equivalent to it 64,6.Moreover, isclearthat conditions conditions 64.5are satisfiedsincethe limitexperimentisa Gaussian shift.Let X bethe centralrandomvariableof the limit experiment. Thenevery likelihood of the limit experimentattains process its supremum exactly at X. Hence,the assumptions of Theorem64.15are and 69.11 and 80.12 o f satisfied, by every sequence asymptoticmaximum likelihood is central.Conversely, estimates by Theorem64.8and Corollary 64.13 maximumlikelihood is a sequence of asymptotic every centralsequence
estimates.
0)))
448
13:Asymptotic
Chapter
Normality)
The conditions (1) and (2) of the precedingtheorem can be simplified
For this,leL (XII) be any central sequence of (E,,) and definethe considerably. residuals sET,,)n E N, of the stochastic expansionby rll(s\302\273)
-
dP\".,= exp
\302\253.'i,
XII)
-
dP..\037
n.O)
1
2
+ rll(s)L sET,.,Il EN.
\"2l1s11
dP\" have likelihood processesdR n.O)IE Tn ( n E N. Then condition 84.2(2) holdr;iff inuous pathsP\".o-a.e.,
84.3Lemma. Supposethat
lim p\".o{supr,,(s)> 1
II
,eI()
'\302\2531
1
I
the
\302\243}
=
cont-)
0, f: > 0,
for every compactK c T.)
Proof Assume first, that 84.2(2),is fulfilled.Then it is clearIhat alsothe in (p\",o)-probability. n E N, are equicontinuous Since residuals 1',..00 for every fixedsET,equicontinuity the assertion. implies rn(s) that the residuals Now.assume conversely satisfy the uniformity condition. It is sufficientto proveequicontinuity of IOg dP\",, , nEN, in (1',..0)( dp\",o)st.Tn 2 Xn ) IIs1l probability.Sinceit is clearthat )sE Tn' tI E\037, is equicontinuousit remainstobeshown that But this T..)n E N) isequicontinuous. is an almostobviousconsequence of the uniformity conditionand of the continuity of the residuals. 0 (rn(s\302\273UTn'
I
-
\302\253s,
\037
(rll(s\302\273'E
dP,.,, have continuou.r; 84.4Lemma. Suppose that the likelihood proces.r;es d1',..o (exists )a IE I; n N. all > > there E 0) 0, compactsetK 5 T pathsP,..o-a.e., rrfor T\"
\037
such that
.
11msupP\",o sup dp\"'s II {seT\"'l( dP.n,O)
.
u \037
\037
eLl
}
< e,
-
then cOtldition 84.2(I), issatisfied.)
ProofLet e > O. Since0 E 1;.,n E N, we have -) 1. sup dP\".,_>
dp\",o setK This implies that for every 17 > 0 there existsa compact $Ii
T\"
P\",o
dP. II,S sup d IET\"'K
{
\037
p,.,o
-}
dP. II,S . (1 s) < '1) fl EN, sup d lilT\" P,..O)))
\302\243
1'suchthat)
84. Charactcrizalionof central sequcnces
449)
For convenience, denote) A,,(K,e) =
r-
dP,..s
-
dP,..s sup . (1 r.) . sup \037 scT {HTn,K 1',..0 n 1',..0) } \037
lET there existsa compactsetK T such that lim supPn.,(A\" (K, < 1;.
Let us show that for any
\037
1;\302\273
,,-QO)
Let t1 > 0 be suchthat For this,we apply (I',. , ,) <{ (p,.0) throughLemma 18.6. ,)
lim supP,.,o(A II) <\" implies limsupp\"., (All) \"...
,,-
Now, let K
\037
<
\342\202\254.)
00)
CI))
T be a compact setsuch that)
lim supp\".0(A,,(K,
\"....
\302\243\302\273
<\".
Q())
Thisimplies lim supP\".,(A,,(K, < e.)
\".... G())
-
e\302\273
Now the proofis finishedsinceby contiguity)
.
hm \"\"'cx,
= 0 = 0, P,..,ddP,.., { 1',..0) }
and we obtainfinally
.
that)
dp\" s dP,.s \037. \037 sup sup ,,-ex, {so:Tn'K dp\"\" SET\" dP,..r lim supP,..r(A,,(K,< I: \"....
hm supP,.., \037
\037
(1- } \302\243)
1:\302\273
Q())
which provesthe assertion. 0)
of Theorem84.2isestablished Usually,the assertion by verifying the assumpof Lemmas 84.3 and 84.4.) tions in the independent, identi84.5Discussion estimates (Maximum Likelihood Fix 77.4. xE M o f some the situation Example case).Consider callydistributed
and define) E\"
= (Q\",.tI\",{P:c\"+rivn: (E 1;,}),n EN,
where T\" = {lE T:x + l/V\037 E M},n E N. If conditions 77.4 (1)-(4), are satisfied then the experiment(Q,.r.(,{P E M})is differentiable. FromExample))) x:''(
450
Chapter
13:Asymptotic
Nonnality)
80.8,(1),it followsthat (E,,) is asymptoticallynormal.In the presentcasethis can alsobe seenfrom the Taylorexpansion)
;:
. = L (t(Wh x + II1 n) - t(Wi' log--dP.)(IVft\" i=
d
t
P:%,,+
\"
V
(Q,\302\273
t
=
x\302\273
1 i:.t\"(wi, x,,(Q,>, i:.('(Wi'X). t - _ 2ni-I
t\302\273.
Vni.1
I\037
i:.t'(W ,X).t+2Px(t\"(.,X)'
(t,I)
1
= I\037
V
1'l
f;;
i
(/,/\302\273+
rll(t)(cg),
1)
where)
1 rn(t) (QJ) = _2
i:.t\"(w , x,,(QJ,
\037
(
t
n 1_ I
I\302\273
. (I,I)
-
PAt\"
(.,x) . (I,
1\302\273\\ \037)
and)
::;
III, ne 1\\1, Ie T.)
Ix -x,(ro,1)1
\037
Let)
-
x). B,,(s,1)'= P,,(t\"(., and X,,:Q\"
(S.l\302\273)
-..T be such
B,,(t,
X,,(QJ\302\273
=
that)
1
I;: V
\"
L ('(Wi'x).I, t;;
1'l
q}E U\",
lET.)
1)
Then Bx is the canonical and (X..) is a centralsequence. covariance
Now, we strengthen the conditions by the additional requirements)
(5))
lim PX<sup It\"(.,x + h)
C\"'O
Ihl
- t\"(.,x)l)= 0
1 limsupsup - i=L t(w;.x+h)
(6))
\".\302\253)
1\"I\037cS
n
I)
than 77.4(4), and condition Condition condition (6) isa typical (5) isa stronger of condition which is usually appliedin orderto achieveconsistency \"global\" fact that under method.We shallprovethe classical the maximumlikelihood F or ofTheorem84.2isvalid. this.we verify the conditions (1)-(6)the assertion the assumptions of Lemmas84.3and 84.4. of Lemma 84.3is valid.For this,we (a) Let us show that the assumption the residuals estimate r,,(l)in the followingway:)))
84. Characterizationof central sequences
I rJl
(t)(QJ)I
451)
\037
\037ltl2
1
1
\037
itl/\"(Wj,XII(Q),/\302\273-PA(/\"(\"x\302\273)
II
\03721/!2;;j\0371 1\"(Wj,x)-PAI\"(.,x\302\273
+
1
1
!
21/12.n
It
11\"(w\"x,,(cy,
I\302\273
-
I\"(w\" x)1
j\037l
2 < !1/1 f. t\"(wlox)-\037(/\"(.,x\302\2731 2 ni-l
1 1 + -1//2.L n j::1 2 II
It\"(Wj,x+h)-I\"(wj,x)/.
SUp Ihl\037I'I/Vii)
folJowsfrom the law of large numbersand condition Now, the assertion (5). (b) Next, we show that there issome00 > 0 such that)
dP:..,tV\" hm p; 11-(1) 1'1 <6oVJI { dP. x)
\"
n
\"
\037
XII) exp(BAI,
-
I
.
4 Bx(t,
I\302\273
}
=1
\"
Theestimation of the residuals under (t) showsthat forevery e > 0 there exists
o(e)> 0 such that
P:
I'!n
}-1.
{lr(t)I<eI/12 ll
< d(t:) V\
Il isclearthat there existssomet > 0 such that) Bx(/,I), IE T.
2 81/1
\037 \037
obtainedat the beginningof this example,the Togetherwith the expansion assertion follows. (c) We show that there exist 00 > 0, c > 0, C < 00,such that for every a> 0 supP
I1m It-CD \"
II
.x
{
. < C2 e = sup dP:+r1ViiCQZ a
G\037I'I\037dovn
tiP:
\037
}
Choose00 > 0 accordingto part (b).Moreover,let)
Ci::min BAI,I) > 0,p:=max BJC(/, I) <00. I'I-t
1'1::1)
Then)
P:
{
sup
G\037I'I\037\"oVII
dP,x+,:vn\037e-Q2aI8 dP,x)))
}
.)
452
Chaptcr
\037
13:Asymplotic
P;
Normality)
{
-
.(BAt, XII)
SUP
a\037I'I\037\"oVII
\037
-a a/s} + 0(1) 2
t\302\273
\037
64.p4 . + 0(1). \037(IXIl12) 2 2
\037
SincePx\"(
4
aa/8p2}+ 0(1)
P.\" {IXIII
\037
1
_ BAt,
a
a)
t'(.,
(.,
the assertion follows. 12)= tr (Px(!' x) @ we in a arc t o the of Lemma 84.4.Let (d) Now, position verify assumption f. > 0, b > O. In view of (c) it isclearthat for sufficiently large a > 0 I
I XII
.
1m
II
supP\"
sup
{a:.I'I\0374oV;;
II-\037
x\302\273)
-.-.11-.;;; < I:-. dP:r. }
dPx\"+,,,/1 ui:
2)
Thus,it remains to show that)
.
hm sup
sup
P,,\"
d\037II+'IVII
{l'I>\"\"v.;
11--<>:;
- } < -.
\037
/I
6
{)
2)
d\037
This is doneby means of condition (6).Togetherwith the stronglaw of large numbers this condition implies)
1 Iimsupsup - L (t(wj!x+h)-t(Wj,x\302\273
II-tO
Ihl\037\"o
j:.
n
P\"N-
1)
-Z we have
Abbreviatingthe left hand sideby {Z> O} = 1,
-
J\037I\\I
and there existsa randomvariableN: QI\\I
1 sup .- L II
(t(Wi' x + h)
11I1\0376onj-l
whenevern
\037
\037
x\302\273
1 < ;, Z(ev) -)
N(C!J).Now, for B > 0 let neB)E
\037N({N
Then n
- t(w;,
\037
nee)}n {Z
\037
-
Nand \"(6)> 0 be such that)
l:::
,,(e)}) 1 2') \037
neE) implies
Px\"
-
dP: sup dP x {
+
II
\"
\037
exp(
-
n
11I1\03760
. t/) < t . } 2
Thisyieldsthe desired
.
such that)
\037
Itm
dP,,\"+'\"',;
I supP; sup -\037
JI....\037
{
Itl\037a
\037
dP\)
s:
CJ
}
< l:::.
84. Characterization of ccntral scqucn<:e$
453)
but islimited Thefollowingexampleneedslessrestrictiveregularityconditions,
to the one-dimensional case.)
and Has'minskii, Consider 84.6Discussion [1972J). Ibragimov (LeCam[1970], x M F ix and define of Discussion 77.5. some E the situation = (Q\",d\",{P:+r/v'iI: IE 1;.}),where 1;.= {IE IR: x + IIV;;E M}, n E N. If then the experiment(Q, {Px: x EM})is conditions 77.5 (1)-(3), are satisfied differentiable. FromExample 80.8(1),it followsthat (E,,) is asymptotically E\"
d,
normal.
(2) of Theorem84.2is valid.For this,we (a) Let us prove that condition K R and someno E N have to show that for every > 0, every compact \037
\302\243
lim supP.\"
-
dP:+$/V\". dP:+,/vft
SUP
6\"0,,\037\"O)( - { s,'e\"
JS-'1<6)
dP:
> e = O.
dP:
}
thisis provedfor someno E N then it is alsovalid for no = 1, sincethe likelihood ratiosare continuous for every fixedn E N. Choosean openneighborhood Ux of x wherey t-+ I(y) is boundedand let If
no E that
N
be such that x + 1\037 K 5 U\" for n V
\037
no.FromLemma 77.2it follows
n)
-
supndi(P\".siVn,P\"fr/v.;) C 'Is 112 \037
,,\037\"o)
whenevers,IEK.This impliesin view of Lemma 2.17)
f
s(
. P:-:.IV\037 dP! \"
- \037P:+'lvft dP.\" \ )
2 d\037 .x
- C' s\037
l
1 12
\",s IE K
n
\037
n0
.
and Prohoroff Now, the theorem of Kolmogoroff yields)
Is-'1<6:c-
lim SUP > = 0, 6-.0sup { .,re\" dP. dP.\" VdP''+,7Vii V\037!:!\"Vii } P,,\"
\"\037\"o
\302\243
\"
E
>0.)
:c)
Moreover, insertingI = 0 we obtain) . lim supP; SUPI /dP. d
41\"0.(0
\"\037\"o
{
$I.\"
> a = O.
}
x\"+\037\037
V
P\
Thisimpliesthe assertion.
84.5(6),the assump(b) Let us provethat under the additional hypothesis tionof Lemma 84.4is valid.Copyingthe argument of 84.5(d),we needonly showthat there exists constanta >0such <>0> 0 and for > 0,l> > 0 some \302\243
that)))
454
Chaptcr
t
3:Asymptotic
lim SUp ,,-+
P1(\"
{
U
Nonnality)
\037
+
dP)(\"
SUP
\037 \037v\"
60Vii 111.5
dPx)
b
}
< t:.
Choosebo > 0 such that B(x,O/\037) U1( and that l(y) c2 > 0 for y xl < bo - Then Remark 76.7implies) I
-
\037
dl2 (I'\"
2 C P;t)
\037
It followsthat for \" P\" x
dP:+11 { dP:
v7-
\"2
I
y
\037
-xl , - xl < 0 2
I
y
0
,)
\"0' t < Vn <>0') I
\037
I
- ex ( _ ('2 p
\037
\037
exp
\0372
4
-4-
)}
c2(2
(
)
J)
dP:+1,V\" dP\"
--
(1- di 4. ( ) - exp c24 1- 2n
\037
exp
c2 12
(\037+tIVii'
(2
\037
(
exp
\037
x
dP.\" x)
C2(2
)(
4 C2 / 2
\"
)
C2(2
exp
( ) (
2
\037\302\273\"
)
= exp
( 4) C2(2
.)
In the followingwe denotefor convenience) II.',=
Y.:
We
' 0, if ( < 1In V dP. Vd'\037'+\"y. if o (I vn 00
!
I
\"
I
\037
,)
shallprovethat for every a > 0)
P:
\037
:
exp
{\037\037fy\037.,1
We
<>
I
\037
consider only the case(
Fix mEN and define) T
=
O. The caset
;)'}
3 p _ <: [ex (
Thenwe have)))
\037
I, = m +
< 0 is treatedsimilarly.
;.'0
\037
i
\037
T.
84. Charactcrizationof central sequences
P;
{
I
m\037x
Y\".,.I
\037
exp
O\037I\037')
c2m2 ( -4--)}
\037
455)
c2m2 ( \037)
(r + l)exp
which implies)
P:
sup 1
{
c2m2 exp - \037 ( )} (r+ l)exp( \037)) c2m2 -16 -ex c2m2)} p 1 n,',- n,':- ex -. ) ( , ( C2m 2
I
y\",,1
lII:Sr\037lII+
+
x
\037
{
\037
\037
6)
su
I
y.
y.
I
p
\037
I'I-'\037r::
p
-4
I)
lII\037r\037r:\037m+
The lasttenn can beestimatedby means of the theorem of Komogoroff and
Prohoroffin the same way asit isappliedunder (a),but making the constants
Thisyields) explicit.
P:
SUP
{
\"I
m\037'1
-':1;;;
-
Y\",,:I
\037
\037
;i'\037\037III+
exp (
C2m2
16 )
- exp
(
-- (;2m 4)
2
)}
1)
c
-
\037
I Yr.\",
( (
2'
-16) -exp(-\037))) c2m2
r exp
c2m2
Thus,we obtain
P: {
SUP
1II\037'o!m+
\037
1
!
\037\"I
(r + I) exp
\037exp
\037
c2m2 _ exp 16
(
- )+ ( e':'
)}
--( e'm')
C
r cxp
(
16 -cxp
(e'm'))'
-. +2Cexp - -16 ( 16 ) +exp( -.4-) ( ) c2m2 --
c2m2
c2m2 -
providedthat mEN is sufficiently large.Hence,for a> 0 1';\"
I
Y...I exp _
(
\037
{\037\037\037
\037
i P:{
III\"'Q
\037
--
SUP
e::' )}
I Y,.\"
I
\037
1II::i'\037III+l
2(C+ 1)m=a exp _ 16 ( )
(
\0372\037\0372
\037
'
c2 n _ exp 16 \037
\037
)}
C1 exp _
(
c\037_a2
16)))
)
.
4)
,)
456
Chapter
13:Asymplotic
Nonnality)
-
dP. IR and E = (IR, 84.7Example.Let {P9:9 E en where d; = f(. 9) continuous distribution andfisan absolutely densityofa probability vanishing
e
at infinity, with
\037
\302\243I,
Fisher'sinformation)
1=J
(})'IdJ..) Then the assumptions of 77.5 (1)-(3),are satisfied. Assume further that > 00. In the results o f the orderto discussion we Po(logf) preceding apply needonly show)
1
limsup sup ,,-C()
\"
L logf(x ni=-l) j
'''1\0376
- h) < JlogfdPo.P:-a.e..> O. (j
This is easily doneas follows.We repeata classical argument due to Wald
[1949]. By
strictconcavityof logand Jensen'sinequaJity we have) O.) o if h Slog/(x h) Po(dx) < Slog/dP
-
=to
By continuityoff, for hER,h such that)
-
J sup logf(x
tl) Po
!JEW\
* 0, there existsan openneighborhood
WIt
of h
(dx)< J 10gfdPo.
of thoseneighborThe compactsetR\\ U can becoveredby a finite collection that) hoods,say W\"I....' W\"Ie' The law of large numbersthen implies
-
\"
1
lim sup sup-
L logf(Xj h) i-I) 1 lim sup max - L sup log/(x j n isI n\"oo t
n'oo
Ihl\0376
n
-
\"
\037
liE WIt))
\037j\037k
= max lim 1
tl)
-
-L /I
sup 10gf(Xj tl) i=l !JEW,,)) = max S sup logf(x 11)Po(dx)< JlogfdPo P\037-a.c. l\037j\037k
l\037j\037k
\"-+00
11
-
\"tW\"1)
of Bayesestimates.) Now, we turn to the discussion
84.8Discussion. Supposethat e
IRk is an opensubset and let = of continuous experiments. .91\", G\" <'1\", {P,.,\037:9 E e}),n E N, be a sequence 0 such that for every fixed 8 E the Assume,that there existsa sequence (j\"
localized experiments)))
\037
!
e
\0374.
En.S=
Characterizationof central sequcnces
457)
tt 7\037(,9)}), nt N, (!l\"\"\",,, {P\",3Jd\"t:
In the usual Let .ula(e)bea probability measure. are asymptoticallynormal. a sequence estimators is a seof nonrandomizcd g,,:Qn terminology, if) of estimators Wand for .u quence Bayes \037
-
II w G.(g. =
inf (1 f!
9t(Gn
e,
dP,...I'(d.9) 9\302\273)
;
.8) HSW ( \
(0'-8\302\273
\"eN. ) Q(.,d(])dP\",s.u(d8), thisismoreorlessequivalent
of section37 we know that Recallingthe results to)
IW
U.
-
-
(g.
Iw
F,(d9)
\037\037\037
9\302\273)
(\"
G:
-
(d9)p.(1')-..\"
Jo;
,)
9\302\273)
where(F;.)are the posterior of the experiments to distributions (G,,)with respect thisis even equivalent to the 11.We shallprove that under someconditions centrality of the sequence)
-
1 1;;= b\ (9\" 8), n e
\037,
of localized sequence experiments (E\".S)\"E\037 for every 9 E e. The essentialdifference to maximum likelihood estimationis that for Bayes on estimators centrality dependsrather integrabilityof the underlying experiments than on smoothness. a general resultand then specialize to the independent, First,we establish c ase. identicallydistributed for the
Let
W:
T
- [0,sequence be subconvex, level-compact, separating of order of Borelmeasures \"e denote
p > O. Given a
and
ex;)
distributions of E\" and 11\",Il eN,) posterior
84.9Theorem.Supposethat
11,,1\037(T,,),
N,
by
F\"
the
is equicontinuous. and uniformly (JI,,)011 Then a sequence ).,strongly compacts. of is randomvariables( Y,.) iscemral it a estimates sequence iff ofasymptoticBayes i.e.,f for W alld(11,,), lim W( t) r\037(dt) > inf J W(s t) f\037(dt) + t} = 0 \"-a;P,..r{J T\ imegrableof orderp, where Ji\"
\037
(E,,)
\037
-
-
5\037
for alllET,c > O.)
67.6and 67.7if we can show that of Corollaries ProofThisis a consequence the limit experimentsatisfies of 67.6(2),and if the general1zed))) the conditions
458
Chapler13:Asymptotic
Nonnality)
with the Bayes estimateof the limit experiment coincides
centralrandom variable. Let F bethe posterior distribution of the limitingGaussianshift with respect to A and X itscentralrandomvariable.Then, by Example36.11 (1),we have F= NT * F.x.It followsfrom Discussion 38.24that Xisthe uniquely detennined for the limitingGaussianshift with respect to Wand A.. 0) Bayesestimate in the independent, 84.10Discussion (Generalized Bayesestimation identically
that e IRk is an openset.Let distributedcase).Suppose E = (.0,sI, {P,9:9 c en be a differentiableexperimentwhose derivativesare injective for each 9 E e, and which satisfiesPa * if (1* 'to Then the \037
\037
the followingcondition: experimentsatisfies U(8)of8 and constants (1) Forevery 8 E e there isan openneighborhood a(9)> 0,b(8)< 00,suchthat a(9)10' tl d2(\037'\037) b(9)10' tl if 0',t E U(9).
-
-
\037
\037
::
Fix somep> O. Let pl\302\243f(e) bea u-finitemeasuresuch that P <{ ).k and gl= and positiveon Assumethat for somen E N the mixture iscontinuous
e.
A
P t-+ J P:(A)ltiI Jl(d(1),A E
Ie
sin,
isa-finite. Thenthe consistency Schwartz[1964] theory developed by Lorraine and LeCam[1973] leadsto the followingassertion: (seealsoStrasser[1981]) For 9 E S IRk and e > 0 there are a constant a> 0 and a E e, (2) every every of criticalfunctions ElF (.0\",si\,")n E N, such that) sequence P:+stVi> > 1 e and < J P:(CPn)IV;;(u9)IPJl(du) (/)11
-
Vllla-,9Ii;D
CPr!
-
-
1
\037
Vn
e.
assoonas 9 + s/01Ee, {UEe:01/ti 91< a}c Let W: IRk [0,00) be a function which is subconvex,level-compact, distribution of (P,,\.,@8 separatingand of orderp. Denoteby the posterior estimate for .0\" is a with respect to p.n E N. Thennil: e (generalized) Bayes (u,t) e e2, n eN, \037: (u,t) t-+ Wq/n (u
-
-
F:a
-
and p
t\302\273,
if
neN.
JW(V;;(lq\"-u\302\273\037(du)=infJW(01(u-9\302\273\037(dti), 9\302\253:8)
In the followingwe discuss the localasymptotic behaviour of the sequence For this,fix some9 E e and consider the experiments
(-9\"11)'
En.3=
.flI , {P:\037I!Vi>: t E \037(9)}),n EN.))) (!l\", n
\037.
Characterizationof central sequences
459)
-
Yn := shallprovethat the sequence .9), n E N, iscentralfor (En .s). V;;('\037n of Bayes This is doneby verifying that on the one hand (Y,.) is a sequence the assumptions estimates for (En .s) and on the otherhand that (E... s) satisfies of theorem 84.9. Let (J,ln.S)/'IEN be measures on definedby
We
[j(\037(9\302\273
Jl...(B)= f<;)Jl({<7Ee:)Iii(\"-.9)E BI)
!-
g(8+ t/ \037-) dt, BE n EN.) g(9) Forevery n e N let bethe posterior distribution of (J\037:\"'t/V';)tF Then it is easy to seethat) =
f18(\037(.9\302\273,
f\037,s
8:
T\"(9) and
J,l..,S'
= F,,{o\"E vn(O\"-9)EB}. BE nE N, F\".s(B) of Bayesestimates and it followsthat (Y,.)is a sequence for Wand (J,l..,s). It remains to be shown that the assumptions of Theorem84.9are satisfied. [j(\037(8\302\273.
....
First,it isobviousthat J,l..,9 ),Ie,stronglyoncompacts. Secondly, equicontinuity of (En,s) followsfrom (1)since) d2 (PS+ \037
J/V\"ft
, Ps+t/t/ii))
h(9)ls-tl, Vndz(P,Hs/Vn.PSt-tIVn) :::;
whenever 8 + s/vn e V(8),8 + tlvn e V(9).Finally, we have to show that isunifonnly (Jl..,s)-integrable oforderp.Thispropertyisa consequence (E...s) of (2) which can easilybeseensubstituting t 1= Vn (0\" .9)into(2) and recalling of jl.., the definition s, n EN.)
-
84.11Example(Pitman estimation in the independent, distributed identically E en where case).Let t9 = RandE = iB,{P..,:.9 =f(. 9) andfisan \037 (\037,
-
with Fisher's absolutelycontinuous density of a probabilitydistribution
infonnation)
1=
d!J'fdl
E
(0.00).)
77.5 the experimentE is differentiable and Ihecondition Then, by Discussion
issatisfied. Assumefurther that (1)of 84.10 2 J 1 f(t)dl<00. issatisfiedwith))) Then,by O\"-finitenessof the mixturealsocondition (2) of 84.10
460
Chaplcr13:Asymplotic
Normality)
= 12, the generalized ,q\", n E t\\I, for W (I) Bayesestimates p = 2.Now, consider 1E IR, and J.l = )..FromExample37.6(1),we know that)
n I(x,- I)dl \"
,\037,,(,!)
=
J1
, ,!E illl f(xj I)dl J ifl ='
-
1R\",
nE N.
1)
and the are usually known under the labelPitman estimates Theseestimates
for the 39identify them asthe optimalequivariantestimates resultsof Section that (Vn(gll .9))IIEI'II is a central it follows experiments n E N. From84.10 for the localized (E .s), .9 E 6J.))) experiments sequence
-
\302\243II,
II
Appendix:Notationand Terminology)
It Thispart isintendedto fix the mathematicalterminology and somenotation.
seemedto us the best way for achieving this is talking about parts of In thisway, we mathematicswhich are presumablywell-knownto every reader. hope that the readerconveniently becomesacquaintedwith the specific and the basicnotationof this book.Thus,oneshouldnot be terminology that or in the logical surprised our remarks here are far from beingcomplete orderof a textbook. Nevertheless,we have triedto give a survey of mathematicalknowledgewhich is indispensable for understanding this book.)
Setsand mappings. If X isa setthen x E X meansthat x isan elementof X. The the elements of X then emptysetisdenotedby 0.If E isan assertion concerning the subsetof thoseelements having the propertyE is denotedby {xE X: x satisfies B Let of X, i.e.A X, B X. Then Au B is the be subsets E}. A, union of A and B, A (')B the intersection, A \\ B the differenceand A 6.B the Two setsA. B are disjoint if A n B = 0.Thecomplement symmetricdifference. of A isdenoted by A'.If A isa setthen IA isthe cardinalnumber ofA. The set of all subsetsof X isdenotedby 2x. for the set of natural numbers, 0 Frequently used notationsare = u {O},iR for the set of real numbers,IR.. for the set of positivereal numbers.Q denotesthe setof rationalnumbers,C that of complexnumbers and = R u {oo} U { oo}.The most importantsubsetsof IR are intervals \037
\037
I
1'\\1
1'\\1
1'\\1
-
\037
beingdenotedby)
[a,b] = {xE R:a [a,b) = {xE R:a
\037
\037
x b},(a,b) = {xE IR: a < x < b}, x < b},(a,b] = {x E R:a < x b}.) \037
\037
Moreover, we use the notations [a,(0)= {xe IR: x > a}, [a,00] = [a,00)U {oo} and soon.Every subsetA IR has a leastupperboundand a greatestlower bound,denotedby supA and inf A. Let 1:4=0 be an arbitrary set.Consider .fd = (A/)/E I of a a family of subsets setX. Then the intersection of ,fd is nd:=n A i ,= {XEX: XE Ai for every ie I}, \037
iel)
and the unionof
d
u ,\037::::: U iel)))
is)
AI
:={xE X: x E Ai for at leastonei E I}.
462
Appendix: Notation and Terminology)
0.A partitionof X isa family d of subsets ofX, which are pairwisedisjoint andsuchthat U d = X. A coveringof a setA isa family of setsEJ suchthat U EJ;2A. Let \037I' f!d2 besubsets of2x. If than fA 2' and 2 is finer than I' 1 2 then a 1 iscoarser TheCartesian o f product A and B isthe setofall orderedpairs{(x,y): x e A, y E B}=:A x B. Let C A x B. For every x e A the fibre of C at x is C = {ye B:(x,y)E C}.If(A{)tel isa family ofsetsthen the Cartesianproduct isgiven by If 1= 0 then we put n ,cf =
\037
\037
X and U ,rI/ =
\037
\037
\037
\037
IC
n
i{;I)
If At =
A,
Aj
=
{(x)iel:XjE A j
j
if
iE I}.
ie I, then we denoteAl 1=n At. iE
1
relationR betweentwo setsA and B isdescribed by itsgraph G\" which is a subsetofA x B.Thegraph GR isinterpreted asthe setofpairswhich are related R on a setA is transitiveif (x,y) e GR and (y, z) e GR implies by R. A relation . If it is transitive, reflexive(i.e.(x,x)eGR for every xeA) and (x,z)eGR symmetric (i.e.(x,y) e GR implies(y, x) e GR ), then R is an equivalence An equivalencerelation definesa uniquely detennined relation. partitionof A intosetsof equivalentelements. Thispartitioniscalledthe setof equivalence classes and isdenoted by AIR. A relation which is transitiveand reflexiveisan orderrelation. Let (/. be an orderedset.It is directedfrom aboveif every paira,bEl has a leastupper boundaub ( = sup(a, It isdirected from below if every pair a,bEl hasa lower bound ('\\ b a (= inf(a, An orderedsetis a latticeif it is greatest A
\037)
b\302\273.
b\302\273.
directedfrom belowand from above. A relation f betweenX and Y isa mappingor a functionfrom X to Y if for
such that (x,y) e G/. In thiscasewe denotey = I(x) or f x \037 y. The fact that is a mappingfrom X to Y is by f X -+ Y. ThesetX isthe domainof and Y isthe range off The expressed imageoff is imf= {f(x):x e X}.If A X, B Ythenf(A)= {f(x):x E A} is the image of A and I (B)= {xE X: I(x)E B} is the inverse image of B. A I ({y}) I = 1 for every y E imf surjectiveiff(X) = Y, injectiveif mappingfis if it is bothsurjectiveand injective. It is bijectiveorone-to-one Iff I X -+ Y is 1:Y -+ X satisfyingf(} = y, bijectivethen t there isan inversemapping} = x, x EX.If X, Y, Z are setsandf X -+ Y, g:Y -+ Z are y E Y, andr isgof x H g(/(x\302\273, x E X. If (X, \037), (Y, \037) mappingsthen their composition are orderedsetsthen a mappingf X -+ Y isisotonic or increasing if XI Xz if XI 5:X2 implies I(xt ) I(xt)\037/(xz)and anti tonicor decreasing implies \"? f(x2)'x I> X2 E X. There are somefrequently usednotationsfor specialmappings.The indicatorof a setA X every x E X there exists exactlyoney E
r
Y
\037
I
I I} \037
(y\302\273
(f(x\302\273
\037
\037
is)))
Appendix: Notation and TcnninoJogy lA
if
I
:xt-+
EA
X
{0 if
463)
,
xftA.)
linear combination of indicatorsis a simplefunction.The identity is idx: X X: x t-+ x. Considering a productX x Y we define the projections = The function pcI:X x Y X by pr1(x,y) x,(x,Y) E X X Y, and pr2 similarly. iscalled defined the signfunctionand is by sgn:R { 1,0,I} A
-. -. -. -
1 if x> 0, 0 if x=O,
sgn(x)=
-1
1 If f
Xx
E
X, Y E
X
Y
XE R,)
x < O.)
if
-.Z is a mappingthen f(.,
Y,
y)
denotesthe mappingx t-+f(x,y),
similarly,x E X. Iff,g are real valued f(x,.)hasto beunderstood
functionson X then fvg:x...... max {f(x),g(x)}, fl\\g:x......min {f(x),g(x)}, = XE X. We denoter=fvO,f (-f)uO,If
+j. =r as sequence set is mapping.!: subsets called sections of , ..., fen), }, I
N A, usually written (X\\"\037N") where x x n E N, are the x\" {xl> 2 n of the sequence to an infinite subsetof N iscalleda sequence. restriction The notationsx\" r and mean that the sequence subsequence. (x,,) is and increasing decreasing, respectively. A setiscountable if it admitsa bijective mappingto a subsetof N. x Let (A\\"EN") 2 be a sequence of sets.Then) A
=
in a
nE A
a
A
N. The
x,,!
\037
GO
GO
liminfA\" = U n AJ;, \"-'00 II-IIe-\ GO
limsupA\"= n....
n
GO
AI:'
U \"=1J;,-,,)
if liminf A\" = limsupAn =:A and A iscalled the limit The sequence converges if All if isincreasing (i.e. lim All)' Thesequence A1I+1'n E N. anddecreasing
.. ;2 \"
\037
GO
It is monotoneif it is increasing or decreasing. Every = monotonesequence converges.and we have lim A\" Cl.All if (A,JuN is and Jim All = U All if it is increasing. decreasing of /. denoted Let Xi 0,i E /, and denoteby A (J) the setof all finite subsets A\"
A\"+l' n E N.
ItE
\"IEN)
.. ::t=
n Xi onto n Xi' P.are denotedby We abbreviate =: E (/).A cylindersetin n Xi with basein E (J) is ier a setofthe form x Il Xi. X i.e. inverse n any imageofa setin n Xi under Pa.' If I is finite then setsof the fonn n Ah Ai X ;E I, are called from by a.p,. The projections
(X
Pl\302\253
P\302\253.
A
(X
P,\302\253.
A
\037
A
i\037l\\\302\253
\037
;\037\302\253
;\037/l
it
\037
l\037\302\253
iE\302\253
\037
rectangles.
A
i IE
r)))
it
464
Appendix: Nolalion and Tenninology)
Linear algebra.Suppose that H is a linearspaceor vector spaceover R. For A subset H there is a smallestsubspace L containing A, calledthe every linearhull or the linear spanof A and denotedby L =:spanA. A subsetA H islinearlyindependent if it isnotthe linear hull ofa propersubset of itsown.It a i f is basis it islinearlyindependent and itslinearhull isX. Givena linearspace H the cardinalityofevery basisisthe sameand iscalleddimension of H, dim II for short. If A. B H are subsetsthen A + B = {x+ y: XE A, YE B}.If LI' Lz c:II are subspaces such that LIn Lz = {OJ and LI + Lz = H, then /I is the direct L 1 c:H there are sum of LI and L 2 , i.e.H = LI (1)L 2 . For every subspace = complementarysubspaces Lz J{such that H LI G) L2 . Thesecomplemenare all of the same dimension which iscalled the codimension of tary subspaces \037
\037
\037
\037
LI' denotedby codimLI. A setof the form M = x + L, whereL H is a subspace, iscalleda linear manifoldin H. The dimension of M is dim M:=dim L, similarly codimM :=codimL. IfcodimM = 1,then M is a hyperplanein H. Supposethat H and If2 are vector spacesand f Ht -+ H2 is a linear mappingorlinearfunction.The kerneloffis the setkerJ:={xE HI:f(x) = O} and the rank off is rk (f) :=dimimf A bijective linearmapping f HI -+ H2 is an isomorphism. Vectorspaces are isomorphic if there exists an isomorphism betweenthem. Finitedimensional vector spaces are isomorphic iff they are of t-+ A translation the same dimension. isa mappingof the form x x + a, x E H. The sum of a linear mappingand a translation is an affine-linearmapping. If dim H < 00 then the setof all linear functionsfH -. isthe dualspaceof II and isdenotesby H*.If B = (bi)idis a basisof II let bt E 1/*be such that (bj) = bjj , i E /,j E /. Then B*= (bj*)jE1isa basisof II, calledthe dualbasis to B. Let Ht. H2 be two vector spacesand f HI -+ H2 a linear corresponding mapping.Thenf.:Hf -+ Ht:xf t-+ xf 0fis a linearmapping,too,and iscalled the dualor adjointoff vector space/l is naturally isomorphic Given a fixedbasisa k-dimensional iRk as the vector we consider with and sois its dual H..For distinction spaceofcolumnvectorswhereas(J\037k). denotesthe vectorspaceof row vectors. Within the matrix-calculus the *-opcration means transposition. Iff HI -+ Hz is a linearmappingbetweentwo finite dimensional vector spaceslet A be the matrix corresponding tof with respectto given basesB1 of /11and 92 ofH2' Thenthe transpose A* isthe matrix of the adjointf* with respect to the bases of nr and B! of H1.If A is a quadraticmatrix then we denotethe determinant of A by detA and the traceof A by tr (A). A seminorm on a vector spaceH is a function 11.11: H -+ [0,(0) satisfying E IR, x E H,and IIx+ yll H).xll= j).I'lIxl,). Ilxli+ yll,x,y E /I. It isa norm if it satisfies isa norm then (H,11'1\\)isa normedvector IIxll= 0 iff x = O. Ifll.1I space. -+ (H2'1I.lb) be a linear mappingbetween normedvector))) Letf(HI'11.11.) \037
I
\037
b\037
IRA:,
B\037
\037
II
Appendix: Notalion and Terminology
I
465)
spaces.Then is bounded 11/(x) \\b <00. sup_ iff)
11/1/1.2:= xf!ll
II X
-
11 I)
Let is the nonn of.r (with respectto 11.11and 11.112). The number Ilfll1.2
-H .
(HI,II.lit), B:HI X //2
XI E
1
(1/2 ,1I.lb) and (H3 ,1I.lb) be normed vector spacesand H3a bilinearmapping,i.e.B(x1, .)and B(.,X2) are linearfor all
HI' x2 E
2
ThenB is boundedif)
IIB(xI,X2)1I3 <00. XI eH..X2eH2 IIxllll 'lIx2 lb) and 11.113)' The number 1181112.3 is the norm of B (with respectto 11.111,11.112 Let H be a vector spaceand p:H --+ a seminorm. By the Hahn Banach L /I satisfying theorem any real-valued linear function on a subspace III p can beextendedto a linearfunction on the whole of H such that still IIBII 12.3= sup
\037
I
\037
\037
-
1/1\037p.
Let B:H x H !R be a bilinearfunction on a vector space.It is positive if R(x, x) 0, X E H, and it is symmetric if B(x,y) = B(y,x), semidcfinite x,yEll. It isan inner productif it issymmetricand positivedefinite,the latter are usually denoted by meaningthe B(x,x) > 0 jff x O. Innerproducts A vector space endowedwith an inner productis an inner product (II, An inner product space.It is a Euclidean spaceif it is of finite dimension. 2= definesa norm by IIxll (x,x), X E H. Let us return to an arbitrary vector spaceH. A linearcombination is a convexcombination if the scalars are nonnegativeand add to one.A subset H isa convexsetif it contains allconvexcombinations For A of itselements. convexsubsetcontaining A, which iscalled every setA /I there isa smallest the convex hull of A and denotedby coA. The convex hull of a finite setis a A cone(ofvertexzero)isa subset A H suchthat x E A convexpolyhedron. whenever ). > O. X EA. A setA H is centrally symmetric if x E A implies x EA. A convexconeis a linear spaceiff it iscentrallysymmetric. A function f H R is convex if f(ax+ (1 a)y) al(x)+ (1 a)f(y) whenevera E [0,1] and x,y E H. It is strictly convex if equality for some aE (0,1)impliesx = y. The function is concaveif is convex.It is its level-sets are convex sets for if a} quasiconvex every E IR. A centrally symmetricfunction is such that/(x)= I( x),X E fl. Finally, a functionf is if I(i.x)= AI(x), A 0, x E H.) positivelyhomogeneous \037
(.,
(.,.).
=t=
.\302\273
\037
\037
A.
\037
-
\037
I
-
{I
\037
I
-
\037
-I
-
IX
\037
(Bourbaki [1958]).Let X * 0 bean arbitrary set.A subsetff Topology
\037
2x is
a topologyjf it isclosedunder finite intersections and arbitrary unions.The is a of are the opensets,their space,the elements pair(X, .0/'\") topological the closed s ets. subset A X containsa greatestopenset))) Every complements
!7
\037
466
Appendix: Notation and Tenninology)
calledthe interior,and is contained in a smallcst closedsetA, calledthe closure. ThedifferenceoA = A\\.4is the boundaryof A. The topologys-is a Hausdorfftopologyif any two different pointsare contained in disjoint open = sets.A subsetA X is denseif if X. The topological space(X,!/)is if it contains a countable denseset.A subset isa baseof if separable the system of arbitrary unionsof setsin coincides with :T.The topological spaceis an A 2-spaceif there is a countablebaseof Every A 2-spaceis A pointx E X is an inner pointof A X if there is a setBE such separable. that x E B A. In thiscaseA iscalleda neighborhood of x. The systemof all of x is denotedby tfI (x). neighborhoods c Let (XII)IIE N X be a sequence. The sequence to a limitx E X (i.e. converges = -. lim x or XII x) if every neigborhood the complement of X contains of a 11-'x.. sectionof (XII)/IE The pointx E X isan accumulation if pointof the sequence of x containsinfinitely many pointsof the sequence. If every neigbourhood IR then the greatest accumulation (XII)III'N pointis lim supXII and the smaJ1est ......00) is lim.inf XII' Let (Xl' !/l)and (X2, !/2)be topological spaces.A functionf Xl -+ X2 is continuous at X E Xl if for every U E there is V E (x) such that if it It is continuous U. is continuous I(V) everywhere.If is bijective, t is continuous, continuous and and too, then I is a homeomorphism, A bijective and continuous (Xl''\037l) and (X2 , ff2) are topologically equivalent. if the domainiscompact and the range is mappingisalways a homeomorphism The real-valued Hausdorff. continuous functionson a topologically spaceare denotedby lI(X),the subsetof bounded functionsby \037b(X)' IfJE \037(X) then the supportoff is supp(j) = {f>+=O}. A real valued function on a topological space(X,:T)is lower semicontinuousif it isboundedfrom belowand itslevelsets{/\"&ex}, E IR, are closed. if f is lower semicontinuous. It is uppersemicontinuous Let (X\"/Yi ), i E I, be a family of topological spaces.Assumethat 1is finite. is a setn Ai 11Xi consisting of openfactorsAi E :Y\" An openrectangle ic.1 iEI iE I. The openrectangles are a baseof the producttopologyQS)5. Now, I let / be arbitrary. An opencylinder setwith basein E A (I) is a cylinder set A x n Xi' such that A E @ .'i\"i' The opencylindersetsare a baseof the prodA,
\037
\037
\037
!I
!I :r
!/.
fM
s
\037
QC
1\\1'
\037
II
(Ie)
'\302\245t
\037
'\302\245t
(J(x\302\273
r
I
-
I
\037
\302\243
f
i\302\243
<X
i(
uct
I CII
1\\\302\253
topologyIE@I fl:.
5) be a Hausdorffspace.A subset
is compact if every open a finite subfamilywhich stilIcoversA. Thisisthc casciff coveringofA contains A has the finite intersection property.It means that every system of closed have commonpointswith A satisfies subsetsof X whosefinite intersections is in A compactif every sequence n n A a=t: 0. A setA X sequentially if A A setA S X is relativelycompact contains convergent subsequence. Let (X,
A
\037
X
\037
\037
\037
is)))
Appendix: Notalion and TcrminoJogy
467)
function attainsthe compact.On a compactset a lower semicontinuous infimum of itsvalues. Let X isa locally compactspaceif every pointhasa compactneighbourhood. (x,,)c; X tendsto infinity if it (X, ff) be a locallycompactspace.A sequence of all functionswith The leaves s et. set continuous every compact finally function vanishesat compactsupportis denotedby
-
\037
\302\243
=
A), x EX. dist({x},
of openballsare the On a pseudomctric space(X, d) the finite intersections iff (X, d) isa metricspace. baseofa topologyffd'Thetopologyffd is Hausdorff It is an A 2-topology iff it is separable. On metric spacescompactness and
if it iscontained in a coincide. A setA c:X isbounded sequential compactness if it admitsfinite coverings ball,it istotally boundedor precompact by ballsof arbitrarily small radius. A Cauchysequence ona pseudometric such space(X, d) isa sequence that the complements in arbitrarily of sufficiently longsections are contained small balls.A subsetA X is complete if every Cauchy sequence in A in a limit A. If is to (X. d) a metricspacethen A X iscompactiff it converges is complete and totally bounded. Let (XI> dl ) and (X2'd2) be pseudometric spaces.A functionf XI -+ X2 is if forevery e > 0 there exists J > 0 such that dl (x, y) < J uniformly continuous d < f.. The set real valued of implies 2 (f(x), unifonnly continuous, functionson a pseudometric space(X, d) is denotedby
\037
\037
f(y\302\273
\037
\037
h(Y\302\273
E:
dj
If)))
Appendix: Notation and Terminology)
468
)'.')
and (X2 , (d2.J)J.J)are unifonnspacesthen f Xl X 2 is uni(Xl'(dl.i if f (Xl'dl,i) (X2 , d2 ,j) is uniformly continuous continuous for all formly i e I, j e J. The uniform spacesare uniformly equivalent if there exists f Xl -+ X2 which isbijectivexand such that/and!l areuniformly continuous. A filter is a subset y 2 such that 0 f.\037, 0, which isclosedunder and whereFe Y, F finite intersections implies rl e ,CP-.Forevery x e X the in a set of neighbourhoods Cjf(x) of x is a filter. Every filter is contained maximal one.which iscalledultrafilterand which is characterized by the fact that for every A X it contains eitherA or A'.A filter:F converges to a point x X if.\037 iff every ultrafilter (x).A subsetofa Hausdorff spaceiscompact An accumulation converges. pointof a filter!Jiisa pointx E X satisfyingx E F \037
\037
.\037
\037
\037
=t=
r\037
\037
\037
\342\202\254
'\302\245t
for every Fe\037. Let (I,\037) be a directedset.A famity which isindexedby I is a net.IfI = N then a net isa sequence. Forevery net there isa natural filterconsisting of the setswhich contain{Xi:i io},io E J. This is the sectionfilter. A net or a if the corresponding An accumulsectionfilterconverges. sequence converges ationpointof a net isdefinedsimilarly.) \037
Measureand integration(Bauer[1981] and Dunford-Schwartz [1967]).Let u Q 0 be a set.Then,r./ 2 is a field if it isclosedunder complementation finite intersection and finite union.The field isa a-fieldif it iseven closed under u countableunionand intersection. A set At 2 is a monotone classif it ..II. containsall limitsof monotone in I t is a sequences Dynkin system if it containsQ, isclosedunder complementation and under countable union of sets.Let '({s;; Then,concerning each of the structures pairwisedisjoint just a a onecontaining and denotedby defined,there exists smallest m (re) and d(f{j),respectively. A field isa a-field iff it isa monotone class.If f{j is a field then a(<6')= m(
\037
\037
2\302\260.
IX
C\342\202\254
(C\342\202\254),
(C\342\202\254),
\037
d
('.\302\253\037)
denotedby
\037
\037
1
d,
or t!4(IR).
Let Q '*'0 and .91be a field on Q. Then p:.9I IR is finitely additiveset function if p(0)= 0 and p(A v B) = p(A) + pCB) if An B = 0. The set of is denotedby ba(D,.r;1).An element bounded, finitely additiveset functions oc p E ba(Q,d) is countableadditiveor a signed measure if p( U A,,) = L Jl(A,,) for every pairwisedisjoint ,r./whose unionis (A\,,r") sequence \037
,,\0371
(Sj
n=1
in
Ail
\037
,d.The setof signedmeasures is denotedby ca(Q,.fJI).
A functionp:.r4/ Let ,r4/ bea-field.
- [0,00]isa measureifp(0)= 0 and if
it)))
Appendix: Notation and Tcnninology \037
469)
rs;
is countableadditive,i.e.\037(U ..-1 L \037(A..) for every pairwise disjoint sequence (A..)neN d. In this casewe admit \037
..\"\"1
\037
\037
E:\",:
\037
\037-
Let to a measure JlE .1(,,(0, = E ..11(0, if Jl2(A) 0 implies It 1 (i.e.Jll sI).Then dominates 111,112 = If Jll and (A) 0, A E Jll then Jll and Jl2 are saidto be ...... if there is i.e. are or singular(111.1112) orthogonal equivalent, I1t 112'They A E .rdsuch that 111 (A') = O. (A) = 0 and 112 Let d bea field.Thereisa natural orderingon ba(D.d).With this ordering ha(!l,.r:I)is a lattice,i.e.for a,T E ha(!l,.r:I)there exista leastupperbound UUTand a greatestlower bounda(lT.We denotea+ = auO,a- = (-a)uO. + t
extended in a unique
d.
\0371
Then
I
u
=
I
way
u(.s;(\302\273.
\037
\0372
\037
\0372
\0372
\0372)
\037
-
d) is the totalvariation and (J = u
+ (J E ba(Q,
U
U
-. We
denoteII all= la/(D),Two elements u,TE ba(Q,.vI) are orthogonalor singular = 1 . if O. In case we have 10'1 this + It = la + TI. (0' t) 1001(lltl A pair (0,d) consisting of 0 =t- 0 and a (J-field,s;( on 0, is a measurable An assertion space.Let 111 be a measure. concerning points(J) E 0 is valid if isnot valid iscontained in some the set where it p-almost everywhere(Il-a.e.) N E .rd with \037(N) = O. Let.K= {NE sI:\037(N) = O} and let sI0 sI bea sub1
d
\037
a-field.Then a(.r:lou.\302\245) is the Jl-completion of .r:loin .r:I. Let (D ,.r:l ) bemeasurable 1) and (Q2,.r:l spaces.A function/Dl\037D2 is 1 A d real-valuedfunction (sit,slz)-measurable iff (d2) l' f on a measurif it is(.r:I, ablespace t} = 0 for every t; > 0, and l1-a.e. if 11{ lim J\" =F f} = O. On (0,.rd)define g if 11{f=F g}) Z
1
\302\243
\037
-
I
\037
!/!
/-
!i'
\"-00)
-= O. Then we
I'll
denote!/!';(\(Q,") sI):If sI,\037)= {fE!e(Q,
I
\037
(X
and L\037 By IX>O}and Loo(Q,.r:I,I1):=!I:'(Q,.r:I,l1)j-. in E elements !R nonnegative (Qt 11)then) wand Loo. If/ II/11M= inf {(X 0:1/1 \037-a.e.}_)
!/! d, 5\302\243:'
some denotethe
\037-a.e.for
we
cO
\037
\037
IX
and a norm on Loo(D,.vI,11). This is a seminormof !R\037(Q, .91,11) -+ a is a nd Iff Ql Q2 mapping (D2t 2) isa measurable spacethen there isa
d
smallest O'-fieldd (f) on01 such that fis (d(f),d2)-measurable. Ifg isd (f)0 = measurable then there is a factori7.ationg qJ f such that qJ: Qz -+ iii is .r:l 2measurable. If ('2..d I) and (Q2'd2) are measurable spacesand I1dd1 is a)))
470
Appendix: Notation and lenninology)
measure,then
an
(,tit,.tl2)-measurable mappingf 0t
-.O =transformsthc 2
measure Jll into a measure 1l2 1.sc(2' (A 2) accordingto 112 III(f A 2 e .sc(2'ThemeasureJl2 isthe imageof JlI underfand denoted by JlI
I
0]1or, (Az\302\273,
if convenient, 2'(fllll). If (D,.sc() is a measurablespaceand JlI.tIis a measure, then (0,.91, Jl) is calleda measurespace.TheIl-integral ofan \037\"-measurable 0 functionf R is denotedby Jl(f)or JfdJl.It is well-definedfor all nonnegatived-measurable functions. A functionfe if Jl(r+)< co,Jl(r)<00.In !e(D,,91)is Jl-integrable thiscaseJt(f) = Jl(ft) Jl(fO).The Jt-integralhas the Levi-property, i.e.for t we have Il(supf,,) = sup11((,.). If every increasing sequence (f,,) c !R (D,.sc() liEN R is a convex function which is boundedfrom below then Jensen's ((J: Iti
-.
-
-.
IIE'N
inequalitysays S ((J ofdJl
(JfdJl),) whenever Jl(f) is well-defined. If 11 is a boundedmeasure then a if (h)iElS Y(D,.r;;/)is unifonnly Jl-integrable Jim
4 '00
\037
sup i,.1
((J
family
1J;ldJl= O.
r
1/.1>4)
!i'
Thesetof wintegrable functionsin (0,.tI)isdenotedby 2' .r;;/,Jl), and L 1 (D,.91, Jl)'= 1 (fl, Jl)/'--.Thereis a scminormon 2'1 (0,.sc(,Jl) defined
!I! d.
.(0,
by)
IIflll= JlfldJl,) which
is a norm on L1 (0,.sc(,Jl). If
Jl
-
is a boundedmeasure and
!l'
if
!R1 (0,..r;;/,Jl) converges tofin Jl-measurethen we have 1If\" fill 0 is uniformlywintegrable. The setof all functionsin (0,.sc()such isdenoted by !Rz (0,.sc(,Jl). If Jl is a boundedmeasure f2 !R1 (0,d, Jl) We defineL2 (0,sI,11);= !fJ2 (0,sI,11)/ . then !f2 (0,sI,Jl) S; !I:' 1 (0,sI,11). Let vl.9Ibe a O'-finite measuredominating JlI,9I.Then the Radon-Nikodym theorem says that there existsfe.!R(n, d)+ such that Jl = fv, where fv: A J fdv, A E.sc(. (f,,)\"EN iff ((,.)1If(,N that E \302\243;
-
\037
A)
The functionf is v-integrableiff Jl
d,
is
bounded.
measure space 11)is saidto be perfect if for any f e !R(0,.sc()and 1 (A) e ,r;;/ there are BorelsetsBl'B2 E 14t such that any setA s;;; IR such that B1 As H2 and Jl o 1 (H2 \\B.}= O. A
c
J r
(\037l,
and measure(Bauer[J98J]and Billingsley [1968]).Let (X, .9\")be a Topology of X, denotedby at(X).The space.Then (i(Y)is the Borel-(i-field topological
is the functionsare .tI-measurable a-field.tI such that all continuous smal1est In we have 9J Bairea-fieldand isdenoted s o (X) at(X). by alo(X). general, If)))
Appendix: Notation and Tenninology
471)
The Borel-O\"-fieldof isdenoted isa metricspacethen .1Jo (X) = .19(X). of If is a Hausdorff then all setsconsisting (X, ff) space 9.t(X) contains by fII\". singlepoints. Let (X, ff) be a Hausdorff space.A measureJlIIB(X)is a Borelmeasure if K < cI:: for every compact c X. A boundedmeasure is tight if for every /J (K) setK X such that Jl (X\\ K) < A family ofbounded c > 0 there a compact measures independ(J.lI)'Elis uniformlytight if the compactsetcan be chosen = if 1J{x} 0, xe X. isnon-atomic ently if ie J. A measureJ.l1\037(x) Theweak topology of\"'Ib(X, is Let (X, d) bea separable metricspace. for the coarsest topologysuch that Jl t-+ Jl(f),Jl E ..II,,(X,\037(X) iscontinuous Theweak topologyismetrizable. Prohorov's theoremsays that everyfE rt',,(X). ..It f or a subsetof ,,(X, is relativelycompact the weak topologyiff it is uniformly tight.The theorem of Daniell-Stone says that a linear,isotonic = function L:
\037A
1!\\
\037
\342\202\254.
,\037(X\302\273
\037(X\302\273
I.ij
!
!
I
.tM(X\302\273
if)
\302\243i
(X\302\273
!i
\037
\037x
!i)
X})))
472
Appendix: Nolalion and Tenninology)
definesa locallyconvextopologywhich coincides with the producttopology of H s;IIIX' and iscalledthe topologyof pointwiseconvergence. If H s; R X is a functionspacecontaining then the nonnPx = 11.11.\" the only boundedfunctions uniform norm,definesthe uniform topologyofH. Let 9\") bea topological H RX and a function s; are boundedon compacts. space spacewhoseclements Thenthe family of seminorms K X definesthe locallyconvex {PK: compact} topologyof compact convergence. If (X. 5\") is a Hausdorffspacethen (\037b(X)' II./I.,)is a Banach space.Let be a locallycompactspaceand endow \037(X) with the topologyof (X, compactconvergence. By the theorem of Arzela-Ascolia subsetof \037(X) is iff it isequicontinuous and pointwise bounded,If (X, d) isa relativelycompact theorem says that a subalcompactmetric spacethen the Stone-Weierstrass in which contains the constants and gebra rt'b(X)' scparatcs points,isdensein isa Banach compactspace.Then(\037o(X), 11.11.,) \037b(X)' Let (X. :Y) be a locally = in \037o(X) which separates points spaceand
\302\243
:1\"\")
=*=
\037
*.
-
d,
d,
= L<;Q(D, 11). a O'-finite measure spacethen (LI (0, A vector spaceH which is an orderedsetis an orderedvector spaceif the with the linear structure. Then subsetC = {xE H: orderingis compatible x O} is the positivecone.It isa propercone,i.e.it satisfies C,,(-C)= to}. Every properconvex coneC H definesa compatible semiordering by the x y if y - x E C.A vector latticeisan orderedvectorspacewhich is definition directedfrom aboveand from below.It is a Banach latticeif it is a Banach with the orderstructure. A subset A of a spaceand if the norm is compatible vectorlatticeis orderboundedif there isan upperand a lowerboundofA in H. H is ordercomplete jf every orderboundedsubsetA !';; H has a leastupper))) \037
\037
\037
11\302\273*
Appendix:Notation and Tcnninology
boundsupA and a greatestlower bound inf A
in
473)
H. Examples of order
are ba(!2, Banachlattice d) and ca(Q,.\037). A linearmappingbetween complete if x 0 impliesf(x) O. It isa vectorlatticestHI H2 isa positiveoperator of the linear and the lattice if it is an isomorphism latticeisomorphism structure. If (H, is an inner productspacethen there is a natural isomorphism in mind betweenHandH*,namelyx 1-+(x, X E H. With this identification the adjointf* of a boundedlinear mappingf /I /I may be viewed as \037
\037
(..
.),
.\302\273
\037
-
mappingIf intoH.
(.,
arc orthogonal Two elements x,y of an inner productspace(H, (i,e. x y) if (x,y) = O. A family (Xi)iE I H is orthogonal if its elements are if it set isorthogonal and itselements it isan orthononnal pairwiseorthogonal, are of nonnone.If L H is a complete there is a uniquely t hen subspace 1 L which is orthogonal to L, i.e,x y if determined complementarysubspace J = an orthogonal directsum x ELandYELl.Thenthe sum H L Ef) L iscalled of L. E H a unique and L1 is the orthogonal x admits Every complement asx = y + z whereY E L,Z E L!. We writeY = p,Jx)and callPL decomposition x PI'<X) 1-, the orthogonalprojection of H ontoL It is linear,satisfies 0 = : x E H, and IIx pdx)II inf {lix yll Y E L},Moreover, PL PL = PL and pt
.i
.\302\273
\037
\037
.i
-
-
-
1
=PL' and Gaenssler-Stute Probability theory (Bauer[19811 [1977]).In a probabilisticcontexta measurablespace(D,.r:I)is calledsamplespace,probability functionsare arc denotedby capitallettersP, Q... measurable measures lettersX, Y, If PId is calledrandomvariables, alsobeingdenoted by capital a probability measure then (D, .$I,P) iscalleda probability space.Provided that P(X) is well-definedit iscalledthe expectation or the mean of X. The The image :l'(XIP) ofP under X is varianceof X is V(X).=P([X p(X)]2). the distribution of X under P. The function EX: x t--+ P {X< x},x E is the distribution the distribution functionof X and detennines uniquely.Sometimes under P is with the function the distribution X identified of 9\"(XIP) in P-measure of random variablesis f\037 P(foX), fE I't'b(R). Convergence of the distributions caJIedstochastic is whereasweakconvergence convergence, in distribution calledconvergence of the underlying random variables.If % = (XI' ..., XII) is a vector of randomvariablesthen the image !R(%1P) is the of Xl' XII' jointdistribution TheGaussianor normal distribution with meana E R and variance(12 > 0 is
.,
...
-
\037,
....
definedby VQ.\0372:
B\037
1 J exp B 2nq Vh
[
- 21 . (t - a)2]dr, _
q)
2
Be r!41'
The particularcase VO. is the standardGaussiandistribution on R, its I
distribution function is denotedby No.:= tP
I
((X).)))
<1>,
for
(X
E
[0,1] the (X-quantileof tP is
474
Appcndix: Notation and Tcnninology)
A))...,
finite system of sets
A
in
A\"
.$I satisfies the multiplication rule if
P(i\"\"'l n peA,).A family of sets c d is independentif every finite n Ai) = i-I the multiplication rule.A family ofa-fieldsd, d, i E I, is sub-familysatisfies if collection A independent every (Aj)iE I with Ai E ..c1 i , i E I, is independent. ofrandom is if variables is family I independent. (X'}'jII independent (.!If (X'}}jjl Let (X')i\037N bean independent and identically distributed of (i.i.d.)sequence If < randomvariables. P(IXtl) 00,then by the law of largenumbers) \"
\"
\037
\302\243:;
-1 L \"
lim
\"-.\037,,
Let
Xi
= P(X1)
P-a.e.
i\037l)
be independentbut not necessarilyidenticallydistributed. If .sf (D,.!If,P),i E N, then the Lindeberg condition) 1 lim 2 L (Xi -P(Xj\302\2732dP=O, c>O, J ,,-'roS\" i' 1 IX.-P(X;)I!,;tS\
(Xi)iE' N E Xi 2
\"
\"
wheres;:=2: V(X ),
,=
j
I
lim !R
\037
(
\"\"'00
s\"
...,
the ccntral1imit theorem implies .)
i (Xi
j\037
P(XJ) P = \\'0.1'weakly.
)
1)
If Xl'X2 , X\" are independent random variablessuch that \037(X;/P)= V61.1, 1 i \", then ( i..1 xliP) is the non-centrall,;(iP)-distribution function isdenotedby X;.&z.In case(j = 0, the [)2:= (jf. The distribution i= 1 iscalledcentral.If Xl' X,,\\' Yh distribution random 1';,2are independent \037
\037
1: \"
variablessuch that
1 i \"2'then) \037
!/!1: II
...,i ..., , !/!OiIP)
= V,sI.l't !/!(XdP)
\037
\037
n1
and
=
VO,I'
\037
!/!( .1 i=l i xliI 1n2 n1
E lj2 P )
j-=l)
The distribution function isdenoted is the non-central f\037I.lIz(b2)-distribution. = iscalledcentral.Thedistribution by \037I'''];6]'In case[) 0 the distribution which is defined on (R, fM) by) /\037,...
(\302\243'
I:,,,(B)=r () v
J
a.\"t,,-l e-IJ'dt, BE1I1 ,
B,,(O.\302\253))
Its distribution function istalsodenoted iscalledthe r:.,,-distribution. by r:.\". c If .910 .91is a a-fieldand X 0 a randomvariable then a: A....... J X dP, It .910) and (1 PI.91 A E .91 o. Hence,by the Radon-Nikodyn))) 0, is in ..it(!2, \037
\037
theorem there
is
Appendix: Notation and Tcnninology
:;-
=1P(XIdo)E
475)
do) . If X is P-integrablethen
!\302\243(Q,
The random variable P(Xldo) is the P(Xldo)l=P(X+I.!;/o)P(X-I.\037o). of X conditional expectation given do.If A E d then peA 1.91 0),= P(1Ald o) is of A given ..r.I the conditional ofsub-uprobability o.Let (..r.I\\"E") bea sequence := or decreasing, fieldsof .91,increasing and .91 Then for every P.91,,). 0 q(lim integrablerandomvariable X the martingale theorems imply lim P(XI.9l,,) ,,-\037 fIi
P(Xldo) P-a.e. Let (Q\" .91,),1 $ i $ n, be sample spaces.The sets of the fonn A A2 The n, are calledmeasurable A\", Ai E ,r;;'!/i, t :$; i rectangles. measurable rectangles generatethe productu-field <8> sI,.If \037Idi'1 ; n, =
X
I
X
... X
\037
\"
\037
i'\"
\037
1
then there existsexactly one probabilitymeasure are probabilitymeasures This is Po @ sfI,such that Po (A t x x A,,) = J}(A,) for every rectangle.
...
\"
I
i:]
\"
the productmeasure
<8>\037. i\037
1
\"
n 1=] In caseof identicalcomponents (D,.!;/,P) we
denote<8> di = d\"and <8> I';= P\". i-I 1 A measurable Now. let (Qi'.9Ii ). i E I, be a family of samplespaces. cylinder setwith basein e A (I) isa cylindersetA x n ai' such that A E <8>di . The ie iell measurable sets t he u -field d . In case of (Qi,di) <8> i cylinder generate product ieI = (Q,.91),; e /, we denote<8>.9I = A family (\037)nA(I) of probability if'1 i measures @d\"CXE A (I),is projective if fJ implies =\037.If :l'(ppjlllj) i ((I a probabilitymeasure Q9.r.l satisfies =\037.<XE A (I), then it is i.jI j and iscalledthe projective limit of (\037)(IEA(1). In general, a uniquely determined limit neednot exist.In the particularcase,where ie /, are projective = and measures limit existsand is (8)J}, E A (/),the projective probability iEI calledthe productof (I';)iel' denotedby (8)\037. In case(ai'd\" = (D,d, P), iE I i E /, we denote(8) = p'.If the projectivesystem AU) is not of the i I:I abovethen additional particulartype considered properties imply topological the existence of the projective limit. Let (D...>It)and (D2 , d 2) be samplespaces.A kernel from (D],.91])to such that K(.,A2) is d](Q2,d2) is a function K: D]xd2\037[O,OO), measurable for every A 2 E .!;/2, and K(wI'.)E JI({22 , d 2) for every WI E QI.lf kernel,if K(w..D2) = 1, K(w l , (2) 1, WI E DI , then it is a substochastic kernel or a transition WI E Q..then it is a stochastic probability.A measure JlI.r.l 1 @ .r.l 2 admitc;a desintegration alongQ1if there isa kernel K from QI to \"
\"
i:
<X
1\\11
d'.
<X
\037I
\037I
\037
!\302\243(p\037I\037)
\037I.\037\"
\037
<X
\037)
\037
\037
Q2 such that)))
(\037)(l1,;
476
Appendix: Notation and Tenninology)
J4(AI X
A 2)
= J K(Wl' A 2)J4(dwl , !l2)' Al
E
.<41 ,
A
2E
.'01 2,
VI
@ V2
A.)
If VI
E
silt
V2
E
sl2, are a-finitemeasuresthen every Jl
\037
admitsa
which can be given in an explicitway. Otherwise,additional desintegration conditions of a desintegration. topological imply the existence Let (D,sI,P) bea probability A stochastic spaceand 0 .91a sub-q-field. kernel K from (D,.91 ofP given 0) to (0,d) isa regularconditional probability .910 if K(.,A) = peA do) P-a.e.,A E
\037
I
-
-
-
-
\037
rx
:X)
P sup {Is-rl
I
X\037
-, X
4c 2C h Y Iog2 } cIJ(P
-
Y
I
\037
\037
h tJ
- . YCI
y\037))
In particular, has continuous this implies that the process pathsP-a.e. Let (X,),eT be a stochastic processon P). The finite dimensional of the processare 9' P), E A (T). This is a marginal distributions on For there projectivesystem (lie,.\037T). any projective system on (IR\\!!IT) (\302\2431,.91,
\302\253X,),u I
(X
coincide with existsa stochastic whose finite dimensional distributions process the given projective limit. systemiff there existsa projective if A stochastic finite a its dimensional is Gaussian process process marginal distribution isdetermined are Gaussian. Hence,a Gaussianprocess by itsmean value structure I P(Xr), t E T, and its covariance structure (s,I) Cov(Xs X,), S, lET. If T = [0, and P(Xr) = 0, Cov(XS1X,) = min {s,t},then the stochastic isa Brownianmotion.Thedistribution process on CC(T)with probability of a Brownianmotionon aJT) isconcentrated = If T a nd one and iscalledWiener measure. [0,1] P(Xr) = 0, Cov(X$'X,) = min {s,I}-S.I then the stochastic is a Brownianbridge. process a sequence Consider of stochastic spaces processes (X\",r),cT on probability = \". PJ, n E N, where T [0,1]. Supposethat the finite dimensional (Q\",,91/ n E N, convergeweakly to the finite))) distributions of Tt marginal \037.
'
\037
<X\302\273
([R\037
(X\".,),\342\202\254
Appendix: Notation and Terminology
477)
further of a process dimensional (X')'ET, t E T. Suppose marginaldistributions a Y all those have there is metric s uch that that !(T, space(Y, d), processes an one.Thenwe say that (X\".')'ETtnE N, satisfies pathsin Ywith probability of (X\".')'ET, n E N, on (Y, d) invariance principleif even the distributions to the distribution and (Y, d) of(X,),.Ton(Y, d).If T = [0,1] converge weakly = (\037b(T), 11.11..) iff it is then the sequence satisfiesan invariance principle and uniformlyboundedin probability.) equicontinuous \037
!'/
Let G be a groupand a topologyon Topological groups(Nachbin [1965]), G.Then(G,:7)is a topological are compatible groupif the groupoperations A BorelmeasureAtE isa left Haarmeasureif with the topology. Jt(G,{\037(G\302\273
Jf(gx\302\273)'(dx)
= Jf(x\302\273),Adx), fE CCoo(G),gE G,)
and a BorelmeasureArE Jf(xg\302\273),r(dx)
Jt(G,
\037(G\302\273
is a right Haar measure
if)
= Jf(x\302\273),r(dx), fE fioo(G),gE G.)
Forthe followingassume that (G,.r)is locallycompactgroupwith countable baseforitstopology.Thenthere existsright and left lIaarmeasures and each is determined toscalar iff The Haar are f actors. measures bounded uniquely up
(G,.r)iscompact.
If A( isa left Haarmeasureon G then A 1-+At (A g. t), A E f!I(G),isa left Haar measure, too,and hencethere is a constant (g) > 0 such that) L1
).(Ag-1) =
L1
(g) AAA),
A
E \037(G).)
(0,(0)iscalled continuous homomorphism. The function 11: G
the modulusfunction of the group.It is a
If (H,<., is a Euclidean spacethen O(H)denotesthe orthogonal group and GL (Il) the general lineargroupon H. IdentifyingH with the groupof translations operatingon H, we may definethe semidirect productof GL(H) and H. It consists of the mappings .\302\273
(A, a):xt-t Ax
+ a,
XE
H, A E GL(H),aE H,)
and the composition is definedto be the natural composition, the considering
pairs(A, a) as mappingson H. If (G,ff) is a topological groupand Jl.v E .A(G. to) they can be composed
\302\243B(G\302\273
v:fl-+fff(g 0 h) Jl(dg)v (dh), fE \037oo(G),) calledthe convolution of Jl and Jl *
v.)))
areBorelmeasures
References)
The integralofa symmetric unimodalfunction overa symmetric Anderson,T.W. (1955): Proc.Amer.Math. Soc.6, 170.-176. convexset and someprobability inequalities. and statisticaldecision functions. Ann. Math. Bahadur, R.R. (1954): Sufficiency
Statistics25,423-462. Statisticsand subfields.Ann. Math. Stat.26,490-497. (1955): On (1957): unbiasedestimatesofuniformly minimum variance.Sankhya 18,211-224 On Fisher's bound for asymptotic variances.Ann. Math. Statistics35,1545-. (1964):
1552.
Bauer,H. (1981): Probability Theory and Elementsof MeasureTheory.New York: AcademicPress. SchwacheasymptotischeNormalit1itvon statistischenExperimenten Becker,C.(1982): bei unabhangigen, nicht notwendig identisch verteilten Beobachtungcn. Thesis. University ofBayreuth. Remarkon separablespacesof probability measures.Ann. Math. Berger,A. (1951):
Statistics22,119 .120. NormalApproximationand Asymptotic R.N. and R.R. Rao (1976): Bhattacharya, New York: John & Sons. Expansions. Wiley ofProbability Measures.New York: John Wiley & BilJingsley,P. (1968): Convergence
Sons.
ofcompleteclassesoftests ofsomemultipara. Characterizations Birnbaum,A. (1955): metrichypotheses,with applicationsto likelihoodratio tests. ADn. Math. Statistics 26,21-36.
D, (1947): and unbiasedsequentialestimation. Ann. Conditional Blackwell, expectalion Math. Stat. 18,105-110. Proe.2nd BerkeleySymp. Math. StalisticsProb., (1951): Comparisonofexperiments.
93-102.
ofexperiments. Ann. Math. Statistics24,265-272. (1953): Equivalent comparisons On statistical d ecision CR. minimax proceduresand their admissibility. Blyth, (1951): Ann. Math. Statistics22,22-42. Boll,C.H. (1955): Comparisonof Expt:rimentsin the Infinite Case.Ph.D. Thesis, StanfordUniversity.
of Bondar,I.V. and P. Milnes(1981): Amenability: A survey for statisticalapplications Z. Wahrschein1ichkeitsthcoric vcrw. Hunt-Steinand relatedconditionson groups. Geb.57, 103-128. ofthe one-parameter R. and J. Pfanzagl (1963): A characterization Borges, exponential of likelihood ratios.Z.Wahrscheinlichkeitsth. of distributions by monotonicity family
eorieverw. Gcb.2, 111-117. Elementsde mathCmatique.Livre III:Topologie Bourbaki,N. (1958): gencrale.Paris: Hennann. Consistentestimatesand zero-onesets. Breiman,L.,L.LeCamand L. Schwartz(1964): Ann. Math. Statistics35,157-161. Ann. Math. Statistics32, D.L.(1961): Burkholder, Sufficiencyin the undominatedcase. 1191-1200. A theoremon admissibletests and its applicationto an Chibisov, D.M. (1967): asymptotic problemoftcstinghypothesis.Theor.Prob.and Appl. 12,9().-103.)))
References
479)
Lectureson Analysis I-III.New York: Benjamin. Choquet,G.(1969):
Mathematical MethodsofStatistics.Princeton:PrincetonUniversity Cramer,H.(1946): Press. LokaleasymptotischeNormalitiitund asymptotischeLikelihoodDroste,W. (1985): Schatzer.Thesis.University ofCologne. Part I:GeneralTheory.New LinearOperators. Dunford,N. and J.T.Schwartz(1969): York: Interscience. ofBayesproceduresin estimationtheory. Weak limits ofsequences Farrell,R.H.(1967): Proc.5th BerkeleySymp. Math. StatisticsProb.1,83-111. New York - Berlin Multivariate Calculation. Useofthe ContinuousGroups. (1985): Heidelberg Tokyo: Springer. GeometricMeasureTheory. Berlin Heidelberg New York: Federer,H. (1969): Springer. An Introduction to Probability Theory and ItsApplicationII. New Feller,W. (1971):
-
-
-
-
York: John Wiley & Sons.
Mathematical Statistics.New York: AcademicPress. Ferguson,T.S.(1967): P. W. Wahrscheinlichkeitstheorie. Berlin- Heidelberg and Stute Gaenssler, (1977): New York: Springer. Invarianceand Minimax StatisticalTests.SclcctaStatisticaCanadia. Giri,N. (1976): Montreal:University of Montreal. A Coursein Nonparametric Statistics.San Francisco:Holden-Day. Hajek,J. (1969): A characterization of limiting distributionsof regular estimates.Z. Wahr(1970): scheinlichkeitstheorie verw. Geb.14.323-330. and inference. In:FoundationsofStatistical (1971): Limitingpropertiesoflikelihoods Inference.Ed.:V. P. Godambeand D.A. Sprott.Toronto,142-162. Localasymptotic minimaxand admissibilityin estimation.Proe.6th Berkeley (1972): Math. Stat.Prob.,Vol. 1,175-194. Symp. J . and Z. Sidak (1967): Press. Hajek, Theory ofRank Tests.New York: Academic R. P . and L.J. of the theorem to Halmos, Savage(J949): Application Radon-Nikodym the theory ofsufficient statistics. Ann. Math. Stat.20,225-241. Z. Heyer,H. (J969): Erschopftheitund InvarianzbeimVergleich von Experimenten. verw. Geb.12,21-55. Wahrscheinlichkeitstheorie Zum Erschopfthcitsbcgriff von D.Blackwell. Metrika 19,54-67. (1972): Berlin - Heidelberg- New York: Springer. (1982): TheoryofStatisticalExperiments. . . and R.Z. Has'minskii(1972):Asymptotic behaviour of statistical Ibragimov,1A I:Study ofthe likelihoodratio.Theor.Prob.Appl. 17, estimatorsin the smoothcase. 445-462. II: Limit (1973): Asymptotic behaviourofstatisticalestimatorsin the smoothcase. theoremfor the a posterioridensity and Bayes'estimators. Theor.Prob.Appl. 18,76-
91.
- New York: Springer. StatisticalEstimation.Berlin- Heidelberg (1981): W. and Ch. Stein Estimation with James, (1960): quadraticloss.Proc.4th Berkeley
Symp.,Vol. 2, 361-379. The Uvy-Hin\037n fonnula for infinitcly divisible experiments. In: Janssen,A. (1985): Berlin HeidelbergNew York: Springer. Infinitely DivisibleStatistical Experiments. Existenzund Konsistenzvon Maximum LikelihoodSchatzem. Landers,D. (1968):
-
Thesis,University ofCologne.
-
Sufficient and minimal sufficient a-fields.Z. Wahrscheinlichkeitstheorie verw. (1972): Geb.23,197-207. ofthe topologies Characterization usedin the theory Landers,D.and L. Rogge(1972): ofmaximumlikelihoodestimation.Z.Wahrscheinlichkeitstheorie verw. Geb.21,197. 200.)))
480
References)
On someasymptotic properties ofmaximumlikelihoodestimatesand LeCam,L.(1953): relatedBayes'estimates.Univ. Calif.Pub!.StatisticsI, 277330. An extension of Wald's theory of statisticaldecisionfunctions.Ann. Math. (1955): Statistics26,69-81. On the asymptotic theory of estimationand testing hypotheses.Proc.3,d (1956): BerkeleySymp. Math. Stat.Prob.I, 129-156. (1960): LocaUyasymptoticaUy normal families of distributions.Univ. Calif.Publ. Statistics3, 37-98. (1964): Sufficiencyand approximate sufficiency.Ann. Math. Statistics35,1419-1455 Theoneasymptotique de la dcdsion (1969): statistique.LesPressesderUniversitede Montreal.
(1970):On the assumptions used to prove asymptotic nonnality of maximum likelihoodestimates.Ann. Math. Statistics41,802-828. Limits of experiments. Proc.6th Berkeley Symp. Math. Stat. Prob.,Vol. 1, (1972): 245 261ofestimatesunder dimensionality restrictions.Ann. Statistics1, (1913): Convergence
38..53.
Noteson Asymptotic Methods in StatisticalDecisionTheory.I. Publ. du (1974): Cenlrede Recherches Universitede Montreal. Mathematiques. On a TheoremofJ. Hajek.Contributionsto Statistics- HajekMemorial (1979): Volume.Ed.:Dr.J. Jureckova,119-135. Dordrecht:D. Reidel. New York: John Wiley & Sons. Statistical Lehmann,E. (1958): Hypotheses. Testing similarregionsand unbiased Lehmann, E.L. and H. Schetre(1950): Completeness, estimation:Part 1.Sankhya 10,305-340. On the efficiency of a classof nonparametric estimates.Theor, Levit, B.Va. (1975): Prob.and Appl. 20,723-740. Invariante statistischeEntscheidungsprobleme. HabilitationsLuschgy, H. (1984): schrift.UniversitaetMuenster.
Michel,R. and J. Pfanzagl (1970); Asymptotic Normality. Metrika16,188-205. Globalasymptotic normality Statisticsand decisionsI, 401-425. Milbrodt,H.(1983): Limits ofTriangularArrays of Experiments. In: Milbrodt,H.and H.Strasser(1983): Berlin New York: Divisible Statislical Heidelberg Infinitely Springer. Experiments. Millar,P.W. (1979): Asymptotic minimaxtheoremsfor the sampledistributionfunction. verw. Geb.48.233252. Z. Wahrscheinlichkeitstheorie
Theminimaxprinciplein asymptotic statisticaltheory.Lecturenotesin math. (1983): - New York: Springer. Berlin- Heidelberg 7 5-265. 976, and M . W. On the Moussatat, Asymptotic Theory of StatisticalExpcriments (1976): SomeofItsApplications.Ph.D.thesis.Berkeley:University ofCalifornia. The Haar integraL Princeton.Van Nostrand-Reinhold. Nachbin,L. (1965): G. Zur Nolle, (1966): Theorieder bedingtenTests.Thesis.University of Munster. verw. ProduktmeBbarkeit von Funktionen. Z. Wahrscheinlichkeitstheorie (1971): Gcb.17,163-167. Statistik.Stuttgart: Nolle,G. and H. Witting (1970): Angewandte Mathematische Teubner. A Noteon Contiguity and Hellinger Distance. J.and W. R. van Zwet (1979): Oosterhotf, Contributionsto Statistics- HajekMemorialVolume.Ed.:Dr. J. Jureckova,157166.Dordrecht:D.Reidel. New York: Probability Measureson Metric Spaces. Parthasarathy, K.R. (1967): AcademicPress. Z. OberalltrennscharfeTestsund monotoneDichtequotienten. Pfanzagl,J. (1963): verw. Geb.I, 109-115. Wahrscheinlichkeitstheorie ofproductmeasurabledensities. On the existence Sankhya 31,13-18.))) (1969):
References
481)
the asymptotic efficiency of median unbiased estimates.Ann. Math. Statistics41,1500-1509. Medianunbiasedestimatesfor M.L.R.-familics. MctrikaIS,30-39. (1970b): On medianunbiased estimates. Metrika 1 54173. 18, (1971): a):Further resultson asymptotic normality 1,Mctrika18,174-198. (1972
On (1970a):
(1972 b): Further resultson asymptotic normality II, Metrika 19,89-97.
ofsufficiency by power functions.Mc:trika21,197-199. A charactc:rization (1974): m edianunbiased estimorsin the presenceofnuisanceparameters. On optimal (t 979): Ann. Statistics7, 187-193. for GeneralStatisticalModels. BerJin- Heidelberg (1985): Asymptotic Expansions New York: Springer.
J. and W. Wefelmeyer (1982):Contributions to a GeneralAsymptotic StatisticalTheory. Lecturenotes in statistics13,Berlin - Heidelberg New York: Springer. Sometheoremsofminimum varianceestimation.Sankhya 12,27-42. Rao,C.R. (1952): The relationsbetween minimal sufficient statisticsand minimal L. Rogge, (1972): sufficient a-fields.Z. Wahrscheinlichkeitstheorie verw. Geb.23,208-215. G.C. of Measures. Roussas, (1972): Contiguity Probability Cambridge University Press. R. A note on statistical Ann. Math. Stat.38,787-794. Sacksteder, (1967): equivalence. vectorspaces. New York: Springer. Schaefer,H.H.(1971): Topological (1974):Banach Latticesand Positive Operators.Berlin.Heidelberg_. New York: Springer. L. (1963): Someremarkson the power ofa mostpowerful test.Sankhyii Schmetterer, 25,207 .210. (1974):Introductionto MathematicalStatistics.Berlin- Heidelberg New York: Springer. Infor(1978): EinigeResultateaus der Theoneder erwartungstreuenSchiitzungen. Trans.71h Prague mation theory, statisticaldecisionfunctions,randomprocesses. Conf.,Vol. B,Prague1974,489-503. On Bayes procedures. vcrw. Geb.4, Z. Wahrscheinlichkeitsthcoric Schwartz,L. (1965): Pfan7.agl,
-
10-26.
Statisticalcxperimcnts and theirconicalmeasures. Z.WahrscheinlichSiebert.E. (1979): keitstheorie verw. Geb.46,247-258. The admissibility ofPitman'sestimatorfor a singlelocationparameter. Stein,C.(1959): Ann. Math. Statistics30,970-979. ofestimatcs.Part 1.JournalofMultivariate Analysis Strasser,H.(1981): Convergence
It, 127-151.
ofestimates.Part 2.Journalof Multivariate Analysis 11,152(1981): Convcrgence
172. Localasymptotic minimaxpropertiesofPitmanestimates.Z.Wahrscheinlich(1982): keitstheorie verw. Geb.60,223- 247. (1985): Einfiihrung in die lokale,asymptotischcThcoricderStatistik.Schriftcnreihc des MathematischcnInstitutsder UniversiHitBayreuth. University ofBayreuth. E.N. (1970): when the parameterspaceis finite. ofexperiments Comparison Torgersen. Z. Wahrscheinlichkeitstheorie verw. Geb.16,219249. Ann. Math. Stat.43,1383-1399. (1972): Comparisonof translationexperiments. StatisticalResearchReport (1974): Asymptotic BehaviourofPowe\037 ofDichotomies. No.6,Instituteof Mathematics,University ofOslo. Testsofstatisticalhypotheses concerningseveralparameterswhen the Wald, A. (1943): numberofobservationsis large.Trans.Amer.Math. Soc. 426482. An essentially complcte classof admissibledecisionfunctions.Ann. Math. (1947): \037,
Statistics18,549-555.)))
482
References)
Note on the consistencyof the maximumlikelihoodestimate.Ann. Math. (1949): Statistics 595-601. StatisticalDecisionFunctions.New York: John Wiley & Sons. (1950): O. Invariancetheory and a modified minimaxprinciple.Ann. Math. Wesler, (1959): Stat.30,1 20. Mathematische Statistik 1.Stuttgart:Teubner.))) Witting, H.(1985): 20\037
List of Symbols)
2,3 (\037\037,N)
d.(P.Q) 11.11
JF(Q,\037 d2 (P,Q) a(P,Q) N olr
R)
\037r(Q,
7t(P, Q)
(D, .111
oAf)
!T4 '1J4 \037I
\"'. \037. \037
(J Nil
(a,S) L(p) Nil
.11(1') .liT
M(p)
P. A.
ST Y'T H\302\253(I)
1(8) (II,K)
JF.(H,K)
';(H,K) .N1(P,Q) ..\302\245(P.Q)
\037E2 2
E. yE:
E.'2E:
62 (E.,
\302\243:)
.II)))
11 13 13 13
16
0-1-set
A2
5
6, 14,38 8 8 9 9
15 15 15 15
\"'..,
(E , 1
\302\2432)
j1 (QR) (P.J EloAf 0 \037
14 14
'1J.
EI
5
78
PI:
21 21,342 21 22 24 24 24 25 25 28 29 29 38 39 40
41 42 42 74, 260
74,98 74, 99 74
7S 78
C(!P)
T.E
L(E) M(E)
78
86 92 93 99 228 106, 230 106,
%
109 116 124,357,429 124,357,429 124, 358, 431 124, 358, 431 128 131
\037
135,359, 434 135
(1\302\243
P,.. HI
XI
H2 K2
P... Ii
.d.
140
E.
141 141 141
01<,.1.(11)
143
91$
.J;
(H,K)
X.
ba(U,d)
91o(D)
91(E,D) I\037p \302\243I(E,D) (J\302\245,),..
T
(T, D,W)
R(E,D, W)
Ro(E,D,W) .I(\037o(T) 'Pw(m)
a:(M) (M,),cG II Jv.1I
150 228 230 230 230 231 237 237 237 237 238 238.266 238 245 257
F E (D.1t'1 2F ,.. F (D.1t'1
257
E2F
260
E
\037
(D.It')
\302\243
257 257
List of Symbols
484
E2F
E-F 11'1'\302\27311
(IT.)UAtTI
H(E.I\037)
H(EJ
'1'.
91(E,F) RM Lat(\302\243)
T.
Jf'oCT)
Jt'(T)
(Mp,),.T
I(T)/\037(E,
F)
.1(E,F)
260 260 267 267 270 271 272 278 278 285 267 284 284 287 296 296 296)))
318 318
8:(w.) B\"(w)
XI.
!i'
foZ
!i'(t(N n \302\273
H3 K) 5/1 T,.o(\037)
(E) T,.So
Dro (9) F:M L2 (Q,.rJ,\\.)
-.
BI.
L.(h) rll(s)
345 342 349 351
359,434 359,434 365 385 385 385 388 389 409 448
Author Index)
Anderson 154. 194) Bahadur 91,103, 154,170,300 Bauer 468,470,473 Becker378 Berger98 Bernstein408. 426 420 Bhattacharya Billingsley 470
443 James 216,
Janssen377) Kakutani
34)
Landers14.34,91
31.68.9 I,104.154.226.227.228. 232,239,241,255.256,278,279,28 293.298, 300.301,308.311. 284, 285.291,
LeCam 16.
130,133,137 314.315.377.378.408,419.426. Blackwell 91.109, 110,161,226,255.256, 458 167 269.282 Lehmann 37, 115, 261.263,268. 131. Birnbaum
Blyth
155,217,221,224
8011155. 191,283 Bondar 251 Bourbaki 465
16 198 30, 194.
Brciman Brunn
Burkholder 91) Chibisov 130 Choquet 256, 288 Cramer 392)
Droste408
Dunford 15.228.251.468.471)
Farrcll
233 131.
Federer30
Feller 22 Ferguson59, 63 Fisher 208 Frascr 155)
Gsensslcr30, 473 Giri 131) 300, 378, 391, 393, 403.404, Hajek 48, 226.
408,418,423,439,444
Halmos91. 94. 95.96
Has'minslcii196. 453 301. 227, 282 Heyer 56, 103, Hunt
251)
Jbragimov
196, 301,453)
Levit 442 Luschgy 227)
Michel408 Milbrodt 377,408 Millar 340, 445
Milnes 251 Minkowski
v.
198 30, 194,
Mises408,426
Moussatal 37.130,340.361,373.378,41 Nachbin 417 Noelle 1,18,131) OosterholT377) Parthasarathy 187,437,446 154, Pfanzagll,19.52, 56, 91,92, 98.103.
164.300.377.408.439.442
Rao.C.R. 168 Rao, R. R. 420 Rogge 14,91
Roussas130 Sacksteder103
95.96 Savagc 91,94, Schacfer 34, 227.411 Scheffe167 Schmetterer37.47, 154 Schwartz J. 468,471 Schwartz L. 16,458 Sicbert256, 291 Sidak378, 403.404.408,418, 423)))
TIS.
486
Author
Index)
130,131, 132.226, 392,408, 456 5, 131 Witting 37, 11
Wald Stein 155,216,218,219,220,251,443
Strasser301, 377,458 Stute 30, 473)
Wefelmayer 377
Wolfowitz 426)
Torger8Cn91,tOO,155,186,255,256,274, v. Zwet 275, 276, 217,278,283)
377)))
SubjectIndex)
Cohcrent 230
Abstract L-space34, 288 Abstract Wiener space351 Accumulation point 308 Admissible, almost
- decision - power - test 41
function
function
Affinity
Almost
-
Completeclassof tcsts 41 classtheorCtn 131,132, 133.245
- experiment 116,167 order 228 -, --
216 161
4I
-
8
admissible216
critical function 141 39 147,251 Amenability Approximate sufficiency 281 invariant
Alternative
Approximation, Array,
setsof decisionfunctions 161 Conditional Hellingertransform 270 two-sidedtest 122 -. upper test 119 Conical measure 287, 290 Conical measure, resultant of a 281 Connectedpoints 140 Consistent system of standard measures289 Contiguous 86, 305 Continuous experiment 38 - experiment,weakly 172 in the limit 303 Convergencein distribution 308 - of experiments,weak 302 Convolution kernel 189,210 - thwrCtn 191, 211, 252, 283, 372, 444 Covariance,canonical 389 of a Gaussianprocess340 Cramer-Waldconditions 392 Critical function, almost invariant 141
cxponential419
Lindeberg378
318. Bayes -Asymptotic loca1ization427,437 - maximum likelihoodcstimates 318,324, 447 - minimum estimates3J8 estimates
331
-
normality 409 Asymptotically equivariant
443
unbiased cstimatcs438 - median optimal tests 430 - similar tcsts 429 422 - unbiased tests 429, 431,434) sufficient
318
estimate 182, -Bayes estimate, gcneralized 182 - estimates, asymptotic 3I 8, 33J - solution 242 - solution, extended244 Bernstein- v. Misestheorem 426 Better decisionfunction J 57 Binary experiment 38, 68 experiment, Mellin transform of a 83 Blackwellsufficiency 102 Borelmeasure 21)
-
Canonicalcovariance389
- derivativc 389 process344 -Central random variable 345 .- sequence4ft)
-
function,
invariant
. function: seetest
140
203 Crosssection 201. Curved exponentialexperiments398 Cylinder function 348 .-set 342 - set measure 342 set measure, distribution of a 342)
Decisionfunction 155,231 - function. admissible161
- function, better 157 f)ecision function, defined hy a linear pr\037 349 - function, distribution of a J 55 function, equivariant 247, 371 - function, generali7.ed23I
-
161
function,
minimax
function,
nonr8ndomized 155,234)))
488
Subject Index)
. function,
157
optimal non-randomi7.ed263 .-function, simple slriclly equivariant 247 unbiased 158 157 - functions, complete optimal 161 sets of - functions, weak topology of generalized231 . problem 56, 237 - problem, 246 - problem,standard 262 - problems,equivalent 264 - space 155 Lebesgue2 -Decomposition, of p-measures1 function,
function, function,
uniformly
t
invariant
Deficiency75, 296 Deficicnt74, 257, 260 Derivative,canonical 389 in measure 390 Desintegration,eSlimalionby 175 Determined by its distribution. uniquely Differentiablecurve 383 experiment 388 Dilalion 111 Distance,Hellinger8 variational 5 Distribulion, convergencein 308 fiducial 208 of a cylinder scl measurc 342 of a dccisionfunclion 155 on the sphere.uniform 2J
-
-, -,
-,posterior117 -,standard Gaussian342 determined ils 313 -, by
uniquely
Distribulional neighbourhood 313 Dominated experiment 38 Dual pair of operators105) 266 Envelope.lowcr 238. norm of a lower 267 power function 40
--,
- risk
157
function
Equivalenceof experiments74, 98.260 of tests 39 Equivalent decisionproblcms264 , stochastically312 Equivariant, asymptotically 443 decisionfunction 247, 371 decisionfunclion, striclly 247 estimate 190, 210 Error function 68
-
182 2JO) 190,
Estimatc, Bay\037 , equiv3riant
313
-,generalizedBayes 182 318
u, minimum
- of
variance, unbiased 166 331 Estimates,asymptotic Bayes318, maximum likelihood 318,324, asymptotic 447 , asymptotic minimum 318 asymptotically median unbiased438 Baycs 318 maximum likelihood 157,318. 320.446 , mean unbiased t 60, 166 162 median unbiased 160, minimal
uniformly
-, -, -, -, -,
-,optimal168
, unifonn
tightness of 326
- problcm 160desintegration 175 Estimator, median unbiased 160. 162.362 -.optimal mcdian unbiased 362 Estimation
by
.Pitman
215
-,
optimal mcdian unbiased 362 Exhaustive Experiment 38 uniformly
102
-. 38 167 ..,ccontinuous omplete 116, 38 -, Experiment,differentiable388 -,dominated 38 -,exponential 56, 115 binary
-, -,
full
shift
Gaus.\037ian
186
shirr J 24, 343
-,homogeneous38, 305 ., respectto an 175.182 -,integrable 172 -, 140,246, 294 integrability
with
invariant
easurable 172 .,mMellin of a
..-,
transform
binary
83
of orderp.integrable 183 reducedexponential 116 shift 143
-,
.., .,standard I 10 - type 74, 296 -,weakly
continuous 172 Experimcnls,binary 38, 68 curved exponential 398 . ,equivalence of 74, 1)8,260 monotone likelihoodratio 48, 52 projcclivelimit of 293 , projectivesystem of 292 strong topology of 298 uniformly integrable 324 ,weak convergenceof 302 weak lopology of
-,
.,
-, -, -,
-.
298)))
Subject Index Exponentialapproximation 419 E'l:poncnlialc'l:pcrimcnt56, 115 experiment,rcduced 116 experiments.curved 398 Extended Rayessolution 244)
-
489)
- dccisionfunclions, weak topology of 231 - Neyman-Pearson lemma 54 Group of operators246 , represenlation of a 251)
dislance8
Fiducial distribution 208 Finite ordcr 157 Fixed point property 251 Fourier lransform 21 Full shift e'l:perimenl
1!!6
Function, admissibledecision161 -,admissiblepower 41 , almost invariant
critica1141
-,better decision157 -,cylinder 348 -,decision155,231
defined by a linear process,decision349 distribulion of a decision155 envelopcpower 40 envelope risk 157 equivariant decision247, 371 error 68 generalizeddecision231 invariant critical 140 loss 156,237 --,minimax decision161 non-randomizeddecision155
-Hcllingcr transform 29, 270 - transform, conditional 270 Hereditary 286 Homogcncous cxperimcnt 30, 305 Hypothesis 39 linear 144)
-,
Indcpendent rdndom variablcs, linearly 396 Informative 74, 98, 257, 260 with respectto an experiment Integrability
175,182
172
-,
-Integrableexperimcnt experimentof orderp 184
-, -,
-
-, -, -, -, -, -,
Function, optimal
decision157
-,orderof a 157 -,power 39 -,risk 237 -,simple non-randomizeddecision263 -,strictly equivariant decision247 -,unbiased decision158 -,uniformly optimal decision157 -,critical: seetest
Functions, complete sets of decision161 non-randomizeddecision234 weak topology of generalil.eddecision231)
., -.
Gaussian distribution, standard 342 - measure 21 measure.standard 21 - process340 process,covarianccof a 340 process,slandard340 shift experiment 124,343 shift. standard 343 Generali1.ed Rayescstimatc 182
-
- decision
function
231)
.
experiments.uniformly 324 Integral, Wiener 354 Invariant critical funclion 140 critical function, almost 141 decisionproblem 246 246, 294 experimcnl 140, 140 map - map, maximal 140 tesling problem 140 Isomorphic,statistically Isotropicmeasure 21
109
- probability measure 143)
Kernel, convolution 189,210 .-,Markov 102 Kcrnelstochastic 102 uniform tightness of a 320)
-,
Laplacc transform 22
2 LebesgueDecomposition
compact 30, 157 -Level CHest,unbiased 34, 359
Likelihoodestimates.asymptotic ma'l:imum
318,324,447. - eSlimatcs, - processIIImaximum
157,318.320,446
ratio 2 ratio experiments,monotone 47, 48, 52 Limit of cxperiments,projective 293 Lindebcrg array 378 Linear hypothesis 143
- process
340)))
490
Subject Index)
- process,decision
defined by a 349
function
- testing problem 143
Linearlyindependent random variables396 Localization,asymplolic 427,437 Lossfunction 156,237 Lower envelope 238, 266 - envelope, nonn of a 267 S-tesl48
-
228) L-space106,
.
test
41
Nonn of a lower envelope 267 Nonnality, asymptotic 409 Nuisance parameter 116)
One-sidedtesting problem 50, 119,357,429 Operator,Markov 105
140
Map, invariant
Neyman-Pearsonlemma, generalized54 Neyman-Pearsontest 42 Non-randomi1.eddecisionfunction 155,234 decisionfunction, simple 263
140 -,maximal Markov kernel 102 - operator 105
-,stochastic 105,245, 278
invariant
Operators,dual pair of 105
-,group of 246 Maximal 140 157 map Optimal decision - decision Maximin tcst 136, 150 157 - cstimales168 Ma'l:imum likelihoodestimates 157,318, 320, - median unbiasedcstimator 362 - 446 likelihoodeslimates,asymptotic 318, 324.447 median unbiased estimator. - test 40, 358.359 Mean unbiased estimales160, 166 Measurablecxperiment 172 40, 358, 359 Optimaltcst, - tests,asymptoticaUy 430 Measure,Borel21 Orbit 140 -,conical 287, 290 Ordercompletc 228 -,cylinder set 342 156 -,derivative 390 -, of of a 157 a set 342 -, cylinder Gaussian 21 -, p, integrableexperimcnt of 184 struclurc of lcsls39) -,iSOlropic21 invariant
function
function,
uniformly
uniformly
363
unifonnly
in
finite
distribution
function
-,isotropicprobability 143 -,resultant of a conical 287 , standard 29,
267 109,
Pair of operators,dual 105 Parameter, nuisance 116
-,standard Gaussian21
-,
Median unbiasedestimates 160, 162,362 unbiased estimates, asymptotically 438 unbiased eSlimates.optimal 362 unbiased eSlimatcs,uniformly optimal 363 Mellin transfonn 24, 25 transform of a binary experimenl 83 Minimal variance unbiased estimate. uniform-
Pitman
Measures,consistent system of standard 289
-
ly
166
- theoremdeci$ion 239 Minimax
function
161
estimate 318 . estimates, asymptotic 318 Model,structure 203 Monotone likelihoodratio experimenls47, 48, 52 Minimum
M-space106,229) Neighbourhood, distribulional 313 Neyman-Pearsonlemma 42, 43)
16
1 structural Partition of the
unity 31 estimator 215 Point, accumulation 308 Poinls.connected 140 Posterior distribution 177 Power function 39
-
function,
admissible41
envelope40 Probability measure, isotropic143 central 344 Process, covarianceof a Gaussian340 , Gaussian340 likelihood 111 , linear 340 standard Gaussian341 Projectivelimit of experiments293 system of experiments292 P-measures,decompositionof 1,2 P-measures,separablesets of 17,18,98 P-mcasures,uniform slructurc on sets of function,
-,
.,
-,
14)))
Subject Indc'l: Random variable, central 345 variables, linearly independent 396 Randomization 106,278 criterion Randomi7.ations,weak topology of 279 Ratio, likelihood 2 Reduced exponentialexperiment t 16 Representationof a group 251 Resultant of a conical measure 287 237 Risk 156, function 237 function, envelope 157 set 237)
-
Tangent vector 385 Test 38
-,admissible41
-,conditional two-sided 122 -,conditional upper I 19
111.281,282
-,maximin 150 -,Neyman-Pearson42 non-randomized41 -,optimal 40, 358, 359 -,similar 41 -,unbiased40 u,
-
-,unbiased level-a-40, 359
-.
unifonnly optimal 40. 358.359 Tcslingproblcm 39 problem, invariant 140 problem, linear 143 problem, one-sidedSO,t 19,357,429 119,431 problem, two-sided 58,63, 430 Tests, asymptotically optimal , asymptotically similar 429 asymptotically unbiased 429,431.434 u, complete classof 41 equivalen\037 of 39
-
Separablesets of p-measurcs17,18,98 Scparating t 57 Shift experiment 143 experiment,full 186 experiment,Gaussian124,343 standard Gaussian343
-
-,
-,
Similar test 41 tests,asymptotically 429
-
Simple non-randomizeddecisionfunction Simplex,standard 28 Standard decisionproblem 262 . experiment 110 Gaussiandistribution 342 Gaussian measure21 Gaussianprocess341 Gaussianshift 343 measure 29, 109, 267 consislenl measures, systcm of 289 simplcx 28 Statistically isomorphic 109
-
Stochaslickernel 102 operator lOS,245, 278 Stoch\037lically
equivalenl 312
Slrictly equivariant
decisionfunction 247
Strong topology of experiments298 Slructure model 203 Structural paramcter 116 Subconvex 157 set 238 Sufficiency 93,97 appro'l:imate281
-,
263
-,maximin 136 -,ordcr slructurc of 39 of estimates.uniform -Tightness of kernels. 320
Topologyof experiments,strong 298 Topologyof experiments,weak 298 of generalizeddecisionfunctions, weak 231 of randomi7.ations, weak 279 conditional Hellinger270 Tran\037form, Fouricr 21 Hellinger29, 270 Laplace22 u, Mellin 24, 25 of a binary experimenl, Mellin 83 Transilion 105,278 Two-sidedS-tcst58 test, conditional 122 119, 358,431 testing problem 58,63, Type,cxperimcnt 74, 296)
-, -, -, -
Unbiaseddecisionfunction
-, -,Blackwell 102
-
Sufficient, asymplotically422
sequence147
S-test.lower 48
-.two-sided58 -.uppcr 48)
326
uniform
-
Summing
491)
-
-
158
estimate of unifonnly minimal variance 166 estimates, asymptotically median 438 166 estimates, mean 160, 162 estimates, median 160, e5limator. median 362 estimalor.optimal median 362 eslimator, unifonnly optimal median 363 levcl-cx-tcst40, 359)))
492
Subject Index)
- tcst 40
S-lest48 -Upper test, conditional 119
434 tests,asymptotically 429.431, Uniform distribution on the sphere21 structure on sets of p-measures14
-
Variational
Unifonn tightness of estimates326 tightness of kernels320 Unifonnly
function
delenninedby its distribution
convergenceof experiments302 topology of experiments298 topology of generalizeddecisionfunctions 231 topology of randomizations279 Weakly continuous experiment 172 Wiener integral 354 space,abstract Weak
324 - minimal integrableexperiments variance unbiased estimate 166 157 decision - optimal median unbiasedestimator 362 - optimal optimal test 40, 358, 359 Uniquely
distance 5
313)
-
..
351)))
