This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
PsoCP> PSz
= 1. Proof(1):If
o = 1
cP\302\253
ii>\302\253
-
p\037
_\302\253
Fromt 1.7we obtainthat) d Y H o Psipy dO' 9=&0)
iscontinuous on [0,(x].It followsthat there issomej E [0,(X] such that d
_
=
Ps
d8
9=90)
p,
Thisprovesthe assertion.
0)
:
for all (x, p E [0,1] there donotexisttests 13.2Remark. Let 90 E e.Ingeneral, = p. Hencethere neednot existtwocP E Y such that PsoCP= (x, dO' PalfJ 9
sidedS-testssatisfying IX
E
.. theseconditions. But for
d.
90
[0,1]there are two-sidedS-tests
Pc.= 90q>
IX,
d8P9q>)
cp* E
every
fF satisfying
80 E e and every
=0.)
8=80)
lemma choosing This followsfrom the preceding qJ =:
(X.)
13.3Lemma. Supposethat E isan exponential experimentofrank I. Let8 90
- 00< + k Sf ' (h ) a.e.
S-testsatisfying and letq>* be a two-sided k 1 e R, k2 e litsuch that
..-
cP
_
I if fa > k 1fao 2 90 {o if fa < k 1f90 + k 2 Sfao')
t1
\037
=+=
t2
<00.Then there are
v _
ProofWe have to show that there are kj E n, k 2 En, such that I if t > k'iebS + \"2SeIlS, _ (Ilv )_a.e. q> {0 if 1 < \"Ie\"s+ k'2 Se\"5, where b = .90 9 * O. the system of linearequations))) Case1:Assumethat '1< 2 , Consider
-
'
13.Two.sidedtesting for exponential experiments:Part 2
ebll + It ebll = I, It.eb/2 + k;12 ebl2 = 1 The system has a uniquesolution since) e'\" t 1 ebll ..12)(t 2 = el'111 D.= evIl) .. k't
65)
k\037
.)
-
I
12ebrz
t
I)
) > O.
The solution is)
- ( e - II') e ) I-D 1 .1t - -D(e e '2) 1
k'
t 2 bl2
bll
bl
2
lo
.)
we notethat sgn b sgnk\037. Then,there is no intervalwhere Moreover, g:t 1-+k'li\" + t eb, , I E R, \037
k\037
isconstant; in addition, the property) g'(t)(s t) 0 if g(l) g(s) that g isa quasi-concave function. Thisimplies that {g::::: of implies I}consists = at mosttwo elements and {g>1} (/ 1,/2) since{II'12} {g= 1}.Hence 1> k'ieb, + k\037tebl impliest < II or t> 12 , and 1
-
\037
\037
\037
The system has a uniquesolution since)
Ie .- be (:1 D'0 + t)e blo
CblO
blo
b10)
is) The solution
-1
(blo + l)eb'O, k'i ::::: D) k'z
=
1
be'\"\302\260.
D)))
= e2b'O(bto +1
-bl > o)
o.)
Chaptcr2:Elementary Theory of TestingHypotheses)
66
notethat sgn b * sgn and argue as in case1 : h'. t k'I e'\" + k' ((I' t E IR ,) isa quasi-concave Thusa function,and there isno intervalwhereh isconstant. localextremumof h isa strictglobalmaximum,and from h(to) = 1,h'(1o)= 0 and h\" (/ 0) t' 0 we get that 1> k 1 e + kiteblis equivalent to t * 10'This provesthe assertion. 0)
2'
We
k\037
\037
bl
13.4Theorem.Supposethat E is an exponential experimentof rank I. Let 90 e andcp .'F.If cp* E.'Fis a two-sided S-testsatisfying \342\202\254
\342\202\254
= Psocp. d p, * do sCP I}'
PSolp,)
-
= d
df:)
9\03790
P\037cP
9\03790')
then)
Pscp.2:PsCP if 9 * 90 , classof testsfor (H,K).) S-Iestsisa complete Thus, the setof alltwo-sided
'
S-testwith t l = -00.Letfo.=fso ProofCase1:Assumethat cp* isa two-sided and fl'= Sj\037. We observe,that lp* isan upperS-testand from Theorems(11.6) i t follows that is a N P-test the for (!0'!1) in senseof cp* generalized (10.4) The same is true of cp, and cp* and cp coincide Definition(10.2). (hv)-a,e,on {ft * t 2fo}.Hencecp is(hv)-equivalent to an upperS-testand from (9.9)(2) it followsthat the power functionsof cp and cp* coincide. is provedsimilarly. Case2:If t 2 + 00,then the assertion there Case3:Assumethat 00< t l t 2 <00.Let 9 * .90 , By Lemma(13.3) are k I E !R, k 2 E IR, suchthat
::
* cp
1 {o
=
if if
-
\037
fs>klfso+k2 Sfso. (hv)-a.e. fs < ktfSo+ k 2 Sfso'
we obtain From(10.3)
Pscp.
\037
PsCP if
9 =F 90 ,
0)
13.5CoroUary.Supposethat E isan exponential experimento.frankl.lfvalue
at a pointof e S-testscoincide andslopeof the powerfunctionsof two two-sided on S-testsare admissible then they coincide Moreover,two-sided for (H,K).)
e.
is an exponential experimentof rank I. Let = Assume that cp.E '\037a(H, K) {90},K {.9:[) .90}.
13.6Corollary.Supposethat (1.E
[0,1],80 E
\342\202\254o),
and H =
is a two-sided S-test.Then)))
E
=1=
13.Two-sidedlesling for exponcnlial cxperiments:Part 2 P\037tp*
= sup{Pscp: cpE .rp;a.(H, K)} if 9 =f 90
67)
,)
= 0 sincethe power K) satisfies !Fa.(H. PsoCP= (I.\037 Ps'P 3=80) has a localminimumat 90 , 0
ProofEvery
cp E
13.7Corollary.SupposethaI E is an exponential experiment01rank I. LeI (X
E
[0,1],90 E e, and H = {.90},K = {.9:9 .90}.If a lestcp E !Fa.(H,K) i.'i =4=
K) at a single optimalin S;a.(H, poill/.9 =t: .90 then it is uniformly optimalin \037a.(H, K).)
the assertion istrivial in case = 0 or = 1.Let ProofSinceE ishomogeneous E (0,1), Further let cp E !F,AH, K) and 9 E K be the pointwhere cp is optimal. a two-sided i .e. E be S-test, cp. :Fa.(H, K) I jf S < t 1 or S> t 2' v-a.e. . cp = {o If t 1 < S < t 2 . (X
(X
(X
.
-
'
(0,1),Lemma 9.7 implies 00< 'I :::::; 2 < + oc. lienee,Lemma 13.3 k l E 1R, k 2 E !R such that) givesconstants As
(X
E
* cp
- {0 _ I
This shows that
if
I> k1/80+ k 2 S/80
if f< kd8.0+ k 2 Sfso ' tp*
is a generalizedN P-testfor
(j\037o'
Sf90;
j\037).
From the
with tp* optimalityof cp it followsby Theorems10.4and 13.4that cp coincides v-a.e.on {Is k1fso + k2Sj\037}.Accordingto the proofof Lemma (13.3) the and coincide v-a.e. where t S t teststp S 1 and tp* 2' Thus,tp isa two-sided S-testand thereforeuniformlyoptimalin (H,K). :f=
=+=
=+=
0)))
!F\302\253
Chapter3: Binary Experiments)
The power and simplicityof the lemma of Neyman and Pearsongives riseto the development This theory is a of a general theory of binary experiments.
caseofthe generaldecision seemsto simple special theory,but itsisolation in severalrespects. beadvantageous Generaldecision o n relies theory powerful In contrast,the theory of binary experiinstruments of functionalanalysis. ments can be basedexclusivelyon the lemma of Neyman and Pearson.A betweenexperiments, the so-caHed decision-theoreticaHy meaningfuldistance deficiency,is easy to handlefor binary experiments. In Section 14we definethe errorfunction of a binary experiment. Section 15 containsthe discussion of the natural semi-ordering of the classof binary In particular. we show it can be expressed experiments. by the pointwise The topological structureis treatedsimilarly, orderingof errorfunctions. and convergence of binary experiments Finally,it is provedthat semiordering can alsobe expressed functions o f by power optimallevel-oc-tests. Sections 16and 17 arc concerned with equivalence classesof binary experiments and their characterization. It turns out that besides the errorfunction boththe distribution of the likelihood ratioand itsMellintransform determine A the equivalence classes iscompactness of the spaceof uniquely. consequence binary experiments. It is convenient to introduceat this point the conceptof contiguous of probabilitymeasures. theseare sequences of binary sequences Essentially, The whoseaccumulation experiments. experiments pointsare homogeneous essential in Section 18. facts are presented the of 3 is Theconcept ofcontiguity is whole well-known. Practically chapter very
dueto LeCam[1960].)
14.The error function) Consider a binary experimentE = (D,.9I, (P,
Q\302\273.)
14.1Definition.The errorfunctionof the binary experimentE is the function g:[0,1] [0,t] which is definedby g(oc)= inf [(1 oc)Pcp+ ocQ(1 cp)]t oc E COt 1].
-.
-
-
.F\037
00 we definefor reasons In the followingassertion ofconvenience: = 1.The t +00)))
14.The error function
69)
to Q and of Q with respect to Pare ofP with respect Lebesgue decompositions denotedby and M) (\037; . N).) (\037\037.
14.2Theorem.Let g be the errorfunction of E. if 0 :5k \037.
areequivalent: e:F then the followingassertions
(1)
\037.
(2)
g(IX) = (1
E
'\302\245J.;(P,
-
Q) for
\037
IX, 0
\037
IX
\037
- - 1.
= 1
k
-
<X
+ exQ(l
ex) p\037.
\037.).)
Proof (1) (2):If k = 0 then Q\037* = 1andtherefore \037
+
C 0) Pcp\" C 0) Q(I \037
-
= o.)
rp\")
\037
On the otherhand we have)
g
C ) \037o
=g(l)
Q(I-cp)
\037
O.)
\037
:\037\037
If k = 00then p\037. = 0 and therefore
C:'00)
Pcp\"+
C:00) Q(I-rp*)=
O.)
On the otherhand we have)
1 g 1+ 00 = g(O) = inf ( )
-.
P\037
= O.
tPE\037)
Let 0 < k ex
<00and =
1
l+k For any iF we have dQ dQ J dp dP+ M'r-.N' J (l-IX)-IXdP 1 [ IX
\037dP
(1-
+ ex! = (1
-
\037)dQ
IX)
The infimum isattainedfor
1 if \037.
\037E
=
0 if
+
(1- L ex)
P
(x
( 11;11 ).
- 11;11
-
seethat the preceding isnotdiflicultto assertion isalsovalid if W = 1 1ro.c), l'> O. The charm of the argument liesin the fact that it restsonly on the It
Neyman-Pearson lemma in the version of Lemma 28.1.The proof of the in its fulJ generalityrequires assertion someelementaryBayesiantechniques.
Thereforewe postpone it toSection 40.
resultsto a slightly moregeneral situation. Finally we extend the preceding = For tEL and <1 > 0 let Pr.u !f(<1id+ tl Nil)' In the followingwe consider the = teL.<1> O}). experimentF (H,fR(H). {.P,.u: For every <1 > 0 the experiment (H,fII(H),{P,.C1: teL})is of the same type as the experiment E if the vector spaceH is endowedwith the inner 1 1= <., It is natural to calJ an estimate() e .1t(F, product<.,.)\" \037
f\037
.). median-unbiased for F ismedian-unbiased for every R(e,t, (1)= II W(lx -f(t) I) e(',dx)dP'.C1'teL, i
\037)
(]
iff it
f\037,
<1
(1
>
> O. We denote) O.)))
166
Chapter 7: Theory of
Estimation)
34.6'Ibeorem.Every medianunbiased estimate (!off satisfie.'i for every tEL, (J > 0,)
-
[/(1) <,f(I)+ d]) P'..\037(., and)
R(q,t,
0\")
\037
:;; (,,:,, ) <1>
<1>
Clf\037
)'
< > 0,d> 0,
v O,1S1 (ds).) J W(lIfll'ls!)
Proof.Apply Lemma 34,2and Theorem34.3to F;,(J > O.
0)
34.7CoroUary.The estimate K. = foPL isa medianunbiased estimate off and estimates optimalamongallmedianunbiased off)
35.Mean unbiasedestimation) that e 0 is an arbitrary setand E = (fl,.flI,{Ps:.9 E e})an experiSuppose ment in S(8).Denote = {Ps: 8 E 8). For the purposeof thissectionthe of n !R2 (Q,d,Ps) are calledestimates. elements =*=
(fjJ
91'9)
35.1Definition.A
e
admitsan unbiasedestimateK if K = 9 E e. The set of all (mean)-unbiased estimates off is satisfies f(9) Ps(K), denotedby H(f).) function f
\037!R
An esti35.2Definition.Supposethat f e -+ R admitsunbiasedestimates. mate K* E H(f) iscalledan unbiased estimate ofWliformly minimalvariance
-
f(9\302\2732)
PS\302\253K.
= inf \"E
Ps\302\253K
if)
-
f(9\302\2732),
ll(f))
From Steiner'sformula it followsthat for K2 E HCn,and every 8 E
e
-
f(8\302\2732)
\037\302\253Kl
\037
PS\302\253K2
35.3Lemma. If K and K2 K = K2 Ps-a.e., 8E e.) 1
in
-
f(8\302\2732)
iff
9E
e.
estimatesK 1 E H(!),
any two
Ps(KD
\037
Ps(K\037).)
H(f) are of uniformly minimalvariance then
1
ProofSince (K1 + K2) E H(f) and) \037
J
K1
(
1 2 1 2 + K2 2 K dP 9E e,))) dPs KldPS+ J 2 J 2 s, 2 ) 2 \037
35.Mean unbiasedcslimation it
167)
followsthat this inequalityis even an equality and that
-
j(KI K2)2dPs= 0, ge
e.
0)
35.4l.emma.Supposethat .9I .91is E-sufficient.Forany K e H(f) let ge E(Kldl ) = Ps(Kldl)Ps.-a.e., Then E(KldI) e J/(j) and t
\302\243;
e.
2 ge e.) t )2) $ Ps(K ), Pa(E(KI.9I
of Jensen's inequality. ProofThis is an immediateconsequence
0)
if for every estimate K An experimentE iscalled 2-complete
e.
9E 0, ,9 E e impliesK = 0, Ps-a.e., Thisisa slightly weakercondition than completeness, definedin Theorem16.4.) Ps K =
and Lehmannand Scheffe[1950]). 35.5Theorem(Blackwell[1947]. Suppose
d)
is2-complete. that .r.() .r.(isE-sufficientand that EI If H(f) i.'t an unbia.'ted estimate 0/ uniformly minimalvarianceK*, and K* \037
K*
=
* 0 then there satisfies
Ps(Kldl ) Ps-a.e.,ge e,KE H(f).)
of H(f).Since Proof Let KI and K 2 be elements if gee, f(E(Klldl)-E(K2Id))dPs=0 of d l implies that E(Ktldl ) = E(K2 Idl ) Ps-a.e., ge completeness there is a function K* E H(f) satisfying
e.Hence
= E(KI..r.f 9E e, whenever KE H(f).) t) Ps-a.e., to the preceding lemma,K* is of uniformlyminimalvariance, According K*
0)
estimateK is an unbiasedestimateof In otherwords,every .9I1-measurable unifonnly minimalvarianceof itsmeanf: 91-+PS(K),9 E e.) Then 35.6Corollary.Supposethat the conditions are .';atisfied. of Theorem35.5 the unbiased estimateof minimalvarianceK* e H (f) satisfies)
s J W(K)dPs, 8 E J W(K*)dP \037
for every convexfunctionW: R
-
IR
H(f),) which is bounded frombelow.)
Proof The assertionfollows from inequality. 0)))
e,
K*
K
=
E
Ps(Kldt ) Ps-a.e.,and Jensen's
168
Chapler7: Theory of Eslimalion)
35.7Example.Let (H,( ., . bea Euclideanspace,L H a linearsubspace. \037
\302\273
sufficient mapping for the experiment E PL is a = (H,(JI(H),{NH*tQ:aEL}). Let us show that 9= {NIl*cQ:aEL} is 1 0 = = for the a-field.r41 pi Porthis,let g h PL besuch that 2-complete = for all L. This E that) 0 a \037g implies
Then
(\037(H\302\273.
o=
\037g
II h(x)e -re .-yIIxII 2
11/2
=
(21t)
=
C(a)f. hex)e<x,Q)e
dim
IIYII
2
e.
IIQIF
e<:r:.a)
2
dxdy
-\037 2 dx)
for every a E L. FromTheorem5.7we obtainthat) h+ (x)
e-u.;u:= h - (x) e-
l!!1I: 2
AI.-a.e.)
a E L. Henceevery function/0PL is an unbiased provesg = 0 \037.a.e., estimateof unifonnly minimal variance of its mean.If is linear then fo PL H(f).) which
/
\342\202\254
35.8Example.Keepingthe notationof the precedingexample,considerthe = id+ alNil)'It experimentF = (H,P.4(H),{\037.cJ:a E L, (1 > O})where x EN, is sufficientfor is again easy to show that S:x.-(pdx),!lxPI.(x)II), = a E (J > for and that is S-1 Therefore, F, L, O} 2.completc {\037.a:
-
\037
\037.cJ
.9\"\"'\302\253(1
(\302\243B(H\302\273.
estimates of unifonnly minimalvarianceare exactlythe estimates the unbiased K which dependon x E II through S. For short,Jet us callany unbiasedestimate of uniformlyminimalvariance Theorem35.5statesthat under the conditions imposedthere simply optimal. the setofall optimalestimates isa linearspace,which isgeneratedby a q-field. of Theorem35.5are necessary. It is our aim to show that the assumptions we information on the structure of the setof want to obtain some Moreover, in such caseswhen the assumptions of Theorem35.5are not optimalestimates satisfied.)
K* 35.9I.emma(Rao[1952]).An estimate optimaliff Ps(K*v) = 0for every i.\"
9 e e and every ve H(0).)
Proof An estimateK* is optimalifT for every 8 E e and every v E H(O) the function A.'+ ).V)2), e R, attainsits minimumat = O. This is the A.
\037\302\253K*
caseiff Ps(K*v) = O.
A.
0)
K such that there exists an optimalunbiased Let D be the setof all estimates H be a If K Diet 1t(K) version of the optimal estimateof 9 Ps(K), 9 \342\202\254
e.
estimate of 91--+Ps(K), .9 E e.)))
\342\202\254
35.Mean unbiased estimation
169)
35.10Lemma. ThesetD isa /inearspaceandn isa linearidempotentmapping. is a /inear space.too.) The.'letxeD)of alloptimalestimates in D and let i.., Ji E 'R. It followsfrom Lemma Proof Let \"1'\"2 be estimates that is an + optimal estimate.This proves the (35.9) ;,X(\"I) JlX(\"2)
assertion.
0) .rtf, n !{'2(!1,
In the followingwe topologize the spaceof estimates family
of semi-nonns K t-+ Ps(K2)1/2, 9E
e.
I\037)
by the
\037t:8)
35.11Lemma. The /inear spacex(D)isclosedand x: D -+ x(D);s uniformly
continuous.)
Proof FromLemma 35.9it isclearthat n(D)isclosed.Thus we need only Let \"I'K2 be arbitrary clements of D. provethat x is unifonnly continuous.
Then)
\037\302\253n(KI)
- n(K
2
\302\2732)
=
P9\302\25371(K
\037
P9\302\253K
which provesthe assertion.
1
1
-
K2\302\2732)
K2)2),
9 E e,)
0)
the class..w Now we introduce 0:={A E.9I:l A E n(D)}.It followsfrom Lemma
35.9that
A
E
.vI0 iff J vdPs =
0 whenever
vE
H(O), 9 E
e.
A)
35.12Lemma. (1).910 isa (I-field. (2) (3) (4)
e.\037timate is in 7l(D). Every.91 o-mea.\037urahle EI.9I0 2-complete. If .911 is E-sufficientthen .910 .91 for er;ery9 E e.) i.\037
\037
J
(P\037)
if A E .vI0 Proof (1) It is easy to seethat sf0 is a Dynkin system. Moreover, and a (I-field. .s.I0 ist1-stable then 1A V E H(0) if v E H(0).Therefore, usingLemma 35.9. (2) This is provedby induction, If P9 (K) = 0 for every 9 E e then estimate. (3) Let K be an .91 o-measurable K
E
H(O) and hence
J KdP9 = 0 whenever
A
E
.910 , 9E
e.
A)
e.
It followsthat\"= 0 P9-a.e.,9 E This meansthat EIsf0 is 2-complete. If A E.9I0 then 104 E n(D)and (4) Assumethat .911 isan E-sufficient(I-field. from Lemma 35.4we obtainthat alsoE(1A1.91 1) E xeD).lieneewe have I...= E(1AI.s.I 1)PS-a.e.,9E e.)))
170
Chapter 7: Theory of
Estimation)
Ifwe denoteB= {E(IA I.Iit.)= 1},then BE .Iit.and A = B
(J\037\037),
e.
.9E
0)
shows that the mapping1t is almosta conditional The followingassertion
expectation.)
35.13Theorem(Bahadur[1957]).Themappingn:D -+ n(D)has thefal/owing
properties: then Kh E D and 1t(Kh) (1) If KEDand if h is.91o-measurableand bounded. = 1t(K)h. (2)
If
KE D and
8E e.)
then n(K) = if n(K) is do-measurable,
P\037(KI.9Io)P\037-a.e.,
and bounded. Firstwe notethat) Proof(1) Let\"ED and h bedo-measurable s = 0, 8 E e.) J (Kh 1t(K)h)dPs= J (K
-
-
sinceK
- (,,) H(0).It follows 1t
E
g:9...... Ps(Kh). If v E
1t(K\302\273hdP
that n (Ie). h H(O) then hv E R(O) and
is a mean-unbiased estimate of therefore
0, 8 E e.) Thisimplies that n (K). h is optimal. s= J n(K)hvdP
(2) The propertyjustprovedimpliesthat for KEDand
A E
s = J n(K) l...dPs = J n(\" l...)dP s ;;;;; J \"dPs, 9 E J 1t(K)dP
A
.910 we have
e.
A)
it is a version of Hence,if n (K) is o-measurable, .QSf
\037
(K I.QSf 0)'
0)
The second assertion of Theorem35.13 leadsto the question, whethera given
is .91o-measurable.) optimalestimate
then it is 35.14Theorem(Bahadur[1957]).If an optimalestimateisbounded,
.r;Io-measurable.)
ThenKV E H(O)for every v E H(O). ProofLet K bea boundedoptimalestimate. we obtainthat K isoptimalfor all n E N. Ifp isa polynomial on By induction R then po\"is optimal. SinceK(D) iscompactthe Stone-Weierstrass theorem If U iii isopen impliesthat cp 0 \" isoptimalwhenevercp:!R -+ IR is continuous. 0 then it followsthat lu K is optimaland therefore{K E U}E .Iito. HenceK is .91o-measurable. 0) ft
\037
converseof Theorem35.5.) Now we arrive at the desired
35.15Theorem (Bahadur[1957]).Assume that E is dominated.If for every)))
35.Mean unbiasedcstimalion function which
admit.v
171)
a bountkdunbiasedestimatethere (\037xivt.v an optimal
estimate then ..cI 0 is E-sufficient.)
Proof We prove that .910 is pairwise sufficientfor E. The rest followsfrom Theorem (20.4).Let PI and Pz be arbitrary elements of r!J= {Ps:9 e 8}. such that Q:= L 2-1 Completethe pair(PI'P2) to a sequence (P,J\"\"N d is equivalentwith Then the densitiesfi , i = 1,2,can be assumedto be dQ bounded. \037
\037
\037
l\342\202\254N
\037.
We have
i=
Let h,= n , j = I, 2, are d.-measurable.
to show that \037ci
1,2.Then Ps(hiv) = 0
if
8 E e,
ve H(O).On the otherhand we have v E H(0).This implies \037
Q((h.-
\037ci
But hi
0,
VE
\037ci
).
\037(v)
= 0 and hence Q
v
dR
( dQ)
= 0,
H(O).
- :ciisitself
an elementof H(0) which implies)
Q((h,-
= O.
\037ci
''IT vye
)V)
(
ve H(O),and thereforeQ(hjv) = 0,
. 0btam
t h at
),)
. dfi.IS optima and I
measurable. 0) dQ
.
m view 0f ItS b ound ed ness sI0-
35.16Corollary.Assume that E is dominated. Iffor allA E sI the functions
9.-.P,,(A) admit optimalestimatesthen do isE-suffcient.)
to 35.15 i s estimate off Since\" bounded - -0 ProofLetf e By
R
Theorem
such that /lK.. co
K/lu
-
L /I\"..
,,=
and such that there is a boundedunbiased K off estimate we have show that there existsan optimalunbiased we may find a sequence of stepfunctions(K,J and (eventually by choosing a subsequence))
\"\"+l/1u
<00.
1)
By
for every Kn there is an optimalestimateK: of [) assumption,
9 E 8,n E N. It followsthat for every 9 E e /I
-
-
Ie: \":+1112,9 /I \".. \037
\037
and thus) t;I)
L 11=1)))
-\"..
\"\"+1/12,.9II \"..
.9Ee. /lK:-\":.llb,9<00,
+
ll1u,
n
EN,
\037
Ps (\"..),
172
Chapter 7: Theory of
Let Po E
Estimation)
C(9)such that 96 --Po.Then,obviously,) 00
L IIK\037-K\037+llb.Po
and it followsthat (,,\037) has a Po-a.e. limit \"*.Now, it
,,*is an optimalunbiasedestimate off
isnotdifficult toseethat
0)
36.Estimationby desintegration)
that (1',d) is a metric spacewhich is locallycompact and separable, Suppose the we a Let &f(T) be its BorelO'-field. For following, choose Borelmeasure < if K Tiscompact, and keepit fixed.Let us t hat means 00 p(K) plfj(T), = an experimentE (a,d, {p,: t E T})in 4 (1'). consider TheexperimentE iscalledf1I(T)-mea.\037urab/eif \037(A) is.1J(T)-measurab for every A E d. If E is .\037(T)-measurable then we may define the measure pep):A...... J P, (A) p(dt),A E ,rd.TheexperimentEiscalledweakly continuousif I H P'(A) iscontinuous for every A E d.Sincein this caseE is separable, the v with set9 = {p,:1E T} isdominatedand for every O'-finitemeasurev 1.91 the function (w, t) H (w), (w, I) E a x 1', can be chosens1x \037(T)\037
t.-
\037
\037
\037;
measurable (Lemma4.6).) Then E is p-inlegrable if for 36.1Definition.Supposethat E is measurable. function qJ E (Q,.91)such that every SETand every f; > 0 there is a critical PsCP< f,
-
and J P,(1 T)
!/
cp) p(tlt)<00.
if every testingproblem({s}, In otherwords,E is p-integrable sET, 1'\\{s}), admitsa testof arbitrary smalllevelwhosepower functiondiffersfrom oneby for every hounded a p-integrable function.It is clearthat E is p-integrahle
measureplat(T).)
definition 36.2Remark. ThecritiC<11functionof the preceding may bereplaced
criticalregion.Indeed.choosea criticalfunction cp such that < f.2 and P'(1 p(ti/)<00. ThenA = {q> > f,} satisfies and J P,(A')p(dt)<00. \037(A) < f, T by a
\037q>
-q\302\273
\037
to checkp-integrability criterion Thefollowingassertion isthe mostconvenient of
E.)))
36.ESlimation
36.3Theorem.A isa-jinile.)
by
desintc:gration
weakly continuousexperimentE isJ.l-integrable iff
t 73)
P(J.l)I,r# is
and let seT. From 36.2it followsthat Proof Assumethat E is J.l-integrable .s.Isuch that \037(AII) 11 and P(J.l)(An) < ex; for there is a sequence (AII)lIc N ,....,1,such that P(J.l) sf\" n is n N. Hence there a set isCTDJ E .YI, every E finite.Let S T be a countable and densesubset. Then D = U Ds issuch that seS SinceE is weakly continuous it \037(D) = t jf S E SandP(Jl)I.r#n D is a-finite. followsthat P'(D)= 1 for every t E T. HenceP(J.l)(D')= 0 which provesthe \302\243
I
\037(\037)
\037
\302\243
assertion.
Assume converselythat PC/L)I.ctis u-finite.Then there existsa sequence such that A\" T Q and P{Jl)(All) < 00, n EN. This provesthe (AII)\"\037N \302\243;.91
assertion.
0)
36.4Corollary.A
weakly continuousexperimentE isJ.l-integrable iff any
isfulfilled: followingconditions dp' (1)J dP. J.l(dt)<00\037-a.e.for every SE T. J
of the
dP.
f (2) Thereis a dominatingmeasurevl.ttsuch that J dv d\037
(3) J
tl(dt)< 00 v-a.e.
. .
J.l(dt)< 00v-a.e. .forevery dommatmKmeasurevl.r#.)
d:
Then assertions (3) and (2) follow Proof Assume that E is J.l-intcgrablc. immediatelyfrom) J
dp'
d; (d/) = J.l
-
dP(tl) dv
v-a.e.)
sinceP(J.l)isCT-finite.Similarlyit isshown that (1)isvalid.Nowassumethat (2) d issatisfied. To provethat E is J.l-integrablelet C,,:={J P'.J.l(dt) n},n E N, dv n v < n N. Then A :=CIIn DII' and let D,.i fl, DII E .ct, E N, such that (Dn) 00, E < 00,n E N, and v(Q\\ U A,,) = O. HenceP(J.l)is CTn E N, satisfy P(J.l)(A,,) \037
II
liE '1/
finite. be a probabilitymeasure which is that (1)holds.Let \\' 1.91 Finally, assume -k t E T}and isof the form v = L 2 Ps\" where(SjJkfN isdense equivalentto {p,: te in T.Then we have
-
N)
-
dp, k dp' J dv J.l(dJ) 2 J - J.l(dt)< 00 \037
dPs\
kEN.))) \037Ic-a.e.,
174
Chapter7: Theory of Estimation)
It followsthat N = {J p(dt)= oo}satisfies PSIc(N)= 0 for every kEN and \037; hencev(N) = O. 0) in subsequent sections let Although much moregeneralcaseswill bediscussed us illustrate theseconcepts some by typicalsimple examples.)
.
36.5Examples.Suppose that (H.< . . is a Euclidean space,L H a linear \302\243;
\302\273
and n = dim H.k = dim L.The Lebesgue measureofH isdenoted subspace by A.
H
.
.
(1) Let P )'H be a probabilitymeasureand denoteh = d; . Consider dAB the shift E = (II,f!J(H),{\037: a E L}),where = P.Choosep = L. Then) \037
\037
\302\243\"
-
dP(A.J(x) = dA. 1I I hex a)A.dda) = hex pdx)+ a\302\273).dda), I
-
A.
XE H.)
Since)
J f hey LJ.i) it
+ a) A.dda) L1(dy) = P(R)= A.
1
followsthat dP()'L)/dA. H < 00 )'H-a.e. This provesthat E is L-integrable. In the specialcasewhere L = H we A.
obtaineven P().II)= ).11' For laterpurposeswe notethat a t-+\037. a E L. is continuous for the variational distance in any case.This is a consequence of Remark 3.1.
Now.assumethat dimL < dim H and considerthe shift with scale a E L.a > O}). where P\".rJ = !R(aid+ alP). parameter F = (H.PA(H).{Pa.o: 1 a E L. a> O. Choose @ da.Then p(da,da) = ).,.(da) (J) (2)
-
\037\037(Jl)
(x) =
dAH
f
R+
JL -i-rr a
h
X
(
= J f }+1h x R+La
(
-a a
-
) Adda)da
PL(X) + a a)
dP(p) = 1__ (x) IIx - pdx)II\"-k
Now, we put y =
\037,
ex
=
\037
a)
(J
IIx PL(X)II.which leadsto
dA.H
k
-
f f '!II\"'cx\"
1
X
h(cx
) )'L(da)du, x Ell.
- pdx) +Y)AL(dy)dcx,xeH.)))
llx_pdx)1I
36.ESlimation
by
175)
dcsintegration
on S. We want Let S = {zE H:IIzll= I} and let t be the uniform distribution we that < for this show and to show that dP(J.l)/dA.H 00 lH-a.e.
l j.i
ex
n
- -1h(exz + y) k
A.
L (dy) dexr:(dz)
= 1.)
is almostimmediatesince(ex, z, y) 1--+ n k h(exz+ y) is a version of It followsfrom Remark (3.1) @ t @ AL)' HenceF is J.l-integrable. dP/d(J.'R' that for shifts with scaleparameter the mapping (a,0-)1--+\037.(1' (a,0-)E L x +, is for the variationaldistance.) continuous But this
I
ex
\037
36.6Definition.A measurablefunction f T --. is (E,J.l)-integrab/eif E is (If J.l)-integrable. IR
I
I
then every boundedfunction is (E,J.l)-integrable.) If E is J.l-integrable.
36.7Remark. Let f T IR be a measurablefunction.Theorem 36.3and are equivalent: Corollary36.4imply that the followingassertions functionf T --.IR is (E,jl)-integrablc. (1)The measurable \037
isa-finite. A E.>1, A 1--+J P'(A)I/(I)IJ.l(dt), (2) The measure P(I/IJ.l): (3) S I/(t)I
dp' J.l(dt)< 00\037-a.e.for every sET. dP. s dp'
measurevld suchthat J 11(t) J.l(dt)< 00 v-a.c. (4) Thereisa dominating dv 1
(5) J 1/(t)I \037; J.l(dt)<:x>v-a.e.for every
and f T If E isJ.l-integrable
.- dP(J.l)) dP(J.l))
then the RN-derivative) is (E,p)-integrable
----
\037
dP(IJ.l) dP(f+J.l)
measure vi d.) dominating
\037
dPCIJ.l) dP(J.l))
ofIcouldbe iswell defined.Theideaof using this RN-derivativeasan estimate to the measure with respect calledestimation by desinteRration J.l1\037.)
36.8Examples.(1) Considerthe shift of Example 36.5(1).Let p> 0 and that P(II.I/')< co.Then) suppose
-
-
AL) dP(II.I/' (x) =
+ Ilia pdx)I/'h(x p,,(x) dAH 2'lUx pdx)+ all'hex PL(X)+ a) A.dda) + 2'I/xll'J hex pdx) + a)A.dda),x E
\037
-
-
a\302\273).,,(da)
-
H.)))
Chapter 7:Theory or Estimalion)
176
Thesecond term isfinite AII-a.e. (1).Thefirst term isalsofinite by Example36.5 smce)
J L\037
f
II
< 00, + all\"h(y + a) Adda)).v. (dy) = P(II.II\")
y
I.)
Hence11.11\" is (E.AL)-integrablc.
(2) Assumethat dimL < dim 1/.For the shift
is valid.Let p > 0 and J IIx
\037\037r;:.\037nasscrtion
\037P
F with scaleparameterthe IIxll' P(dx)< 00, Then we _ PL(X)II\"
Q!.!I'Jl) (x) dlH X 1 pdx) + a . S J n+l-lix-pdx)+all\"h IR-L(J ( (1 ) A.L(da)d(J
-
\0372\"
x t 2\" R.Lu\" J J \037 IIxll'h (
_
PL(X) + a (T
) lL(da)d(J
- +yl/\" J San-k-1-\"lItXPL(X)I! IIx - pdx)lIn-k 2'
x .PL(x)
A+
-
\\Ix
-- -
lx pdx)
II
2'I\\xl\\'
+. 11
XE
L
x PL(X) . + Y)A.ddy)d(X _
h\302\253(Xi
We
-
/i\"=k
PL (X ) 11
X
J R\"
I
(X
n-k-11 I(
X
(X
II
X
- pdx) + ) . (d ) d - ) PL ( X
y
A.
L
Y
(X,
II
II.)
know from Example36.5(2) that the secondterm is finite i.H-a,e. For the
first term thisisalsovalid since) S f J (Xn SR+ L)
=
It-l--,
\\ltXZ
+ yll'It(tXZ + y)lddy)d(X!(dz)
xII' P(dx)<00. J--x--------pdx) II
1\\')
\\I
Hence11.11' is (F,Jl)-integrable. (If we only assumethat POI.II')< 00 then followsthat (a.a)
..... aII)'.(a.a) II
E
(3)
If we specialize P=
We have))) (E..A.,,)-integrable.
N H in
is (F.I')-integrable.) every linearfunctionf L -t iii is
L x R+
(\037
(1) then
it
36.Estimation \037P(fAL)
dP(A,J
dP(fl.J
=
-
dA H
-
--
desintcgralion
117)
dP{A.J
/
dAll
- llix-a1l )
ff(a) ,. exp(
by
1
2
--.
AL(da)
J
J,.)exp( 2 11x aW) l,.(da)
-
and from IIx all2 = IIx
- p,Jx)
11
2
-
+ IIpdx) all2 it followsthat
-
dP(fA.,) f o PL') dP(A,J If we specialize P = Nn in (2) then it is not difficult to seethat the conditionis satisficdiff n k > p. If k > 1 then every linearfunction and we obtainin a similar f L IR if (F,p)-integrable way as for (3)
-
(4)
-.
,,-
dP(fp)
dP(P)=foPL')
The basicideaisas follows. The distributions. Now, let us introduce posterior
!
mixture)
A
x B\037 P'(A) p(dt),
A
E.9I,Be &I(T),
x isdecomposed along(0,sI) by a kernel F:!1
\302\243i(T)
JB P'(A) }l(dt)= J F(w, B) P(p)(dw), A
- [0,1]
such that
E.\",BE(}I(T).
A)
Any kernel with this
distribution.) propertywi1l be calledposterior
-
A 36.9Definition.Supposethat E is weakly continuous and jl-intcgrablc. x lI(T) [0,1]iscalleda posterior stochastic kernel F:!1 distribution (with
respectto p)
if)
. dP(1BP) F(B):=F(.,B) E dP(p) , BEfJ (T).) 71lenthere that E isweakly continuousand JJ.-inteKrable. 36.10Lemma. Suppose
existposterior distributiolls.)
Proof Let v d be a dominatingmeasure and let Q114be an arbitrary measure. Let) probability I
N=
{
f
;
l'(dOj
(O,W)})))
178
Chapter 7:Theory of
Then P(j.t)(N) =
Estimation)
O. For every BE(JI define)
F(B)= JB
/
dp,
j.t(dt) dv-
I Q(B) Thenwe have for every A
E
dp' J dV\" II(dt) on Q\\ N, on
N.)
sI with P(j.t)(A) < 00and every BE14(T))
d = J r P' JI(dt)dv JA F(B)dP(j.t) dV A\\N' = = J P,(A\\N)p(dt)
P(1BP)(A).
B)
Thisprovesthe assertion.
0)
The precedinglemma also gives us a methodfor computingposterior
distributions.)
36.11Examples.(1) Consider the shift of Example 36.5(1).It is easy to see that a
distribution is) posterior
_dP.
JB d
A.
- a)
(x
-
H
F(x,B) = dP J
(x
AL(da)
(x,B)E H x .\037(L).
,
a) )'L(da))
dA.H
If we specialize P = Nil then we obtainthat F(x,.)= Epr.(JC) * Nt\" x E H. the shift with scaleparameter of Example(36.5) (2) Now, consider (2).Then
a
distribution is) posterior
-
1 dP X a d(J L (da) H nrr d B (J I\\H ( (J ) , (x,B) E H x rill (L x F(x,B) _ x a H (J}+1 d H ( ) (/(JJ..,.(da)
-
\037
A.
\037
+ \037
).
d\037
(J)
A.
to P = NH doesnot leadto a considerable Specializing simplification. which doesnot rely on a distributions We needa representation of posterior in Theorem36.14 after somepreparations.) measure. It isobtained dominating
36.12Lemma. Supposethat E is weakly continuous. Then) \037
JB
{
dp'
I p.(dt)< 2 j.t(B)
II
\037
d\037
}
4\037\037\037
\037
- P'II, BE
tM,
sET.)))
36.Estimation
by desintegralion
179)
is trivial if J.L(B)= n. If Il(B)> 0 then we have) ProofThe assertion Ps
JB
{
=
I Jl(dt)< 2 Jl(B)
dp' dPs
}
dp' dp, { dp' {, dPs
1 2
J.L(dr)> J.L(B)
J.L(B) r
\037
\037
\037
\037
\037
r
\037
Recall,that
1
J.L(dr)> J.L(B)
t
, --
d\037J.L(dt)
d\037
0
P, II.
4 supII P,
}
\"2
2 :r 1 dp' J Jl(B) uB)
}
if r....... definitionE is continuous for the p, is continuous
by
variationaldistance.)
36.13Corollary.Supposethat E is continuousand Jl(U) > 0 for every U c T, U 0. Then) dp' [ dp,J.L(dt)> 0 \037-a.e.
open
=*=
fo.revery S E U and every openU c T, U:+:0.) Let U T be open, S E U, and B(s,c):={tE T:II P,II < c} U.Then)
Proo!
\302\243;
\037
P,
{!
\037;,
\037
\037
-
choose t > 0 such
that
\302\243;
I'(dl)=
o}
dp' < _1 J.L(B(s, 4c. J.L(dt) 2 {B(s,e) dP. } S
e\302\273
\037
o)
oS)
36.14Tbeorem.Supposethat E is continuous, Jl-integrah/eand that J.L(U) > 0 distribution F satisfies) for every open U c T, U * 0. Then every posterior dp'
_
. J - .J.L(dt)
p.s-a.e.,B E /A (T), S E T. F(B)= !,_d\037 dP, J dP.J.L(dt) s)))
\037
J
80
Chaptcr7: Thcory of Estimation)
ofa posterior distribution we have for every ProofLel BE91(T). By definition
Ae.91)
= P(IB Jl)(A) = J P,(A)Jl(dt). J F(B)dP(Jl) B
A
dp' P'(N) 0 and P,(A\\N) = AJ d - dp', t e T, The P, existence of such a setN followsfrom weak continuityof E. Let t E T. From >0 Thus we obtain it follows that J Corollary(36.13) dP,) Jl(dt) p'-a.e. Let
Ned be such that
o
'\"\"'
\037l}
J
A\\N
!.
dP'
=J J Jl(dt)d\037 F(B)dP(Jl) 8A\\N dP. s = J
J8
A\\N
f
dp'
dp'Jl(dt) dp,Jl(dt)dp,. dp' dt J dp' Jl ( )
dP. s)
-
. followssince . dP(Jl) dp' --- - on Q\\ N. The asserlton Jl (dt) = J dp.s)
dP,
o)
36.15Corollary.Supposethat the conditions aresatisfied. A of Theorem36.14
measurablefunctionf T -+
\037
is (E,Jl)-integrable iff
SE T.) flf(t)1F(dt)< 00 p'-a.e.,
In this case)
dP(hl)= dP(Jl))
sET. SfdF f's-a.e.,
and 36.14. combine36.7(3).36.13 The second ProofFor the first assertion
followsfrom Lemma 36.10.
0)
lemma.) Finally,we prove a useful approximation
36.16Lemma. Supposethat E is continuous.Let
...,
K
\037
T be compact and
K -+ R !?I(T)-measurable and bounded.Then for every t > 0 there are a measurablepartitionBt, BM ofK,pointst i e Bi . 1 i M.and <Xi E lit 1 i
f
\037
M, such that) \037
-
dp'Jl(dt) lcf(t)dp' f
\037
M (J.j j\037l
\037
dp' Jl(Bj) < E;, SET.))) dP;
\037
37.GeneralizedBayesestimates
un)
If Jl(K) = 0 or IIfllu:=supIf! = 0 then Proof Let c> 0 and K s; T compact. K the assertion is trivial.Therefore,we may assumethat Jl(K) > 0 and IIfllu > O. SinceE iscontinuous there exists{) > 0 such that) dp\" \037
dPs
- - -P. 1< whenever Itt - < dP,2 dPs
\037
II P\"
/2
I
I;
2JL(K)IIfllu)
t21
f>,
t..
t2E
K, SE T.)
!of
Sincefisboundedthere isa stepfunctiong = i=L !Xi lBI such that IIgll.. IIfllu and IIf gllu < ' The sets Bi , 1 I. M, may be chosensuch that 2Jl(K) diam BI < f>, 1 ; M, and {HI:1 ; M} is a partitionof K. We obtain) \037
-
I
\037
\037
-
\037
\037
dp'
(t) d
K
=
-
\037
-
M
\037
dP'.Jl(BI) Jl(dt) ,-1 i .L d Ps
If
\037
\037
ex
\037
dp, (f(t)Jl(dt)+ I}If(t) g(t)ldp'Jl(dt) r k
\037
M
d\037
i\037t
M
\037
+
i\0371
\037
\037
. Jl(K) +
2Jl(K)
\037
.
2Jl(K)
M
dp' ( ) Jl(dt) dP. dP. Ig(t)1 dPsJl(dt)) g(t) ki r
get)\037
\037\037
Jl(B,)< E;.)
dP,
dPs
d\037
d\037
d\037
o)
i\0371
37.GeneralizedBayesestimates) and assumptions 36. of Section keepthe notations In the followingwe considera loss function W = (U0/ET' given by IE T, suchthat (s,t)...... (s,I)e Tx T, is \037(T)x \037(T)J-Y,: T\037 [0,(0), measurable and W(O) = O. If E Jl(E,T) isan estimate then the risk of (!with respectto W is given by We
\302\273-;(s),
\037
tE T.) U(ro,ds)P'(dro), H the notations) Forconvenience let us introduce \302\273-;UP,:=
\302\273-;(!:
\302\273-;(S)
w...... J
\302\273-;(s)
(!(w,ds),
WE
fl, t E T,)))
182
Chapter 7:Theory of
and) l?
ESlimation)
p,:Bt-+ f (!(w, B)
\037
(dw),
BE94, t E T.
discussfirst Bayes estimates assumingthat IIIgJ(T) is a finite measure. Generalized are the extension of this conceptto non-finite Bayes estimates t he Recall that F denotes measures w.r.t.Jl.) distribution 1l1\037(T). posterior
We
37.1Definition.Supposethat JlIEl(T)is finite
is Jl-intcgrable. An to Jl and W) if estimate e*E fJl(E,T) iscalleda Bayesestimate (with respect P'Jl(dt) for a1l e Yt.(E, T).) J J.v,q*P'Jl(dt) J \037
J.v,\037
and
W
\037
37.2Theorem.Supposethat E is weakly continuous, JlI5l(T)isfiniteand W is thenfor every SET) (!*e {Jt (E,T) isa Bayesestimate If an estimate Jl-integrable. J
(J\302\245,U*)
(w) F(w, dt)
\037
J J.v,(s)F(w, dt) P(Jl)-a.e.)
Proof Assumethat there existsSoE T and A E .91with P(Jl)(A) > 0 such that f
Definean
< f(W;(!*)(w) F(w,dt) J.v,(so)F(w,dt)
if
WEA.)
estimate C!E fJt(E,T) by
(!(w, .) =
(!.(w, .)
if
w rf A,
I
ClJ
'f
{
\302\243\"0
EA.)
Thenit is easy to seethat
<J
P'Jl(dt). 0) If E is weakly continuousand Jl(U)> 0 if U c:T is open, then the exceptional setis assertion P(Jl)--{p,:t E T}.In thiscasein the preceding J J\302\245,e\037Jl(dt)
J\302\245,(!*
of \037-measurezerofor every t E T. The assertion of Theorem37.2is the startingpoint for the definitionof generalized Bayesestimates.) if for every SETthe 37.3Definition.A lossfunction(U0u;Tis(E,Jl)-integrable
function / t-+ J.v,(s),/ E T, is (E,Jl)-integrable.)
and (Jt;)'ETis(E,Jl)-integrable. 37.4Definition. that E isJl-integrable Suppose to Jl and An estimate (!*e fJt(E,T) isa generalized (with respect Bayesestimate W) if)
(w) F(w,dt) J (J.v,(!.) for all .'IET and rE T.)))
::;J
J\302\245,(s)
F(w, dt)
\037-a.e.
37. Generalized Bayesestimates
183)
37.5Remark. Let usmake somecomments on the preceding definition. First we notethat the inequality may
be written as)
O.(w, ds')
f (f J-v,(s')F(w,
\037
dt\302\273
J
F(lt),dt)
J\302\245,(.\037)
r E T, sET.) p\"-a.e.,
Assumefor the moment that
W'-'inf J J-v,(s)F(w,d/),
Q,
WE
SE T)
isd-me:asurable. Letting) B(w) 1={s'E T:J J-v,(s')F(w, dt) = inf J '/fiT) we seethat
F(w, d/)},
J\302\245,(s)
WE
Q,
0.isa generalizedBayesestimateiff
o.(w,
B(cd\302\273
=
p'-a.e.,t E T.)
1
The preceding of generalized remark showsthat the existence Bayesestimates
depends on regularity properties of the posterior risks the:mostgeneralset J J-v,(s) F(w.dt),sET.w E Q,We: donotintend to state: of generalized of conditions the existence since which implies Bayesestimates, for most specialcasesexistence followsjust from a moment'sreflection. of which is useful in many cases. we will work a out set conditions However,
s.-.
First,letus consider examples.)
37.6Examples.(1) Let us consideragain the
shift on a Euclideanspace beenintroducedin Example 36.5(1).Define the loss function by \037(b)= lib a1l2, a,bEL, and assumethat P possesses finite momentsup to the order2.Then)
(H,< ., .
\302\273
which has
-
\".(x) = J aF(x,da) =
- a) (x - a)
:; dP
-T
Ja
J
(x
d)..H
is a generalized Bayesestimate. of (1)but let (2) Keepthe situation I if b a > C. = W a (b)
if
(da)
,
xE
H,
Adda))
a e L.
IIb-all
{o
-
AI.
II
The pertaininggeneralized are calledMaximum Probability Bayes estimates
Estimates.
Nil'Then it is easy to see that for bothtypes of lossfunctions,,*= PL is a generalized Bayesestimate. the caseunder (1)to P = (3) Now we specialize
It)))
184
Chapler7:Theory of Estimation)
will turn
outin Section38that this iseven vaJid for a considerably larger class
of lossfunctions.
a Gaussian shift with scaleparameter and define (4) Finally let usconsider
-
\037
as in Example 36.5(2).Definethe Jossfunction by , a(b,t) = (J2 I1b aUZ, a,bE H, (1, > O. Then it is not difficult to seethat K. = PL is a generalized Bayesestimate. existence theorem. Now we approachthe announced In the followingJet (H,< ., . bea Euclidean spaceand assumethat T = H. = W(s t) where W: T....., Moreoverlet Jl = All and [0,(0)is a measu= rablefunctionwith W(O) O. An experimentE E 8(T)is\037-integrableoforder p 0 if it isJ\037-integrableforf t r+ t liP, t E T. If W isoforderp and E isAIIintegrable ojorderp then (Jt;)u;H is (E,AH )-integrable.) \037
'\037
or
-
\302\273
J\302\245,(s)
II
\037
37.7 Lemma. Suppose that E;scontinuous and ;,w;ntegrable oforderp.If W is riskfunctions of orderp then the posterior
st-+fW(s-t)F(dt),sEH, arecontinuousPr-a.e. Jorevery r E H.)
-
Proof ChooseS E Hand r E H. Let B(s,b l ) H be such that W(u t) CJ \" IliP + C4 if u E B(s,01)and IEH. SinceE is AII-integrab]eof orderp it \037
\037
followsthat
JiltjlPF(dt)<00 J:,-a.e.,rEH.) of r E H. Now Jet > 0 Theexceptional setN E .91can bechosenindependently arbitrary and fix somew N. Thereexistsat 0 such that) \302\243
\037
\037
II tll f IJrJIP>a.
P
- -. I;
. F(w, dt) < 3C
J)
Define u-(c),=min{W, CJa + C4 }.We may find bE (O,bl ) such that) C
II u-(t)(U
-
t)\302\243(w,
dt)
-
- J u-(t)(s-
I)F\302\253(I),
dt)1 <
\037
3)
Hencewe wheneverII u s\" < O. Thisfollowsfrom continuityof convolutions. obtainfor every II E B(s,b)) I
J W(u \037
- t) F(w, dt) - J W(s -
-
+ J (W \302\243
\037-
3
-
\037t\302\273(u
F(w,dt)1
-I - t)F(w, - F(w, + J - -
If \037C)(u t)F(w,dt) t)
t)
dt)
(W
+2CJ f II/JlPF(w,dt)\037\302\243. 11'11' >41.)))
dt)1
\037C)(s
\037t)(s t) F(w, dt)
0
37.Generalized Bayescslimates
HS5)
37.8Theorem.Supposethat E i...continuousand l,,-integrable of orderp. there Assumefurther that W is separating, level-compactand of orderp. 111en existgeneralized Bayesestimates.)
of Remark 37.5issettledby Lemma37.7.If ProofThemeasurabilityquestion we can provethat B(w)* 0 on a set A E d with \037(A) = 1,S E H, then the assertionfollowsfrom the existencetheorem on measurable selections,
Theorem6.10.
1SI step:Let.'iE Handt > O. Firstwe notethat there existsa b < supW such
that) \037{J W(s
- t) F(dt)> b}<
To provethiswe
\342\202\254.
setK integrablethere existsa compact r
\037{
is not bounded.SinceW is (E,AH)Hand l> > 0 such that
first assume that W
W(s
-
\302\243
t) F(dt)> l>} < t.
H\\K
is boundedon K. -.) isboundedand separating. Let c< sup
The assertion followssinceW(s
W besuch that that W Now, assume = B {tET: W(s-t) c} is a neighbourhood of s. From Lemma 36.12we obtainthat there isan 11> 0 such that
::;
< 11} < t. \037{F(B) The number 11can be chosensuchthat c< supW
-
t) F(dt) = supW' (1
f W(s
-
\037
= supW
-
We
ohtainthat)
supW' F(B')+ (supW '7)F(B) + supW' F(8) '7F(B)
-
F(8\302\273
11F(B).
Henceit followsthat)
-
- 11.
W(s
t) F(dt)> supW
-
2 '7 }
{F(B)< 11}< E. 2nd step:Let.rE Hand > O. We showthat thereexista compact setK Hand a number 0 < l> < such that) inf f W(x t)F(dt)> inf J W(y t)F(dt)+ l>} > 1 c. \037(f
\037
\037
E;
\302\243)
\037{
\037
-
-
XE II\\K
)I\037
-
H)
Since > 0 isarbitrary thisimplies to the first that B(w) * 0 \037-a.e.According W a b < such there exists that) step sup \302\243
\037(J
W(s -t)F(dt)
\037
b} 1 \037
- 2'l: -
Let 0 < l>
C c H satisfying)))
Choosea compactset
186
Chapter7:Theory or Estimation)
-o}< 2 - t)F(dt) b}rI{F(C) 1-o}.)
<1 Ps {F(C) and let)
A,.::::; {f W(s
\302\243
')
\037
Then we have \037(Ac) > 1
\037
- t. Since
is level-compact there existsa compact setKc H such that tEe and x rt Kc imply W(x t) > (b + 0)/(1 0).Then we obtainfor every x rt Kc that on the setAI:)
-
W
\302\243;
J W(x
- t) F(dt)
\037
> b+b F(C)
1-=b.
\037
\037
cJ W(x b
-
t)
-
F(dt)
-
+ 0 J W(s t) F(dt)+ 0 \037
inffW(y-t)F(dt)+o. D
yeH)
proofof the precedingtheoremshowsthere existgeneralized Bayes which are even non-randomized estimates estimates.) As the
38.Fun shift experimentsand the convolutiontheorem)
l,
Let G be a locallycompact groupwith countablygeneratedtopology.Let = ).tif and At be a pairof right and left invariant measures such that Z(O'IJ.,)
u:x.-.X-I, X E G. The Borel-u-field of G isdenotedby &I(G).The modulus function of G isdenoted by A.)
. .
38.1Definition.Let PltI(G)be a probabilitymeasure onG.Thenthe experiP:t e G})iscalledthefullshift experimentgenerated ment E = (G,{M(G), {\302\243,
by
P. For shortwe denotePc:=
\302\243,
P, t E G.)
A measurable 38.2Remark. The followingfonnulasare immediate. function
is P-integrable iff it is Pc-integrable and JfdP,= Jf(lx)P(dx), (E G. we have Pc(A) = P(t-I A), t E G, A E 14.If P )'1then In particular, \037
tE G, and)
-
dp' _ dP -I (t x) (x) dJ..;
A.r a \"
.c.,
t
E
Pc
\037
I'b
G.)
d)'\037
38.3Lemma. (Torgersen[1972]).Supposethat E is the full shift experiment
areequivalent:))) generated by P. Then the followingassertions
3M. Full shift
experimentsand the convolution theorem
187)
(1) P l\" t E G} is dominated, (2) {p,: IE G} l\" (3) {p,: (4) E is weakly continuous, (5) E is continuous.) \037
\"\"
Proof'(1) (2):Let A e \037(G).Then (I:,+ P)(A) = J 1A (Ix) P(dx) = P(t 1 A), \037
t
E
G.
If AtCA) = 0 then AAt -t A) = 0 and hence(t,* P)(A) O. t e G}.If measuresuch that Q {p,: (2) => (3):Let QIEM(G)bea probability 1 A E &iJ(G) and t E G then Q(t- A) = 0 implies \037(t.-I A) = 0 for every s E G. = Hencewe obtainP'\037(A) 0 for all s E G and there Q(A) = O. It followsthat Q(A) = 0 iff
-
\037
0= J Q(t-lA) A(dt)
= J I lA (IX)Q(dx)A,(dt) = I A,(Ax-l) Q(dx)= l,(A)jL1(x I) Q(dx),)
where L1
is the modulusfunction of G. Since > 0 we obtainQ L1
(3) => (1):Obvious.
(5) \037 (4) \037 (2):Obvious. (3) =- (5):It is a well known fact that
limII/(x)
'-e)
-/(t- x)1).tCdx)= O.
.
Since)
for every IE
dee, - P)
- l(.
!e(G,.'!I(G), It) 1
1
dP_ (I 1 x),
=
d;.,--(x) d).,
the assertion follows.
XE G, Ie G,)
0)
that G isAbelianand let G be the charactergroupof G,i.e.the group Suppose of all homomorphisms x: G {zE c:Izi = 1}.The Fouriertransform of a measure PIBI(G) is probability
-.
P:
X
t-+ J XdP.
X
E
G+.
It iswell known (seee.g.Parthasarathy [1967]) that two probability measures iff their Fouriertransforms on fM(G) coincide coincide.)
38.4Lemma. Supposethat G is Abelian.If p(x)* 0,X E G*,then the fullshift measures.) experimentE consists 01pairwisedifferent probability ProofAssume that
\037
=
\037,
s,lEG.Thenit
foHowsthat
for every X E G*)))
188
Chapler7: Theory of Estimalion)
=
f x(x)P'(dx) f
(x) \037(dx) and thereforeXes) = X(t) for every X E G*.This provess = X
t.
0
Estimationfor full shift experiments dependson the methodsdevelopedin 36and 37.) Sections
38.5Lemma. Every fullshift experimentE is A,-integrahle.) ProofLet usshow that J P,A,(dt) = l,.If A E ll(G)then we have = f P'(A) A., (dt) ffl A (tx) P(dx)A, (dt) = HIA (tx)A.,(dt)P(dx) = A,(A). a)
38.6Example.Themain examplein the presentsectionwill bea full shift ona
Euclideanspace(H.<'. For thiscasethe precedingassertioncan be improved to the followingfact:If P has finite p-th momentsthen E is J.H integrableof orderp. This is the particularcaseof Example 36.8(1),where '\302\273.
H=L.
Throughoutthe followingwe supposethat E is a dominatedfuJi shift measure experimentgeneratedby the probability Let us compute the posterior distribution given A,. A particularcaseof the PI\302\243f(G).
in Example36.11 followinglemma iscontained (1).)
38.7Lemma. The posterior with re.'IpeCl to A, is F(x,.)= ex.P, distribution = XE G, where P .!f(uIP).) Proof Let A E \037(G).Thenwe have
!
dP
dP
d)'\037
d)'\037
(t-Ix) A., (dt) = J 1,4(/-1) dP =A(x)J 1A (xt -1) d).
(IX)At (dl)
\037(t\302\273).(dt)
= A (x>J1,4(xr ) P(dt)= L1 (x) 'p(x-I A). Henceit foJlowsfrom the proofof Lemma 36.10 that 1
F(x,A) = J(x)
A) =
:(\037-I
P(x-I A).
0)
thispointthe readershouldrecallDefinition32.1where amenablegroups of certain areintroduced. Amenabilityis the reasonof the optimum properties We have beenconcerned beforein Section 32. situation with a similar estimates. distributions.))) w e statesome lemmas for First, approximation posterior At
38. Full
shift
cxpcrimcnlsand
convolulion lheorem
the
189)
38.8Lemma. Supposethat G isamenable.Theilfor every probability measure (K\ QI\037(G)and every summingsequence Jim
*
At) , J (ex
\"-00
Q) (K\037)
K\
K\"
I.,(dx)= o.
followsfrom) ProofThe assertion
i (ex
*
Ar (\037II)
Q) (K\037)
1,(dx)
\"
1
= A,
1 (1 J.r(dx) (K ) J KJx) J -lKJxt\302\273 Q(dt) II
1
= J Q(dt)
J OKJX)
A,(K,,)
-l Jx) l K
KJxt\302\273
A,
(dx)
1
\037JQ(dt)A
,(K) J 11,,(x)-l,,(xt)IA,(dx).0 K
K
K\
II
Let F bethe posterior distribution 38.9Lemma. Supposethat G isamenable. of E given J.,.Thenfor every summingsequence) lim \"....00
l/,
KII)
= O. HKJIF-F(.IK\lIdP,A,(dt)") )
-
ProofSince F F(.IK ) = F(K\037). n E N, and sinceF(x,.)= II
II
II
only show that)
lim
1 \037 JJ \"-00\"', ( \"K\ )
(\302\243.1
\302\267
P)
(K\037)
P)(K\037) J (ex.P)(K\037) (e, P)(dx)= (E, and thereforewe may apply Lemma (38.8) with Q = Every
.
P we need
;.,(dt)= o.
.P.
.
We have)
P, (dx)
\302\243;(
P.P.
0)
measure QI9I(G)definesan estimate probability e e \037(E, G) by (l(x, Q. XE G. .)I=\302\243X.
Such estimates satisfy e p, =
P, *
Q, t E G.)
38.10DefinitiOD. An estimatee E Yl(E,G) is a convolution kernel if e(x,.) = tx * Q.x E G, for someprobability measure Q E 91(G).) 38.11Remarks. (1) An estimate(l E gr(Et G) is a convolutionkernel
iff)))
190
Chapler7:Theory of Estimation)
the condition is necee(tx,tA) = e(x,A), X e G, t e G, A E gI(G).Obviously, we define which To show ssary. t sufficiency, Q:=e(e,.) impliese(x,A) = Q(x- A) = (ex.Q) (A), x e G, A E \037(G). of a convolution kernel satisfy P, t E G, the distributions (2) Sincep, = = t E latter G. The an invariancecondition, eP, namelyeP, propertygives riseto the followingdefinition.)
.
\302\243,
\302\243,.
38.12Definition.An estimate e E gr(E,G) isequivarianrif eP,=
\302\243,.
eP, t E G.)
38.13Remarks. (1) An estimatee E 9t(E,G) is equivariant iff (eP,)(tA) = (eP)(A), t e G, A E the special casewhereG = (H,< ., . isa Euclidean (2) Consider spaceand K: H P = NH . In this casea non-randomized estimate Hisequivariantiffit is of the form K(X) = x + a P-a.e.for somea E H. To see this note that \302\2438(G).
\302\273
equivarianceimplies SI(K(x\302\273P,(dx)
= SI(K(s+ x)
-.
- s)P'(dx)
for all .'1,tE H and every IE ljjb(H).Sincethe Gaussian shift experiment is = it follows that K(S+ x) K(X) + s P-a.e., .'Ie H. An application of complete Fubini'stheorem yields p2{(.'I, x):K(S+ x) = .'I + K(X)}= o.
Hencethere existsXo E H such that K(S+ xo) = S + K(Xo) for P-almost every S E H. Now, taking a = K(XO) Xo provesthe assertion. The last rcmark showsthat sometimes non-randomized equivariant estikernels.Can a similarassertion mates are almostconvolution be provedfor
-
arbitrary equivariant estimates? In the followingtheoremit is shown that even the distributions of nonare \"in the mean\"convolutions.) equivariant estimates
38.14Theorem.Supposethat G isamenableand let(K,,)bea summingsequence. measures Qn I\037(G),n E N, e E 91(E,G) there existprobability
Then lor every .'iuchthat)
lim II ,,\037oo
,(
A
r
\302\267
) \"in)
\037
\302\243,-1
Proof Recall that for every
h:G2
-.
IR
-
eP'J.,(dt) P.Q\"II = O. boundedfunction
.\037(G) @ .\037(G).measurable
we have)
= H hex,t)F(dtIK,,)(x)Ps(dx)A,(ds),) hex,t)P,(dx)A,(dt) LJ L n E N. We obtainfor every A E 84(G))))
Ful\\ shift
3\037.
x,(
experimentsand the convolution thcorcm
191)
(aP')(tA)i.,(dt)
f
) \"K\"
\037
=
J If e(x,tA)F(dtIK,,)(x)\037(dx)i.,(ds)
) ( \"K\"
I\037 r \037
=
l)K,,)L If e(x,lA)
* (\302\243;1;
P) (dt) \037(dx);',(ds)+ R,,(A))
where) IR,,(A)I
\037
l,(\037,,)
By Lemma 38.9it
LJ
IIF- F(.IK,,)/IdJ:;',(dt).)
followsthat
lim sup IR,,(A)I = O. ,,-\302\253t
If
A..aI(G))
we denote)
Q,,:A
1
\037
l,(ds), J e(x,xA) l's(dx) l,(X\"K\ ) f .
A
E \037(G),n
EN,
then we obtain)
1
A
,(K) inr (eP')(tA)l,(dt) \"
= R,,(A) + 1 A
xtA) P(dt)f>s(dx);',(ds)
,(K) Ie\f If e(x, \"
= R,,(A) + J Q,,(tA) fl(dl)= R,,(A) + which provesthe assertion. A
first
(P.Q,,)(A),)
0)
is the convolution theorem.) importantconsequence
38.15Theorem.(Boll[1955]).Supposethat G is amenable.Then for every eE equivariantestimate that Qp, = p, * Q, t e G.)
91(\302\243,
a probability measureQ19I(G) such G) thereexists
of Proof SinceQ is equivariant we obtainfrom Theorem38.14a sequence measures probability Q\"I&f(G),n E N, such that
- P.
lim lIeP
Q\"II=
O.
\"\"'<10)
Let QI\037(G)be a vague limit pointof
(Q,,),,\302\253:\037.
Then for every fe \037oo(G)
we)))
192
Chapter7: Theory of Estimation)
have)
foP = ffd(P+ Q).)
Since(}pis a probabilitymeasurethe same is true for P.Q and Q.
0)
38.16CoroUary.Supposethat G is amenable.Then for every equivariant estimate there existsa convolutionkernel with the same distribution. Let W: G [0,(0)be a lower semicontinuous function and definethe loss function by \037(x):=W(t- x), x G, t e G. Then the risk of an estimate G) is given by o
-
I
\342\202\254
\342\202\254
91(\302\243,
\037(}p,
= Sf W(t-Iy ) e(x,dY)P'(dx),tE G.
kerneldefinedby If (!isa convolution
QI\302\243\302\245(G)
then
tEG. \037(!P,=SWd(P.Q), theoremforamenablegroupsthe risk of any cquivariant By the convolution
estimate isalsoof thisform.The maximalrisk of arbitrary estimates can be boundedfrom below by the risksofconvolution kernels.)
38.17Theorem.Supposethat G is amenable.Then for QE
!Jt(\302\243,
.
such that measureQI.1J(G) G) there is a probability
\037(! P, sup IE (i)
\037
f W d(P Q)
functionW: for every level-compact
G
every
estimate
- [0,(0).)
of probabilitymeasures Proof FromTheorem38.14we obtaina sequence
11E N, such that Q\"I\037(G),
lim supJ fd(P* Q,,),IE \037,,(G). sup \"6f(}P' \"-'00) \037
If the right hand sideis zerofor all fE \037oo(G)then it followsthat sup \037QP,= W(oo) 'fiG) and the assertion is valid with arbitrary Q.Otherwise,let Qol\037(G)bea vague limit point of (Q,,),,( N, Qo O. Then Qo is a vague limit point of =t=
p.
(P.Q\UN.")Let Q = Qo1(G). Qo.Then QI\037(G)is a probabilitymeasure.If fE fCt (G) then we have lim inf f fd(P.Q,.)
\037
\"\"'00)
f
fd(P.Qo)
\037
f
fd(P.Q)
and thereforefor every levelcompact functionf E fCb (G)we have (since)))
38.fuJl
shift
experimentsand thc convolution thcorcm
193)
supf-IE'Go
(G\302\273
lim SUpJ fd(P* Q,,) J fd(P* Q)
,,-
\037
co)
We
obtainthat)
supf(?P' J fd(P* Q) I( for every levelcompactfunctionfE\037,,(G) and the proofis finishedby Lemma \037
G)
6.4.
0)
and suppose that 38.18CoroUary.Let W: G -+ [Ot 00)belowersemicomillUOUS there is a pointYo E G such that)
J W(x'yo)P(dx) J W(x.y)P(dx) if yE G.) Then the non-randomized estimateK: x \037 X. YOt x E Gt \037
ha.\037
the following
properties: (1) K is optimal(for W) amongallconvolutionkernels. (2) If (i isamenablethen K isoptimal(for W) amongallequivariantestimates. (for (3) If G isamenableand W islevel-compactthen K isa minimaxestimate W).)
and (3) followsfrom Proof (1) is obvious,(2) followsfrom Theorem38.15
Theorem38.17.
0)
Now we turn to the casewhere G isa Euclidean space,denotedby (H,< ., .
\302\273.)
38.19Example.Let W(x) = IIx1l2, X E H. Thenthe assumption of the preced-
ing corollaryis fulfilled with Yo = J xP(dx),providedthe first moment exists. Hence,K: x \037 x + Yo' x E H, is a minimax estimatewhatever PIPA(H) looks
like.If PIEI(H)iscentrallysymmetricthen the first moment iszeroif it exists and hence,the identity is minimaxfor quadraticloss.This easy fact can be extendedvery far as we shalldo below.) 38.20Lemma. Let C, D 5 H be convexsetswhich are centrallysymmetric. Then) l\302\253C+
y)r-.D) l(Cr-.D),YE H.) \037
It is that C and D arc closed. Proof In view of Theorem(6.1)we may assume clearthat we may even assumethat C and D are compact.For any convext compactsetA II let SeA) A + ( A). Then it is easy to see that \302\243
-
I\"\"\"
\037
Cr-.D,using that S\302\253C+y)r-.D)!;;
\037
C and D are convex and centrally sym-)))
194
Chapter 7: Theory of
Estimation)
it issufficientto show that metric.Hence,forthe assertion ).(A) for convexsetA 5 H. But this isan easy consequence of the Brunnany compact, MinkowskiTheorem6.2.For this,put k:=dim H and notethat ).(S(A\302\273
1 ).i\"
=
(S(A\302\273
11
).1(2A + 21(
-
1
A\302\273
\037
11-
1
2).i(A) + 2 ).k(
38.21I\037mma. (Anderson[1955]).LetP <( ,t.Supposethat W:
H -+ [0,(0)aresubconvex.Then) f WdP
\037
f W(x + y)
A)
=
\037
1 ).i\"
(A).
0
- dP di and
P(dx), ye H.)
ProofLet C 5 H beconvexand centrallysymmetric.Thenfrom Lemma38.20 it
foltowsthat for every ex
P(C+ y) =
\037
0,y E H)
1.t(C+ y)n {
>
\037\037
.(
;:;;I Cn Putting C = {W
=
{\037\037
>a})da
P},P 0, we obtain P{W>P} P{W(.+ y) > P}, \037
da
a})
P(C), ye H.
\037
\037
{J
\037
0,ye H,)
and therefore) QO
QO
P{W>{J}d{J f0 P{W(.+ y) > {J}d{J =fW(.+y)dP, yeH. 0) At this point we couldapply Corollary38.18 to obtainthat under the of Lemma 38.21 the identity is the optimalequivariant estimate. assumptions J WdP = fo
\037
However,for the verificationof the minimaxpropertywithout assuminglevelof Wwe have to go back to Theorem38.14.) compactness
- dP
38.22Theorem.LRt P <{)..Supposethat di..and W: H -+ [0,(0) are subconvex.Then Ie:x....... x, X E H, is a minimaxestimate (for W).) Let QE .\037(E, H) we may assume that Wis lower semicontinuous. ProofW.l.g. of Then from Theorem(38.14) we obtaina sequence be an arbitrary estimate. measures n E N, suchthat probability Q\"I\302\243\302\245(H),
sup H'\"e IIEH
\037
;;;; lim
supJ (W (\\ a) d(P* Qn)
n-QO)))
38.Full for every
a> O. Sincewith
W
experimentsand
shift
the
convolution theorem
195)
and also W r.a, is lower semicontinuous
that it follows subconvex, w,.eP\" J (W r.a)dP, sup heR) \037
a> O.
From)
J WdP
--=
sup J (Wr.a)dP 0>0)
the assertion follows.
0)
Thequestion whetherthe identityisthe only equivariantestimate with minimal risk is answeredby the followingcorollary.)
38.23Corollary.Let P
\037
A
and p(z)
=+=
0,Z E H.
.\\\037uppose
that
W:
H -+ [0,oc)is
andsuch Ihat) level-compact J W dP < J W(x + y) P(dx) if y * O. Then an equivariantestimale (l E H) has minimalrisk (for W) 91.(\302\243,
= 1,4(x) P-a.e.,A
E
(ff
(l(x,A)
&f(H).)
measureQI\037(H)it followsthat Proof.Since(lP= P * Q forsomeprobability J WdP = J Wd(P*Q). First,we show that Q = f:o.For C( 0 let DII = {xe H:!Ix C(}.Define A(.I= inf J W(y + z)P(dy). \037
II
\037
z.D.)
Theseinfimaare attainedsincelevel-compactness implies Jim z..\037)
J W(y + z) P(dy)= supW.
HenceMil > J W dP for every > O. We have \037
J Wd(P.Q) = H W(y + z)P(dy)Q(dz)
+ Q(D(J)Mo + Q(D(J)Mo.) It followsthat Q(DIl)= 1 for every C(> 0 and thereforeQ = to. that for every t E If and Thus,we obtain(!p= P. Equivarianceof (!implies \037
=
A
E
fM(H)
\037
(1-
Q(D(J\302\273Mil
-
P(dx)= peA t). II be compact. SinceshiftsareA-integrable J P(K t) )'(dt)< 00,))) J (l(x+ I, A)
Let K
Q(D\037)Ma
-
196
Chapter 7: Theory of
Estimation)
and hence)
-I(z,r -x) J (}(I,K) P(dx) (dx) = J e-I(z,,) ZE H. P(K
Je
-
A.
t\302\273)'(dt),
Fubini'stheorem thisyields p(z)J e-i(z,r)e(t,K)/.(dl) = p(z>Je-i(z,r) which provesthe assertion. 0) By
lK(t\302\273)'(dl))
The idea of the followingproof has been used also by Ibragimovand
Has'minskii [1981].)
38.24Discussion. Lemma). If P = Nil then (Equality in Anderson's J WdP = J W(.+ y)dP impliesy = 0,) for every subconvexfunction W: H -+ [0,(0)which islevel-compact, separating and of finite order.It shouldbenotedthat noof theselatterconditions can be omittedwithout compensation. Lemma 38.21. In To provethis,let y 0 and recallthe proofof Anderson's W view ofthisproofit issufficientto show that there issome < such that) Po sup =t=
P{W>P}< P{W(.+ y) > P} for all P Po. ChoosePo in such a \037
way that {W
\037
0 as inner Po} contains
sinceW is separating. Let Po P < supWand denote point.This is possible = C {W P}.Then we have \037
\037
P(Il\\C) < P(H\\(C iff)
J.
-y)n
((C
>
{
\037\037
-
y\302\273)
_})
d (cn{
\037\037
>_}))
for someex > O. This is the inequalitywe shallprove below. Let
_.=sup
C S;
{Y:
{
>
\037\037
Then
r}}.
it
followsthat C s;;
{
\037\037
Moreover,we show that
oCn
{
\037\037
For thislet
=
,,}
\037
0.
b.!0 and choosex.
E
Cn
{
\037\037
<\"+
n E
b.}.
N. Then)))
;<;
,,}.
3M. Full shift
(X
(X,,)< a
\037
+ b\",
nE
experimentsand the convolution lhcorem
N, and there is an
197)
accumulation point Xo of (x,,)
\037\037
a. Since C is dosed we
(xo) =
satisfying
have
Xo E
C, and since
\037\037
t
we have Xo E ac.Thus,we obtain
>
\037
\037},
{\037\037
XOEacn{
\037\037
=+
Next, Jet us show that \037
a}).)
\037 Xo\037s\302\253c+y)n{
Assumingthe contrary,namely)
y)n
XOE S\302\253CI
\037 {\037\037
dP
1
=2(C+Y)11 {d)'
\037(X
dP 1 + 2(C-Y)fl d)' it
{
would followthat) Xo
a}) }
\037
(X
}
-
,
= 1 (Xt + y) + 1 (X2 y) = 2 2
where x,.X2 E C,x, + Y E
1
;;;
{
a}.x,
\037\037
(X2 + X2),
\"2
-
YE
{
SinceC
\037
\037\037
\037
\037}.
and sinceXo isa boundarypointof the strictlyconvexset
;;; {we obtain a} that \037\037
Xo
= necessarily XI
+Y E
\037
{
\037\037
a}
{\037\037
X2 = Xo. But
and
Xo
-
Y
E
{
\037
\037\037
\037
a}
+
isa contradiction to the fact that Xo is a boundarypointof the strictlyconvex dP set dl 2.a .
{
-}
.)
It followsthat for somet > 0
>
B<Xo,8)\037
(s\302\253c+y),,{ a}))'.))) \037\037
198
Chaptcr7: Theory of Estimation)
On the otherhand Xo E acf'I
=
{
\037\037
Therefore
a}.
xoetf'l{ > a}.
(1-\037)
\037\037
This is due to the fact that 0 E (; and 0 E
(1 n
separate
>
{
\037\037
Xo
onecould
a}.(Otherwise
a contradiction to Xo E from (;andproduce
clearthat the openset U,\037 B
((1 i)xo.n satisfies =0. >a})
Uf'lS(C+Y)f'I{
\037\037
C).Now. it is
but
uus(c+Y)f'I{>a})\302\243cf'l{>a}.) \037\037
\037\037
This implies)
>
'(S(C+Y)f'I{ >a})d(cf'l{ a})).) \037\037
\037\037
Another application of the Brunn-Minowski theoremasin the proofof Lemma 38.20provesthe asserted inequality.)
solutionof the estimation Now, we are in a positionto presenta complete with linearrestrictions. problemfor Gaussianshift experiments and consider the experimentE = (H,1I(H), Let L Hbe a linearsubspace {NH 6,:tEL}),This is not a full shift experimentif L 4= /I. We dealwith the L IRk wherewe assume a linear function! problemofestimating w.l.g.thatf issurjective. Thismeans, that the decision spaceisD = and the lossfunction is of the fonn W; (y):=W (y [0,(0).We y E IRk, tEL. where W: iii\" estimateK = f 0 PL is the best possible shallprovethat the non-randomized
.
\302\243;
-
-
\037k
f(1\302\273.
-
choice.)
we shallfrequently 38.25Remark. In the proofsof the followingassertions
usea reductionof the problem. Let F = (H, {Nil.6,:t L kerf}) and letG:=K.f: To describe the experimentG we endowR k \302\243f(H),
with the inner
product) \302\253Yl'Y2\302\273:=(X
.X2)
1
jf
y,=f(xi),xi.lker.f.;=1,2.)))
38.Full With
shift
experimentsand the convolution theorem
199)
thisinner productwe have \037(KINH.6,) = NGlIc.6/(,), t
1.kerf,
and thus
G = (RJ:,EI\", {NRIe.6/(1):t 1.kerf}).) The pointis that G is a full shift experimentwhere the theory of the present sinceK = f 0 P,.isa suffcientmapping,G and can beapplied. section Moreover, F are statistically whether E and G are (Thequestion isomorphic experiments. makes no sensein ourframeworksince and G have isomorphic statistically \302\243
differentparameterspaces.)
Forconvenience Our first resultisconcerned with the convolution theorem. kerf) Nil. IE L, Q,= NRIl.Cf(')'1.1
we denote.P, =
C\"
38.26Theorem.Let e E g,(E,IRk) be an estimate H..hichis \"equivariantfor the in the estimation off\" followingsense: * (ePo), tE L. eP,= Then there existsa probability measureR such that e.P,= (f 0 Pd'p')* R, tEL.) \302\243/(t)
I \037k.
!i'
ProofSincee E fJt(E,IR\") and 9t(F,IRk) = 9t(J::,[Rk) we have e E !1i(F,IRk). The F and G are statistically and thereforethere exists experiments isomorphic t E 9t(G,IR\"} such that tQ,= eP\" 1.1 there existsRial\" kerf By Theorem38.15 such that
eP,= tQ,= Q,.R, t.lkerf, or in otherwords) (ePo)= NAIe. * R. t 1.kerf. SinceIE L occurson bothsidesonly throughf the equationis valid even for \302\243/(1).
every
\302\243/(')
IEL.
0)
Next, we show that
K
= f 0 P,.is a minimaxestimate.)
38.27Theorem.Assume that inf \"F\037(F..R\")
Proof Let e E Then)))
W
i.v
lowersemicontinuous and subconvex.Then)
sup JYreP,= J W(foPL)dN\". ,pl.)
91(\302\243,
H\")
and
choose't E 9t(G,IR\"} as in the precedingproof.
200
Chapter 7: Theory of
sup \037e\037 r L
\037
Estimation)
SUp
r.l ker f
E
= SUp
\037e\037
r 1 ker f)
\037tQ,
= J WdQo = J W(f\302\260PL)dN H
This provesthe assertion.
.)
0)
the uniqueness FinaJly,we consider problem.)
-
38.28Theorem.Assume that
the subconvexfunction W: IRk [0,(0)is leve/compact,separatingand of finite order.Then every equivariant estimate (!E 91(E,1R ) (in the senseof Theorem(28.26))with 1t
\037eP,= J
W(fopJdNll , tE L,)
satisfies U (x,B) = 1B (f 0
p,.(x\302\273
A.-a.e.,BE fill<
.)
as in the proofof Theorem38.26. ThenCorollary 38.23and Discussion 38.24imply that Proof.Chooset E 9t(G,
\0371)
'[(.,B)=1 8 Qo-a.e.,BE\037k, or in otherwords) J (1(.,B)dNH = J IB KdNH O
A
A)
for all BE gr, A c:K-1 (81\.")If we put
(1(.,B)dNll = NH(K-
=
A
1
f 1(--1(81)
(B\302\273,
,,-1(8) then it followsthat BEgr,
and hence)
e(x,B)= 1
if
fopdx)EB
A.H-a.e.
0)
carriesover to the casewith an It is clearthat the precedingdiscussion
scaleparameter (J > O. additional
since If P NH then the reductionusedin Remark 38.25is not possible This isoneof the reasonsfor the \" = f 0 P\" neednotbe a sufficientmapping. of the theory of the next section.) development =*=
39.The structuremodel) In thisparagraphwe consider which generalizes the caseof full shift a situation 38.))) considered in Section experiments
39. Thc slructurc model
39.1Example.Let P
201)
and considerthe n-fold product E\" IE n E {(P t,)\": !R}), N, of the full shift generatedby P. The on the samplespacen = Ii\" by translation groupG = IR operates gw = (WI + g,W2 + g. w\" + g), g E G, wE Q. ofQ. Each orbitisisomorphic Theorbitsof the groupactionform a partition can we with the operatinggroup.Therefore, identify the pointsWEn by the Let M = {wE L Wi = O}.This is a crosssectionof followingprocedure: i-I each orbit in exactly onepoint.Each WEn can be orbits,i.e,it intersects = (R\",\037\",
\037
AI\302\243itl
*
...,
!1: /I
in the followingway:) decomposed W
=
-1\" .L n
+
Wi
1=1
Denoting
-1 i-L
(
WI
- 1\" .... - -1\" .r ) . .1: 1-1 .-1 WI'
n
w\"
WI
n
\"
T(w) =
Sew) = we obtainW
\037
n
and
Wi>
t)
- nl=1 .L 1
(
WI
- nl=l) -1 L /I
/I
Wi>\"\"
W\"
Wi
),
T(w) + Sew), where
T(g + w) = g + T(w),
w
\037
n, g E G,)
and)
S(g+ w) = Sew). we n, ge G.) In otherwordsTisequivariant and S is invariant. Let (n,\037)be a measurable spaceand let G bea groupoperatingon Q.We assumethat G is a locallycompact groupwith countablygeneratedtopology and the operation Let ).,. @ ..
(\302\243I(G)
q:g 1-+g 1,g E G.)
The followinganalysisis basedon onefundamentalassumption.)
39.2Assumption.There is an (.>1.
\302\243I(G\302\273-measurable
satisfies T(gw) = gT(w) wheneverg E G and WE Q. The followingexamplegeneralizes Example39.1.)
map T:Q
- G,
which
39.3Example.Supposethat Q = H where(H,< ., . isa Euclidean spaceand \302\273
let L H be a linear subspace. Then G = L operates on H by the operation xw :=X + w, x E G, WE H. It isclearthat the orthogonal T = PL projection satisfies the condition of Assumption 39.2.))) \037
Chapter 7: Theory of
202
Estimation)
39.4Example.As beforelet a = HandL s;;; H bea linearsubspace satisfying = x dimL < dim H. But now let the semidirect G IR. L product operateon H + by (a,s)w:=aw + s, (0',s) E R x L, WE H. If we define = idH P,. and (.)II, 1'2)satisfiesthe conditionof Assumption39.2for 1'2= PL then l'= (1111 WE H\\ L. To seethiswe have to show that) 1\037
T(O'w+ s) =
-
(0',s) T(w)
= (O',s)(lll1 (w)lI, T2 = (0'11(w) II, a T2 (w) + s).) (w\302\273
\037
But thisis immediate since)
111; (uw + s)1I= lIu11 (w) + 11(s)1I= ulll1(w)1I) and)
1i(uw+ s) = uT2 (w) + T2 (s) = uT2 (w) + S.) ForW E L the function Tdoesnotmap into1R+ x L However,for ourpurposes it doesnot matter to repJace Q = H by Q = H\\ L sinceL is a subsetof show that Lebesguemeasure zero.Denotek = dim L. Easy computations da @ ).L(ds) is a left-invariantmeasure on G and ).,.(du, A(da,ds)= ds) = da@ ).L(ds) is right-invariant.
!
ak\0371
0')
39.5Lemma. The mapping S:WH (1'W)-IW , under the operation ojG, i.e.it satisfies) (1) Sew)= S(gw), (2)
WE
WE
a, is maximal invariant
a, gE G,
S(wl)= S(W2) implies Wt = gW2 Jorsome g E G.)
Proof (1)This followsfrom
-1(gw) = -1(gw) = (Tw) -1g-Igw = (Tw) -1W = S(w).
S(gw)= (2) WI
(gT(w\302\273
(T(gw\302\273
If 8(wl ) = 8(w2) then (Tw1f WI = (Tw2)\"
= Tw I
I
W2 . '(T0>2)-I
I
W2
and therefore
0)
each orbit in exactly onepoint.Every 39.6Lemma. The set im S intersects (J) = g . S where g E G and S E im S,namely admitsa unique decomposition WE !1
g = T(w),S = Sew).)
Proof The definitionof S showsthat Sew) and ware on the same orbitfor that im S intersects eachorbitat mostat one))) every WE a.Property(1)implies
39.The structurc
model
203)
of the asserted point,i.e.im S is a crosssectionof orbits.The existence = w Q. To follows from W E let w T(w) Sew), proveuniqueness decomposition = gs,g E G, se im S.Thisimpliesthat Sew) = S(gs)= S(s)= s and therefore T(w)s= gs.Applying T we obtain) T(w) T(s)=
T\302\253Tw)s)
and thereforeT(ro)= g.
=
T(gs)= gT(s))
0)
39.7Corollary.The setimSisin.PIsinceimS= T-1 {e}.) T(w) = e. If conversely Proof If s = Sew) = T-1 (w)w then T(s)= T-1 (w). .1 . = = sET-I{e}then T(s) e and thereforeS(s) T (s) S = s. 0)
39.8Examples.In the caseof Example39.3the maximalinvariant is S:x x x PL(x),X E 1/.For Example39.4we have S:x........ PL (x) -'x E /I. In the x PL(X)) secondcaseim S = L1 {xE H:!Ix = I}.
-
(\\
_-II
II
\037
11
39.9Remarks. At
the first glanceAssumption 39.2might seem to be very Morefamiliarare thc followingtwo assumptions: restrictivc. (1)Thereis a crosssectionM of the orbitsof G onQ. This means that for every wED the setM (]{gw:g E G}contains exactlyoneelement. The of all WE Q coincide. (2) (Theisotropic isotropic subgroups subgroupof WE D is (gEG:gw= w}). Let If G bethe commonisotropic SinceH isa normal subgroup subgroup. we may replaceG by the factorgroup G/ /I. Then the commonisotropic subgroupof every WE Q is H = {e}. It isclearthat our Assumption 39.2implies conditions (1)and (2) with M = im Sand H = {e}.However, apart from measurabilityquestions alsothe converseis true sinceeachro e Q admitsa uniquedecomposition as w = g . s with g E G and s EM, Defining T(w) = g iff ro = g . s yieldsa mapping T:Q -+ G which satisfies T(gw)= gT(w),g e G,we o.) \037
39.10Definition,Let PI.saf be a probabilitymeasure.For p(g-IA), A E d. Then the experiment E = (D, let\037: A
every
gE G
g E (i}) is calledstructuremodelgeneratedby P if the groupoperation satisfies condition \037
.\037, {\037:
39.2.
The casesconsidered in Examples39.1, 39.3and 39.4are structuremodels. In the followingwe denotePs:=\037(SIP).It is clearthat Ps = \037(SIPg),
g E G, (by
(39.5\302\273.)))
204
Chaptcr 7:Thcory of
Estimation)
39.11Remark. We definethe measures) A.t * Ps:A H H l (gs) ;.t(dg) Ps(ds), A E.\037, and A, * Ps:A H H 1...(gs) ).,(dg)Ps(ds), A E .91.) A
are O'-finite sincel(and A, are O'-finite and P::;is It isclearthat bothmeasures finite.Moreover, we observethat) (l,* Ps)(A)= J t1 0 T del,* Ps), A E .tI. ...)
whereA isthe modulusfunction of the groupG.Sinceim * Ps.) that At * Ps '\"1..,
A
\302\243
(0,00)it follows
39.12Lemma. Every structure modelisJ.,-integrable sinceP(A,) = i.,* Ps.) Proof If A E .91and g E G then we have A) J p(g-1
A,
(dg) =
;.,(dg)P(dw) IS 1...(gw) = JJ 1... (gT(w).r) A., (dg) Fs(ds) = JJ 1... (gs)i.,(dg) P.r;(ds)= (A., * Ps)(A).
0)
39.13Lemma. Themeasurei.(* Ps is invariant, i.e. (A,.Ps)(gA) = (A.t * Ps)(A), g E G, A Ed.) Proof For gl E G and A E.c{we have
Ps)(g1A) = H l\"...(gs)A. t(dg)Ps(ds).) Since1\",... and sinceAI. is left invariant on G the assertion (g.\\') = 1...(gj'\"lgS) (A., *
follows.
0)
39.14Theorem.For
every structure
equivalent:
(1) P 4.A,. Ps, (2) (3) (4) (5)
.\037
i.\\'
dominated,
9 l, * P.r;, \037
E is weakly continuous, E is continuous.)
Proof (1)<:>(3):Let A E
d.
Then)))
model the following assertion.\\' are
39.The struclurc model = Peg-1A) = \037(A)
(hs) II 19-IA
205)
.
dP deA, Ps) (hs) A,(dh) PS(ds)
-
- (hs)i.t
d(I., Ps) dP -1hs\302\273).,(h)Ps(ds). = SI1A (hs) d .(g * P d(A, s))
'.
Thisimplies \037
\037
i'l * Ps.
-
Qld
measuresuch that Q {P,:gE G}.We be a probability (2) => (5):Let claim that Q Q(A,), Indeed,if Q(A,)(A) = 0 then there existsagEG such that Q(g-1 A) = 0 for every hE G and therefore A) = O. This impliesp,,(g-1 = PlICA) = 0 for every hE G.HenceQ(A) O. By Lemma 39.12we have Q(I.,)= A, * Qs and Remark 39.11implies * '\" * therefore At * Qs. A( * Qs A, Qs. It follows that Q i'l Qs and that * Qsis an invariant measureand therefore Lemma 39.13 implies \037
9
\037
\037
).\"
(w) =
d\037
d(\037(-\"Q)
dP
d().,* Q)
(g
. Q-a.e.,g 1.(.
-1w)
showsthat g\" The Lemma of Riemann-Lebesgue
.
hm \"\"'00
I
.
.dP .dPQs)(g-1hs) - d(A(.
G.)
-g implies
=0 hs) ;.,(dh) (g;1 Q.(})
-dCA,
E
for Qs-almost every S E im S.Hence
.
I1m \"... (0
. Qs) _ (A,d()., Qs) d(i., Qs) dPg
d\037\"
*
*
lemma (Remark and Scheffe's
proves(5).
(3.1\302\273
(5) \037 (4):Obvious. (4) \037 (3):Weak continuity implies P(A,). Since follows. PC),,)= A, - Ps'\")., Ps the assertion (3) ::>0(2):Obvious. 0) \037
\037
\302\267
we see immediately that condition(3) may be consequence to 9 A., * Ps sinceA, * Ps = P(A.,) f!J and A., * Ps\"'\"A, Ps.) strengthened 39.15Examples.(1) When we are dealingwith a situationlike Example39.3 dP P )'H'Let h = - -;-.It followsthat then we usually assume
-
As an easy
\037
\037
peA) = J h(PL(X) + x
- PI.
d\"'H
(x\302\273
A)
A.H(dx)
!
= J J hey + z) i.ddy) L (dz) = I hd(AL.;'L!) A A.
A)))
\302\267
206
Chapter 7: Theory of Estimation)
for all A
\342\202\254
we have \037(G).Therefore
dPs (y) = hex + y)J.ddx), YELl.. I
dIL\037
with Example39.4. Herewe observethat (2) Thingsare morecomplicated peA) =
=
!
-
x
-
h(lIx p,Jx)H _ PL(X) + Ilx PL(X)
)'H(dx)
pdx\302\273
II
z
+ J!h(lIzlI'lI\037
Y)
),L(dy)ALJ.(dz),
A
e 14.)
Let t be the uniformdistribution on {ze [): II z /I = I}.Usingtransformation to coordinates onesees that is with polar 1'-density ,91(II.III),L\037) a measure q...... C q\"-lc-I, q> 0, where n = dim Handk = dim L. Thenit followsthat) peA)
1 = JJ J CrL'h(as + y) k+T dat(ds)),L(dy), A
A E
oc)
a,
and therefore)
dPs
dt
(s) = SSCoc
1
\" h(oc.\037
+ y) (rc +l doc),r.(dy),) sEimS.)
39.16Lemma. Supposethat G is Abelian.If .!t>(TIP)(x) * 0, X E G*,then the SlrucluremodelconsislS mea!1'Ure.r.) 01pairwisedifferenlprobahi/ily Proof.Assumethat
\037I
=
\0371
for someKI'K2 E G.Thenit followsthat for every
XE G*) IX(T(w\302\273\037I(dw)
=
fX(T\302\253(I)\0372(d(l)))
and thereforeX(gl) = X(g2)for every X E G+.Thisprovesgl = g2'
0)
39.17Examples.Theassertion of the preceding lemmacan be verifieddirectly in particularcases.Let Ie(10 (H).If in Example 39.3there existsan aE L, a * 0, such that
= P then PM = P for all n EN which implies JI(x + na) P(dx)= JI(x)P(dx), \" EN. \037
-
-.
SinceI(x + na) 0,x E H, as n 00,it followsthat P = O. leadto) considerations As for Example39.4similar
u\"-1
II(a\"x + q _ .-a)P(dx)= f/(x) P(dx), n EN,) which admitsonly
P = 0, or P = 1;\"/(1 - a) if q *
1.)))
39.Thc structurc
modcl
207)
39.18Lemma. The invariant a-jie/d
d = {A Ed:gA = j
A
if g E G}
isgeneratedby the mappingS,i.e.A c .f1f, iff A = S-1(A).) Proof Followsfrom Lemma 39.5.
0)
which is provedin the followingtheoremisbasicfor the The decomposition
modelisa mixtureof full It isshownthat a structure rest of the presentsection.
shifts.)
structure 39.19Theorem.Supposethat E = (0,.>1, {\037:g E G})is a dominated a im exists there model.Thenfor every s E S fullshift = (G, that: E such {P;:g GJ) \302\243S
\302\243I(G),
(1)
.91i'
under distribution is a version of the conditional
\037
of T given
A, = {hE
G:hs E A},
(J) 1-4!'i(tIJ)
(2) Pg(A) = J J>;(AJPs(ds), g E S E im S.)
G,
A
E.>I, where
Proof First,we notethat for every CE 5B (imS))
-1dP
= P{SEC}= Ps(C). (hs)).Adh)Ps(ds) H C d(Af * Ps))
Thisimplies that
.1.(dPP ) (hs) ).Adh) = 1
J d(
\302\267
s)
Ps-a.e.
For thoses E im S for which the equality holdsdefine)
P;(B)=
!
(g-Ihs) ).,(dh), BE!M(G), g E G.)
d()':\037 Ps)
For the restdefine {P\":g E G} to be any shift experiment. It is then clearthat for every s E im S the experiments) ES:=(G,fM(G), {J>;: g E G}))
areshift experiments. and Ce:\037(im S) (1)We have to show that for every BE \037{TEB,SEC}= J P;(B)Ps(ds),gE G. C) Thisis easily seenfrom))) \302\243I(G)
208
Chapler7: Thcory of Eslimalion)
dP -1 (g hs) ).,(dh) t dO..,... Ps) = \037{h.\\': he B,.\\'EC}= Pg{TEB,Se C}. needonly be provedfor setsA e .91of the form assertion (2) The second A = {hs: hE B,sEC}, Be \037(G),CE\037(imS).
Ps(ds)= cJ J>;(B)
r r \037
But thishasbeendonealready under (1).
D
39.20Discussion (Fisher'sfiducial distribution). (1) Let us first discussthe caseof a full shift.HavingobservedWE Q onemight beinterested in particular . the covering of the randomset(J) B where Be 8d(G).It isobvious probability a moment's reftectionthat by = \037{WE Q:ge w. B} PCB), Be ii,ge G. Theinteresting feature of thisequation is the fact that the right hand doesnot dependon g e G. Therefore,PCB)can be interpretedas the probabilitythat w. B coversthe unknown parameterg E G. (2) Now, we turn
a similar to structuremodelsand attempt to establish
relation. The preceding Theorem39.19 showsthat
P:{hE G:g E h . B} PIl(B), B E
g E G,S E im S. Again, the right hand doesnot dependon g E G and thereforethe number of the random setT(w) . B,given Sew).We PS(WI(B)isthe covering probability into may put the covering probability =\"
PS(WI(B)=
\037\037:'I
\037,
(T(w). B).
has beencalled It is by R.A.Fisher. fiducialdistribution of the mostimportantcasein statistics wherea posterior an analysis experiment is possible without using a priordistribution. of fiducialdistributions and posterior distri(3) Thereis a strongrelation butions.Let us provethat,independentlyof the choiceof SandT the fiducial with the posterior distribution for Ar. distribution coincides = Choosew h.'iand B E (G).Then
Thedistribution
\037\037:\037
\037
J B
dP
d{;.,-.i>s)
-1
(g w) ).r(dg)= J s
= J I B (g -I)
dP
d().(.A)
-1
(g hs) )'r(dg)
dP
d(;',,,,Ps) (ghs) A(dg) dP
= L1(h)J 1s(hg1) (gs) A.,(dg) d(A,... Ps) =
.1
(1'(w\302\273
J ls(T(w)'g-I)
d(A\037=
Ps)
(g'S(w\302\273;'(dg).)))
39.Thc structure
model
209)
F for ).,is) distribution Hence,the posterior dP r w ).,.(dg) (g * f's) d(J.( F(w,B) = dP -I ' (g w ) 1_,.(dg) J
, -_.
I
\302\273
d(J.;-:-P.<;)
-
J 1B(T(w)g-1) d()'\037\037-P\037
_.-.-
J
d(
)'\037\037
(g'
(g.
S(w\302\273
S(w\302\273
A(dg)
).(dg)
\037<;)
= J 1B (T(w)g t)PS(I4)(dg) =
P\037\037c:J)(B),)
Be a. This leadsto a simplerformula for the covermg probabilities: p.<;(6))(B)= F(w, T(w) . B), WE !l, BE forA,. satisfy an invariancecondition, distributions namely (4) The posterior where we fl,
a.
. gB)= {>S(w) (Tw' B) = F(w, B), we fl, Be \037(G), ge G.
F(gw,gB) =
pStgW)
I
I
\302\253T(gw\302\273'
of the coveringprobabilities: This leadsto a further modification PS(C/)) (B)= F(S(w),B), wED, B E \037.
It must be stressedthat the fiducial methodis only a computational T satisfying Assumpestimator methodand doesnotdistinguish any particular choiceof T shouldbesuch that the coveringprobabilities tion39.2.A suitable T are large in somesense.E.g. if for someB E \037(G) there existsan estimator (5)
and satisfyingAssumption(39.2) F(w, T(w) . B) = supF(w, g . B), leG)
wED,
then the confidence regionl'(w).B hasmaximalcoveringprobability. of remarkscan beextendedliterally to the computation (6) The preceding risksgiven S(w).Since(4) implies conditional 1 J1B(h-g)P;(dh)= J1B(h)F(s,dh),SE imS, Be \037(G), ge G, risk with respect to a lossfunction W: G [0,00)is the conditional
-
I)F(s,dh),seimS, geG. fW(g-lh)PgS(dh)=JW(h In otherwords,given Sew) the conditional risk of T(w) is = J W(h- I . T(w\302\273F(w,dh).))) J W(h-I)F(S(w),dh)
210
Chapter7: Theory or Estimalion)
But the latterisnothingelsethan the posterior risk of T with respect to A., and lossfunction (W(g-l.\302\273gI!G' Thus,in caseof structuremodels,we have
the
risk as a conditional risk without of the posterior obtainedan interpretation In this sense, with Bayesestimates any needof a priordistribution. generalized conditional risk functions. can viewed be asestimates respectto A., minimizing In the followingit is our aim to find estimators which are optimalwith risk functions. It will turn outthat thisisthe same to the unconditional respect risks.) classwhich minimizesthe conditional
of estimates). Remark (Decomposition We keep the notationof (39.21) Then we have for every bounded,\037-measurablefunction Theorem39.19. fQ-+1R \037(fldi)= Jf(hS)\037s(dh), gE G.
another Consider an estimate e E (E,G).Forevery SE im S we may construct s x estimate e':(h.B) e(hs.B).(h, B)e G a.It isclearthat (le fJl (E G),and
.
\037
\037
we have)
ePg = Je'P;Ps(ds),ge G.
Thisfollowsfrom)
'
I';
= JJ e(hs,B) (dh) Ps(ds) (B)Ps(ds) J (e 1':> = J e(en,B)Pg(dw) = (e\037) (B), BE91.)
.
39.22Definition.An estimate e E (jt (E,G) is equivarianriff e\037 = eg (eP)for every g E G.)
39.23Definition.An estimate e E [Jt (E,G) isa convolutionkernelif e(gen. gA) =
\037(en,
A)
for all en eO,g e G, A e (G).) \037
-
It is clearthat every convolutionkernel is equivariant.A non-randomized estimate K: 0 G is a convolution kernel iff K(gen)= g K(en),g E G, WE Q.)
39.24Remark (Decomposition of convolutionkernels).If e E at (E,G) is a
convolution kernel then we may write e(en,B) = e(T(w)Sew),B) = e(S(w),T-1 B), (l) E Q, BEtM(G). If we denote Q'(B):= u(s,B). SE imS, Be \037(G) then we have e(hs,B) = Q'(h-1 B), hE G, se imS, Be a (G).It followsthat in this case(with the notationof Theorem39.19).) \302\253(1)
(B)= JJ e(hs,B) J>;(dh)Ps(ds)
(e\037)
\037
HQ\037(h-lB)J>;(dh)Ps(ds)
=J
(I';.Q')(B)Ps(ds).)))
39.The slruclurc model
211)
Thisexplains the term \"convolution kerne)\".Conversely, eachstochastic kernel defines convolution k ernel a to .\037EimS, .n-+Q$I\037(G), according e(w,B) 1 =
QS(W)
{T- (w).B}.
The followingis an extension of the convolutiontheorem to structure
models.)
39.25Theorem.Suppo.\037e Ihal G i.'I amenable.Then for every equivariallle.'Ilikernel s Ho QSIfJI (G),s E im S,such mate e E 9t (E,G) there existsa stochaslic that) Q\037
=
J(p;.QS)Ps(ds),gE G.)
Proof For Q e (Jt (E,G),se im S,define Q\037:
BHo A.
1
Be a(G), ne\037, ,(K,,) H e(hs,hB)P;(dh\302\273).,(dg), K...)
where (K,,) is a summingsequence. Then it followsfrom Theorem38.14 that)
.
hm \"-o\037
1
sup
1
BI!!I(G)A, (Kit )
j (e$P;)(gB)
A.,
..)
(dg)
- (ps..Q:)(B) = 0,
S E im S. Let IE
denote
AltU'.
q\302\273
= \037 J II ) \"1
A,(K
and)
\"..)
S
[JJ/(g-lh) (e P;)(dh)cp(s)Ps(ds)];',(dg)
(F.Q:)(dh) q>(s)Ps(ds).)
B,.U:cp) = JJf(h)
of bilinear. functionsand by By weak compactness
Theorem6.11there is an
ne N, accumulation pointQ:sHoQ$I\037(G)of the sequence QII:sH>Q\037I&f(G), t x on ttoo(G) L (imS,Ps) and a for the topologyof pointwise convergence such that) s; N subsequence No lim A,,(f,cp) = lim B,.(f,cp)
\"eNo
liE
=
1\\:10)
H/(h)(PS.QJ (dh) q>(S)Ps(ds))
for allIE ttoo(G)and q> E V (imS,Ps).Sincee is equivariant we have P, fE\037oo(G},nEN,) A,,(f,1)=le and therefore) P = Hld(PS * QI)Ps(ds), fE '6'00 (G).)))
le
212
Chaptcr 7: Theory of Estimalion)
Since)
1= 1q P = JJ 1d(?* QS)Ps(ds) it foHows that Q:s Q\" can be adjustedPs.a.e.to be a non-degenerate kernel.This provesthe assertion. 0) stochastic \037
39.26Corollary.Assume that G is amenable.Then for every equivariamestimate there existsa convolutionkernel with the same distribution. Let W: G -+ [Ot (0)be a lower semicontinuous functiont let the lossfunction be (W(g-l. and definethe risk of an estimate l!E fJl (E,G) by \302\273g6G'
W(g-ly) e(wt dy) \037(dw), gE G.) It isclearthat the risk of an equivariantestimate isindependent ofg E G.If e is S a convolution kerneldefinedby Q:s\037 Q I.1l(G) then \037e\037:=H
\037e\037
= II W(g-ly)(\037 * QS)(dy) Ps(ds) = H f W(g- 1xy) E>;(dx)Q\"(dy)Ps(ds) = f If W(xy) PS(dx)QS(dy)Ps(ds) = IS W(y)(ps QS)(dy) Ps(ds).)
.
39.27CoroUary.Supposethat G is amenable.Then for
every estimate
kernel Q:s QS (G) such that e E (jt (Et G) there isa stochastic supU-;e\037 JJ W(y) (PS * QS)(dy) Ps(ds) \037
I \037
\037
gEG)
fUllctionw:) for every level-compact
copythe first part of the proofof Theorem39.24.Then we obtaina possiblydegenerate S E im 5t satisfying) kernel s lim A,,(J: 1)= Hfd(}>\"* Ps(d.\037), fE rrJoo(G).
ProofWe
may
\037
Q\037t
Q\037)
\"t;
No)
Let QS:=
Q1\037G)
measure Qbt whereQb(G)> Ot and QS an arbitrary probability
on \037(G),elsewhere. Then it followsthat inf fe\037
\037
gEG
for every
lim A,,(f,1)
\037
\"eNo)
ffd(ps
fe fto(G)+.Hence,we
fE \037b(G) supf()\037 g.G)
\037
.
QS)Ps(ds)
have for every
Jfd(ps.QS)Ps(ds).
The proofis finishedby Lemma 6.4.
0)))
function level-compact
39.The structure:
model
213)
of the lossfunction W. Note that Q is independent in as Section 3 8 we conclude that in caseof an amenable groupan Similarly We have to discuss the optimalequivariant estimateis a minimax estimate. how to find uniformlyoptimalequivariant estimates.) question
39.28Lemma, Supposethat there is a measurablefunction tp: im S -+ G such that for every y E G
J W(xrp(s\302\273 PS(dx) J W(xy) PS(dx) \037
Then for the convolution kernel K:
m.-T(m)
Ps-a.e.) tp
(S(w\302\273,
WE
fl, the following
assertions holdtrue: K is (1) uniformly optimalamongallconvolutionkernels. (2) If G is amenable then K is uniformly optimalamong all equivariant estimates andisa minimaxestimate.) Proof (1)The risk of K is J
W(K(W\302\273
P(dw)= II
W(Xlp(S\302\273
PS(dx)Ps(ds).)
From the assumption it followsthat K minimizesthe risk over all convolution
kernels.
of (1)and of Corollaries 39.26and 39.27. (2) This is a consequence
0)
It is rather obviousthat the estimates definedin Theorem39.24are (\037)!HGunbiasedif \037:=W(g-I.),g E G.) The final assertion of thissectionwill be that uniformly optimalequivariant estimates can be obtained as generalized for A,. In view of Bayes estimates Discussion 39.20(4) and (6),theseare just the estimatesminimizing the conditional risks.)
39.29Theorem.If a non-randomized generalized BayesestimateK with respect to ..t,is a convolutionkernelthen it is uniformly optimalamong allconvolution kernels.Hence,if G is amenable then K is uniformly optimalamong all and minimax.) equivarianrestimates
we obtainthat a dominated structure modelis ).,Proof FromLemma 39.12 = Discussion 39.20 shows that F integrable. \037. Now, let\"bea non-randomized, generalized Bayesestimate. By assumption, K is even a convolution kernel.We notethat for all WE and hE G)
!l
f
W(gT-t (w)h)
(dg) = fW(g-th)Pf\037\037(dg) = J W(g-th)F(w,dg).))) pS\302\253(O)
214
Chapter 7:Theory of
Estimation)
The defining propertyof generalized impliesthat for every Bayes estimates
heG)
PS(tAJ) (dg) J W(g TJ W(gh) p\"\037(tAJ)(dg) &'-a.e. SinceK is a convolutionkernel,we have T- 1(w) K(W) = where II' the condition of Lemma 39.28. satisfies This provesthe assertion. 0) t
\037
K(\302\243O\302\273
\302\253(0)
Ip(S(\302\243O\302\273
Let us consider someexamples. the caseof Example39.3. 39.30Example.Consider DenoteII = dd:. 11
2 = 11.11 and that F satisfies appropriateintegrability Thenit followsfrom Example37.6(1),that conditions.
(1)Supposethat K(X)=
W
- a) J.dda) xEH,)
f ah(x
fh(x\037a)l.dda)-' isa generalized and a convolution kernel.Thereforeit isoptimal Bayesestimate in the senseof Theorem39.29. If W is notboundedthen assume that F (2) Let P = NHand W subconvex. satisfies conditions. In view of Example 36.11 appropriateintegrability (1),a estimate isobtained generalized Bayes by minimizing) z'
W(z' - a) e-'2llpd.x)-aIiZ A.L(da). z' 1
L. we seethat for every x E H. From Anderson's Lemma (38.21) \037
f
E
is a a i t estimate and convolution kernel. Therefore is Bayes optimalin generalized the senseof Theorem39.29. K
= PL
39.31Example.Now we consider . the caseof Example39.4.Let II = d d\037
2 and (1)Supposethat W = 11.11 conditions. Thenit followsthat)
=---( - a ) JIh
F satisfies appropriateintegrability
X a ---1 \037d(1).,Jda)
----------.
fJ ah
K2(X)
I.H
that
(J
X
-
0-\"+3
1
XE
H,
(--(1 ) (;n+.jd(1).dda))
and a convolution is a generalized kernel.thereforeoptimalin Bayesestimate the senseof Theorem39.29. If W isnotboundedthen assumethat (2) Let P = NHandW besubconvex. Then a generali7.edBayes F satisfiesappropriateintegrability conditions.
isobtainedby minimizing))) estimate
40. Admissibility Z'
-a
1
of estimators
215)
1
2 Z'H H W ---;;--e-a}IIP1.(Z)-1I11 a,,+1ldda)dG, z'eL, ( )
for every x E H. From Lemma 38.21we see that the integralover L is minimized by of (1 > O. Hence\"2 = PI. is a generalizedHayes z' = PI.(x), independently and thereforeoptimalin the senseof Theorem39.29.) estimate
40.Admissibility of estimators) In this sectionwe discussbriefly somewell-knownfacts concerning admissibility of estimates. Let E = (Q,.r:I,{Pc:tEL})bea dominated structure modelwherethe operatwe denotethe a Euclidean is For convenience ing group space(L,<., of L on by + \".Let W: L [Ot 00) be a subconvex function.) operation
!l ..
.\302\273.
\037
40.1DefinitioD.A generalized for (W (t Bayesestimate a Pitman estimate for W.)
-.
\302\273,
ELandA L iscalled
40.2Examples.The followingis a particularcaseof Example 39.29.Let Q = and L = IR. Definethe operation by \037\"
t
+ W = (t + Wit
..., t
+ w,,),
t E IR,
WE
R\".)
is) (I) If W(t) = t 2, t E R, then the Pitman estimate
K(W)
=
Jt
If
(ll)
(jAH
dP
S
- I)dl
-, -dP
d).H)
(w
wEQ, tE IR.
t)dt
=
Itl, tE R, then any Pitman estimateIS a median of the hencesatisfying) distribution, posterior (2)
Wet)
a(w)
dP
(w
JI)di;, + 0() dP
- I)dt
-f -(w-t)dt d).H)
1
2'
wEQ.)
0()
-
If Wet) = 1 1(0.()(I), IE IRt then the Pitman estimates are calledmaximum probability estimates sincethey satisfy))) (3)
216
-
Chapter 7: Theory of
J
dlH
x\302\253(J))-c
(4)
IfP =
- t)dt= uR J-CJ dl'dP,- - t)dt, !1. s+c
dP
X((J)+(
Estimation)
\302\253(J)
SUp
-
!
(J) E
\302\253(1)
H)
N R \" then in any
caseK(W) = n
i:.
i-t)
Wi>
WE
a, isa Pitman estimate.
40.3Remark. Extending the precedingexample,supposethat Q = H, If P = NH then (H,< ., a Euclideanspaceand L H is linearsubspace. of K = PL within the classof equivariant Theorem38.28provesadmissibility estimates and (takef= idL).Thisresultcouldbeextendedto P * NH if \037
.\302\273
-
:\037
of Corollary38.23,usingthe decomposition of assumptions But evenfor the Gaussian caseP = NH it isknown that K = PL Theorem39.19. isinadmissible assoonasdimL > 2,(Jamesand within the classofallestimates W
satisfy the
Stein [1960]). We are interested in this sectionin admissibility within the classof all estimates. In view of the precedingremark we confineourselvesto the case = dimL 1,i.e.L = lit)
40.4Definition. An estimate if for any further (l e .t1t (E,R) isalmo.rtadmissible (1E rJi (E,IR) the inequalities) estimate J.v,0'J; J.v,eJ;, tE!R,) \037
imply)
=
J.v,(1p,
all t E R.) 0P, for ).-almost
J.v,
40.5Lemma. Let W(t) = t 2, t E R. Supposethat (l E fJl (E, isalmostadmissibleand satisfies H (x t)2e(W, dx) P'(dw)< 00, t E IR. and uniquely n01lrandomized Then it followsthat (l E 9t (E,R), (l is admissible, determinedby itsriskfunction.) \037)
-
-
momentsare finite it followsthat e(.,{+ 00, co})= 0 ProofSincethe second (E R. Hence(!E 91(E,IR). Then p'-a.e.,
-
J (Jlx \037
e(w,
P'(dw) H (x 1)2e(w,dx) (w) < 00, tl
dX\302\2732
P,
that implies Jlxlu(w,dx)< co p'-a.e.,IE IR.) estimator))) Hence,the non-randomized
t E IR,
40. Admissibility
of tstimators
217)
,,=Jxe{.,dx) is well-defined. yield Easy computations -- W; K .p, = SUX2(l(W,dx) (f X(l(w,
-
W; Q.p,
P'(dw), t E\037,)
dX\302\2732J
we arrive at) and,since(l isalmostadmissible
p'-a.e.,tEfR.) Jx2Q{.,dx)-<JxQ(.,dx\302\2732=O This means,that Q is non-randomized sinceit coincides with K. Now, let (1E .rJt (E, be an estimator satisfying) \037)
W;O'.P,\037W;K'p'. tEIR.)
Then a is almostadmissible, too,and can be replacedby a nonrandomized estimator \"1:Q R. It followsthat
-
-
I(K1 -/)2d'p'= J
J
K1
(
+K 2
-I)
-.-!-.f
2
dPr + J
=
K
(
I
2 f (\"t
K
2 I
) dp'
- I) d'p'+ 2 s 2
I
(K
t)
2
dp',)
IEI1i.)
This implies of K and K l' that by admissibility S (K 1
whenceK 1 =
-,,)2dp,= 0, ).-a.e.,) K
Ie IR. 'p'-a.e.,
0)
The followingassertion containsthe basicideaof Blyth'smethodfor proving
-
Let h(t) = s(lt I).IE IR, be a positiveand continuous admissibility. probability such that s: i s w e Later, shallput density [0,00) (0,00) decreasing. h(t)
\037
n(1
\037
/2)
' IE R,
and h(/) =
vk
exr
( ), \037
40.6Theorem.Lei(!be the Pitmanestimatefor W If lim (J J.v,eP,' h(8t)dt inf J J.v,a P,'h(a)dt)= O. ,-0
-
tI (\037(E,R))
then Q is a/most admissible.)))
IE\037.)
21H
Chapter 7: Theory of
Estimation)
ProofAssumethat (1E tit (E,A) satisfies)
tER.
J.Y,G.p,\037 J.Y,u.P\"
If (!, denotesthe Bayes estimatefor Wand the prior distribution Jl\" : t 1-+th (tI), 1E R, then we have
\037'
J J.Y,eePrJl.(dt)J J\302\245,a.P,Jl.(dt) J J.Y,e\037Jl.(dt). \037
\037
the sets) Consider Ma.h =
{III< a, J.Y,eP,> J.Y,u P, + b},
Then)
a> 0, b > O.)
-
J J.Y,qP,JlI(d/) J J.Y,C!tPrJle(d/) \037
J.Y,(1.P,)Jle(dl) f(J.Y,e.P,-
\037
bJ..(MII,b)
\302\243h(\302\243a).
The assumption then implies that
l(Ma.b) = O. Hencethe assertion.
0)
Let us consider 40.7 Discussion the caseof Wet) = 12, tE lit If (Stein[1959]). and has a finite second is) moment,then the Pitman estimate PI\037l iscentered dP
K(x)=Jtdl (x-t)dt=x-JzP(dz)=x,xEIR.)
The Bayesestimates K. for the priorsJl.are given by) KC(X) =
J If;(x,dt), x E
IR,)
the posterior of E and /J.e' Thisgives) where f; denotes distribution J J\302\245,KPrh(\302\243/)dt- J J.Y,Kc.P,h(et)dt
- (dx)Jl. - -t1 JI (x) - (dx) 1 -2t(x= - H (x t 1 - 2/(x- f;(x,dt)Ps(dx)/J.,(ds) = - HI (x - \":(X) t 1 - - (x = ./J..(ds) t II (x = -1H (x P'(dx)/J.c(ds). = 1 JJ (x t
-
1)2.P,
2
(dr)
K;(X)
(K.
t)2 .P,
Ke(x\302\273)'p'(dx)/J.l(dt)
2
K.(X\302\273)
2
K;(X) 2K,(X)
Ke
(X\302\273
t)))
2
K,(X\302\273
\037(dx)
/J.,(dt)
of estimators
40. Admissibility
Since)
219)
..
-I)dP d).(x I)h(el)dl X-K\302\243(X)=J(X-I)\037(x,dl)= ---iP----.J (x -I)h(el)dl) J (x
.
d).
we obtain
J I-\302\245,KP'h(tl)dl-J I-\302\245,K,P,h(et)dl =H
- t) dP (x - l)h(t/)dl dP dJ. -------n. ) Jl (x - t)h(tt)dtdx dj>- J (x t)h(r.I)dt
J (x
2
-
(
fiX
h(t(z-
= J ($zh(e(z f
p(dZ\302\2732
x\302\273
P(dz))
x\302\273
dx.
in a suitable choiceof h such that the last Thus,an admissibility proofconsists tendsto zerofor t O.) expression
-.
Let Wet) = t 2, t E [R, and PI\037I such that 40.8Theorem(Stein [1959]). J z P(dz)= 0, J Z2 P(dz)< 00, Then the Pitman eSlimateK = idR is admissible and uniquely delermined by itsriskfunction.) the preceding discussion and notethat) Proof We continue
-
- -
= (Jz(h(e(z h(tx\302\273 (J zh(t(z h(tX\302\2732 P(dz).) J Z2 P(dz)'J (h(t(z x\302\273
p(dZ\302\2732
\037
- -
x\302\273
p(dZ\302\2732
x\302\273
for t Moreover, J
h(\302\243(z
\037
-
1 we have)
x\302\273
P(dz) P( \037
-1,+ 1).)
-I
h(tx+y).
inf \037)I'\037
1)
Now, we specifyh(x):=\037.' _l_-i'XE lit Since 1t
1+x
1+ (z = 1, 1+ +21)2 z) z-\037 lim
there issomec> 0 such that) inf
h(z+y) c'h(z), ZEUi. \037
-1\037)';:;1)
Hence,we may
estimate)))
220
Chapter 7: Theory of Estimation)
-
(J zh(r.(z dx J P(dz) J Const.J Z2 P(dz). JJ (h(t(z x)
-
h(\302\243(z
p(dZ\302\2732
x\302\273
- -
x\302\273
\037
h(tx\302\2732
/ h (tx)dxP(dz).)
we have) Moreover,
- -
J (h(e(z x) = Jh 2
h(ex\302\2732
(\302\243(z
h
x\302\273j
/ h(ex)dx
2 (ex) dx ----J
= J \"--
h(r.(x+
-
-
h(f;x)dx 2J h(f;(Z
x\302\273dx
+ J h(f;X)dx
h(\302\243x)dx
z\302\273
1 1 1 1+ e2 (x + Z)2 2 2 =it (1--t-e)y.dx-\037S-f+\037xidx 1 1 xz + Z2 dx = =1 dx = J 2 \"2-') Z2,t (1+ e2 2)2 n (1+ X )2 \"x
;
\302\2432
2\302\2432
\302\2432
\302\2432
Thus,we obtain) J
(J
zh(\302\243(z
-
h(e(z_
x\302\273
x\302\273
z.)
dx = e.Const.(J z 2 P(d
p(dZ\302\2732
P(dz)
Thisprovesthe assertion. we obtaina As a consequence
\302\243Z2
2
\302\273
0)
model.) generalresultfor an arbitrary structure
40.9Theorem(Stein[1959]).Let E = (fl,d, {p,:t E R}) be a structure model convolution kernel 10:Q R, and assume that there existsa non-randomized for W (t) = t 2 . t C IR, whichhas afinile fourth moment.Then the Pitmanestimate and uniquely determinedby itsriskfunction.) is admissible
-
Proof Let So be the maximal invariant basedon and F the posterior is (by Discussion for Then the Pitman estimate distribution dz) F(So(oo), dz) = J (z + T{oo)= J zF{oo, = 10(00) Po (lOld) (00). = J z\037o(W)(dz) + 10(00)
l.
\037
(39.20\302\273)
-
1O{oo\302\273
j
It followsthat T has a finite fourth moment,too. Let S be the maximalinvariant basedon T and denoteby p's,t E IR, s E im S, of T given S = s. We have to show that) distribution the conditional
-
-
lim (H(T 1)2dP,h(tt)dt H (K,
,-.0)
-
= t)2 dP,h(tt)dt)
for the prior#l\" where K, is the Bayesestimator
0,
d;.!(x) = eh(ex),x e IR.
We)))
40. Admissibility have)
-
of cstimalors
221)
-
H (T 1)2dp,h(8t)dl= J H (z t)2P'$(dz)h(t;t)dtPs(ds),) and)
H (Kt
-
= J H (Kt(Z + S) t)2 dP,h(r.t)dt
-
t)2 P,s(dz)h (tt)dtPs(ds).)
z H Ke(Z + .\\'), z E R is the It isclearthat for every fixeds E im S the estimator for the conditional t E R}) and Pl' experimentEs = (R,\037, {P'$: Bayesestimator 40.7to each of theseexperiments, it remainsto show Thus,applyingDiscussion that)
dx Ps(ds)- 0. PS(dz) J h(t;(z 1 1 and considering the proofof Theorem Specifying (x) = . l+x2' lim JJ
\037J
\037
-=
\037\037(\037
l\"'O
\037\302\273
\037$\037d\037\302\2732
x\302\273
h
X
E
IR,
7t
(40.8)it sufficesto provethat Ps(ds)< 00.) J (J Z2 But this is an immediateconsequence of 4 H Z4 PS(dz)Ps(ds)= J T dP < 00. P'(dl\302\2732
0)
casewhere Q = andP = Vo,,' In this case,the Now, we turn to the special result of Theorem40.8can beextendedto a considerably larger classof loss \037
functions.)
40.10Theorem(Blyth [1951]). LetE = (R, a, {VI.,: t E \037}),f= that W is separating and offinite order.Then the identity uniquely determinedby itsriskfunction.)
andassume is admissible and id\037
ProofThe proofis dividedintothree parts. of Theorem40.6is (1) In the first part we shallshow that the assumption 2 . = satisfiedfor h (x) 1/V2n exp ( x /2),x E Sincein the prescntcasethe we have to show that) identity is the Pitman estimator
-
\037.
-
= OCt). inf J U';Qp, v o.1!c(dt) J W(ltl)vo,ddt)(IE 91(F.,/II))
To compute inf
Vo , IfL (dt) J U';e.P,
(l'\037(E.A))
we consider the mixtures) Pr(A x B):=J B)))
V\"
I (A) Vo. \037_
(dt),
A
E
g4(R),
BE91(R),
Chapter 7: Theory of &timation)
222 F.
> O. The marginal distribution for the first coordinate is Jlt(A x R) = VO,I-t/t(A),
and desintegration of with J..lc
J..lc(A x
A
E
\302\243i(\037),)
yields) respectto the first coordinate
B) = J \037(x, B) J..lc(dx, R),
A E
.sf(R), BE .sf(\037),
A)
is given by) wherethe posterior distribution
\037(x,
B) =
JB f
dV,.1
d)' (x) Vo.t/l(dt) = v
dVr 1
di
it
X
E
IR,
(x) VO.1/c(dt))
\302\243i(IR).
(1+c).I/(1 +c)(B),
_
x.-.1x+t) , x E
The estimates Ke:
BE
xf
-
IR, t;
> 0, minimize
SI-+J W(ls tl)f;(x,de), SE R,) for every x E R and therefore inf
J J.\302\245,l!p'Vo,t/c(dt)
QE \037(E,A))
= J W;Kc1;vo.1It(dt)
= II
I: w( ) .0.1
=J
W
I
(dx).0.1{,(dl)
\302\243
x
( }I1+t)) vo..
Integrationby partsyields
-J ( = 2 J (4)( + - + 2q/l + - 1) J a'
VO. 1 (dt) J W(111)
f. 1 V\037.
W
!X)
4>(
+
) Vo,t
a}l1+t\302\273
W(d!X)
[O.a\302\273)
\037
f.
(O.a>
cp(+a) W(da) =
))
-
sincex> y > 0 impliesIcl>(x) identity isalmostadmissible.)))
cl>(y)j
\037
Ix
-
YI
O(e)
cp(y). This provesthat the
40. Admissibility
of estimators
223)
In the be such that supJY,eoP,\037JW(ltl)vo,,(dt). C?oE9t(E,R) fER = 0 A-a.e. secondpart of the proofwe shallshow that eo(x,{oo}) Thecasewhere W isunbounded isobvious. that W isbounded. Let Suppose = = 0> 0 be arbitrary and defineA {Oo(.,{oo})> c5}.We provethat A(A) O. For every x E A we obtain) (2) Let
JJ W (Iy ;;;;
-II)
+c),l/U+cl (dt) Qo(x,dy)
v x/U
.IW(co)+ (I
-
D)f
-
c5)J W
= 0 W(oo) + (1
w(
i:-.-I )
(dt) V'/lI+.\"1/1\"')
III
(V1+E) vo,ddl) ,)
which implies, again for x E A,)
III
JW HW(ly-ll)vx/U+C),l/U+c)(dl)Qo(x,dy)(VI .t't:) vo..
\037
\037
- III ) (vt\037 b(W(cc)- J W(ltl)vo,1 o(W(oo) J W
vO,!
(dl\302\273
-2 J
c5(W (00)
(0,
(d!\302\273
\037(-o:)
W(do:\302\273
=:017> 0,
1)
sinceW is separating. If we integratetheseinequalities by vO.1 + lIt then we obtain)
-
J JY,eoP,vo.tl,(dl)
( ) _'1 0,1+1/,)
> c5n . v
inf j.>u'<E.llh
J \037QP'VO,I/C(dt)
A
-
part (1)of the proofthat VO,t + lIt (A) -= OCt), t O. Hence = O. A{A) In the third part we may thereforeassumethat eoE Yf(E, R), For (3) simplicitywe denote) which implies by
g(C1,Z)::: J (\037(Z-C10:)+\037(-Z-C1:x)-2\037(-u<x\302\273W(d<x), (O,
where(1> 0,Z E IR, It iseasy toseethat this iswell-definedsincethe integrand is and forfixed(J > 0,Z E IR, dominated non-negative by a W-integrablefunction. calculation and Easy integrationby partsyield)))
224
Chapter 7: Theory of Estimation)
H W(I Y
-II)
V#n
=HW(
+').lIn+.)(d/) Qo(x,dy)
y--- t\037 ) x
1+8
V1+e
= J C?O(X,dy}g(Vl+\037, vT+ ey
-J W( I:. -) I
v./n +0,.I/('+0'(d/))
Vo.l(dt)(}O(X,dY)-JW I\037 VO'l(dt)
-
(Vl+e)
. \037)
XE
R.
1+\302\243)
If we integrateby J
J-\302\245,C?OP'
vo, I
\037
lIt
then we obtain)
-
inf
1/c(dt}
V O.
/?
E
tI(E.GI)
J
. (dt}
O 1/c
J-\302\245,(}p'v
= JJg(1f1+8, 1f1+8y-
\037) C?O(X,dY)VO'l+l/C(dX). 1+\302\243)
By part
(1)of the proofand Patou'slemma we arrive at
-
H lim infg(vr+e.V1+ ey c-O
+
1 J/TX
f.)
)
llo(x,
dy\302\273)'(dx)
=
o.
It is not difficult toseethat lim inf g(Vi+ r.,0\"+ ty
c-O
---
\037-
.co
)
V\"f+t
= g(1,y
- x)
continuous and that z 1-+g(1,z), Z E R, is a non-negative function which vanishesexactlyat z = O. This provesthe assertion. 0)
.
Now we return to the generalcase.Let E = (H,.\037(H), {p,: tEL})where H isa = Euclidean N and L a is linear function.) space,p, f., H , tEL, f \037
40.11Theorem(Blyth [1951]).Supposethat
\037
the lossfunctionW
and of .finite order.Then fo PL is admissible for determinedby itsriskfunction on (ker/)1.)
is separating
El(kcrf)I and uniquely
Proof Choosea unit vcctor e.lkerf, eEL, such that fee)> O. Define the F = (H,Y4(H), {NH * E IR}).It iseasy toseethat,,:= subexperiment fo PL eA\037:)'
is sufficientfor F. We have)
G1=\".(F)= (R,
\302\243f,
A e R}). {vAllfll.1I/1I2:
Thereforethere existst E 9t (G,
\037)
QA\302\253(}O(.,
such that
B}IK)= t(K,B), Be a,)))
40. Admissibilily of eslimalors
where Ql:=NH * Sle.-,A E
If W(ly \037
225)
lit The decision functionf satisfies)
-A.llflll)t(.,
dy)dv J W( yl) vO.llfll2 (dy), I
A
II/II,1I/1I2
;.E
\037.
it followsthat t ( ., B) = By Theorem(40.10)
BE .41,and we obtain 18;'-a.e.,
Il = J1B oKdN H Jeo(.,B)dN
A
A)
wheneverA J
E
K-1 (\037), BE
\037.
t particularlyA = K (B)it followsthat Choosing
(/0(.,B)dNH = N H (,,-I(B\302\273
1(-1(8))
and hence
(}(x,B)=l if fopdx)EB An-a.e.,BE\037.
0)
isobvioushow to generalize Theorem40.11 tothe situation considered at the endof Section 34.))) It
Chapter8: GeneralDecisionTheory)
decision in orderto classify Statistical theory has beenfoundedby Wald [1950] of the vast variety statistical methods The basicideaisto by generalprinciples. consider statistical as in the sense o f von Neumann's experiments games theory of games.In specialcaseswe have introducedthe ideasof decision theory 7 and 33.Early importantresultsof the theory were the already in Sections minimaxtheoremand the complete classtheorem. Decision theory becamea after it was possible to formulate general framework for statistical purposes importantideasof classicalstatisticssuch as sufficiency and asymptotic terms.Bothproblemshave bec:nsolvedby optimalityin decision-theoretic LeCam[1964 and 1972]followingpreviousattemptsby Blackwell[1951 and
and Hajek[1972]. 1953],
consider the problemof sufficiencyin Chapter9, and the foundations of i n 1 0. I n the w e asymptoticoptimality Chapter presentchapter only prescnt thosebasicfacts ofclassical decision which have already beenknown to theory Wald, but we do this as generallyas requiredlater on. Theobjects of statistics are statistical Toapply linearfunctional experiments. the of families have to beembedded m easures analysis underlying probability intosuitable linearspaces. In caseof a dominatedfamily thisisdonein Section 24by taking the L I-spaceof an equivalentC1-finite measure. Thegeneralization this of procedureto an arbitrary experimentleadsto the L-spaceofDefinition in Section 41.3.The basicfacts on L-spaces are presented 41. In Wald's decisiontheory the strategies of the statistician are given by kernels. functionsis not topologically However,this setof decision complete. it is convenient to i t Therefore, completc by considering arbitrary bilinear decision 42. functionsas generalized functions. This is the subjectof Section However, only decisionfunctionsin the narrow senseadmit a statistical 42.5and 42.7dealwith the roleof kernelswithin the Theorems interpretation. classof generalized decision functions. functionsare bilinear decision Generalized functionson the productof the continuous decision o fbounded functions on a space spaceandthe topological In thiscontextthe boundedcontinuous functions L-spaceof the experiment. Forstatistical however,it isnecessary play the roleof lossfunctions. purposes, to admit alsothe upperenvelopesof boundedcontinuous functionsas loss the class o f to the setof functions. h as tobeextended loss functions Therefore, In Section lowersemicontinuous functions. 43we showthat after thisextension the classof decision function in the narrow senseremains sufficiently large We
(Theorem43.5).)))
41.Expcrimcnlsand their J,-Spaccs
227)
Section45 containsa functionalanalytic version of the minimaxtheorem in Section 46 to the framework of decisiontheory.In which is specialized 47 we prove a versionof the complete classtheorem. Section
1b a
large extent we follow LeCam [1974].For the theory of Banach
In orderto avoidthe Mackeylatticeswe refer to Schaefer[1970and 1974]. topologyusedby LeCamin the originalproofof Theorem42.5we employ 45 we followHeyer [1969]. Lemma42.4.In Section of thischapteris devotedto the general theorem of Hunt The final section and Stein.Insteadofa transfonnation groupoperatingon the samplespacewe startwith a groupof stochastic on the L-spaccof the experiment. operators This generalizes the first situation,but will turn out later to be almost We it. to distinguish equivariant from strictlyequivariant decision equivalent In caseof shift experiments orstructure functions. models,the latterconcept with that of convolution coincides kernels. We provea versionof the theorem ofHuntandSteinwhich coversthe originalversion in testingtheory (Section of Sections 38and 39.Practicallythe whole theorems 32)and the convolution of this sectioniswell-known.Further information on thissubjectiscontained in Luschgy [1984].)
41.Experimentsand their lrSpaces) Let T:t=0 be an arbitrary
set and E = (Q,.t'/,9) with 9' = {p,:lET} an
experiment. t E T, assertions of decision Measuretheoretic theory often are valid p'-a.e., i.e.9'-a.e.If E isa dominatedexperimentthen by Lemma(20.3)there existsa measure Poldsuch that Po\"\"rJI.Thus,the expression rJI-a.e.may probability be replacedby Po-a.e.The suitablefunctionalanalytic frameworkis then the
pair L1 (Q,d, Po) and LCX)(Q,d, Po},i.e.the L-spaceand the M-spaceof section24.However, if E is not dominatedthen the questionfor a suitable functionalanalyticframeworkhasa morecomplicated answer. It is the aim of the presentsection to find the right generalization of the conceptof L-space. Let ca(Q,.flf) bethe setof all boundedsignedmeasures on (Q,d).We begin with a reinterpretation of our originalconceptof L-spaces for dominated experiments.)
41.1Lemma. Let EE 8(T)bea dominated experimentandlet \\lId bea q-finite \\'. Then) measuresuch that L(E)= {JlE ca(Q, \037
\"\"
d):Jl
Proof Obvious.
0)))
\037
v}.)
228
Chapter 8:General DecisionTheory)
we needa representation of L-spaces which does For a suitable generalization not rely on dominating measures. For this,we prove)
41.21beorem.LeiE E cf(T)bea dominated The'lJ1 e L(E) iff for experiment. every u e ca(D,.9I)
a 1. lET, implies u 1.JL) P\"
Proof Let Po E C(fJ)be such that Po u
1.1;,T, tE
iff
- 9.Thenwe have
u 1.Po')
Indeed,if u 1.Pothen there isA E .sIsuchthat Po(A) = 0 and u(A') = O. This implies1;(A) = 0 for every t E T and therefore.P,.lu for every t E T. If converselyp, 1.u for every lET then let AI Ed be such that 1;,(Ai) = 0 and 00 a: = :;: = i Then A where E 0 , N, Po(A) = 0 and a(A;) Po (Xjp',\" () A, satisfies i=1 ;-'1 u(A') =0.Hcnccu1.P o. we Therefore, needonly provethat J1 E L (E) iff (11.Po implies (11.Jl. If JlE L(E) then Jl Po,and u 1.Poobviouslyimplies (1.1Jl. If converselyJ1 satisfies the condition then let A E d be such that Po(A) = O. Since J1IA 1.Poit = followsthat J1IA 1-J1 which is only possibleif J1(A) O. HenceJ1 Po, i.e. \037
\037
\037
J1 E
L(E).
We
take the preceding assertion as startingpointfor our conceptof L-spaces.)
0)
41.3Definition(LeCam[1955]). Let Ee t!f(T)be an Then the L-spaceof E is
arbitrary
experiment.
L(E)= {,uE ca(D,.w'):(11.,uif (11.p',lET}.)
41.4Corollary.TheL-spaceL (E)ofan experimentE isa Banachlatticefor the
variationalnorm. if every subset which has an Recall,that a Banach latticeis ordercomplete upperboundhasa leastupperbound.It is a well-knownfact that baeD,.w') Banachlattices and ca(D,d) are ordercomplete (seeDunfordand Schwartz,
[1969]).)
41.5l..emma. The L-space of an experimentis ordercomplete.) Then it is clearthat Proof Let {Jli:i E l} L(E) and denoteJ1 = sup i,I ,ul' issuchthat u 1.p, for every t E Tthenwe have .w').If (1 e ca(D,.w') ,u E ca(D, \302\243;
to)))
41.Experimentsand
their
L-Spaccs
229)
showthat u .1p.SincePi E L(E),i E /, it followsthat (1.1Pi'i E /, i.e. (1 n PiI = 0, iE I. Now, from P- Pi- Ipi!.iE I, we obtainJl-nIO'I= O. On the I
\037
I
I
\037
otherhand) P
+
n 10'1= suppt v 10'1= suppt +
id
id)
I
(1 = P + + 10'1 I
+ 0) implies P 1-0'.Hence0'.1Jl. Our next aim isto obtainmoreinfonnationaboutthe relationbetweenL(E)
and ba(D,.PI).)
41.6Lemma. Each O'E baeD,.91)admits a
such that)
0'= unique decomposition
O't
+ 0'2
(1) O'IEL(E), u 2 EL(E)1, (2) if u 0 then U 1 0, (12 O.) \037
\037
\037
of any decomposition satisfying (1)isclear.To showthe ProofThe uniqueness of the decomposition we need only consider 0' O. We define) existence u 1 = sup{tEL(E):0 t O'}, \037
\037
\037
(12=0'-(11') It is immediate that 0'1 0, (12 O. We have to show that 0'21L(E).Let e + (11E L(E). P E L(E)and e = (12n pl.It isclearthat e E L(E)and therefore = But e + (11 (12+ Ut (1which implies by the maximalityof U 1 that e = O. Hence0'2 .1Jl. 0) \037
\037
I
\037
41.7Theorem.There existsa positive/inear operatorT: ha(!l,.flI) -.L(E) satisfying the followingconditions:)
(1) I'll= 1. (2) (Tu) (1)= 0'(1) if (1 (3) TIL(E)= idL(E)') II
\037
O.
Proof Let 7t E L(E)be arbitrary such that 7t 0, 7t(I) = 1.For0'E baeD,d) define T(O')= 0'1 + \"2(1)n, using the notationof Lemma 41.6.Then T is a For(1)we note (2) isobvious. positivelinearoperatorsatisfying (3).Condition \037
that) II
Tull= \037
+ u 2 (1)nl (1) 11'0'1 (1)= 10'1
10' (1)n(1)-Iul + 0'2 (1)= 10'1 1 (1)+ 10'21 (1)= 110'11 1
which implies II I'll
\037
1
1.Togetherwith (2) it follows(1).
a)
The topological dualof L(E)iscalledM-spaceofE and is denotedby
M(E).)))
230
Chapter8:GeneralDecisionTheory)
41.8Lemma. Supposethat E 8(T)isa dominated experimentand let a u-finitemeasuresuch that 1/ v. Then M(E)= L\302\253)(fl,.9I.v). \302\243
- definesan element
\037'
I,vi be
In general, to every f !Rb (D,d) cPf M (E) according = epr(a} a (f), a E L(E).If the:mappingf...... CPr issurjectiveontoM(E)then E iscalledcoherent. are Dominated experiments coherent. \342\202\254
We finish this
\342\202\254
the relationbetween sectionwith an assertionconcerning
for arbitrary experiments.) M(E)and !i',,(D,.PI)
41.9Lemma. LetE bean arbitrary experiment. Then) {cp/EM(E):0
\037
f l,fe!R,,(D, .PI)}) \037
isdensein) {q>EM(E):0
\037
cp(u)
\037
u(D) wheneveru
\037
0.UE L(E)}
for the L(E)-topology of M(E).) Proof If the assertionis not true then there existssomecP E M, 0 (1(D)if (1 0, (1E L(E),and (J E L(E)such that .PI).) u(f) S c t < c cp(u) if 0 t.fe !i',,(D, for someC E R, e > O. This implies q+ (D) C e < c cp(q) cp(q+) q+ (D) Hencethe assertion. 0) which isnotpossible. \037
\037
-
\037
-
\037
cp\302\253(1)
\037f\037
\037
\037
\037
\037
42.Decisionfunctions) (fl,.91,\037)be an experiment and let D be a topological space.The D We consider as spaceof possible \037o(D). Recallthat a Markov kernele:D x \037o(D)-+ [0,1]iscalleddecision decisions. functionsisdenotedby functionforE and D.The setof all decision D).) Let E =
Baireq-fieldof D isdenotedby
91(\302\243.
42.1Remark. Let e e 9t (E,D).Forconvenience we denote IfJJ.l= H/(x)l!(w. dx)J.l(dw). function e E rJt (E.D) definesa bilinear L(E).Every decision to functionP : \037\"D) x L(E) -+ R according if IE rc,,(D), J.l E Q
J.lEL(E).))) fJqU:J.l)=fQJ.l if fE<:&,,(D),
42. Dcci\037ion funclions
231)
42.2Definition. A generalized decision function for E and D is a bilinear function fi: \037b(D) x L(E) R satisfying the fonowingconditions:) (1) IP(f,Jl)! IIfilullJlIIif fe (G,,(D). Jl e L(E). if f 0, Jl O. (2) P(f,Jl) 0 if JleL(E).) (3)) P(1,Jl)= Jl(Q)) \037
\037
\037
\037
\037
The setof an generalized decision functionsisdenotedby 11(E.D).It isclear that for every (1 E D) we have fiQE 91(E, D). The weak topologyof 91(\302\243,
on \037b(D) x L(E).) dI(E,D) is the topologyof pointwise convergence
42.3Theorem.The set91(E,D) of allgeneralized decision functionsis convex and compact for the weak topology.)
@ L(E)., Proof.dI(E,D) isa convex,closedsubsetof the unit ballin \037,,(D). which is compactfor the (i,,(D)@ L(E)-topology of \037,,(D)+@ L(E)+. 0)
The next theorem isbasicfor the whole theory.For the proofof this theorem of Lemma41.9.) we needthe followingextension
42.4Lemma. Let E be an arbitrary experiment.Then the set of all linear functionson L(E)k which are of the form) Ie
(OJ)1
-iSk \037
i'\"
Ie
o /; 1, L It functions \037
:E (1i(/;)'
\037
\037
i.\037
\037
1)
1,hE !I' ,,(D,.cI),1
\037
i
\037
k, is densein
the setof aI/linear
1)
\" \302\253(1i)1
L
\302\253(1i),
(
M(E)\",
1)
\"
which satisfy
i
L =1
\037
Cl'j\302\253(1)
L(E)-topology of
(1(1)if (1 0, (1E L(E),(forthe k-foldproductof the \037
M(E\302\273).)
Proof First,we notethat
every
continuous linearfunction on L(E)\" is of the
fonn) \302\253(1')1
--
\037'\037k......
k
L
--) E M(E)\".
(
\037,\037\"
It isthen clearthat every linear we may identify M(E)\" = (L(E)\..") Therefore, function on M(E)k which iscontinuous for the k-foldproductof the L(E)topologyof M(E),is of the form)))
232
Chaptcr 8:Gcncral DecisionTheory) k
(
L CPi(O'i), (O'i)1i-I
\037iq
E
.)
L(E)\".
is not true then there exist(
i-I
\037
\037
c -c
k
L
S:i\037a
O'j(/,)
\037
Ie
\037
j\0371
L
cPj(O'j)
i=1)
\"
wheneverO
\037
f,\037
1 \037i\037k, 1,i-I L f, 1,f,e!R\"Q,d), \037
and for someceIR,
\"
6> O. Denoting0'0:=U = i
0';(1)S; c
-
O'i
\302\243
we obtainfrom Lemma (6.7)that
I)
Ie
\037
Ie
L cp/(O';) 0';(1) L cp/(O'j) i-I) i-I \037
\037
which is not possible. Hencethe assertion. 0)
42.5Theorem(LeCam[1964]).Theset{{3(/E 1I(E,D):(!E
91(\302\243,
1I(E,D)for the weak topology.)
D)}isdensein
{[I,lz,..'
...,
Proof Let fJ E (}I(E,D).Choosee > 0 and finite sets ,1m} rc,,(D) and {Jt),Jt2. Jt,,} L(E).We will construct some E 91(E.D) such that \037
\037
fJ
l\037i\037m, l\037j l1.)
if 1{3(f;,JtJ)-f,(1JtJ/
\037
n. By Lemma 6.6 \037j a finite partitionof the unity (p,,)I there exists on D such that each/;, 1 i p n, varies lessthan on the supportof the Pie' 1 k p. By Lemma (42.4)
W.l.g.we may assumethat JlJ(Q)= 1 and Jlj 0, 1 \037
\037
\037
\037,,\037
j
\037
\037
\037
choosefor every k E {I,2,..., p} somehIe E \037 (Q,d) such that) t
IP(p\",Jlj)-Jtj(h,)I<
if l\037j\037n.
m
3p(L 1=
IIhllu)
1)
This impliesthat)
-
p
L hit) JtJ(Q) Jlj(1e=1
-
p
= JlJ(1 L hit) 1e=1
<
\302\243
m
3
L
i'\"
Ilhllu
1)
to Lemma42.4the functionshIe may bechosensuchthat 0 According p p 1 k p, and hIe 1.Let us denote110= 1 L hIe' \037
\037
\037
k=1
\037
-
Ie-I)))
\037
hIe
\037
1,
42. Decisionfunclions
Let x\" e Supp(PIe),1 p
Q(w, B) =
L
\037
k
\037
P, and let Xu be arbitrary. Then we define)
h,,(w) I B (x,,), BE \037o(D),we Q.
1:=0)
it followsthat for every we have (!e 9P.(E,D).Moreover Obviously, \037
n,) I
233)
-
-
p
J\037)
1 \037j
p
L {J(hPI:' fJ(Ji,Jlj) Ji(1Pjl = \"=1 Pj) L .f,(x,J pj(h,,)! \"-0) I
-
p
I
\037
L ({J(J;P'\" Pj) {J(h(x..)p\", Pj)1 1e=1
-
p
P
max 1IJ;lIuJlj(1L hIe) + L 1J;(x..) -J;(x,,) P(P..,Jl) Jlj(h..)+ l;:Ii;:l'\" 1:-1) 1:=1 I
P
<
Ui-.f,
(x,,\302\273
IP(I:\037I
P
\342\202\254
I:\037I
Ie
Ie
1
+_
\342\202\254
I
e
1J;(x\IIP(p,,,Jlj)") Jlj(h,,) + 3)
P
P( L:;t Pt.Pj) + L= IJ;(xt ) 3
_ \037
-
P p\", pj)1+
\302\243
P
3 L J; \"=1) II
3
\037
I
e.
0
II\"
e \037(E, D):Q e [}teE. D)}are not identical. The followingassertions describe the mostimportantcaseswhere they are.) In general, the sets (E.D) and \302\243I
{P\037
42.6Lemma (Farrell[1967]). and D is a locallycompact If E i.'i dominated with b ase then countable e space for elJery P \037(E, D) there existsa kernel
- [0,1]
(1:Q x &I(D)
such that
P(!\"tp) = f(/}l
if
Ie
thisisTheorem6.11. ProofEssentially,
0)
42.7Theorem.Supposethat E is dominated and D is a locallycompact .t;pace with countable base.If P E \037(E, D) issuch that fl-+P(/' P,) isa Daniellintegral on \037,,(D)for every t e T, then thereexisl.5(/ E fJt(E,D) satisfying) P(!\"tJl) = feJl
if f E CC,,(D),Jl E L(E).)
Proof Apply Lemma42.6and notethat 1(1P, = sup{f(!p,:0 '5.f '5.1,f e
1)
for every t E T.Thisimp1ies (!(w,D) = 1 \037-a.e.Eventually improving() ona set of 9\"-measurezeroyields(/ E 9t(E,D). 0)))
234
Chapter 8:General DecisionTheory)
42.8Coronary.If E is dominatedand D tM(E, D) = {Ptl : (l E 9t(E,D)}.)
IS a
compactmetric spacethen
ProofThisfollowsfrom Theorem42.7sinceon a compact spaceevery Radon measureis a Daniellintegral on (jb(D). 0) of non-randomized thissectionwith a characterization decision function E fJt(E,D) is non-randomized if functions. Recall that a decision E t E T and all BE f or (D).) (l(.,B) to,I} \037-a.e. every We finish
f1
\302\243to
42.9Theorem.ut e E fJt(E,D).Thefol/owingassertions areequivalent: (1) (1 isnon-randomized. and E L(E). (2) f2 eJ.l= (fe)2J-lforevery fE .P,,(91 o (D\302\273
(3) f2(lp'= (f(1)2 p'forevery fE \037,,(D) and t E
J-l
T.)
Proof(1) (2):Firstwe note,that by assumption lET. f2(1\037 = (f(l)2P, if fE .P,,(9I o Fix fE .P and let A = {f2(1 (f(1)2> O}.Since[2(1-([(1)2 O it = followsthat P'(A) 0, t E T. Let E L(E).ThenJ.lIA.iP, for every t E T and by definition of L(E)thisimplies that J.l .iJ.L. HenceJ.L(A) = 0 and therefore f2eJl= (f(l)2J.l. (2) (3):Obvious. \037
(D\302\273,
-
,,(ato(D\302\273
\037
J-l
L\037
\037
(3)
\037
(1):It followseasily that
= (18(1)2 lET, (1B)2(1p' P\"
wheneverB is\037,,(D)-opcn. Thesystemof allsetsBE\037o(D)with this property is a Dynkin systemand thereforecoincides with Ylo(D), 0)
43.Lowersemicontinuity) Let f D
-.
IR
be a lower semicontinuous function which
is
boundedfrom
below.If D is a separable metric spacethen there is a sequence (
J.lEL+(E).) feJ.l= sup{q>eJ.l: q> \037f, q>E\037,,(D)}, This motivatesthe followingdefinition.)
43.1Definition.Iff D
-. islowersemicontinuous and boundedfrom R
below)))
43. Lower semicontinuity
235)
then)
P(f,J.l):=sup{{J(ep, J.l):ep \037f. ep E tjb(D)},Ji E L+ (E).) and D if a compactmetricspacethenfor every 43.2Theorem.If E dominated P E \037(E, D) there existsfl E rJt(E,D) such that P(f, J.l) = f (l functionf D ;R which is for every J.l E L+ (E) and every lowersemicontinuous i\037
-
J.l)
bounded frombelow.)
if D is a compactmetric spacethere existsa Proof Sincetj(D) is separable as in Theorem f f. If (!E 9t(E,D) ischosen (cp,,)S CC(D), \037f. sequence it a nd E then follows t hat (42.7) J.l L(E) ep\"
ep\"
= sup{J(ep\",J.l) = P(f,J.l). fflp.= sup ep\"flJ.l liE liE '\ I\\:
The lastequality holdssinceP(.,J.l) is a Daniellintegral on \037(D).
0)
The followingtheorem is relatedto Theorem42.7.)
43.3Theorem.Supposethat E is dominated andD is a locallycompactspace
-
countable base.Let P E 91(E,D). If there existunbounded level compact functions 1,:D IR such that 0< p(/\" P,) < 00 for every t E T then fJ = p\037for somee E Yt(E, D).) with
Proof In view of Theorem42.7we needonly show that 13(., P,) is a Daniell for every t E T. Let t E T and choosef. > O. Then integral on rc,,(D) 1 K:={I, - PC];,p,)} \037
\302\243)
iscompact and)
g.=-B 13(1\"P').(t - tIC) 1
\037
/\"
Sinceg is lower semicontinuous we obtainfJ(g, P,) P(1 1 , P,) E. Thisprovesthe assertion. 0)
-
IC
\037
{J(f\"
P,) which
implies
\037
In general, an element {J E rJI(E,D) neednot beof the form P = {J(/ for some (!E fJt (E,D). Nevertheless,onemay provean assertionrelatedto Theorem 43.2.Recall,that if D is locallycompactthen \037o(D)is the spaceof all)))
Chaptcr 8:General DecisionThcory)
236
continuous functionson D which vanish at infinity. We have \037o(D)= \037oo(D) for the uniform topology.)
that E isdominated 43.4(.emma.Suppose and D isa locally compactspacewith countablebase.Then for every {J E :1l(E,D) there existsa decision function E such that (J (E.D) P(f,J.L) fQJ.L if ferco(D)+,J.LeL(E)+.) \037
\037
Proof Let P E (E.D) and chooseQo accordingto Lemma 42.6.Let A = {weD:eo(w,D\302\273O} and let PlfB(D)be an arbitrary probability measure. \302\243I
-
Define)
-l!o(w, --..B) eo(w,D)
e(w,B) =
I PCB)
.
tf
OJE
A
,
if w, A ,
Be \037(D).Thenit followsfor fE CCo(D) \"
J.L
E
L(E)t,that
= (J(f,J.l)=f(}oJ.l fLf(x)eo(w,dx)J.L(dw) = Lf(x)eo(w,dX)J.L(dw) dx) Lf(x)Q(ro, J.L(dw) =feJ.L. 0 J Jf(x)Q(ro,dx)J.L(dro) \037
!!
\037
D)
43.5Theorem.Supposethat E is dominated and D is a locallycompactspace base.Thenfor every p E !fI(/::,D) there existsa decision with countable function (J E
\037
(E,D) such that feJ.L
\037
PU:J.L), /l E L(E)+,
for every level-compactfunctionf D
-
\037
IR.)
-
and in 'tfb(D).Then supf f is in ProofFirst,letf D IR be level-compact isan immediateconsequence of Lemma43.4.Now, rco(D)+and the assertion
be arbitrary tevet-compact. Let
= {gE
letf D
\037
R
.Yf
+)
J.L):g .Jf'} {J(f,J.L) sup{{J(g, sup{geJ.L: g E K} = fQJ.L) Levi-theoremis valid. sincefor eJ.L the generalized \037
\342\202\254
\037
0)))
44. Risk funclions
237)
44.Riskfunctions) Let T 0 bean arbitrary set.Assumethat D is a topological space.The space D will play the roleof the decision space.) =t=
= (J-v,),(T of functionsJv,: D -+ lET, which iscalleda lossfunclion. A are boundedfrom belowand aJo(D)-measurable. = W if is attributed to each has the (w;),,,T W;, lET, property.) property
44.1Definition.A
family
W
\037,
44.2Definition.Let {3E f:I(E,D) and let
be a lower semicontinuous loss function.The risk of p at lET with respectto W is p(W;, The risk of (!E fJt (E,D) is the risk of {3t1 E (JI (E,D) and is denotedby Jv,(!\037, lET. The functionsI P(W;, -Pr), lET,and I W;Q lET,are calledriskfunclions.) W
\037).
\037
\037
44.3Definition.If problem.The sets
W
P\"
is a Jossfunction then (T, D, W) is cal1eda decision
R(E,D, W) = {(PCW;,
T: p E
P'\302\273'E
.r:\037
(E.D)}
\037
IRT)
and)
Ro(E,D, W) = {(W;Q P,),c:T: Q E 9f (E,D)} IRT are calledthe risk selsof the experimentE for the decision problem(T, D, W).) \037
44.4Lemma. If W isa bounded continuous lossfunclion Ihen R(E, D, W) s; isconvexandcompaclandRo(E,D, W) = R(E,D, W).) 42,3and 42.5. ProofThisfollowsfrom Theorems
\037T
0)
Let us consider the behaviour of risk setsunder restrictions of the parameter T -+ IJ(To the projection.) space.Let To s; T and Jet PTo: R
44.5Theorem.Let W bea bounded conlinuous lossfunction.If To c T then) D, PTo(Ro(E, PTo
(R(E,D,
W\302\273
W\302\273
= Ro(ETo' D, W), and = R(ETo D, W).)
'
isobvious. It fol1ows that) D) the first equation ProofSince(}leE,D) = (}l(ETo' R(ETO' D, W) =
Ro (ETo'D, W)
2 PTo(Ro(E, D,
= P.rD(Ro(E, D,
W\302\273
W\302\273
= PTo(R(E,D,
W\302\273)
sincePTo iscontinuous. On the otherhand compactness of R(E,D, W) implies that PTo(R(E, is obtain))) and we D, too, compact, W\302\273
238
Chapter 8:Gcncral DecisionTheory)
D. R(El,o' D, W) = Ro(ETo,D, W) = PTo(Ro(E. = PTo(R(E,D, C PTo(R(E. D, Thisprovesthe assertion. 0)
W\302\273
W\302\273
W\302\273.)
45.A generalminimaxtheorem) In thissection we present a generalversionof the minimaxtheorem in a purely functional to decision analyticframework.Theapplication theory.in particular the minimaxtheorem of decision in the next paragraph. theory isconsidered Let T be a locallycompactspaceand let ..ItAo(T) be the spaceof all o n measures with compactsupport.Each m E probability dI(T) (T) definesa linear functionon fIl(T).The spacefIl(T) is topologized with the on compacts.) topologyof uniform convergence ..Jt\037o
45.1Example.Let T 0 be an
arbitrary = topology.Then ...If\037o(T) STand \037(T) = =t=
set endowedwith the discrete
\037'f.)
45.2DefinitioD.Let M
the function) m
H tpM(m):=inf{Jldm:IE M}, mE ...ItAo(T).)
45,3DefinitioD.Let M c
f.'\
45.4Remark. If M \037(T) is compactthen isclosedin CC(T).To see this,let g E (M).Then for every compactK T and every I:> 0 \037
e\302\253M)
(X
\037
g(/)+ if IE K} * 0.) The family of closedsets isdirected from belowand therefore n {Fx.c:Kcompact,> O} + (/). Fic,e -= {hEM:h(/)
\037
\302\243
FK..E
\302\243
This provesthat g E a.(M).
Recall that a set M c \037(T) is calledsubconvex if for 11E M, 12E M and e (0,1)there is13e M such that 13 (XII + (1 (1.)/2'Every convex setis if M isdirected from belowthen it is subconvex, subconvex. too.If Moreover, M is subconvexthen :x(M)is convex,)))
(X
\037
-
45. A general minimax theorcm
239)
45.5Remark. Forevery M 'I/(T)and m E JI\037o(T)we have \037
tpM(m) = tpa(M}(m).
isthe basicseparation theorem.) The followingassertion M2 S; <6'(1'). Assumethat M2 45.6Theorem(LeCam[1964]).Let M1 S; <6'(1'). ns areequivalent: issubconvex.Then Ihefollowinga.(}serlio (1)Forevery fE M1 there;ssomege fJ.(M2) such that g \037f
for every m E \"I(\037o(T).)
(2) tpM:(m) 1I'M.(m) \037
(m) for every Proof (1) => (2):Assertion(1) impliesthat tp\037(m) 1I'M. m E \"I(\037o(T).Remark 45.5 proves(2). => (2) (1): Assumingthe contrary t hereexistsfeM1 suchthat g \037ffor every ge(J,(M 2). Hencef\037fJ.(M2)' SincefJ.(M2) is convex and closedthere is a continuous linear functionL on <'G(T)such that) \037
L(f) < inf {L(g):g E (J,(M2)}.) Since(J,(M2) = M2 + (1')the linearfunctionL must bepositiveand henceis \037+
a Radonmeasure on \037oo(1'). in such a way that It can benormalized L(h) = J hdm, hE f({(1'), for somem
E \"I(\037o(T).Thus we
obtain
tpM.(m)< 1I'\302\253()f2)(m) = tpM2(m) which contradicts (2).
0)
f({(T),M2 WeT). Assume that M2 ;s subconvex and (M2) isclosed. Then the followingassertions are equivalent:
45.7 CoroUary.Let Mt
\037
\037
fJ.
(1)Forevery fe M there issomege M2 such that g \037f 1
(2)
1I'M2(m)
\037
tpMI
(11'1) for every
m E
(1').
.J(\037o
apply the precedingresultsto obtain a version of the well-known minimaxtheorem.) We
45.8Theorem (Minimax Theorem). LeI l' be a convex, compact subsetof locallycomJexspaceandlet Y bea convexsubsetofa vectorspace.Assume that the followingconditions: f l'x Y .....IR satisfies t-J. I is continuous andconcaveon l'for every y E Y. (1) f(t,y) (2) Y t-J.f(t, y) is convex011Y for every t e T. Then)
inf )leY
inf f(t,y). sup f(t.y) = sup leT leT )IcY)))
240
Chapter 8:Gcneral DocisionThcory)
Proof The inequality ex
E
( - 00,00]suchthat
..
inf f(l, y) sup ICT )'c
is obvious.To prove the reversed inequality let
\037\"
\037
a.
Y)
We have
to show tha
inf .HY
t)
SUp[(I,y)\037a. 1\302\2431\
If = 00nothingisto be proved.If a < 00 let M1 = {a} WeT) and ex
\037
M2 =
{[(., y):
Y
E
Y} <;; \037(T).
The setM2 issubconvex. For m E J(\037o(T)let 1m be the barycenterof m on T. Then we obtain = inf {Jgdm:g E M2} 1S'M:(m) inf f(1,\", y) a = 'PM.(m). \037
\037
)IE
Y)
of gEa(M2) such that g Theorem(45.6)implies the existence e > 0 there isYc E inf y\037
Y
suchthat
[(.
sup[(I,y) a + t. \037
r lET)
, Yc)
\037
\037
+ e and therefore
\037
a. For every
0
45.9Corollary.Assume Ihat the condition... Then o[Theorem45.8aresatisfied. there is to E T such that inf suplet,y) = inf [(to,y). lET YEY
y\"y)
on T. If there is ProofThe function t 1-+infY let,y) is uppersemicontinuous YI: If to E T such that inf [(to,y) = 00 then the assertionis immediate. )'f: r
< Cl), lET, thc:n the function is boundedon T and attainsits followsfrom Theorem45.8. 0) supremum.Now, the assertion inf f(t,y)
)'E Y
46. The minimaxtheoremof decisiontheory) sectionto the particularcasewhich of the preceding Now, we apply the results = t in Example45.1. Let E (U,sI, {p,: isdescribed T})bean experimentand an arhitrary subset.<11 .<1I(E, D a topological decision D) space.We consider which is weakly compact and convex.))) \342\202\254
\037
46.The minimax
46.1Lemma (LeCam[1964]).Assume that
function.Let To
\037
W
theorem of dccisiontheory
241)
is a bounded,continuousloss
T be an arbitrary subset.For every fE IRTo the following
assertions areequivalent: (1) ThereexistsP E such that f(t) {J(\037,P,)for every t E To. (2) Sfdm inf {JP(\037,p')dm:P E for every In E STo') \037
\037
\037
\302\243j}
ProofIt isobviousthat (1)=> (2).To prove(2) => (1),we defineM. = {f}and M2 = {(P(\037, that 'PM2(\037 ) 'PM1(m)for (2) implies J;)'ETO:PE \037}. Condition we obtainsome every m E STo'FromTheorem(45.6) g E (M2) suchthat g f -+ it followsthat M2 is Since[JI is compactand PTo:IRT (RTo is continuous compactand hence <x(M2) = <x(M2)' Thisprovesthe assertion. 0) \037
<X
\037
46.2Corollary.The assertion of Lemma46.1;salso(,alidif W isan arbitrary lowersem;continuous lossfunction.) (1)=> (2) isobvious.Toprove(2) => (1)let 1/ bethe set ProofThe implication of all bounded, continuous lossfunctionsV W. Condition (2) implies \037
Jfdm inf{J{J(V\"p')dm:PE fM} HenceLemma(46.1)impliesthat for every V E 1/) for every m E SToand V E \037
f.
{PE\302\243j:P(V\"P')\037f(t)if
Since iscompactand \302\243j
n
VE\"\
{PE
\302\243j:
tETo}*0.)
f isdirectedfrom abovewe
P(V\"
P,) \037f(t)
if
tE
1\037}
have)
*0
and any elementof thisintersection satisfies (1).
0)
46.3Theorem(MinimaxTheorem). Forevery lowersem;co1ltinuous lossfunction
W)
inf 6E9I
supP(\037,P')= sup inf IE T
..
ProofThe inequality we denote) ex
\037..
SP(J.\302\245\"p,)m(dt).
\"'EST 61:91)
isobvious. For the proofof the reversedinequality
= sup inf S fJ(\037. p')m(dt). \"'EST (te!!l)
If = 00 then nothinghasto be proved.If < 00then we put f(t) = ex, t E T, and apply Corollary46.2. 0) ex
rx
46.4CoroUary.For every lowersemicont;nuous lossfunctionW there is some Pw E
\302\243j
such that)))
242
Chapter8:General DecisionTheory.)
sup Pw (H-;,P,) = inf supP(H-;,P'). reT reT) 'c\037
Proof The assertionhas been actually provedIn the proof of Theorem
46.3.
0)
andD is a compactmetric space. 46.5Theorem.Assume that is dominated Then for every lower semicontinuous lossfunction W the followingassertions hold: inf supH-;\037l{= (1) SI(E,D) sup inf JH-;uP,m(dt). r r: T \"'E ST DI(E. (IE \302\243
(IE
(2) There issomeqw E H-;ewP, = sup re T
91(\302\243,
inf QE\037(E.D)
D))
D) such that) sup H-;ep,. IeT)
of Theorem43.2if ProofThis is an immediateconsequence Theorem46.3and Corollary46.4. 0)
it
is appliedto
46.6Theorem.Assume that E is dominated and D is a locallycompactspace countable base. T hen with for every level-compactlossfunction W the following hold:) assertions (1)
inf
sup H-;eP, = \"'EST sup inf
Qell(E.D)lET
(lEtl(E.D))
(2) Thereis someQw E
= sup h.T \037eW P,
91(\302\243,
inf
S H-;ep'm(dt).
D) such that) sup H-;ep,. T)
q..Iit(E,D)
I\037
..
Proof The inequalities are obviousin both cases.The oppositeinequalitiesfollow from Theorem43.5if it is appliedto Theorem46.3and Corollary46.4. 0) \037\"
47. Bayessolutionsand the completeclasstheorem) function PoE .f!4(E,D) is 47.1Definition.Let mE ST' A generali7.eddecision calledBayessolutionfor mE ST and W if -Pr)m(dt)= inf J P(Jt;,p')m(dt). J fio(H-;, e91(E.
,
D))
lossfunction Wand forevery))) It isobviousthat for every boundedcontinuous
47. Bayes Soil/lionsand
the
complete
theorem
cla\037\"
243)
mE ST there is a Bayes solutionfJo E :!I(E, D). This is due to the fact that on the compactsetf1I(E,D).) iscontinuous fJ t-t> J p(w\" p')m(dt)
lossfunction.Let 47.2Lemma. Assume that W is a lower semicontinuou.'i E mE ST' Thenfor every {J &feE, D) J {J(W\" where
p')m(dt)= VE'f\ sup J fJ(V\" p')m(dt)
f is the setofallbounded,continuow'lossfunctions
V
is obvioussincefor every I:> 0 we may ProofThe assertion p(w\" P,) P( P,) + for every t E supp(m). 0)
that
v\"
\037
\037
w.)
find
V
E
f such
\302\243
note the easy fact that for boundedloss functionsW the function m f P(w\" p')m(dt) iscontinuous on E A (T),for every fJ E [fI(E, D).) We
t-t>
(X
S\302\253,
47.3Theorem.For every lower semicontinuous lossfunctionWand every
me ST there existBayessolutions.)
lemma it followsthat the mapping Proof Fromthe preceding p t-t> J P(w\" p')m(dt),fJ E &j(E,D), is lower semicontinuous. Henceit attainsits infimum on the compactset f1I(E,D). 0)
47.4Corollary.For every lower semicontinuous lossfunction Wand every
meST)
inf
,.,9I(E.D)
J (J(W\"
- sup 1Ie!l(E.D)) inf J
\037)m(t)
fJ(V\"
VE1\"
p')m(dt),
where .y denote.'i the setof allbounded continuouslossfunctionsV
..
ProofThe inequality (X
= sup Ye'f\"
We
inf
\037\"
..
isobvious. To prove
J {J(V\"
\037\"
\037
W.)
let
p')m(dt).
6\342\202\254iI(E,D))
observethat for every
VE
\"j/'
the sets
{pe(}I(E,D):J {J(V\" p')m(dt)
\037
(X}
* 0.
Hencealsothe intersection of all thesesetsisnon-empty and therefore contains somePoE (}I(E,D) such that sup J Po(V\" p')m(d/) ex. YC'f\ \037
Now the assertion followsfrom Lemma47.2.
0)))
244
Chapter 8:General DccisionTheory)
--
If W is an arbitrary lower semicontinuous lossfunction then in general Ro(E,D, W) :f::R(E,D, W). The lower envelopesof Ro(E,D, W) and The preceding that however,implies assertion, R(E,D, W) neednot coincide.
-
at
leastthe followingis valid.)
47.5Corollary.Forevery lowersemicontinuous lossfunction andevery mE ST inf Jlf9l(E.D)
where
J fJ(W;,
p')m(dt)= Yet\" sup
inf
J V,ep'm(dt),
\037Etit(E.D))
l'denotesis tosetprove of allbounded class complete
continuouslossfunctionsV theorem.)
the
Our next aim
the
\037
W.
A generalized 47.6Definition. decision function fJo E fJd(E, D) is an extended Bayessolutionfor the lossfunction W if for every e> 0 there is m, E 81, sueh that)
J fJo(W;, p')mr(dt) inf {JP(W;, \037
+ e: P E 91(E,D)}.) p')ml(dt)
Thefollowing isa rather weakversionof the complete classtheorem. However, it
is valid for unboundedlossfunctions,)
47.7 Theorem.Assumethat
pE
W
isa lowersemicontinuous lossfunction.Forevery
D) there existsan extendedBayessolurionPoE gJ (t:,D) such that
14(\302\243,
Po(W;,P,)\037fJ(W;,P') if
tET.)
Proof Let C1E .
-
meS\",)
J (P(W;, P,)
(U0f\302\243
tp(m\302\273
-e)m(dt)
\037
tp(m) for every mE ST')
we obtainPoE \037(E, D) such that Now from Lemma (46.1) Po(W;,P,)\037P(W;,P')-\302\243if
tET.)
of e it followsthat) Fromthe definition inf
(J Po(W;,\037)m(dt)
-
tp(m\302\273
= O.
\",eST)
HencePo isan extendedBayessolution.
0)))
48. The generalizedtheorem of Hunt and Stein
245)
47.8Corollary.Assume Ihal W is a bounded, lossfunclowersemiconl;nuous lion.Thenfor every p E \037(E, D) and every E A (T) Ihere is a BayesSolulion Pa E 91(E, D) such Ihal Pa(W\" P,) fi(W\" P,) if IE t;X
t;X.)
\037
ProofThe proofof Theorem47.7remains vaJid if Sr is replacedby particular E A (T).We obtainplJ E (E,D) such that PaO'v', P(w\" P,) if t E rJ., and rJ.
S\037
for a
,\037
\037)
inf
-
\037
(JPIJ(\037' \037)m(d()
IIIt:S.)
tp(m\302\273
= O.
Theorem47.3it followsthat for boundedloss In view of the notepreceding and thereforethe infimum functionsW the function'I'isuppersemicontinuous is attainedon Sa' 0)
ClassTheorem.) The followingis the Complete lower 47.9Theorem(Complete ClassTheorem). Assume that W isa bou1lded. Ihere exisls a weak T hen E loss semicontinuous function. for every P f1I(E,D) limit of Bayessolutions PoE \037(E, D) such that Po(\037, .P,)
Proof For every
t;X
\037
(J (J.v\".p,)
EA
for every t E T.)
(1')there is a Bayessolution suchthat
(JIJ(\037' P,) \037 (J(J.v\"
fJlJ
P,) if
tE rJ..
Let 910 be the setof all Bayessolutions in \037(E, D).Hencefor every
t;X
EA
(T)
Iion {O'E tM(E, D):O'(\037,P,) P(\037,P,) if (E rJ.} =F 0.) For reasonsof lower semicontinuity thesesetsare closedand thereforetheir intersection is non-empty. 0) \037
48. The generalizedtheoremof Hunt and Stein)
d, {.P,:t e T})be an experimentand G a locallycompactgroup countablebasefor its topology.The Borel-a-field of G is denotedby
Let E = (D,
with
\037(G).)
-
48.1Definition.A positivelinear mappingM: L(E) L(E) is a siochastic operatorif
II
MO'l/ = I/al/
if aE L(E),0'
\037
O. A family (Mg)geGof stochastic)))
Chaptcr 8:Gencral DecisionTheory)
246
isa groupif Mg, 0 g2 = Mg,\"Mg1 for gl'g2E G and Me = idL(E)'It isa operators if g 1-+(MgJl)(A) isa (G)-measurable measurablegroupof stochastic operators for all Jl E L(E) and A
E
d.)
on L(E).The 48.2Definition.Let (Mg)gFGbe a groupof stochastic operators experiment is invariant under (Mg)gcGif MgP,E sET}for a11g E G and {\037:
tE
T.)
48.3Example.Let Ebea dominated experimentand G a groupofmeasurable transfonnations g:Q Q where (g,w) 1-+gw, (g,w) E G x D, is .
-.
E a G-invariantexperimentif p, 0 g-lE sET}forall t E T,g E G.Particular are the shift experiments of Section 38and the casesof G-invariantexperiments 39. structuremodelsof Section of Section 31isa specialcaseof the situation Let us show that G-invariance described in Definition 48.2.ForJl E L(E)defineMgJl:=Jl 0 g-l,g E G.Firstwe of Ghave to show that M,llE L(E),g E G. This is an easy consequence in varianceand Lemma 41.1.It isclearthat (M')'1; G is a groupof stochastic that is and E invariant under operators (Mg)gf:G' Since the operation Fubini'stheoremimplies (g,(0)1-+goo,(X, (0)E G x Q,is1I(G)@ d-measurable that (Mg)gcG is a measurable group. We introduce the notionof a G-invariantdecision problem.) {\037:
Let (T.D, W) bea 48.4Definition. Assumethat E isinvariantunder (M')'E(i' decisionproblemwhere G operateson D continuouslyfrom the left. The decision problem(T, D, W) iscalledG-illvarianlif P'=M9\037 implies \037(x)=\037(gx), xED, wheneverS, lET and g E G. thisconceptby the mostimportantexamples.) Let usillustrate
48.5Examples.(1) Let (H,K) bea testingproblemin the classical sense,i.e.a o f t E into K. To it into the tenns two setsHand of general put partition {p,: T} decisiontheory let D = {O,I} and \037(1) = IH (P'), \037(O) = l K (P'),t E T. If M,Hs;Hand M,Ks;K for a11g E G then obviouslythe decisionproblem (T, D, W) isG-invariant. Specifyingto the caseof Example48.3we seethat Ginvariant testingproblemsin the senseof Definition31.1are G-invariant decision problems. that p, = Mg Ps iff t = g.';,wheneverg E G, (2) Let T = D = G and assume t E G. This is the casewith estimation or problemsfor shift experiments structuremodels,Let \037 (h) t= W (g-th) where W: G IR is a EI(G)measurable function which is boundedfrom below.Then .p, = Mg\037 implies
-.
t
= gsand hence)))
48. The generalizedtheorem of Hunt and Stein \037(x) =
Thus (G,G, W)
247)
W(s-tx) = W(r-t gx) = J+;(gx), X E G.) is a G-invariantdecision problem.)
Assumethat E isinvariant under(k!,),e 48.6Definition. G and that G operates on D continuously from the left.A generalized decision functionP E .\037(E, D)is equivariantif) tE T, gE G.) (J(fog,P,) = (J(f,M,P'),fEfjb(D),
It isstrictlyequivariantif) P<J0 g,p) = (J(f,Mgp),f E
\302\253fb(D),
PE
L(E),g E G.)
betweenequivarianceand strictequivariancebecomes clearif The distinction we specialize to the caseof Example48.3.)
48.7 Example.Considerthe situationof Example (48.3).Let Q E 9t(E,D). ThenQ is equivariant in the senseof the preceding definitioniA') Hf(gx)Q(w,dx)P'(dw)= Hf(x)Q(gw,dx)\037(dw)) and structure models for all fE CCb(D),g E G and t E T. For shift experiments thisisexactlythe conceptof equivarianceconsidered there.Now, let vld'bea
u-finitemeasure such that fjJ..., v. A decision function E .\037(E, D) in the sense of Definition 4 8.6 iff) equivariant Hf(gx)(l(w, dx) hew) v (dw) = H f(x)Q(g(I),dx) hew) v (dw) \037
is strictly
for all fE fCb(D),g E G and hELl(v). Thisimplies Jf(gx)Q(w,dx) = Jf(x)Q(gw,dx) v-a.e.) for all fe ttb(D)and g E G. In Sections 38 and 39 a convolution kernel was definedby the condition) (!(w,B) = (!(gw,gB),
WE
fl,g E G,BEg.f(D).)
The followingassertiongives conditions where strictequivariance can be
this is a lifting theorembut we improved to the lanerproperly.Essentially, Thesituation presenta complete proof,notrelying on generallifting theorems. issimilarto that of Theorem31.7. of measurability.) First,we have to discussbriefly a question
48.8Lemma. Considerthe situationof Example48.3andassume chat D is a basefor topology. Then))) locallycompactspacewith countable it.fj
248
Chapter 8:General DecisionTheory)
(g.(0)....... e(gW, gB), (g,(0) e G x Q. istJI (G) @ ,r.(-measurablefor every fl e 91(E,G),Be
\037
(D).)
on D and !/u ProofForrc00(D)let Yp the topologyof pointwise convergence the uniform topology of D. Then ffp c ffu. but since every ball baseof fYy {fet'Coo(D): IIflly < a} p) and sincethere is a countable = of ballsit followsthat !M(.'T For every consisting \037(!Yy) on t'Coo(D). p) \302\243
\302\243f(!i
fe \037oo(D)the map
g.......f(g-I.) Fj: G\037t'Coo(D): is5\"p-continuous and hence ll(5\"p)-. and ll(5\"y)-measurable. Moreover, (w,!) ....... x Q is .91 dx), (w,f)e f6'oo(D), @ \037(!Yu)-measurable. Now, the Jf(x),,(00, that) ofmeasurable composition mappingsimplies is
\302\243I(G)
(g) w)....... Jf(g-lx)e{gw,dx),
(g,(0) e G x Q,
for every fe @ ,fI-measurable
(too(D).
Finally, a standardargument provesthe assertion.
0)
the situationof Example48.3and assumethat D is a 48.9Theorem.Consider 111ell basefor itstopology. locallycompactspacewith countable for every strictly D) there existsa strictly equivariant decision function equivariant e e e \037(E, D) such that (1) flo(W, B) = fl(W, B),BE:M(D), and (2) eo(gw,gB) = eo(w,B), g e G, Be tI(D) on a .\\'et A e .91with \037(A) = 1,t E T.) 91(\302\243,
\037o
is a u-finite measure being equivalent to = {p,: A is u-finitethere t E T} and let l be a right Haar measure of G. Since existsa probabilitymeasure A.o ,\037(G) which is equivalentto A.. This implies) Proof Assume that
\037
\\'I.!1Sf
I
A.o(B)=Oiff .A.o(Bg)=O,Be.<1I(G), geG. field Let 910 9I(D)be a countable Let e e {)I(E,D) be strictly equivariant. orintersections of familieswith which generates \037(D).In the followingunions with the are countable unionsor Be indices !N(D) equal corresponding on910 necessarily which coincide intersections with BE 910 , sincetwo measures coincide on $feD). \037
We have v
{I
lIe);(D))
{w E Q:e(w,B) '* q(gw,gB)}=
for every g e G which implies)))
0
48. The generalizedtheorcm of Hunt and Slein (v
249)
= O. U {(W,g)E D x G:(!(w,B) =F (!(gw,gB)}
@ A.o)
BE9I(D))
Hencewe obtain A.o
I
{gE G:e(gw,gB)=+ e(w,B)}= 0 v-a.e.
J
B.?t;(D))
setof \\I-measurezerowhere the equationdoesnot Let N be the exceptional hold.Define = f e(hw,hB)A.o(dh)A.o-a.e.}. A:= {WE Q:e(gw,gB)
n
BEDI(D))
soonwe have shown that) (1) AEd, (2) Q\\N A and hence
As
\302\243
(3) (4)
A
E
d,
v(Q\\A) =
0,
j
f e(hw,hB) A.o(dh)=
J e(hgw,hg B) i.o(dh)if
WE
A, g e G, Be .r?I(D)
the proofcan be completed as foJlows. Define
- { e(hw, e(w, B)
eo(w, B) _
f
hB\302\273'o(dh)
if
WE
if
w f A,
A,
BE 91(D),
Be fA(D).)
Inorderto showthat eoisstrictlyequivariantwe notethat WE
A
implies by (4)
that)
(lo(gw,gB)= J (l(hgw,hg B\302\273'o(dh) = J e(hw,hB)Ao(dh) = Uo(w,B)) for al1 Be .c;!d(D).The
we A\\N we have)
decisionfunction ao solvesthe problemsincefor
e(w,B) = e(gw,gB) A.o-a.e.,Be \037(D),) and therefore) = eo(w,B), Be 84(D).) {!(w,B) = Je(hw,hB)lo(dh)
It remains to proveassertions (1)-(4).
Assertion (1)foUowsfrom the fact that A isthe setwherethe d-measurable
functions) W
H JIJ e(hw,
hB\302\273)'o(dh)
- e(gw,gB)llo(dg),weD,)
BE910 , vanish. For the proofof (2) we notethat w ri N implies g He(gw.gB) isconstantA.o-a.e. andthereforewe A. Next let us prove(4).)))
250
Chapter 8:General DecisionTheory)
If (I) E A then we denote = II {kE G:e(kw,kB)+ J e(hw,hB)Ao(dh)} Nt)) B@)/(DI)
and observethat
= o.Since)
Ao(Na\302\273
= O. For arbitrary ge G it followsthat Ao(N(I)g-l)
t Nwg- = {kE G:kgE N\"J = U {kE G:e(kgw,kg B) * J (!(hw,hB) ;.0(dh)},
B@9'(D))
we obtain)
J e(kgw,kg B) ).0(dk) = J e(hw,hB)Ao(dh) for all BE\037(/).This proves(4). Finally, we show that (3) istrue.If WE for every g E G
A
then Ao(Nwg-1) =
0 implies that
e(kgw,kg B) = J e(h(l), hB))'o(dh),BE:!I(D),I'o-a.e.)
thiswith (4) we get for all g e G Combining e(kgw,kg B) = J e(hgw,hg B) Ao(dh), B E \037(D), Ao-a.e., B by g-I B and replacing e(kgw,kB)= J e(hgw,hB)Ao(dh) Be \037(D), )'o-a.e.
Thisimplies gw E A.
0)
48.10CoroUary.Assumethat E isa shift experimentor a structuremodel.Then
for every strictly equivarianl I!E riI (E,G) there existsa convolulionkernel eoe fit (E,G) such that l!o(w,B) = e(w,B), BE (D),on a set A E J;( with P'(A) = 1,t e T.) \037
ProofFromthe proofofTheorem48.9it isclearthat eocan bechosenon Q\\A kernel. 0) to be a convolution theorem. Now we approachthe proofof a general version of the Hunt-Stein if G G there exists E such that For convenience, callsand t connected by g classes are Thisisan equivalence relationon T and the equivalence .P, = M,Ps. calledthe orbitsof G on T. Let usdenotethe setof orbitsby TIG.)
48.11Lemma. Assume that E is invariant under (Mg )9EG and (T, D, W) is Ginvariant whereW
isbounded andcontinuous. If P E fA(E, D) isequivariantthen
I........ P'),1e T, is conslanton Ihe orbitsin PO'Y\"
TIG.)))
48. The generalizedtheorcm of Hunt and Slein
251)
Proof Let P, = M 9 Ps.Then = P(\037.Ps). P(Jt;,l{)= P(Jt;.M9\037) = P(Jt;og,.P,)
0)
The main technical 48.12Discussion. toolfor the followingis a fixed point
property.
vector spaceand ..Ita family of continuous linear (1) Let X bea topological mappingsq>: X X. The family hasthe fixedpointproperty if every compact, convexsetK X such that q>(K) K.qJ E At.contains a fixedpointof vii, i.e. -\037
\037
\037
a pointXo E K satisfyingqJ(xo)= Xo for evcry qJ E .II. of a groupG isa homomorphism (2) A representation g \037 1;of G intothe linearmappingT: X -+ X of a topological vector space spaceof all continuous X. of G has (3) ThegroupG has thefixedpointproperty if every representation
the fixedpointproperty.
It is well known and will sufficefor our purposes that every Abelian the fixed the has T his is Markov-Kakutanifixed group pointproperty. point theorem. For a proofseeDunford-Schwartz[1967. TheoremV. 10.6]. relatedto amenability. (5) Thefixedpointpropertyisclosely Amenabilityis a weakercondition than the fixedpointproperty.If G isconnected then both conditionsare equivalent to each other.For a survey discussion of the between various conditions relatedto theseconcepts. see interrelationships Bondarand Milnes[1981].) (4)
Theoremof Hunt and Stein).Assume that E is 48.13Theorem(Generalized
on D continuouslyfrom the left. invariant under a group(Mg),\037G and G operates then has the JfG fixedpointproperty for every fJ E \037(E. D) thereexistsa strictly
functionPoE equivariantdecision
11(\302\243,
D) such that)
SUpP(jog-l. MgJ1) Po(j,J1) ,f:G) \037
+ functionf' D fur every pEL(E) and every lowersemicontinuous
-
IR.)
ProofForevery g E G let Pg: (j,}J) P(jog-I,Mg}J),fE \037b(D),P E L(E).Let 91 K be the closedconvex hull of {P,,: in E (E,D). If X denotesthe g G} x L(E), vector spaceof continuous bilinearfunctionson rc,,(D) topological -+ X X definedby then for every g E G the map 1;: \037
1;(Y):(f,}J) y(jog-I.M,J1),u:J1)E \037,,(D) x L(E),I'EX.) of G and satisfies is continuous and linear.Hence,(1;)is a representation K contains a fixedpointPoof (1;).SincePo is Tg(K)S K,g E G.By assumption, a fixed pointit is strictJyequivariant and sincePoE K, it satisfies the asserted \037
condition.
0)))
Chapler8:General DecisionTheory)
252
48.14Remark. (I) The resultsof the previouschapterswhere we have used much easierproofif the condition of amenability amenabilityadmit a similar, isreplacedby the fixedpointproperty.Thisisparticularlyvalid of the classical Hunt-Steintheorem (Theorem 32.5)and of Bon'sconvolutiontheorem (Theorem38.15).
the preceding assertion can be provedunder the assumption Conversely, of amenability by the technical means of previouschaptersprovidedthat 0 1-+ g /3(/ g-I,M\"Jl),g e G, is &t(G)-measurable, Ie\037b(D), Jl E L(E).Let us sketchthe prooffor this case. Let (K,,) be a summingsequence of compacts G, n EN, and define K\" (2)
\037
A.\"
= Ar(.1Kit)' n E N. Denote)
if,Jl) 1-+J fJ(10 h-I, M J1)
L(E),n E FromTheorem42.3it followsthat there existsa weak accumulation point inequality is PoE (E,D) of (P\flEN'") If Po is equivariant then the asserted obvious. Let us provethat Po is even strictly equivariant. Thechoice of impliesthat for every g E G and allfeI'Gb(D),J1 E L(E), fJlt:
II
A\"
(dh), IE 'tb(D),J1 E
f\\J.)
\037
(..\\\II4IN")
t lim IJfJ(/og-loh=0. , M,.,Jl)i.,,(dh)Sp(foh-I,M\"J1\302\273)'\"(dh}1
,,-
co)
It followsthat
-
lim fJ,,(/cg-I,M,J1) fJ,,(f,Jl)1= 0 -. 1
\"
CIO)
for all g E G, IE't,,(D)and J1 e L(E). It is now clearthat every weak accumulation pointof (P\\"E") isstrictly equivariant. remark canbe appliedto Q E fJt (E,D),sincein thiscase (3) The preceding (/0g-I)(lM\",g e G. is \302\243B(G)-measurab)c for an Ie(lb(D). e L(E).For the proofconsider the function F definedby (g,h)eG2 .) F(g.h)=fog-I(}M,.J1, fit;
g.-
J\037
Ie
(D),J1 e L(E),e e D).Then it isclearthat g 1-+F(g,h), for for every he G,and h 1-+F(g,h), hE G, ismeasurable g E G, iscontinuous and dense,and let d be a distance every g E G. Let (g,,) G be countable for some
<6'\"
91(\302\243,
\037
the topologyof G.Then generating
{(g,h) E 02: F(g,h) ex} = n u (g,h) e G2 : F(g\",h) \037
It= I
,,=
1)
{
\037
ex
-l
+ k , d(g\",g) < k , \037
}
Sincefor every e R, which impliesthat F is 91(G) @ 91(G)-measurable. follows.))) D E \037 (G) @ !M(G)the set{gE G:(g,g) e D}is in !J4(G),the assertion
ex
48. The generalizedlhcorem of Hunl and Stein
2S3)
48.15CoroUary.Assumethat E isinvariant under(M,)geGand G operates on D
left.If G has the fixedpointproperty then for every D) there existsa strictly equivariantdecision function PoE \037(E, D)
continuouslyfrom the
PE such that) 91(\302\243,
P sup rEr
(J.Y\"
\037)
sup Po rEl)
(J.Y\"
\037
\037)
lossfunction W ofan invariant for every -;E T/Gandevery lowersemicontinuous deci.vionproblem (T, D, W).) IfE isdominated basethen the and D isa locallycompactspacewith countable be a ssertions can preceding improved.) and D isa locallycompactspace 48.16Theorem.Supposethat E is dominated with
countablebase.Considerthe caseof Example 48.3and assume that G
on D continuouslyfrom the left.Assumefurther that there existstrictly operates D).If G is amenablethen for every P E PA D) equivariantelementsin there existsa strictlyequivariamdecision functione E fJt (E,D) such that supP(fog-l,Mgp.)\037fllil 91(\302\243,
(\302\243,
leG)
for every
J.L
E
L (E)+ and every levelcompact functionf D-+ R.)
without lossof generalitythat ProofIn view of Theorem43.5we may assume of a strictly P E [Jt (E,D).ThenRemark 48.14(2) and (3),impliesthe existence E rM such that (E,D) cquivariantPo
supP(fog-l,M,Jl) Po(/'Jl), \037
Ji E
-
gEG)
L(E)\\
forevery lower semicontinuous fD R We prove that there existsa strictly equivariant decisionfunction 11E Yt (E,D) such that
Po(/'p.) \037fllP.,
J1 E
L(E)....)
D R For Pochoose a kernel110according forevery levelcompactfunction! to Lemma42.6.Let IIIbeany strictlyequivariantdecision functionin Y1(E,D). \037
Define)
-
e(w,B) 1=Ilo(w,B) + (1 ()o)(ro,D)el(ro,B),
ro E
Q, BEfleD).)
It is then clearthat 11E (E,D).It is alsoclearthat 11 is strictlyequivariant sinceIlo(w,D) = (!o(gw,D)\037-a.e.Iffe ((f,,(D)is levelcompactthen and for J1EL(E)+we obtain supf-fE\037;(D) .\037
-
-
-
P(supf /, Jl) = (supf f)eoJl (supf \037
f)UJl.)))
254
Chapter 8:General DecisionTheory)
This implies
P(f,Jl) \037feJ1. Now Lemma 6.4provesthe assertion. 0 It is obviousthat a similarconclusion as in Corollary48.15 is possible.)))
Chapter9: Comparisonof Experiments)
The main differencebetween the usual decisiontheory and the theory of
can be explained as follows.It is the aim of decision theory to experiments rule.In a fixedstatistical e.g.to find an optimaldecision investigate experiment, the relations considers contrastto decision theory the theory of experiments In this mostof the betweendifferentexperiments. sense, precedingtheory in thisbookcouldbe subsumedunder the labelof decisiontheory.However, of the sufficiency and invariancepointintothe direction problemsconcerning theory of experim\037nts. The startingpointof the theory of experimentswas the problemof comBlackwellintroduced initiatedby Blackwell[1951]. parisonof experiments, which a statistically natural concept forthe comparison of generalexperiments with binary experiextendsthe ideaswe have usedpreviouslyin connection 49 ments (Section (Section 15)andwith dominatedexperiments 22).In Section we discuss the relations betweenthe various definitions.
The essential is that the basicorder pointof the theory of experiments first definedin rather general terms, can be relationbetween experiments, In Chapter3 transformedin such a way that it becomes analyticallytractable. in an elementaryway, using this has already beendonefor binary experiments the Neyman-Pearsonlemma, only.Now, in the generalcase,things are more and we dividethe procedure intoseveralsteps. complicated Firstof all,by means of the minimax theoremthe orderrelationcan be in terms of the Bayesianrisksforpriorswith finite support. It iseven expressed iscarriedthrough in sufficienttoconsider only uniform priors.This reduction 49.At this pointthe orderrelation isstillbasedon all possible Section decision for a A suitable of s tandardization decision problems given parameter space. matters considerably without changingthe orderrelation. problemssimplifies It turns out that we needonly considerdecision spaceswhich are convex 50and 51).The polyhedra,and lossfunctionswhich arc projections (Sections next stepis to expressthe Bayesian risksfor uniform priorsand standard decision and to express problemsin terms of integralsover concavefunctions, the orderrelationbetweenthe experiments in terms of theseintegrals (Section52).This procedure of transforming the order relationgoesback to and 1953], It is Blackwe))[1951 and hasbeen generalized by LeCam [1964], alsodiscussed [1970]. by Torgersen In Section53 we show that the Hellingertransformscharacterize the classes As o f a we obtain that also equivalence experiments. consequence the)))
256
Chapter 9:Comparisonof Experiments)
of the likelihood distributions have this property,thus extending processes r esults o f Section 2 5. previous We distinguishtwo different order relations between experiments. The coarseroneconsiders testingproblemsonly whereas the finer one admits of experiments. In Section54 arbitrary decision problemsfor the comparison we presenta few resultsessentially back to Torgersen[1970]. Among going for binary experiments. coincide others,it is shown that bothorderrelations For arbitrary experiments relationsare only the corresponding equivalence identical. 24and 25 we have provedthat for dominatedexperiments In Sections the in terms ofstochastic orderrelation basedon testingproblemsean beexpressed This characterization is exL-spaces. operatorsbetween the corresponding tendedin Section55 to the randomization criterionwhich is the general of the ideaof sufficiency.Initiatedby Blackwell[1951] the final formulation version isdueto LeCam [1964]. with finite parameter spacethere Forevery equivalence classof experiments is a standardmeasure on the unit simplex.There is even a one-to-one and standardmeasures. This can betweenequivalenceclasses correspondence be shown in an elementaryway and is known sincethe papersof Blackwell [1951and 1953].In caseof infinite parameter spacesthe situationis more difficult.For every equivalence classof experimentsthere is a uniquely which corresponds determinedprojectivesystem of standardmeasures to a the time of It took some conicalmeasure, using terminology Choquet[1969]. until it was possible to give a correctproofof the converse. Our proof,given in we observethat Sections 56and 57 followsSiebert[1979]. As a consequence a limit. every projective system of experiments possessesprojective In Section 58we consider the ideaof invariant By means again experiments. of the preceding oneconceptof invariance theory it turns out that essentially all definitions considered before. subsumes on the spaceof experiments leadsto a definitionof a The orderstructure This isdoneby means of a distance. calleddeficiency. With natural topology. is a complete the spaceof equivalence classesof experiments this distance metric d iffersfrom the t he metricspace.Forinfinite parameter spaces topology to finite weak topologywhich is the coarsest topologysuch that all projections are continuous, Endowedwith the weak topologythe spaceof subexperiments is a This 59,goingback to experiments compact space. isthe contentof Section LcCam[1972].)))
49. Basicconcepts
257)
49.Basicconcepts) E E 8 (T)and FE S(T). Let T 0 bean arbitrary set.We consider experiments =t=
aredescribed Theseexperiments by) E= (!11 1, tE T})
,d F= (!1,d , {Q,:tET}).) {\037:
and)
2
2
-
of a topologdecision Furthermore we consider problems(T.D. W) consisting = W D t and a loss function where D icalspace R, E T. For any J-Y,: (JY,),& T'
lossfunction W II J-Y,II
we define)
= sup{IJ-Y,(x) I:xED},
t E T.)
49.1Definition.Let (T,D. W) be a decision problemwith a lower semicontinuouslossfunction Wand let t O. Thenthe experimentE ise-deficientwith respectto the experiment F for the decisionproblem(1',D, W) if for every fJ2 E 14(F,D) there is PI E Ee (E,D) such that \037
PI(J-Y,,\037)
\037
P20-y',Q,)+ ell J'Yr1l,
t E T.)
In this. casewe denoteE ;2 F. IO,W) definitionmakes only The readerobservesimmediatelythat the preceding sensefor E>0 if the lossfunction W isbounded. o If E ;2 F then we denoteE 2 F. The relation\" ;2 ,. is an order r;
(D, W)
relation.If E
(D, W)
(D.W)
2 F then E iscalledmore informativethan F for (1',D, W). If
(D.W)
E ;2 F and F ;2 E then E and F are calledequivalent and we denote (D.W)
E
..\"
(D, W)
F.
(D, W))
49.2Lemma. Assume that W isa bounded, lowersemicontinuous lossfunction. t Then E ::::> F holdsiff E ;2 Ffor every e > O. (D,W)
(D, W))
of the weak compactness of Proof This IS an immediate consequence \037(E,
D).
0)
In Chapters3 and 4 we have considered orderrelations betweenexperimenls to which are closely related the concept of the preceding definition. Let us recall
intoslightly more general tenns. t In Chapter3,Section15,we have considered the relation\"2n for binary
what we have doneand put
it
2)))
258
Chaplcr9:Comparisonof Expcrimcnts)
and in Chapter4, Section the orderrelation 22,we have considered experiments \" ;2 for general dominatedexperiments. Recallthe definitions: 2 Let t 0 and let E, Fe 4(T) bedominatedexperiments. Then the experiment E ise-deficient with respect to the experimentF for testingproblemsjf for every testingproblem(H,K) and for every lfJ2 E :F (!2 2 , .r;;/2) there is d I) such that) CPI e /Pil (.QI'
..
\037
P'([JIS; Q,([J2+ 2
if
tE H.
if
tE K.)
\302\243
P'CPI
\037
-t
Q,CP2
2
e
In thiscasewe denoteE 2 F. This relationis relatedto Definition49.1as follows. 2)
49.3Lemma. Let E 0 and let E.FE & (T) bedominated Then experimems. ( ( E ;2F iff E 2 Ffor every decision prohlemwherethe decision !ipaCeD contains \037
2
(V.W)
exactly two elements.) (
F, Let D = Proof.(1) Assumethat E;2 2
{O\037
I}and Wan arbitrary lossfunction.
We define)
If = {tET: J.v,(1) J.v,(O)}, K= {tET: J.v,(1)< J.Y,(O)}.) \037
In view of Coronary42.8we choosesomee2e fjt (F.D) and denotecpz ( = e2 1). Obviously,CP2 E !Ii'(Q2'd2) and accordingto E ;2F there 1S CPI E (QI\"\037I) such that)
(..
2)
\037
:::
\037CPI
Let el =
CPI
e
Q,CPl + 2 if
\037CPI
\037
Q,CP2
-
+ (1
\302\243\\
resp.O. We notethat) J-Y,\037I
-
E
2)
tE
H,
IE K.
if
CPt)f.o where
\302\2431
at 1 and to denotethe Diracmeasures
-
= J.Y,(l)P'cp. + J.v,(O)P'(1 CPt> = J.v,(O)+ P,([J.(J-Y,(1)
Q,= J.Y,(1)Q, + J-Y,(O)Q,(1 = + Q, (J-Y,(1)-
P,
J.v,(O\302\273
and)
J.Y,e2
CP2
1\302\245,(O)
CP2
CP2)
J-Y,(O\302\273.)))
49. Basicconccpls
It followsthat) Jv,Ul
-
\037
-
Jv,UzQ,-= (Jv,(1) I
\037
Jv,(1)
-
Jv,(O\302\273(\037q>1
259)
Q,q>2)
{;
Jv,(O) \\.
2
1;11Jv,1I.)
\037
I-
Hencewe have E ;2 F. (D, W)
e
problemwhere the (2) Assume converselythat E ..:J F for every decision (D.W) If (II,K) isan arbitrary testing decision spaceD contains exactlytwo elements. defineD and) {O,I} problemthen we \037
--21
Jv,(O) =
Jv,(1) =
1
+-
Let q>2 E !Ii(!J2 , 2)' Then for such that) \037Q/
+
Q2 =
q>z
-
i:1 + (1
2
if
IE H,
if
[EK.
q>z)i:othere is QI E 9t(E,D)
t
2' lET.)
11
Jv,U2 Q,
\037
1
if IE K,
2)
Jv,e..P,
+-21
IE H,
if
Denotingq>J = (>1(.,1)we obtain)
-2-. -P'III P'III '''t'l + -. 2 '''t'l (1
JoY,
eJ .p, =)
1 2 1
'p'q>1+
and)
J.Y,(>2Q,=)
1
-
2 (1
-
1
if
IE H,)
'p'q>.)) if
[e K,)
)
-:2(1 Q,q>2)+ 1 1 + Q,q>2 2 1 2
-
1 \"2
Q,lf>z if
(1-Q,q>2)
if)
[E H,) rE K.)
It followsthat) 1
-2:(1-
1
and)
- 21
'p'q>1+ r
ThisprovesE ;2F. 2)))
1
+\"2 P'epl
-2
1 i (1 P'q>I)
-2
\037q>I)
\037
\037
0
1
(1- Q,q>2)t 2 Q,q>2 2 jf [E H,) 1
I;
-+
1
Q,q>2+\"2
(1-Q,q>2)+\"2 \302\243
if
re K.)
260 We
Chaptcr 9:Comparison of Experimenls) c
dominated generalizethe notionE;2F to the caseof not necessarily 2)
experiments.
49.4Definition.Let t;
with O. Then the experimentE ist;-deficient
respectto the experimentFfor testingproblems (E 2 F) if E 2 F for every decision 2 (D.W) wherethe elements. decision contains two problem space exactly c The fonowingdefinition extendsthe spiritof the relation\" => \" to general decision problems. \037
c
t.
2)
49.5Definition.Let E O. The experimentE is t;-deficientwith respectto the problem(T, D, W) with a experimentF (E:;1f) if E :;1F for every decision (D,W) lossfunction W. bounded,continuous t t. is not true. It is obviousthat E 2 F impJiesE 22 F. In genera], the converse Later it will beshownthat the converse is at leastvalid for binary experiments (seeSection 54). o If E;? F then we denoteE;?F. The relation\" ;?\"is an orderrelation.If E ;?F then E iscanedmore informativethan F. If E :;1F and F:;1E then E and F are calledequivalentand we denoteE F. c It is obviousthat E:;1F iff E;2F for for every > O.) \037
t.
t.
\"'\"
E;
49.6Corollary.Let 8 (T, D, W)
with
C
O. If E;2F [hen E
t.
;2 F for every decision problem (D,W) a lowersemicontinuouslossfunctionw:) \037
problemwith a Proof Choose > 0 arbitrary and let (T, D, W) be a decision lower semicontinuous lossfunctionW. Let be the set of an bounded, V W such that inf t E T. It continuous lossfunctions Jt; J.v,
f-
\037
II
\037
JoY,
\037\"
II
\037
\037
JoY\"
f
isclearthat for every bounded, lossfunction V, W there is V E continuous suchthat VI V\037 W. Moreover,each Ve f\" satisfiesJt;1I J.v,1I (1+ <5), t E T. If {J2E (F.D) then E F implies) \037
\037
II
\037
II
II JoY, II
}
l1
\037
\037
n {pe :M(E,D):
P(V\"
lET)
1;)
\037
P2(J.v\" Q,)+ e(l +\037)
:+=
(/J
f6' Since isdirectedfrom aboveit fonowsthat) n n {fJ E fJI (E,D):fJ 1;) P2 Q,)+ t; J.v,II} :t 0. VE'fI'6 ,eT)
for every
VE
\"f'\037
(V\"
This provesthe assertion.
\037
(J.v\"
II
0)
for))) By means of the Minimax Theorema necessaryand sufficientcondition
49. Basicconccpts
261)
t
E ;? F can be established in terms of Bayesianriskswhich is the key of all (D.W)
followingconsiderations.) Assumethat (T, D, W) isa decision 49.7Theorem(Blackwell[1951]). problem I; F holdsiff for lossfunction.Let e O. Then E with a lowersemic:ontinuous \037
\037
(D, W))
every mE ST
inf fJl.\037(F.. D)
::: inf fJ2C
\037
(F, DI)
J PIOf;,\037)m(dt)
JP2(\037,Q,)m(dt)+efll\037lIm(dt).
Proof Apply Corollary(45.7)to M2 = {(PI(\037, PI E !M(E,D)}, and = 0) Mt {(fJ2(\037,Qr)+ell\037ll)rI'T:P2E\037(F,D)}. \037\302\273rET:
continuous lossfunction W. If (T, D. W) isa decision problemwith a bounded, theorem may be restricted to 9t(E,D) and then the infima in the preceding Yt (F.D),respectively. of E to be parameterspace(x. It turns For E A (T)let be the restriction of experiments outthat the comparison may be phrasedin terms of the finite (X
E\302\253
subexperiments, only.)
49.8Corollary.Assume that (T, D, W) is a decision problemwith a bounded continuouslossfunction.l.etr. O. Then the followingassertions are equivalent:) \037
c
(1)) E(D. F. \037
W)
t
(2))
\037
t.\037
(D.W))
f\037
for every
(X
E
A
(T).
Proof It isclearthat (E.D) = .(jt (E(I'D) and .rJt (F,D) = .rJt D) for every E A (T). Hencethe assertion is an immediate consequence of Theorem 49.7. 0) \037
F;\"
(X
m may bereplaced rinally,we:show that in Theorem(49.7)the priormeasures
by the unifonndistribution.)
Let c 0 and let D be a decision 49.9Corollary(Blackwell[1951]). space. \037
Then the followingassertions are equivalent: t
continuouslossfunction W. (1) E(D.=> Ffor every bounded W))))
Chapter 9:Comparisonof Experimcnts)
262
(2)
Forevery
ex
inf
continuous lossfunction W) (T)and every bounded inf L J-Y,QI P, L J-Y,Q2Q,+ t L J-Y,II. EA
II
\037
l/2c8t(F,D),CCl
l/1\0378t(E,D)'(:\302\253
'\037CI)
ProofIt isclearthat (1)=> (2).Let us prove(2) => (1).Let m E ST and let W be a bounded. continuous lossfunction.Let E A (1')be such that me We defineanotherlossfunction W by ex
S\302\253.
xED, tET. W(x.l)=lexlm{I}W(x.t), Then it followsthat inr l/If!Sf(E.D)
J J-v,Ul p'm(dt)=
_1 L -
.
Inf
\037
l/z5Sf(F,D)
I ex I
=
inf hf!\037(F,
D))
inf
\037
l/1\037Sf(E.D)
\037U2Q,+
u\302\253
-L 1
\302\243.
1
L Jf;Ul
\037
I ex I '@CI
ex 1
rTJ II yy,1I
u\302\253
0 J\037Q2Qtm(dt)+eJIIJ.-Y,lIm(dl).
50.Standarddecisionproblems) 50.1Deftnidon.A decisionproblem(1',D,L) is a standarddecisionproblem for a parameter spaceT if J) c; is a convex.compactsubsetand if ..... D is the the coordinate of HT, t E T. onto L,: projection \037T
It\"
IJ.I
It will be shown in the next paragraphthat comparison of experiments ofstandarddecision problemsonly.In this may bebasedon the consideration of sectionwe discussthe reduction arbitrary decision problemsto standard decisionproblems.Let us consideran experiment E E 8 (T) suchthat E=
(D,.flI.{p,:lET}).)
50.2Lemma. Let (T,D,L) be a siandarddecision problem.Then for every Q
eat(E,D) there is a nonrandomizeddecision function Qo e {Jt (E.D) such that
L,UoPr=
L,U\037,
leT.)
Proof For every we D letb(w)be the barycenter of Q(w,.)in D.SinceL, is decisionfunction affine-linear on D for every t E T the non-randomized
eo:W'-'b(w) satisfies = (L,Uo)(w). (L,e)(w) = J L,(x)e(w,dx)= is Sincew......(L,Qo)(w) It remains to show that (Jo is (d,\037o(D\302\273-measurable. of for every t E T and since{L,:t e T} separates .9/-measurable points (b(w\302\273,
the)))
SO.Standard decisionproblems
theorem impliesthat fo flo compactset D the Stone-WeierstraB measurable for everyfe \037(D).Henceflo is.ilo(D)-measurable.0)
IS
263)
,t;/-
A nonrandomized 50.3I>efinition. decisionfunction Uo E 9t(E,D) is called
simpleif it attainsonly finitely many values in
D.)
50.4Lemma. Let (1',D,L) be a standarddecision problem.Then the set is i s {(L,Uo1D,,, T:UO e 91(E,D) simplenonrandomized} densein R(E,D,L).) Proof It foHowsfrom Lemma 50.2that the set {(L,UoP')'ET: UO e tit (E,D) is coincides with {(L,UP,)UT: nonrandomized} (!e .\037(E, D)}which itselfisdense in R (E,D,L). Hencethe assertion followsby a standardargument usingthe fact that D is totally bounded.
0)
showsthat,given an experimentE e & (T),for every The followingassertion decision problem(1',Do,L) problem(T,D, W) there is a standarddecision such that the risk setsR(E,D, W) and R(E,Do,L) coincide. Thus,the analysis
of an experimentE may be basedon the consideration of standarddecision problemsonly.)
50.5Theorem(Blackwell[1951]). Let (D, W) be a decision problemwith
a
continuouslossfunction convexhull of bounded, If Do denotesthe closed, = R (E,Do.L).) {( T: xeD} then R (E,D. W) \037
J-Y,(X\302\273,c
Proof (1) Let us prove R (E,D, W) s;R (E,Do,L). By Lemma 44.4 it is sufficientto prove Ro(E.D, W) Ro(E,Do,L).If (!E Yl(E,D) then for every WE Q we defineUo(w,.)to be the one-point measurewhich isconcentrated at Then we have T' (J J-Y,(x)(!(w, dx)) J J-Y,(x)e(w,dx)= J L,(x)Uo(w, \037
dx\302\273,c
for every (I)E Q and tE Twhich provesthat (JY,(!P,)'.TE Ro(E,Do,L). Let show that us To this endwe show that R(E,Do,L) R(E,D, W). (2) Ro(E,D, W) isdensein R(E,Do,L) and apply Lemma44.4.Sincethe risksof the simple decision functionsare densein R (E,Do,L) it is sufficientto prove that every neighbourhood of the risk of a simpledecision function contains of Ro(E,D, W). Let E .91(E,Do) bea simpledecision elements funclionand \037
\037o
let)
if IE U={YERT:I>I-L,uoP,/
for some > 0, \302\243
!X
EA
C!o(w,B) =
(X}
(1').The decisionfunction flo isof the form) \"
L 1.4;(w)IB (x;), WE Q, BEfJI{D), '.0; 1)))
264
Chapter 9:Comparisonof Experiments)
...,
where (At.A 2 . partitionof!1and XjE Do,1 i Arc) is an .\037-measurable k. Fromthe definition of Do it followsthat thereare mj E SD, 1 i k, such
:;
that) I
J J-v,dm j
-
:; :;
< r.
(Xj),I
if
IE
ex,
\037
1 i k.) \037
\037
We define) rc
q(w,B) =
,-L l ,(w)1nj (B), we 0,
BE 91(D).
A
1)
U.Thisisthe (JE D) and it remainsto show that (J-Y,Q\037)'ETE Obviously, casesincefor every t e 91(\302\243,
rx)
I
-L,QoP'1=
J-Y,QP,
k I
k
L J-Y,dm i P, (Ai) i= I J \"
\037
L i=
- L (xi),P'(Ai)1
IS J.Yrdmj
i
-
t)
- (Xj),IPr(Aj)< E.
0
I)
It isconvenient to calldecision (T, D1, WI) and (T, D2 , Jt2) equivalent problems = if R(E.D1, Jt;) R(E,D2 , \"'2)for every experimentEE I(T).Thus we havc provedthat for every decision problem(T, D. W) with a boundedcontinuous lossfunctionthere isan equivalentstandarddecision problem.)
51.Comparisonof experimentsby standarddecisionproblems) Let us consider E = (QI' .rdI'{p,:t E experiments
T})and
F= (Q2.d2'{Q,:IE T}).The reductionwhich has been achieved m the
preceding paragraphyieldsimmediatelythe followingcriterion.)
51.1Theorem.Let E
\037
51.2Theorem.Let
\037
O. Then
C
c E;2F ho/d'iiff E ;2 F for every .'italldard
(Do.L) decision problem(T, Do.L). isconcerned with testingproblems.) Another consequence
\302\243
O. Then E
c \037
2
F iff E
Z
Ffor every standarddecision
(DQ.L) Xz},Xl E JRT, Xz E
problem(T, Do,L) with Do = co{Xl' Proof We know that E 2 F iff E
t \037
e\037T.)
2 F for every decisionproblem(T.D. W)
(D.W)
containsexactly two elements. By Theorem50.5for every such decision problem(Do,L) with problemthere isan equivalentstandarddecision = co x X E Let E Do {xl' 2}, x\\ 2 conversely (Do,L) be a standard))) where D
[\0371..
\0371'.
51.Comparisonof expc:rimcntsby
standard decisionproblems
265)
decision problemwhere Do= co{Xl'X 2 }.This is by Theorem50.5equivalent with the ordinary problem({XI'Xz},L).
0)
In the followingwe apply Corollary49.9tostandarddecision Let us problems. recallsomenotations.For every A (T) the set consistsof all points from R T to W isdenoted in Theprojection X E IRT with support contained by !X
IR\302\253
\342\202\254
IX.
A,.
We
testingproblems.) beginwith a simplefact concerning
51.3Theorem.Let
E;
O. Then
\037
c
F iff for every E;2 2
IX
EA
(T) and for every
standarddecision problem(T, J>a., L) where Da. = co{XI'X2},XI ERa, X2 E inf inf + 1; L IIL,I/. L L,(}Ip, L L,(}zQ,
\037:a)
\037
f4:a Qz.\037(F.D..)
f4ia QIf!91(E.D.)
fECI)
I
I
iff forevery :2 ProofWe knowfrom Corollary49.8that E :2F 2 z 49.9the secondassertion isequivalentto) By Corollary E\302\253
inf Qle\037(E;.DC.\302\273)
L
J.\302\245,lh
P,
L
inf
\037
/l2 E !I(F.D(.\302\273)
fC:\302\253
,e\302\253
\037(}2
\037
Q,+ 1;,ea) L
IX
E A
(T).
II J.Y,II
for every A (1')and every decision problem(/)(2), W) where D(a.) contains W is lossfunction.In view of two elements and a bounded,continuous exactly Theorem50.5for every such decision problem(D(QI;',W) there isaan equivalent standarddecision problem L) where = co{Xl'X2},Xi E n , j = 1,2,and = II J.v,1I, t E IX. 0) II L,II IX
\342\202\254
(D\302\253,
D\302\253
feature of the followingresultis the fact that only convex The interesting
to obtaina criterion for E polyhedraneedto be taken intoconsideration
SI.4Theorem.Let e
c
\037
F.)
c
F holdsiff for every C(e A (T) andfor every standarddecision problem(T,D:a,L) where Da. isa convexpolyhedronin \037
O. Then E
\037
RCI
inf /lIE\037(\302\243.D.,
L L,QlP,
\037
'Eal
inf Q2G\037(F,D..)
L L,QzQ,+ e L IIL,II. 'E\302\253
'Eal)
showsthat E 2 F iff ProofA similar argumentasin the proofofTheorem51.3 for every C(e A (T)and every standarddecision problem L) where c inf L L,(lJP, /l1(:9t(E.D..) '\037aI)
DCI
(D\302\253,
isconvexand compact)
\037
inf \0372f1:dt(F.D..)
jR\302\253
L L,(}2Q,+ E L L,II. II
,e\302\253
'4iCl)
of the presenttheorem is nt:Ct:ssaryfor E F. To prove Hencethe condition Forany > sufficiencylet C(E A (T)and let Da.S;; RCI be convexand compact. \037
0)))
\302\2431
266
Chapter 9:Comparisonof Experiments)
we choose a simple decision function e2E .'1t(F,Da) such that)
I:L,U2Q,
\037
rp\302\253
I:L,qzQ,+ 8 I: L,II.
inf
1
ht9l(F,D.) rt\302\253
,ta)
II
Let Da.lbethe convexhuH of the imageofQ2'Since 1 isa convexpolyhedron it follows from the condition of the presenttheorem that) D\037,
inf L Lrql P, 'u) QII9I(E,D.) \037
\037
inf L Lrql P, 'Ea) \",.\037(E.D...,)
L LrC?zQ,+ t 'ea) L IILrll D..,)rea inf L L,QzQ,+ (8 + 81)L IILrll. 0 rea inf
Q2_!lI(\"', \037
Q2e9l(F,D.)
u\302\253)
52.Concavefunctioncriteria) of is the setof all x E Let T * 0. The dual (IR\037)+
IR\037
R\037
Recallthe tenninology of Section 45.)
with finite support.
52.1Definition.Let D s;(iR be a boundedset.Then 'PD:Rr.--.R+:y.-.inf{(x,Y):XED}) \037)+
is caHedlowerenvelopeof D.
Notethat forany D s; we have 'PD = 'P\037D' Henceany lowerenvelope is the lower envelopeof a closedconvex set.The lower envelopeof a convex } is given by 'I'D(y)=min{<x;,y>:t\03 polyhedronD=cO{x l ,X2,''''X It iseasy to seethat) n},YE 'PD,(\\ 'PD2 = 'Pt.5(D,v D2)' 1 V'D, + A.2V'D2 = 'PA,DI+A2Dl' (\037\037)*
II
IR.\037.
\037
A.
for any
V'D(-Y)= V'(-D)(y),
YE\037T,)
setsD1, Dz, D
and
\302\243
(R\037).
A.
1
\037
0,
).z
\037
O.)
52.2Lemma. Any
lower envelopeis upper semiconrinllolls, concaveand positively homogeneous.)
Proof.Obvious.
0)
is even continuous.))) It isclearthat the lower envelopeof a convexpolyhedron
52.Concave function
52.3Definition.If lI'D is the D
\302\243
(IR.\037)*
criteria
267)
lower envelopeof a convex,compactsubset
then the norm of 'PD is defined by
II 1IJD II
= L supIx,I. 'frT x.,D)
-
It is easy to seethat I11J'DII= L max {-1J'v(e,), -lpv( e,)}.If (T,D,L) is a 'f!T) standarddecision problemthen 1111' D = 2: L,II. II
II
,.T)
52.4Lemma. The lower envelope11'0of a convex,compactsubsetD = (IR\037)* satisfies
-
-lI'D(12)1111IJDIIIYl Y2Ico,)iE 11I'D(YI) \037
Proof Easy.
Rt i = 1,2.)
0)
52.5Definition.Let E = .rd, t E T}) be Q ..... be such that) et E A (T)let 1;.: dp' (w) I.f tEet, = d 'L. \302\253(,1,
{\037:
an
For experiment.
every
R\037
$I'Q
(\037(w\302\273,
Io
\037
if
t
;
et
.)
Then = .!l'(1;.1
is calledstandardmeasure of = (Q,.r;I,{p,:tEet}) and of E. A(T) is the systemofstandardmeasures It isobviousthat the mapping1;.. is E(I-sufficientfor every et e A (T).) \037
(1\302\253
\037)
E\302\253
.sE\302\253
\302\253(1\302\253)\302\253\037
52.6l.emma.Let EE SeT)be an experimentandJet (T, D,L) be a standard decision problem.Thenforevery et E A (T)andevery nonrandomizedII E .
of given d(\037). Proof Let el be any version of the conditional expectation We will provethat P,{el;D}=Oif tEet. \037
Let M be the family of all affine-linearfunctionalst:
t(x) 0 if XE D.Then {e, D}= (r;,M) U {t0 (!I> O}. \037
;
If Mo c M iscountable and densein M then)))
R\037
.....R
+
satisfying
268
Chapter 9:Comparisonof Experiments)
;
{el D}= \"Mo) U {t0 fll > O}. notethat toele \037(toQI.Qt(1;\302\273 if IE and thereforeto(l1 0 \037-a.e.if 1E ex. Henceal can be adjustedto obtainqo. 0)
We
(X
\037
beenshown in Theorem51.4the comparison of experiments may be basedon the Bayesianrisksfor standarddecision These problems. Bayesian risksmay be expressed as integralsof the standardmeasures over concave As has
functions.)
52.7Theorem(Blackwell [1951]). Let Ee I(T) and let (O'(I)UA(T)be the associated Thenfor every C(E A (T)andfor every systemof standardmeasures. standarddecision problem(T, D(I'L) where D(I \302\243
R\037
= 1pD..d(1t,. inf L L,(!p, 1..) lEa QE.(E,D.)
Proo! It is
sufficient to considernonrandomizeddecisionfunctions D. (l e 9t(E,Da) which in the followingare viewed as mappings(l:Q ..... 0 = to Lemma52.6we may assumethat (} (lo It isthen clearthat According Q
1;.
L L,(lP,= J (eo(y),y)
C1\302\253(dy),
lell)
Thus we obtainimmediatelythat
L L,(lP, J f.PD.(Y)
inf
\037
D.) '\"a)
Q\".(\302\243.
To provethe reversed inequality choose > 0 and let (X\/lt'N") be a sequence \342\202\254
which isdensein Da.Forevery y e Sa define(It(Y)E D in sucha way that (l,(y) = Xk if keN is the smallest integersatisfying (xt,y) f.PD'
\037
Then(!,:8v. ..... function asis easily seenfrom) Da. is a measurable {Ya
e Sa:(le(Y)= Xk}
=
1-1
n {ye 53:(Xi,y)> V'u..(Y)+ t}
i=
I)
f\"'I
+ t}. {ye SQ:(Xk'Y) 1pD.(y) \037
we have) Moreover, inf
=J
L '.v.L,(}p, lev.) \037
IlE9l(\302\243,D.)
\302\253(le(Y),
\037
Y) t1a(dy)
L,(at.\302\260
\037
TJP'
0)))
52.Concave function
52.8Corollary.Let f. 0 and let \037
critcria
(O',')UA(TJand (ra:).:!EA(T)be the t
system of
standardmeasures of experimentsE and f: 71,ellE F holdsiff for E A (T) andfor every convexpolyhedron/)(1: \037
rx
\037
269)
every
IRa)
+\302\24311'1-'0./1.) Jtpo.dO'('( Jtpo..d'r('( \037
Proof CombineTheorem51.4and Theorem52.7.
U)
t
It is easy to formulate a similarcriterionfor the relationE 2 F. Related in Section 54.) will be considered questions \037
52.9Lemma, Assumethat M s; ST iscompactandconvex.Thenfor every upper
concavefunction f M -+ IR there is a closed,convex set semicontinuous. J)
\037
([RT)* such that
f(y)=inf{<x,y):xED},YEM.) Proof SinceM is compactthere exists E A (T) such that M s; Sa:.We will is true for D == {XE(IJf/\*:") <x,y)\037f(y) if yE M}. show that the assertion = Let W {(y,{J)E M x IR: f(y) {J}.Then W s; IRT X IR isclosedand convex. Chooset> 0 and Yo E M. Then (Yo,f(yo)+ r.),Wand hence there exist rx
\037
Xo E
(IRT)*,Y E
such that)
IR,
<Xo.Yo) + ,(f(yo)+
whenever}'E M, P We
\037
\302\243)
\037
C < c + <5
<xo,y) + yfJ
f( y). Putting y = Yo and P = f( Yo) it followsthat}'< O.
normalize Xo in such a way that y = (Xo,Yo)
\037
-f()'o)- t
\037
-
-1and obtain)
<xo,y) f(y)
if
ye M.
T SinceYo E Sa:and M S; Sa:there is a translation XI E (R )* of Xo such that)
(XI,yo)-f(Yo)-C=O \302\253
We
obtain
assertion.
Xt
E
x 1,y)-f(y) if YEAf.
D and f(yo) <XI'Yo) =f(yo)+ t. This proves the \037
0)
52.10Corollary(Blackwell[1951J). Let (0'a:)a:(IA(T) and (ta)(I:(IA(T) he the system,; ofstandardmeasuresofexperimentsEandF. Then E => F iff for every andfor every concave,upper semicontinuous functionf Sa -+ R) JfdC1('(
\037
rx
EA
(T)
Jfdta:.)
issufficientfor E ::>F ProofFromCorollary52.8it followsthat the condition sinceevery supportfunction 'I'D.with D(I R(I: isconcaveand uppersemicontinuouson Sa. Conversely,the conditionis necessary in view of Lemma \302\243
52.9.
0)))
Chaplcr9:Comparisonof Expcriments)
270
53.Hellingertransformsandstandardmeasures) Recall that if E E SeT) is an experiment then E A (T), denotesthe restriction of E to the parameter set s; T. In the followingwe define = 1.) E\037,
(X
\037
0\302\260
53.1Definition.Let E = (D,d, {p,:IEI'})be an experimentand let 91 .91 be a finite subfieldof d. If BI , 82 \"\",B\", are the atomsof 91 then the \037
conditional tran:,formgiven 91is) Ilellinger m
19I)(z):=LI tn 'p'='(J\037)
H(E\302\253
if
(X
E
A
j=
E\302\253)
(T),Z E S-z..)
53.2Remark. Recall Holder'sinequality for nonnegativenumbers atJ , lET, 1 j $ m:) \037
m
L
j= I
n 0:;
m
\037
rECI
fI (L atl' J=
rECI
if
E Sex'
Z
IJ.
EA
(1').
I)
53.3Lemma. Let E E 8 (T)bean experimentand!Ms;.f/I a finitesubfield of.f/I. Then 0
\037
H(Ea.I 91) 1 on S(lJorevery \037
ProofIt is obviousthat
(X
EA
(T).)
for every H(Ea.IfJ4) O. Holder's inequality implies \037
ZE SCI)
H(E I9l)(z)= cz
= L te-z. fl p,:'(\037) tea. fl (L P'(\037)y' 1. 0 j-I j=1) m
m
\037
53.4Lemma. Lei E E S(T) be an subfields of ,rtI.Then) fJll\037fJl2
for every
(X
EA
experimentand let 911 and
!M2
be finite
on Sa implies 1l(EaIJll)\037H(Ea.I!M2)
(1').)
ProofThe relation911 912 impliesthat the atomsof ,tjl2 are a finer partition of than the atomsof 96..This meansthat every atom of 8ft 1 is the unionof followsfrom the fact that for pairwisedisjointatomsof912 , Now the assertion unionC C of sets E v E C C disjoint 1 any I 2 2 sI, Holder's inequalityimplies)
!l
\037
\037r4,
fl'p'r'(CI U C2) = fl ('p'(C I ) + .P,(C 2)Y') n p,r,(c1) + fI p'r,(c2). /(
If'CI)
I\"a.)
\037
a.
0
tEer)
53.5Definition.If E E 8 (T) then for every
(X
E
A
(T) the Hellingertransform)))
53.Hellingertransfonns and
slandard measurcs
271)
H(EJ:S2-+ [0,1]is definedby (z) = inf
H(E\302\253)
I
{//(E\302\253
&I) (z):fJI
\302\243
d a finite subfield} .)
53.6Theorem.LetE = (D,d, {p,:t E T})bean experiment. IfrxE A (T) andif vld isa a-finitemeasuredominating{p,:t E ('j,} then)
-
dP. ' n H(Ea)(z)=JlEa: ( d )
%1
ZESa.'
dv,
V)
...,
for probability measuresvlsl. Proof It is sufficientto provethe assertion (1) Let 11c d be a finite subfieldwith atoms(81,82 , 8m). Then we
have for every j E
J
Bj
..., m}, :'
{1,2,
d [1 'Ea: (d )
dv
\037
\037
E
Z
\\
Silo')
dP' = [1 (J d dvy' II P':'(\037) 'Ea: V
BJ
'Ea:)
and this implies)
dP, J ha. dV ( )
%'
[1
-
dv
\037
f.
n P'Z'(\037).
)\"'II(a:)
Thus we obtain)
I n (dP' J 'E d )
Z'
dv
\037
V)
a.
H(Ea:)(z),ZE Sa:.)
(2) To provethe reversed inequality let I; > 0 and z E Sa be arbitrary, and in such a way that the functions choosea finite subfield \037
m
\037
d
p, (\037)
h =.\037 V (B) 1s}, t E
)-1
satisfy)
dP.' dv
Z,
(j\"
> 0,
J)
-[,
t
%. \037
1
\037'
tE
rL,
Z,
> O.)
Sincevld isa probability measurethe possibility ofsucha choice followsfrom Let = martingaletheorems. ('j,
{'1't 2 ...., 'II}'Thenwe obtain)
- }\"1 i: II dP, f1 :,>0( )
n (dd.P,V ) J lea
%.
dv
z'
\037
S I\037II,
'p'z,\037)
I\037a:
dv
n
jz.
lEa:,:,>0)))
dv
Chaptcr 9:Comparisonof Experiments)
272
%'1
\037'!.
'Ie\302\253.:,;>o
\037
dP. J --!! ft'l n n . f/'J. dv ( ) k
L
\037
dv)
dP'I
L \"t!a.:r,>oJ
[( (
%'1
%t
dV
) :' Lz, dv =. dP' ---!. = L /,z, It!a,=,>0(S (dv ) )
-
j, J ( )<. (Usethe fact that Ix - < Ix dP. \037
L
\037
f
A
dV
%tl(
n (f ) k>i
dp'\"dV dv
%II(
)
]
Z'
E;
d\037'
dv)
'..\302\253.%,>0
z
the assertion.
yZI
yl%
if
x
\037
0, y 0, 0 < z < 1.)This proves \037
0)
53.7Remarks. (1) If 'P%:Sa.
dv
-/,:':t ) ' n ( /,1( 1
dv
%11t
-.
IR:
Z
E
Sa. then we denote)
x...... n x:'. lEa)
The function 'P:iscontinuous and concaveon Sa.However, for x
\342\202\254
8Sa the
functionsz 1-+'P:(x),Z E S(%, are notcontinuous. Fromthe preceding theoremit followsthat H(E,J(z) = J 'P%d(fz, Z E Sz,where (fa is the standardmeasure of
s.
Ea, ex E A (T). 53.6it followsimmediatelythat H(Ea.)(z) (2) Fromthe fonnulaofTheorcm = H(E\037) (z) if Z E fJ S;; (1. the terminology of Section 5 it isclearthat H(Ea.)is the Hellinger Recalling transfonnof the standardmeasure (J(%, (1E A (T). This yields the following S\037,
assertion.) uniqueness
53.8Corollary.Let E E SeT),Fe SeT).For
ex
E
A
(T) let (fa be the standard
measure of and let t be the standardmeasure of F(%. 71zenthe following assertions are equivalent.') E\302\253
(1) (2) (3) (4)
02
E-F, E02 \037
-
=
F\",
f02
for every
for every
fJ.
ex
(T), E A (T). \342\202\254
A
H(4J= H(F;.)for every
ex
EA
(T).)
we ProofIt followsfrom Corollary52.8that (1) (2).FromCorollary52.10 obtain(2) => (4), Theorem5.19yields (4) => (3) and (3) => (2) is another of Corollary52.8. 0) consequence \037
53.9Corollary.Assume that T is a finite set and let
(J, t
be the
.\037tandard)))
54. Comparisonof experimentsby testing problcms
273)
are equivalent:) measuresof E andF. Then the fol/owinKassertions
F (1) E...., (2) (3)
(1=
t,
H(E)= H(F).)
Proof It isclearthat (1) (2) \037
I/(E) = lI(F)implies
I/(E\302\253)
followsfrom the fact that (3).The assertion = ll(\037) for every E A (T). 0) \037
ex
The folJowing is a useful lemma which alJows to reduceequivalence of
ratio processes. of to equality of likelihood It is an extension experiments Corollary25.9.)
53.10Theorem.Let E and Fbeexperime1ltsin I(T)andlet P/ = {p,:t E T}and !l= {Q,:t E T} the corresponding setsofprobability measures. Then E F iff)
-
dP. = !I'((dl's )leT p.) !I'((dQ:sleT ) Q) \037QI
$
5)
for every SET.)
of the precedingresultsand becausethe distribution of a isdetermined stochastic process by itsfinite dimensional marginaldistributions it issufficientto prove the assertion for finite setsT * 0.But this has beendone in 25.6. n) already Corollary Proof In
view
54.Comparisonof experimentsby testingproblems) us explorethe relationbetween comparison of experiments by general and decisionproblems by testingproblems. As indicatedpreviously both for coincide To concepts binary experiments. provethis we needa formula to Iransronn i ntegr.Isfor (J.intointegr.lsror Jl. !R where PI) E = (PI'Pl) is a binary experiment. Then we arc in a positionto apply the resultsof Chapter3.) Let
\037
(\037\037
54.1Lemma. Let E be a
binary experimentand 'PD the lower envelopeof a !}(2. 71,en)
convex,compactsetD J 'POd<1E= 'Po(O,1)+ J ['Po(l,t) \302\243
- 'Po(O, t\302\273
\037lt(dl).)
Proof Let E = (PI'P2). Elementarycomputations yield)))
Chapter 9:Comparison of Expcrimcnts)
274
= J 'PO('1h d1 d\"2) + J 'PD('11, '12)ud '12)UE(d'11, d'12) J'I'DduE \". =0 \". >0 = J 'PD(O,1)'12Uf,(d\"\"d'12)+ J !J.'D 1,'12 '11 d'12) '1.=0 '1.>0 ( '11) uE(d'1t, dPl = 0 + dP2 = 'Po(O, 1. 1)P2 'Po f ( dPI ) dP. {d(Pl + Pz ). } = 'PD(O.l)(l $ I + $ 'PD(1. I) J.lE(c/I). 0) 1\"
-
J.l,.;Cdl\302\273
54.2Theorem(Torgersen[1970]).LeIE = (PI'P2) and F = (Ql'Q2) bebinary Thenfor every 0 Ihe relationE F is valid iff E ;2F. experiments. \302\243
\037
\037
of the assertion isnot trivial. Assumethat E F. ProofOnly onedirection \037 2)
Then we obtainfrom Theorem17.1that) JfdJ.lE; JfdJ.lF+ \037
;(f'(0)
+
f\302\253(YJ\302\273)
functionf [0,oc)-+ [0.co).f(O) = O. Let D for every concave, increasing Then) be an arbitrary convexpolyedron. 'PlJ(Y)= min {<x\037,y>: t
if D = co{Xl'Xz. J 1pD
d(]\302\243
...,
\037
\037
\037
n},
R2
ye\037,)
X,,}.We have to show that
J 'PDd(]F+ e II 'Poll.)
Let us denoteXi = (\037i' '1i)'1 + max I\"d.We denoteu = min
\037
1
i
\037
1
\037i\037n
i n. Then \037
\037I
and v
1
\037i\037\"
n_ J UYI J 'I'DdUE: = J 'I'DduF.
we have II'PDII= max I 1 :;iiS\" = min '11' Since
d(T\302\243(Y)
\037d
;li\037n)
+ Ii =
S'I'D(u. O)d(TE+ u
and similarly
+U J 'PDduF = J 'PD-(u.o)duF we obtainfrom Lemma 54.1that) J
-
= ['PD-I\".0)(1, I) - vl]J.lF(dl) t) - vt]J.lE(dl)- f['Po-(u,O)(t, -u) + 1('11- J.lE(dl)- J min - u) + I(\"i-. = J min
'PDd(1\302\243
= J 'PD-(u.o,duE S'fJJD-(u,o)duF
$ 'PDduF
$
1
\302\253\037i
\037i\037n
the function))) Considering
\302\253\037i
v\302\273
1 \037i::':\
v\302\273
J.lF(dt)
54. Comparisonof ex.perimentsby ltslingproblcms
J(/) = 1mm
- u) + I(\"j-
((\037j
I 0, \037
v\302\273,
:!i\037\
we observethat it is concave.It u = \037II' V = \"j2'then we have f(O) = min (C:j u) = 1
\037I\037n)
1'(0)= min (\"i ,:,=u
-
v)
is increasingsincetli
-
\037il
\037
U
275)
=
-
v
\037
0, 1
\037
i
\037
n. If
0,
2 max I\"d. 1
\037j\0371I)
J(oo)= min(\037I-u);;!2 1max '1,=11
I\037d.
\037j\0371I)
Thus,we obtainthat)
-
-
J 'PodrJf. J 'PndrJF= JJdJ.lf. JJdJ.l,.. f.
\037
\037
2 (I'(0) \302\243(
max 1
+I(Cf:)) I\037d
+ max I
\037j\037\"
1,,;1)
\037i\037\
which provesthe assertion. 0)
..
with an arbitrary parameter spaceT 0. Let us return to experiments '\" and\" \" are identical. that t he For at least equivalencerelations\" prove thisis a consequence of Theorem25.8,Corollary25.9 dominatedexperiments and Theorem53.10. Let us consider E = (D I' \037 = {P,: experiments lET})and = {Q,: F= (Q2td2,f2 IE T}).Let (CTa)al'A(T)and (!a)2I\".4(T) be the systems of ofE and F.The followingisan extension of Theorem22.5.) standardmeasures We
=t=
l'
d l'
54.3Lemma(TorgersenrI970]).Let II
L a,\03711 'eT
Jor every a E (IRT)*
\037
II
\302\243
O. Then
\037
E? F iff
L la, leTa,Q,II- ,cT) L
e
I
.)
F iff for every Proof We obtain from Theorems51.3and 52.7 that E;2 2 x <XE A(T) and every pair (X 1,X2)E ;;!J min (XI> y) '01(dy) + L max (xl),I. J min (Xj,y) RIZ
j-1.2
CT\302\253(dy)
i-I.2
R\302\253
E:
I
'E\302\253
i-= 1.2)
It issufficientfor E ;2F to requirethe condition for all pairs(xl'Xz) E W x IRa 2 which are centeredat zero,i.e.x I + X2 = O. For such pairswe have))) &
276
Chapter 9:Comparisonof E'l:periments)
.
(Xi> y)
-2 1
=
i\037:\0372
and)
1
1 max I (Xi),I = - lxI,' 2
i'\"
),2)
.-(xz.y)
(X 1>y)
I
- xz.,I.
F iff for every a. E A (T) and every pair (XI.Xz) E RII x W It followsthat E -2 2 such that Xl + X2 = 0)
-
fl(xl,y) (x2,y)la(l(dy) \037
- (Xz.y)I'tIl(dy) - t L - x2.,I.
JI(xt,y)
Ix)\"
'CCI)
Thisinequalityis equivalentwith)
L (Xl\"
J
'\037CI
-xz,,)d'L,dp'p. d L I's s
s\037\"
.sEll)
\037
-Xl.,)d dQ, LQ
L (XI., J 'cer
S
-
d L Qs e L Ix)\" 'F(I SFCI
-x2.,I.
Hill)
Putting a = x I
-
Xz
we obtainthe assertion. 0)
Let e O. Then E F iff for every pair 54.4Corollary(Torgersen[1970]). ml EST.m2 EST the binary experimentsEo = (J P,mt(dt).J and f'o = <JQ,m)(dt).fQ,mz(dt\302\273 satisfy Eo;2EO.) \037
\037
p,m2(dt\302\273
t
ProofAssumethat E 2 F. Let ml EST.m2E ST' We
have
to show that
z)
\"aJ .p,m)(dl)+ b J 'p'mz (dl) \"
\037
for all a E
IR.
lIaJ Q,m.(dt)+ b J Q,m2(dt)lI8(lal+ Ibl))
bE IR. Fromthe preceding theoremwe know that
lIaJ p'm)(dt) + b J p,m2(dt)
II
L lamI {t}+bmz{t}I. \037lIaJQ,mI(dt)+bJQ,mz(dt)lI-e ,,,T) The assertion followsfrom the fact that)
L lam){t}+ bm2 {t} lal+ Ibl. ,e I
T)))
\037
54.Comparisonof experimentsby testing problems
Assumeconverselythat the condition issatisfiedand let (R T)*.If a, every lET or if a, 0 for every lET then it is easy to obtain) a\342\202\254
\037
277)
0 for
\037
II
L a,\03711 'fT
\037
t
L a,Q,II- '(1') L la,l. ,eT
II
Otherwiselet ml {t}= a,+ /
r a,+,m2{t}= a,-/scL a,-, IE T.
IE T
It followsthat) II
:E at rGT
\037
;;?:II
II
=
T)
rr.T Q,II- t (I :E a at
$lOT
f
s
I
+
I
r as I)
$lOT)
LT arQrll-erELT) larl.
II
rE
Thisprovesthe assertion.
0)
54.5Theorem(Torgersen[1970]).E 2' F iff E
-
-
F.)
E '2 F.Assumeconverselythat E 2' F. Proof It isobviousthat E F implies Then for E A (T) and mESawe put 1 L P, and QI = L m {t} P\" PI = oc_ 'ea ex
_
P2
rCHI
I
I
=. 1-- LQr loci
and Q2=
h.G!)
L m{t}Qr. '\037a)
ThenCorollary54.4implies (PI'Qd ...,(P2,Q2)'Hencewe obtainfrom Corollary
16.5)
!I!(
\037;,,'
p,)
\037
!I!(\037;,:Q2)
which implies
!I!
C\037
m
{I}d \037 p, s&a
i:1
.\037.
\037
p,)
!I!
m{/}d
C\037.
\037Q \037:
s.a)
I:'
.\037.
Q.).
Now an easy argument usinguniqueness of Fouriertransfonnyields)
_!..-L P = !f' dQ, !i'((\037 d :E )tia 1<xI.lu) ((d L Qs) S
\037
sea
sea)
\037
L QS .
'Ell loci sea
)
Thismeans that the systemsof standardmeasures ofE and F coincide.
0)))
Chapter 9:Comparisonof Expc:rimenls)
278
54.6CoroUary(LeCam[1964]and Torgersen[1970]).111e followingassertionsareequivalent:) (1) E 2'F. (2) E F. (3) There is a linear isometryM: span# span!!lsuch that M (P,) = Q, if
-
-
te T.)
ProofApply Theorems54.3and 54.5.
0)
55.The randomization criterion) Let E and F be experiments in G (T).The aim of thissection is to provea generalversion of Theorem25.8.)
55.1Definition.A R: !l'b(Q2' .912) X
randomization from E to F is a bilinearfunction L(E)-+ IR satisfyingthe followingconditions:
(1) IfI\037 0, a 0, then R(f, CT) 0. \037
\037
(2) (3)
IRU:CT)I 1I/11\"IICTIi. \037
R(1,CT)=t1(Ql)' The setof all randomizations from E to F isdenoted by !B(E, F).)
-
55.2Definition.A stochastic operatoror transitionfrom L(E) to L(E) is a
positivelinearmapM: L(E) L(F)suchthatllMCTl1 = IICTII ifCTE L(E),CT O. Every stochastic operatorM: L(E) -+ L(F) definesa randomization RM t 1 E to RM(f, (1)= (M(1)(f),fE (Q2\"w' from E toF according 2)' L(E).E.g. condition (3) followsfrom) \037
\037b
RM
-
(1.a) = (MCT)(1)= (Mq)(!22) -:-:M(q')(!22) M(a-)(!22) = lIa+1I-!la\"\"11 = IIM(a+)II-IIM(a-)1I = a(O.).) = CT+(O.) q-(O.)
-
-
The converseis not true in general.If R is a randomization then there is a
mappingMR : L(E) ba(02'd 2) whieh is defined by MR(a) (f) = R(f, CT), d2), aE L(E).The mapMR is a positivelinearmap suchthat fE 9'b(Q2' = (1E L(E),a O. Thisisan easy consequence of the properties MR (111. of R,) II
\302\253(1)
II
II
\037
55.3Definition.An experimentFE &(T) is a ralldom;ZQt;ml of E jf there is a R E \037(E, F) suchthat Q,= MRP, for every t E T.))) randomization
55.The randomizationcriterion
279)
of dominatedexperiments. In Section24 we have defined randomizations
two assertions are compatible.) The following imply that bothdefinitions
55.4Lemma. For every randomizationR from E to F there ;s a stochastic L(F)such that operatorM: L(E)...... lET.) IIM\037-Q,II\037IIMRP'-Q,II, ProofBy Theorem41.7there existsa positivelinear operatorT:ba(Q2'.912) ...... L(F)satisfying TII= 1,(TC1)(1)= (1)if C1 0 and TIL(F)= idL(F)'Define II
\037
M:==ToMR' Then
-
-
-
liMp, Q,II== TMRP, Q,II= IITMRP, TQ,II = IIT(MRP' Q,)II IIMRP, Q,II.) tE
II
T, which provesthe assertion.
\037
0)
55.5Theorem.An
exper;mentFE S(T);sa randomization0/ EE SeT)iff there i'ia .'itocha.'itic operatorM: L(E) -+ [.(F)such thaI Q, = M P, for every t E T.)
55.6Remarks. (1)The set14(E,F) can be topologized by the weak topology which is the coarsest for every topologysuchthat R \037 R(f,J.l) iscontinuous E \302\243l:'b(Q2,.td 2), J.l E L(E).A similar argumentasin Theorem42.3for .rJI(E,D)
/showsthat 91(E,F) iscompactfor the weak topology. (2)
A
subsetof \037(E, F) is the setfR(E, f) of all stochastickernelsK: If KE 3t (E,F) then [0,1].
QI x .912 ......
R,,:(f, /1)
\037
J I/(wz ) K(fl) , , d(J)2)/1(dwt ),
.912), J.l E L(E),is an element of fM(E, F). A copyof the proofof Theorem42.5shows that {R\"E \037(E, F):K E fit(E,F)}isdensein fj(E,F) for the weak topology.The only differencein the argument is as follows:The we may choose functions/I'''' ,I\",have to bechosenin 2, .91 2) and w.l.g. function. IfC C is an .9I -measurable of , step t z partition Qz suchthat each p h, 1 i m, isconstanton eachC , 1 k p, then we definePic 1=lck, 1 k p. Now the proofis literallythe same as for Theorem42.5. that E isa dominated t E T}) (3) Suppose experimentand F = (Q2,.91 2 , {Q,: is such that Q2 is a locallycompactspacewith countablebaseand .r42 = 9I(f1 M:L(E) ...... L(F) there existsa kernel 2). Then for every transition K 9t(E,F) such that RM = R\". The proofis easy.Indeed,RM E \037(E. F) is such that R M (.,P,) is a Daniell integralon \037b(Q2) for every tE T. Then the existence of the kernel K followssimilarlyto Theorem42.7. we criterion. We followLeCam Now, approachthe importantrandomization
/E
\302\243l:'b(Q2'
...,
\037
\037
\037
\342\202\254
[1964].)))
\302\243l:'b(D
tc
\037
\037
\037
Chaptcr9:Comparisonof Experimcnts)
280
55.7Lemma. Let
e
+ 1]T,If [-1t there isan R
0 and D =
\037
t
E;2F
then (E,F) such that
for
every
E 9t(F,D) and every mE ST (MRp')m(dt) J L,(!z Q,m(dt)+ (;,) J L'(!2 decision function Q2 e d'/(F t D) such that ProofThereis a nonrandomized
(12E
\037
\037
= J L'(!2 Q,m(dt) J L,Q2Q,m(dt) ,)
t
> 0 be arbitrary. SinceE => F there existsa nonrandomized simple decision function el e 91(E,D) such that + t + {). J L,Qt'p'm(d/) J L,Q2Q,m(dt) It may beassumedthat QI takesitsvaluesin the convexhull of the imageof Q2' Let
{)
\037
Let) '\"
01= i=L xi 1Aj 1)
\"
\"
L L (1,lj = 1tYjE im'?zt 1 j nt 1 i m. j=1(1,lj}j, j=1) Forevery j e {1t 2t t n} let wj e O2 be such that Q2(wj) = Y,;. Thenwe define the randomization R by) where Xi =
\037
\037
\037
\037
...
d2), J.l L(E), L L (1,iJf(wj) J.l(Ai)tf E !L'b(Q2t i:;:1 j:;:1) It is obviousthat R E (E,F). Moreover,for lET we have) R(j,J.l) =
In
\"
\342\202\254
\037
= L,Q2(M = R(L,Q2' L,Q2(M R P') R P,) P,)) =
In
\"
L L
,Pr(A i) (1,lj(h.,(Wj)
i\037lj\037l)
which provesthat
'\"
n
=
L L (1,ijYj\"P'(Aj) i=1 j=1
=
L xj.,P'(A ) = L'(!1 i:;:
'\"
j
p\"
1)
+ e + {). J L'(!2(Ml(p')m(dt) J L,(12Q,m(dt) Since{) > 0 was arbitrary we obtain) \037
-
inf
R P')m(dt) J L'(hQ,m(dt\302\273 (sL,Q2(M Now the assertionfollowssincethe infimum is attainedon the compact set(fI(E,F). RE\037(E,
F)
0)))
\037
\302\243.
55.The randomizationcriterion c
55.8Lemma. If E;?F then for every mE ST
281)
there exi.<;tsa randomization
R E f!4(E,F) such that
+ 1] Proof Let D = [-1,
J IIMRJ: Q,lIm(dt) e.) T.
4) (MR
\037
we denote) Forconvenience
-
,(!,m) = J L,(!(A1 R J:)m(dt) J L,\037Q,m(dt)
if R E f!4(E,F),(} E
9t(F,D) and mE ST' Then the preceding assertion implies inf
sup
R.IiI(E,F>)
\037E9t(F,D)
tP(MR ,(!,m)
\037
t.
Fromthe MinimaxTheorem(Theorem45.8)we obtain sup cP(MR' (!,m)
inf
\037
C
Q.\037(F.D')
Re9t(\302\243.I',
for every mE ST' SinceR \037
sup
(F,D)
QE!Jf
4) (MR'
on (!,m) is lower semcontinuous
the compact setf!4(E,F) the infimum isattained.It remains to show that for every R E 91(E,F) and every mE ST) = JIIMRJ:-Q,lIm(dt). sup clJ(MR,(!,m)
QEStCF.D))
Firstwe notethat obviously) clJ(MR , (},m)
-
JIIMRJ: Q,lIm(dt) for every {!E 9t(F,D).On the otherhand,for e > 0 and for every t E T there isa E function.f. b (Q2',$/2),lI.f.ilu 1,suchthat
!e
\037
-
\037
-
J/,d(MRJ:) JlrdQ, IIMRJ: Q,II-t.) Let {!,E 9t(F,D) be the nonrandomized decision function which is definedby \037
{!e= (f,)'ET then we obtain)
\037
J II MR P,
This provesthe assertion.
- Q,lIm(dt)-
1;.)
0)
In the light of Sections The foJ1owingtheoremis the randomization criterion.
22-25it solvesthe problemof \"approximate sufficiency\".)
55.9Theorem.(LcCam[1964]).Let e
t
\037
O. Then E;2F holdsiff there exists
R E f!4(E,F) such thaI II
-
MRJ: Q,II e for every lET.))) \037
Chapter 9:Comparisonof Experiments)
282
t
Proof(1) Assumethat E;2F. Then we obtainfrom the precedingassertion
-
that)
sup
inf
We
f..
J II MR J: Q,lIm(dt)
\"'ESTRE9I(E.F))
\037
apply Corollary45.7with J{\037o(T)= S1\" Let M 1 = M2 = {(IIMRP, Q,II>'ET: RE a(E,F)}.)
-
{e}and
The assumption Clearly,Mz issubconvex. implies 1pM2(m) 1pMI(m)for every mE ST'Theproofisfinishedif (M2) isclosed in IRT. But this isa consequence of Remark 45.4if we put) \037
(X
f(t, R) = IIMRJ:.-Q,II, t E T,
RE
\037
(E,F).
-
(2) Assume that there is R E (E,F) such that II MR;: Q,II t E T. Let D be an arbitrary decision space.Thenfor every (12E .<M
\037 \037
l;
for every
(F,D))
x L(E) R: (j:J-l)\037fe2(M2Jl)) p:rcb(D) is an element of 91(E,D) and it satisfies \037
P(W;, J:)
\037
W;e2Q,+ ell W;II)
for every boundedcontinuous lossfunction W: D x T
.... R
0)
to L(F),thus obtainingthe following The range of MR can be restricted of Theorem25.6.For finite T, the secondpart is dueto Blackwell extension
[1953].)
55.10CoroUary. (LeCam[1964]). (1) Let l;
stochastic operatorM: L(E) II
(2)
-
M.p, Q,/I
\037
\037
tift
\037
L(F)such that
O. Then E
t
2 F iff there existsa
E T.)
E 2 F iff there existsa stochastic operatorM: L(E)
M;:= Q\" t E T.)
ProofTheorem55.9and Lemma 55.4.
\037
L(F)such that
0)
and relatedto the concept Thefollowingversionisin the spiritof Heyer [1982]
of exhaustivity(seeSection 23).)
55.11CoroUary.Supposethat E isa dominated experimentand
F= .\"2,{Q,:t E T}) i.r .mch that Q2 is a locallycompact spacewith baseand.J11 countable 2 = \302\253(12'
14\302\253(12)')))
55.The randomi7.ation
(1) Let E
c
\037
O. Then E;2F iff there existsa
[0,1]such that II
(2)
K P,
-Q,
II
\037
t,
t
E
criterion
kernel K:QI x .flI2 stochastic
283)
\037
T.
kernel K:Q s12 --.[0,1]such that E;2F iff there existsa stochastic I
X
KP,= Q\" tE T.) and Remark 55.6(3). ProofCorollary55.10
[])
we provea generalization of the convolution theorem.) application 55.12Theorem.Supposethat G isa locallycompactgroupwith countablebase As an
.
E=(G,.\037(G),{t:g*P:geG})and F=(G,\037(G), are valid: {8g Q:g E G}).If G is amenablethe the fo/lo\037ing assertions > Let t O. Then E;2F iff there existsa probability (1) (Torgersen[1972])
and P\037lr' Let
-
measureKI\302\243B(G) such that
liP* K QII E F iff there existsa probability measure KI\037(G)such (2) (Boll[1955J)
that
P.K = Q.)
\037
\302\243.
\037
measureK such that ProofIt issufficientto prove (1).If there isa probability K.P Q E then it isclearthat II
-
.
II
\037
P* K
-
-
Q = \"P*K Q
g E G,) c which implies E =2 FsinceIlH Il*t K, IlE L(E),defines a randomization from E to F.Assumeconversely, that E F and let R bea transition from E to F such II
E,
F., *
II
II
\037
E,
\037
that)
IIMR(Sg*
P)
-
(Sg*
Q)II
\037
t,
gE G.
In view of Remark 55.6(3),R can be supposed to be a kernel. Let (KII)IIEN be a summing sequence of compactsubsetsof
randomizations RII from E to F, n E
--
-1 R II (f,Il)I=-:;A.
_
J R(jog (K\"Kn) )
G. Define
by)
\037,
1,C *Il)l,(dg), Q
fE Yb(G,\037(G\302\273, IlE[,(E).Let RoE f!J(E,F) be any weak accumulation point = of (RII)IIEN' Standard arguments yield Ro(fog-l, J.l) Ro(j,Il) if and Jl E L(E).Moreover, sincethe variational norm is lower fE Y,,(G, semicontinuous for the 9'b-topology it followsthat) of ba(G,
1;,.
\037(G\302\273
II
MRo P
-Q
II
\037
lim supII MR\" P
.n....
-Q\
<X>)
\037
supIIMR(tg P) ,eG)))
* (\302\243g
Q)II
\037
t.
\302\243M(G\302\273,
284
Chapter 9:Comparisonof Experiments)
M Ro by a stochastic Next, we replace kernell!E 9t(E,G).Let l!o:G x 11(G) a b e kernel such that [0,1]
-
Ro(f,J1)=f(!oJ1, fEt'joo(G),J1EL(E).) Define)
-eo(x,
l!(X,B):=eo(x,B) + (1
G\302\273
IB(x), x E G, BE 9I(G).)
It isthen clearthat l!isstrictlyequivariantand hencethere existsa probability measure KlaI(G)suchthat (lP= P * K (useTheorem48.9and cf. Remark 38.11 Moreover,we have) liP* K Q = Ill!P Q = Il!P Q (1) ll!oP QI(1)+ (1 I l!o(., G)dP).) (1\302\273.
-
-
to (loP- Q.This Notethat MRoPand that MRoP- (loPis orthogonal that implies 1 - J eo(',G)dP= (MRoP)(I)- (eoP)(1)= MRoP- eoPI(1)) II
II
I
\037
\037oP
\037
I
and hence)
-
liP.K - QII
-
lOoP QI(1)+ IMRoP eoP I(1) = eoP Q + MRo P eoPI!= MRu P Q
\037
II
This provesthe assertion.
II
II
II
-
II
\037
E.)
0)
55.13Remark. The assertionof Theorem38.15is a particularcaseof the
precedingresult.Let E be a full shift on an amenablegroupG then every a full shift experimentF = (G,31,(G), equivariantestimate (} E (Jt (E,G) defines = * Q:=g E G})with Q l!P. Obviously) E F and hence the assertion by {E\" of Theorem 55.12. part (2) t E T})and F = (!12' Let E = (QhSIth {.P,: SIt2' {Q,: t E T})beexperiments in that E F.Thenthere isa transition M:L(E) L(F)such cr(T)and assume = that M P, Q\" lET.It isclearthat the transition on M isuniquely determined span{p,:t E T}.The followingimprovesthis fact.We followLeCam [1974]. of (RT)* and let Let.YfoCT) be the setof all lowerenvelopes of finite subsets Jf'(T)be the vector latticewhich isgeneratedby {p,:t E T},wherep,:fiT R \037
-
\037
ontothe /'11coordinate, / E T.) denotesthe projection
-
55.14Lemma. Jf'(T)= Jf'oCT)-.1'f oCT).) Proof It isclearthat .JIfoCT)-.reoCT) .Jr(T).We have to show that JYo(T) JYo(T) is a vector lattice.SinceJYo(T) is a convex cone
-
\037
the)))
55.The randomizationcriterion
285)
-- .1tJo (T) is a vector space.From difference.1I'o(T)
-
-
(11' cp)+ = tp tpf1cp, (tp cp)- = tpucp cp,)
-
it
followsthat
it
-
iseven a vector lattice.
0)
In the followingwe denoteby Lat(E)the vector latticewhich is generatedby
{p':teT}.)
55.15Lemma. Let
E::: (Q,.91, {p,:
T}) be an experiment.There exists exactly one vector latlicehomomorphismJf(T) --t Lat (E):cp 1-+Pip satisfying tE
Pr 1-+\037, t E T.)
Proof Uniquenessis clearsince{p,:t E T} generates Jf(T). For provmg existence define(seeDefinition 52.5) Pip:=:'(cp c TJ' L
,ea)
l\037
whenever cpe.tf(T)dependsonly on the cp 0 Pa = cpo
coordinates of aEA(T), i.e.
it followsthat Sinceevery cp e f(T) is homogeneous
q>, tp E
K(T)and
E
rJ.
A
(T)issuchthat q>\" = q>, tp\" PI7.
= p.+ '1 = Pip f1 Pip\" If p\"v..' = PlpuP..\ are obvious. Thisprovesthat P\037Ip-trI'\"
=
P\302\253
Pip
tp,
is well defined.If then the relations)
P\"\"
\037
P..\"
q> 1-+Pip
is a latticehomomorphism.0)
Let E and F be equivalent experimentsin 55.16Theorem.(LeCam[1974]).
cr(T).If M:L(E) -.L (F) is a transitionsuch that M p, = Q\" t E T, then M is determinedon Lat (E) and it isan i.wmetricBanachlatticei.wmorphism uniquely \037--
----
from Lat (E) ontoLat (F).)
e .te0(1')then there are {ar .;:1 i m, t E ex} 1R such that = min L a,.i'p'. It followsthat M Prp Q_ for every cp E Jf oCT)sinceMis Pip 1 re\" a positiveoperator.Moreover,we observethat for every cp E .Yf oCT), Proof If
cp
\037
\037
\037
\037
\037i\037,\"
cp Q
p\302\253
=
cp,)
p.(1)= J cpdq\" = J cpd'tl1.= QIp(l).) Thisequation holdseven forevery cP E .K(T).Hencewe obtainfor cp E .KoCT))))
Chaptcr 9:Comparisonof Expcrimcnls)
286
II
-
-
Q., MP.,II= (Q., MP.,)(1) = Q.,(l) MP.,(l)= Q.,(l) P.,(l)= 0)
-
-
thereforealsoif qJ E Jff(T). Q. It isobviousthat Lat (E) = {P.,: E qJ K(T)}and Lat (F) = {Q.,: qJ E Jf'(T)}. HenceM is uniquely determined on Lat (E) and for reasonsof continuityon Lat (E).It isclearthat it is a Banach latticehomomorphism. The proof is finished if we show that M is isometric on Lat (E). Since = Q.,= MP.,it issufficientto showthat Q., P., for every cp e Jft (T).Since we have p..= (P.,)\"\"',etc. It lp...... p.and lp...... Q.,are latticehomomorphisms which implies = MP.,if qJ E \302\243'0(1')and
II
II
II
...
II
followsthat)
+ P., = (P.,) (1)+ (p.,r(1)= P.\" (1)+ P\" (1) = Q.,.(1)+ Q.-(1)= (Q.,)+ (1)+ (Q.,l(1)= Q.,II for every qJ E Jff(T). 0) II
II
II
If E is an arbitrary experimentthen L(E);2Lat (E). It will be provedin the followingsectionsth at for every EE 8(1')there is an experimentF --E such that L(F) = Lat (F). A property of an experimentE isheredilaryif alsoany experimentF E has \037
thisproperty.)
55.17CoroUary.Dominationis an heredilaryproperly,i.e.if E is dominated and F
\037
too.) E, then F isdominated,
-- ---
Then there exists Proof Let E = (Ql'.911, fJ = {p,:t E T}) be dominated. v E L(E) such that p, that we may V, lET. From Lemma 20.3it follows, that for every t E T) assumev E spanfJ Lat E. Domination implies d dP' f1k dv f1(kv) = lim J dP' lim P, V dV dp, k dv = O. = lim \037
\037
II
-
-
II
\037
\"-+
\"-+
J
dv
sIL... -)
A;-+w
4y
-
\037\"
-
such that Let F= (Q2,.91 tE 1'})and M:L(E) L(F)be a transition 2 , {Qt: M P, = Qt. te T. From Theorem 55.16we obtain M(P' p'f1(kv\302\273 = M P, M P, nk Mv = Q, Qt nk Mv, lET,and therefore)
-
lim II Qt
j,;...oo
This implies Qt
-
\037
-
Qt f1kMvll = lim II p, \"--00)
Mv,
t
E
T.
0)))
- P'f1kvll= 0,
-
tE
T.
56. Conical measurcs
287)
56.Conicalmeasures) The conceptof conicalmeasures is the technicalkey for a satisfactory
we consider representation theory of experimenttypes.In the presentsection from a general pointof view. They are discussed from the conicalmeasures view in Section 5 7. o f statistical point the set of all lower Let T * 0 bean arbitrary set.Recallthat jf'0 (T)denotes T envelopesof subsetsof CIR )* and let .K(T) be the vectorlatticewhich is {p,:t E T}.FromLemma 55.14we obtainthat generatedby the projections .K(T)= .Yt oCT) .Yt oCT).)
-
56.1Definition.A positivelinearfonnM on .tf(T)iscalleda conical measure. iscalledthe resultant of the conicalmeasure. We will provethe basicfact that every conical measurecan berepresented by somemeasure on the O'-field Q9 \037(IR) which is generatedby {p,:t E T}.
The point(MP,),eT
\037
IRT
IfiT)
56.2Lemma. Let M be a conicalmeasure, = O} is a vectorlatticeand C = {q;E .tf(T)j%:q>
=
Then.;\302\245'
-
\037
{q> E
.Jff(T):M(Iq>1)
O} ;sa propercone,i.e.a
convexconewith vertex0 satisfying C(I( C) = {O}.)
Proof It isclear (llI't +
is a vector space.The relations)
that.;\302\245'
-
I
I
q>21)
\037
lI't
n q>2
\037
lI't
U q>z
\037
I
q>d
+
I
q>2
1)
iseven a vector lattice.Thefact that C isa convexconeof vertex B followsbecause the mappinglp ijJ is 1inear.We have to show that = Let Cn( C) {O}. ipE C(I( C).Then there are q>1 E ip, q>1 0,and q>2 E cP, O. Sinceq>1 (fJ2 E.AI' we have M( q>2 CP21)= O. But 2 E E which the assertion. 0) + .AI', AI\", CPt 2 implies CP2 proves imply
that.;\302\245'
-
-
\037
I
I
-
I CP
1
\037
I
CPI
-
\037
I CPI
-
CP
1
\037
CPa
56.3Lemma. Let M bea conical measure. Then
\037
q;z
iff
q;z
-
q;1E
C
definesan orderrelationsuch that vectorlatticehomomorphism,)
\302\243'
(T)j.Al'isa vectorlauiceand cP t-.ip is a
is an orderrelation.For Proof It isclearthat the relation..\037\" on .1f(T)j.K the second (CPI U C(2)'\" isthe-leastupperboundof q;1and cPz. part we showthat It isclearthat (CPI U CP2) ipl and (CPI U q>2) ip 2' Let converselyip be any E (: upperboundof {iPl'iPz}'Then ip iPl and ip (fi2 and thereforeiP E ,Ai- and '1'2 E .;Vsuch that and cP CP2E C.In otherwordsthere are '1'1 + '1'1' cP CP2 + '1'2'))) \037
-
(fJ
\037
\037
\037
CPI
\037
\037
-
(fil
Chapter9:Comparisonof Experiments)
288
It followsthat
(cp
-
>
CPl)(')(cp CP2) 1/)1(')1/)2 and A ip (CPt U CP2) \037
-.
thereforecp
-
(CPl U CP2)
> 'PIn 'P2E.tV. Hence similarargument shows that lower boundof CPl and ({>2' 0) (CPt n CP2) is the greatest
56.4Theorem(Choquet).For every conicalmeasure M there is a measure JlI <8'>g(R)such that Mcp = J cpdJl if cp E .Yl(T). 'f!T) Proof The function cp t-+ M (IcpI) is a seminonnon :K(T) which satisfies M(CPl+ cpz) = M CPt + M CP2 if CPl 0, cpz O. Let H(T):=.Yr(T)/AI'be the nonnedspacewith norm ciJlIM 1=M(Icpl).Thecompletion neT)of H(T)with \037
\037
1\\
Hence,by Theorem6.9,there existsa respectto II.IIMis an Abstract L-space. measure space (D,d, and a vector lattice isomorphism Let us t/J: B(T)-+ L 1 (a,.91, define A)o S:J'f'(T)-+ Lt (0,.c(,A.): cp H CP(ip).) For every cPE .1l'. (T)we have Mcp = lIipliM= J CP(ip)dA.= J Scpd)'. We notethe relation Scp = cp 0 (SP,)reT for every cp E Yf) (1').Indeed,the setof isa vector latticewhich contains{P,:t E T}. functions which satisfy the relation = If we defineJ.l Z\302\253Sp')'fT!)') then we obtain \037)
Mcp = J Scpd\037 = J cp 0 (Sp,)u TdA = J cpdfl
for every cp E )f(T).
0)
56.5CoroUary.If III@ tI(R)is constructed as in
densein L 1 (Jl).
'E T)
Theorem56.4then Jft'(T)is
to ProofLet t > O. SinceH(T)isdensein !leT)and neT)is latticeisomorphic L 1().), for every gEL1 ().) there is cpeJf(T)such that IIScp-glll
-gilt<
Scp
II
cP
Eo
Then
-fill = J -fldJl= JI -gldA < E. = !IScp I
cP
q>
-fo(S,q),eTld).
0 (S,q),eT
0)
57.Representationof experiments) Let usconsider Ee 8(T)where T* 0 is an arbitrary parameter experiments of an experimentsatisfies a consistency))) set.The system of standardmeasures
57.Representation of experimenls
289)
tc relation.We notethat a function qJ E: .')F(T)dependson the coordinates 0 = .-. A if R Q denotes the natural only, (T), fP Pa fP. Thereby PQ: ex
ex
\037T
\302\243
projection.)
57.1Lemma. is the systemof standardmeasuresof an experiment E E S(T) then it is consistent, i.e.it satisfiesany of the followingequivalent If\302\253(JQ)QEA(T)
conditions:)
(1) H(<Ja.)(z)=H(<Jp)(z) if ZESar.Sp,ex,fJEA(T). (2) f fPdu = J fPdup iff ex, fJ E A (T),fP E Jf(T) and fP CPo:= fP 0 Pp = fP. Q
(3)
(Ja
Ps) ((-:,
= !R
( L:
Up
p\037)
HIJ)
SE<1
\037
$\037Q)
if
)
ex
\037
p, ex, PEA(T).
satisfies condition Proof First,we show that (2).We may A(T) necessarily assume If in that we mind that is then it p. keep fP positivelyhomogeneous \302\253(J\Q(:")
ex
\302\243;
foJlowsthat
J cpdu(l s.
= J cp ()
((d
i
SEa)
p.S)lEa. )
d r. Ps $(12
d L. dP. r - d L: P.S d d L. (( SFII )ru) Sf'L: Ps 3EfJ \037
= J qJ ()
.
u\037
\037
P)
d r. Ps dP. \037 = J qJ d L P.S () ((d L Ps)ru d L: Ps) U'fI _\03741\037__
S(P)
Sf'1I
=
f fP
h
dp'
L Ps ((d r. Ps) ) dse' rEQ
ufJ)
=
r
fP
tl
k, p.s)r.p) ((dHP) \037
dL
\037
U'fJ
-
=
r ill
Next, letus provethe equivalenceof (1) (3). (1)=- (3):Let ex p, ex, pEA(T),Z E SQ' Then))) \302\243
fPd(Jp.
290
Chapter 9:Comparisonof E'l:pcriments) H\302\253(1(J)
(z) = H(up) (z) = J
f1X:'(1p(dx)
S,IE(J)
=
=
::1
n ( L ) (Ua: L (lp(dx) ) .sc:r) X,
J s, r
t.
x.\037
XS
r\037a:
f1 /Ell
. /;'((L..Ps) ( L p,,) (lp) (dx).
\037
x\037c
SEa:
tEa:
s\037a)
to the proofof necessityof (2). (3) => (2):Thisissimilar => (2) (1):UseLemma 52.9. 0) 57.2Corollary.Let(\037)H A(T) be the systemof standardmeasuresof an experiment Ee 8(T).Then MfP:=JfP d(l(J if
fP
:if(T)
E
and
fP
=
fP
0
p\",)
isa conical measureofresultant MA = 1,t e T.)
57.3Definition.Let Ee G(T).Then the conicalmcasur\037
which
measureof E and isdenoted Corollary57.2is the conical by
is defined in
ME')
57.4Example.If E = (fl,.9I, t e T}) is a dominatedexperiment then its conicalmeasurecan be given in a simplerform.Let Po 1.91be a probability measuredominating\037.Let a. E A (T).Forevery cp E .K(T),cp 0P7. = fP, we have) {\037:
Mt;(cp)= S fP
--dIL_ d L Ps d (( L P\" )I!HI) sea: SEll)
-J
dL UCI
dP. I
fP
((d L ) ) P\"
IF
CI
P\"
dPo
dP.0
SEll)
= J fP
dP. d L 1',, 1
((d L
Po\"
S\037(J)
= J cp
\037\037
0
)
tE(J
dPo
!! (( ) ) dPo. \037
dPo
tEIX)
in terms of their of dominatedexperiments We obtaina characterization conicalmeasures.)))
57.Representation of experimcnts
57.5Theorem.An measureJ.l1 @
291)
experimentE E SeT)isdominated iff there eXL<;ts a bounded such that
\302\243f(R..)
I6T)
::::S rpdJ1, q> E Jt(T).)
M\302\243(q\302\273
take a probability measurePol.sfsuchthat 9 Po Proof(1) IfE isdominated Then Example(57.4)provesthe assertion.) anddefineJt = $I' \037
((
\037\037
(2) Let J.l1 @
t.
\302\243f(\037)
).r Po).
be a boundedmeasuresuch that
T)
= J q>dJ.l,
M\302\243(q\302\273
q> E
.Yf(T).)
tE DefineF:::: (IR\037, (8) \037(IR+), {Qt:
uT
T})by Qt = PtP,t E T.ThenQ\"
t
E
T, arc
since probabilitymeasures MEP, :::: 1, lET. J p,dp :::: we have ME.= MF'sincefor It isclearthat J.l dominates {Q,:t e T}.Moreover, q>
<>
Pa. = q>,
IX
e A (T),
((:rPs)1Ia.) LPsdJl=MFq>.
MEq>::::Jq>dJl=Jcp
st!c:z)
\037
Sf:
a.)
This impliesE\"\" F (Corollary52.8) and
dominated.
by
Corollary55.17,E
IS
0)
57.6Corollary.If E E SeT)isdominated then thereex;sts measure a probability = t e T});sequivalentto E. @ \037(R+)such that F (8) \037(IR +), {PrQo: QollET reT Is every conicalmeasure the conicalmeasure of an experiment? The settlesthis question.) followingassertion (IR\037,
57.7Theorem(LeCam[1974]). Forevery conical measure M un .Jf'(T)with resultant Mp, :::: 1,t E T, there isan experimentE E .t(T)such that M = ME')
Proof(Siebert [1979]).Accordingto Theort:m56.4thert: is a measure J.l on (8) \037(R) such that Mrp = J rpdJ1if q> E Jt(T).Let E = (a,.91, t E T})with {\037:
T
a:::: .!II:::: (8) 1fT rE
IRT,
measure for every
followsthat)))
\302\243f(IR)
tE
and
p,:::: Pt/l,
T. Sinceevery
T. Obviously, is a probability it q> E .Yf(T)is positivelyhomogeneous t
E
\037
Chapter 9:Comparisonof Experiments)
292
Mtp = J ({Jdp= J lp
:.'
((L.Ps)'Ell) L Psdp lEiJ
SCII)
if lp =
0 PII' This
({J
provesM = ME'
0)
The experimentE justconstructed neednotbedominatedsincep neednotbe O'-finite.)
57.8CoroUary.Let(O'JuAITIbea systemofprobability measureson \037 &B(R) 'ET such that O'II(SJ= 1 if E A(T). Then (O'II)2EA(T) is the system of standard in the sense measuresof an experimentE e K(T)iff it isconsistent of Lemma IX
57.1.)
arenecessary. To prove ProofIt followsfrom Lemma57.1that the conditions 0 H> sufficiency notethat M:({J J lpdO'II,lp E Jf'(T),lp ({J PQ, definesa conical measureM of resultant Mp,= 1.t E T.Thereis an experimentE E 8(T)such the systemofstandardmeasures that M = Mf;- If (1 A(T) denotes ofE then it followsby 52.8that =-=
11)11\"
J cpd(J{J = M lp = Mtlp = J lpdt = if tp e .JF(T).({J0 PII = cpo Thisimplies 0'12 Q
1 for every e A (T). IX
12
0)
57.9Remark. Thepreceding theorem isof an elementarynature if T isa finite bea probability measuresuch that J p,dO'= 1 if t E T.Then set.Let (J on t E T})isan experimentwhosestandardmeasureon ST Eu = (ST'\037(ST)'{p,O': with (J. The experimentEu iscalledthe standardexperimentof 0'.) coincides \302\243feST)
of experiments measures.) may be phrasedin tennsof conical Comparison
57.101beorem.Let8 0 andlet E and F beexperimentsin SeT)Then E F \037
\037
iff)
ME qJ
\037
MF lp + 6
if
Proof.Apply Corollary52.8.
({JE
JI{'0 (T),
II
qJ
II
= 1.)
0)
57.11CoroUary.Let E.Fe G(T).Then E\",F iff ME = MF .
of the importantTheorem57.7 is the Another consequence limitsof experiments.) projective
57.12Definition.A IX
E
existence of
system of experiments(E((I)u A(T) where E(iJ) E 8(1X), An experiment))) A (T), is a projective P implies system if (E(/1)iJ\"\"\" EiJ' IX
\037
58.Transfonnation groupsand invariance \302\243E
limit of the projective &(T)isa projective system
(E(CZ\302\273CU,A(1')
every
(X
EA
293)
if Ecz '\" E,cz) for
(T).)
57.13Corollary(LeCam[1972]).For every projective system of experiments there existsa projective limit Ee G(T))
(E(II\302\273CZCA(T)
E A (T),then the system ProofIf isthe standardmeasureof satisfies conditions (1)and (2) of Corollary57.8.Hencethere isan experiment E E 8(T)such that isalsothe standardmeasure of Ecz for every E A (T). (X
O\"(Z
\302\253(7CZ)IIEA(T)
\302\243(7.)'
(X
O\"(Z
Thisimplies Ell '\" E(II) for every e A (T).
u)
(X
Let us return to the problemconsidered below Theorem55.16.) in Theorem57.7satisfies 57.14Corollary.The experimentE constructed L(E)
= Lat (E) = L 1(p).)
Proof Clearly,Lat E L(E).Let us identify LI(Jl) and {fJl:fE L 1(Jt)}.In (J E L(E).(J O. Similarlyas for orderto show that L(E) L 1 (p) consider = (J Lemma41.6we may decompose 0, (J2 0,where e L1 (p) (71 + (72' L L B ut t E and (J2.l 1(Jl). (J2.l 1 (Jl) implies T,and the definingproperty (J2.l (J = (1) E L) (J.l). if (]2 = O. Hence, of L(E)yields(]2.L(] which is only possible Thus it remains to show that Lat (E) is densein {fwIE /.1(Jl)}.With the to notations of Lemma55.15 for every q> E .K(T)we have P\" = q>Jt. According is 0) 5 6.5 the set dense This t he assertion. i n Corollary .tf(T) L) (J.l). proves \302\243
\037
\037
O\"t
\037
\037
0\"1
p\"
57.15Corollary.Forevery experimentEe 8(1')there is an experimentF\",E such that
L(F)= Lat (F).)
of Theorem57.7 Proof Applying to the conicalmeasure ME the construction there is an experimentF such that MI:; = M,..which satisfies L(F)= Lat (F) accordingto Corollary57.14. 0)
58.Transformationgroupsand invariance) Let E =
(0,.91,{p,:t E T})be an experimentand letG be a group.)
58.1Remark. Assumethat there isa group(Mg)gEGof stochastic on operators such that E is invariant under It follows from L(E) Corollary55.10 (Mg)geGo that the experiments)))
Chapter9:Comparisonof Experiments)
294
IE T}), gE G, MgE,=(!l,.9I, {MgP,: are equivalentwith E. If an P, for s * t, then (Mg)geGdefines = Mgp\" ge G,se T. \037
=t=
of G on T by operation
\037s
With
thisnotationthe experiments) geG,) (fl,d,{\037,:teT)),
are equivalentto E. Now we start with a groupG operatingon T.)
58.2Definition.Assumethat G operates on T.ThenE iscalledG-invarianlif E isequivalentto eachof the experiments(fl,d, {Pg,:lET}), g E G.
glanceit seemsthat the notionof G-invarianceis moregeneral than the notionof invariance under the operationof a (Mg)gcGof stochastic At first
However,thisisnotthe case.) operators.
on T and Ihat E is G-invarianl.Then 58.3Theorem.Assume IhalG operales
a groupofstochaslic IhereexislS (Mg)gt: G on L (E)such thaI Pgt = M 9 P, operators
for allg e G, leT.)
Prouf.Let EoE 8(1')be such that Eo'\"E and L(Eo)= Lat(Eo).Such an 57.15. Firstwe showthat Eo isalsoG-invariant. experimentexistsby Corollary = DenoteEo (flo,.910 , {Q,:tE T}).Thereare stochastic operatorsM.:L(E) -+ L(Eo) and M2: L(Eo) -+ L(E)such that M.P'= Q,and M2 Q,= {E T. This impliesthat M.Pgt = Qg, and M2 Qg, = Pg, for all t e T and g E G. It followsthat p\"
(fl.d, {\037,: t E T})'\"(flo.do.{Q,,:t E T})) for every g E G which implies G-invarianccof Eo.Now, since)
.
Eo'\"(flo..910 {Qg,:lET})) there isa stochastic operatorM,:L(Eo) -+ L(Eo) suchthat M,Q,= Q,/t lET. It isclearthat M'IO(J2= Mg,oMg2 on the set {Q,:lET}.But from Theorem isevenvalid on the wholeof L(Eo) and it followsthat the groupproperty 55.16 Now define therefore(Mg)gtGis a groupof stochastic operators.
M,:=M2oM,oM.,geG.) It isclearthat (Mg)gf'Gisa family of stochastic satisfying MgP,= P\". operators g e G,t E T. It remainsto show that (Mg)g,c; isa groupof stochastic operators.)))
58.Transformation groupsand invariancc
For this we notethat M,oM2 = idIL(ED)'This followsfrom (M = M.(.P,)= Q\" t E T, and from Theorem55.16. Hencewe obtain)
1
Mg, ()92
-
- M g'1 9: M - g =
0 0 1 l\\t!. 2 0 91 (J Mg:0 2a M2 0 M 0 M1 0 M2 0 M () M1 = M
This provesthe assertion.
(Q,)
Mt--
9. M\"1
9:
fll
0 M 2)
295)
<,
.)
0)
of G 58.4Remark. Let \037(G) bea a-fieldon G and assume that the operation for in the sense that g Pg,(A),g E G, is,tj(G)-measurable on T ismeasurable Then the precedingproofshows that (MfI)(JEG is a measurable every A E Hencethe situation of G-invariantexperimentis groupof stochastic operators. in Section 48and the Hunt-Stein reducedthe caseconsidered theory completely can be applied. G-invariance it isconvenient to have a criterion Forchecking in teonsof the likelihood processes.) \037
\037c1.
011T. Then E is G-invariantiff) 58.5Coronary.Assume that G operates
!i' , dd (( l's) P,
IF T
!i'(( ) -,
p.f _
dPg, dPgJ ),
E
p. T
flf)
)
for allg E G andSET.) Proof Apply Theorem53.10.
0)
thissectionshowing that every equivalence classof G-invariant an experimentwherethe groupof stochastic is contains experiments operators inducedby a groupactionon the sample space.However,we cannotshow that in the operation is measurable the senseof Example48.3.) We finish
58.6Theorem.Supposethat
E is G-invariant.Theil there existsan equivalent on andQg, (8)&j(!R+),{Q,:(E T})where G operates
,.
experimentF = (R\037, T = IE T, gE G.)
Q,'g-l,
R\037
be the measure which isconstructed in the proofof Proof Let III'E(8)\037(R...) T Theorem57.7.LelM be the conical measureofE and Mg the conical measure of (Q,.51, {P,,:t E T}),g E G.SinceE is G-invariantit followsthat M = Mg,
gEG.
of G on RI by Q,:=P,Il,t E 1'.We definethe operation -+ It is obviousthat g: is (8)\037(IR+)gE G, XE gx:=(x(J-I')'ET' 'ET))) Let F be definedby
\037\037.
IR\037
R\037
296
Chapter9:Comparisonof Experiments)
measurable for every g E G.For every
E
CfJ
1 .11 (1'))
gdp = Mg(cp)= M(cp) = J cpdp,cp E .tf(T),g E G,) and sinceJ'I'(T)is densein L 1 (11)it followsthat p is invariant under the of G. We obtain) operation f cp
0
x)p(dx)= Qg,(B)= J IB (x)pg,(x)p(dx)= J IB(x)p,(g-l = J 1B(gX)P,(x) 11 (dx) = Q,(g-l B))
ifge G, IE 1',Be I(T)
@\302\243B(lR
t
). Thisprovesthe assertion. 0
59.Topological spacesof experiments) Let T + 0 be an arbitrary set. Then S(T) denotesthe collection of all f or the T. The elements o f the experiments parameterspace quotient are called I t be should recalled t hat set8(T)/experimenttypes.
-
59.1Definition.Let E, FE SeT)and let beE,F) = inf {c> 0:E S F}. Then A (E.F) = max {b(E.F),1>(F,E)}iscalledthe deficiencybetweenE and F.)
on G(T).) 59.2Lemma. The deficiency is a pseudodistance L1
have to prove that B inequality. ut this inequality
Proof We
E,F,GE8(T)) CI
C1
0: 8(1')x 8(1')-+ R satisfiesthe triangle is the consequence of the fact that for
E '2 F, F'2 G impliesE
h + Cl \037
0
G.
Ee 8(1')/-be the equivalenceclass with EEE. We denote is a metric space EES(T),FES(T).Then (S(T)/-,L1) L1(E,F):=L1(E,F). = since.d (E,F) 0 iff E F, the deficiencybetweenexperiments.) Thereare severalways to express 59.3Lemma. If E,Fe SeT)then .:1 1, E Jf o(T)}.) (E,F) = sup(lMECfJ MFCfJI : Let
-
-
IICfJII
\037
CfJ
ProofThisfollowsimmediatelyfrom Theorem57.10.
0)))
59. TopologiC
297)
59.4Corollary.If E,Fe cf(1')then
.:1 (E,F) = sup{.1(Ell'f;): e A (T)}.) (X
lemma. ProofNotethat Jfo(T)= U Jro(x)and apply the preceding A(T))
0
te T}) mId F= (D2 ,.9I2 , {Q,:te T}). 59.5Lemma. Let E= (Ql,.9I h {p,: Then)
-
beE,F) = inf {sup MP, Q,II:J'4:L(E) -t L(F)isa stochastic operator}. rflT) II
Proof Apply Corollary55.10.
0)
59.6Corollary.Let E = (O,.sI, lET})and F= (O,.sI.{Qr:IE T})beex{\037:
perimentswith A
the same samplespace(D,.r:#).Then
(E,F) supd1 (P\" Q,). ,eT) \037
from E to F it followsfrom Proof Sincethe identity definesa randomization Lemma 59.5and Lemma 55.4that
beE,F) sup ,eT) \037
II
P,
- Q,II
which implies for reasonsof symmetry
-
.1(E,F) sup liP, Qrll. 0 ,eT) \037
To obtain the topological of experiment spaceswe beginwith properties
the simplest case. considering If T isa finite setthen each E E 8 (1')ischaracterized by itsstandardmeasure 0'E E !fT'FromRemark 57.9(1),we obtainthat the mappingE.,....., 0'E is even SinceE'\"F impliesO'E = O'F the mappingt.,....., from (JF. isa bijection surjective. ontog'T'From Lemma5.20we knowthat ''/Tiscompactfor the weak 8(1');\"\"'
topology.)
59.7Tbeorem.If T is a finiteset then (r1(T)/-, .1)and//T are topologically equivalent.)
ProofThe mappinga H-i:;gisa bijectionfrom the compactspace/I'Tontothe Hausdorffspace(G(T)/ \"',.1).The assertion is provedif we show that the iscontinuous. mapping Let O'n
-
0' weakly where (O'n)IIE'I,I
J q>du\"
-t J q>du
if
q>
E
\302\243;
.efT'Then)
:It'0(1').)))
Chapler9:Comparisonof Experiments)
298
Sinceeachfunctionq> E .Yf oCT) satisfies a Lipschitz with Lipschitz condition constant1Iq>1I it followsthat {qJEJf'o(T):IIcpli I} is an equicontinuous and uniformlyboundedsetand therefore) \037
lim sup(IJ q>d(1\" ,,\037\037)
Thisimplies J
(E\037\",
Ell)
- J q>dql:
-0
cp
E
Jf'o(T), cp II
II
\037
I} = O.
in view of Corollary52.8or Lemma 59.3.
0)
59.8Corollary.If T isafinitesetthen (8(T)! \"',.1)isa compactmetricspace.) Let us return to the casewhen T is an arbitrary set.)
-,
Themetric space(4(T)1 .1)is complete.) 59.9Theorem(LeCam[1972]),
J. for the pseudodistancc ProofLet (E\,,tN")S; 4(T) be a Cauchy sequence E A (T) the sequence is a 8(rx) Cauchy sequence and Corollary59,8impliesthat there is a limit It is easy to seethat limitE E &(T). systemand hencethere isa projective A(T) isa projective We have to show that E is a limit of (E,,)uN' For every t > 0 there is N(t) E such that .1(E\"\"S' Em,lI) E A (T),jf n Forfixedn E and a; E A (T)the functionFt-+A (E\",II' is N(r.),m N(I'.). on 8(ex)which implies .1(E\".II' continuous EJ t if n N(t), E A (T).Now followsfrom Corollary59.4. 0) the assertion Then for every
ex
\037
(E\",\302\253)\"'N
\302\243(11)'
(\302\243(11\302\273111\"
1\\1
\037
\037
rx
\302\243,
\037
\037
\037)
\037
\037
ex
Forarbitrary parameterspaces the .1-topology It oftf(T)neednotbecompact.
will sometimes becalledthe strongtopology of 8(T)iscontrastto the topology of the followingdefinition.)
59.10Definition.Thetopologyon 8(T)j-which isgeneratedby the family of
(E,F)t-+ J (Ell'\037), E A (T),iscalledthe weak topologyof pseudodistances . 8(T)/\"'\" If T is a finite set then obviouslythe strongand weak topologyof SeT) rx
coincide.)
59.11Theorem (LcCam[1972]).The spaceof experimenttypes
compactfor the weak topology.)
&(T)fis
Prnof Let
1l1l(\037)
.\037
be an ultrafilter
-+ 8(rx)/- be the natural
an ultrafilterfor every
ex.
E
-. For projection.
in
S(T)I
A(T).If M
every
As a first
\037
rx
EA
(1) let 1l : 1l
stepwe prove that cf(rx) is an arbitrary setthen
E 1lCl(\037) or M' (M')E $i. Hence M = 11 (1l;1 7t: (M)E.'F or 1l;1 = 1l (1l;l(M'\302\273E 1l(l(ff). This means that 1l (ff) is an ultrafilter. Since 8(ex)f is quasicompact the ultrafilter 1l (..'F)converges.Let))) 1
11
1l
-
(M\302\273
cJ
1l
59.Topologicalspacesof cxpc:rimcnls \302\243(\037)
299)
e 8(a.)/-bea limit of 1t\037(.F). As secondstepwe prove that (E(\037)aeArl') isa
systcm. projectivc
isfiner than the filterof neighborhoods of p.Thefilter1t6(!F) The projection S(p)/- S(a.)/ is 2 t:/1in S(P)/ i.c.1t/1(':;;) Let ex
-, and thereforewe continuous
UU
\302\243;
\037
(\037\037(1t/l\037(E(/l))
1ti\302\2531
-
6(\302\2436)
UU
\037
1tJlIJ:
/1(\302\2436)'
have)
\037/l(E(/l)
\302\243;
1t/l(\037)'
Thisimplies \0371%(1t6\037(E(ft))
c:
=
1t/l\037(1t.8(ji'\302\273
whencewe obtain1t{1\037(E({1)
'\"
1t\037(Y)
E(\037).
Accordingto Corollary57.13there is an experiment Ee SeT) such that '\" 7t\037(E), a.eA(T).Since1ta(\037);2 a.eA(T).we obtain UUI%(7t\037(\302\243\302\273,
\302\243(IJ)
.'F
\037
U
'1
7t a
a (1tA
E\302\273
\037\037A(T))
which provesthat
E is a limit of
\037
for the weak topology.
0)))
Chapter10:AsymptoticDecisionTheory)
statistics relieseitheron weak converPractically, every result of asymptotic on a of or Besidesthe gence experiments strongertype of convergence. randomization criterionit was the secondgreat successof the theory of when LeCam [1972and 1979]was ableto show that the main experiments, the resultsof localasymptotictheory (LeCam[1953]and Hajck [1972and of weak convergence. 1973])can be provedexclusivelyunder the assumption Thisis the reasonwhy we discuss in the presentchapterweak convergence of and itsconsequences from a general pointof view. The resultsof experiments thischapterare appliedin Chapter13to obtainthe classical assertions of statistics. asymptotic The basicideaof asymptotic decision and easily explained. theory is simple Considera weak convergentsequenceof experiments. The objectof the are of decision functions. of investigation sequences asymptotic Every sequence decision functionspossesses accumulation functions pointswhich are decision for the limit experiment. Now, it can be shown that the asymptotic properties of a sequence of decision functionsare completely described by the properties I n of its accumulation a of decision functionsis points. particular, sequence asymptoticallyoptimalin somesenseiff the accumulation pointsare optimalin thissensefor the limit experiment. In Section60 we discussweak convergence of experiments. This type of of the likelihood can be expressed in terms of convergence convergence insteadof nets isjustifiedundermildcontinuprocesses. Considering sequences In case of likelihood ity conditions. contiguityit is even sufficientto consider base. with a fixed T his is in Section 6 1. processes proved in distribution of decision functionsisthe subjectof Section 62. Convergence We show that every sequence of decisionfunction possesses accumulation in that bounds risksof can be distribution and for the the points sequence in tennsof the risksof the accumulation expressed points.This leadsto the on asymptotic importantasymptoticminimax boundand rendersassertions these main o f LeCam are the results Essentially, [1972].Howadmissibility. of LeCam [1979]. Moreover,the ever, we followthe improvedpresentation versionspresented hereare sufficientlygenerallo coveralsosomeclassical local to Bahadur resultsdue and Pfanzagl [1970]. asymptotic [1964] the obtainedin Section62 Concerning asymptotic admissibility, assertions are expressed in terms of convergence in distribution. In this respectthey are which are contained much weaker than the corresponding classical assertions, There it is provedthat in the one-))) LeCam [1953]and Hajek[1972].
in
Chaplcr10:Asymplotic DecisionTheory
301)
of decisionfunction dimensional asymptotically normal caseall sequences minimax boundare stochastically which attain the asymptotic equivalent.In thisphenomenon is put intocompletely LcCam[1979] general terms.LeCam of the fact that the shows that stochastic equivalenceis a consequence for limit admissible decision f unction the experiment is noncorresponding We presentthese randomizedand uniquely determinedhy its distrihution. resultsof LeCam in Section63. Somepracticallyimportantdecision functionsare definedfor each sample a minimum path of a point by propertydependingon the corresponding Thisisthe casewith maximum stochastic e.g.of the like1ihood process. process, From the aslikelihood estimators and with generalized Bayes estimators. o f view the a minimum ariseswhethersuch ymptotic point question property remainsvalid after passingto the limit.Let us put the problemmoreprecisely. a sequence of experiments Consider weakly to a limit experiment. converging For each experiment of the sequence considera decisionfunction having a certainminimumpropertyasdescribed before.Then the questionis whether of decisionfunctionssharethis the accumulation pointsof the sequence minimum property.E.g.the question is whether the accumulation pointsof estimates are maximumlikelihood estimates for the limit maximumlikelihood orwhetherthe accumulation experiment, pointsof generalized Bayesestimates are generalized for the limit experiment. Bayesestimates in a rather general framework.It is In Section 64this problemisconsidered shown that an invariance principlefor the underlyingsequence of stochastic is the technical answer. For the caseof maximum processes key for a positive likelihood estimates such an approachis due to LeCam [1970]. It has been elaborated and extendedto generali7.edBayes estimates and by Ibragimov Has'minskii In bothcasesthe authorsrequire an invariance [1972and 1973]. likelihood themselves. for the sequence However,there may principle processes be somedoubt whether such an invariance principlefor the likelihood isof any statistical of processes meaningsinceit isnota necessaryconsequence of experiments. weak convergence In the caseof generalized the definingminimum property Bayes estimates doesnot dependdirectly on the values of the likelihood but on processes stochasticprocesses which are integralsover the paths of the likelihood In Sections 6567we show that weak convergence of experiments processes. aloneis sufficientfor an invariance principleholdingfor thesestochastic in Section 67 we arrive at a positiveanswer to our propblem Hence, processes. of for the caseof generali7.edBayesestimates requiringonly weak convergence the underlyingexperiments. The resultsare taken from Strasser [1982].)))
302
Chapter
10:Asymplotic
DecisionTheory)
60.Weakly convergentsequencesof experiments) a sequence of subsets Let T * 0 bean arbitrary setand (1;.) such that 1;.T T. In the followingwe consider of experiments sequences (E\,,eN")where E\" E \037(7\037)\037
ne N.)
-
60.1Definition.A sequence ofexperiments En e 4(1\037), n e N, c:onverge\037'
weakly
to E E SeT)if J (E\".II' 0 for every e A (T). It isclearthat in case1;.= T, n E N, this definitionis compatible with the The reasonwhy we notionof weak topologyin the senseof Definition 59.10. admit also T,.:I=T will turn out when we are dealingwith problemsof localization 82and 83).Specifyingthe followingto the (seeSection asymptotic of If (T)/ . case1',.= T, n E N yieldstopological properties forsomeC(e A (T) andfor such Forconvenience which make sense assertions c n E N that which is a matter of T,. are statedwithout the lattercondition E\302\253)
(X
\"'-'
\037
(X
course.)
60.2Theorem.A sequence (E,,)u convergesweakly iff (E\",II)nE N convergesfor '\" lim E\".IIfor every E A (T). every e A (T).In this case(Iim \".oco n o:t:;) P'V
(X
(X
En)\302\253
of of Proof It is clearthat weak convergence N implies convergence for every e A (T) and that in this casethe assertedrelationholds. assume that (En,(I)rre N converges for every E A (T).Thenit isclear Conversely, that the limitslim E\",(I'a E A (T)\037 form a projective system.The projective) \"... limit is a weak limit of (E\\"EN'") 0) (\302\243,,)nli
(X
(E\".\302\253)\"EN
(X
co)
of the precedingassertion is that every weak Cauchysimpleconsequence This of 8(ex)for followsfrom the completeness sequence weakly. converges every e A (T). Another formulation of Theorem60.2is that converges weakly iff E A This (T). leadsto pointfor every (En. N has at mostoneaccumulation Let (U\",II)
(X
(E\\"EPIO")
ex
II)\" Ii
60.3Theorem.A sequence of experimentsE\" E tf(1\037), n e N, convergesweakly
to E E SeT)iff any of Ihefollowingconditions is.tjaliified: (1) convergesweakly to every E A (T). (2) pointwiseon SII 10H(E,)for every E A (T). N converges 10 converges ME cPfor every cpE Jf'o(T).lIcpli1. (3) (ME Ps for (4) !R converges weakly to !R p\",s O'\037for
(O'\",\302\253)\"o:PIO
(X
(H(E\",a\302\273\"E
\037
..cp)\"fii'9;J
( (( 1>,..s)I.cz ))\". \037P\".t
every
(X
EA
(T)andevery S E
(X
N
\037.)))
(( ) \037P,
\037
Il:cz)
)
60.Weakly
convergent sequencesof experimcnts
303)
is equivalenttocondition (1).This is Proof It isclearthat weak convergence to the dueto the fact that (S(a)/ \"',..1)is topologically equivalent space//(1of on S(I (conferTheorem59.7). standardmeasures The converseisvalid since (1)=> (2):The implication (1)=> (2) isobvious. transforms determine standardmeasures. Hellinger a of (1)to (2). Use similar (1) (3): argument as for the equivalence of the (2) => (4):As in the proofof Corollary25.4we seethat convergence Ilellingertransformsimpliesconvergenceof the Mellin transformsof , SEa, 1',.'$' n E N, to the Mellintransformof \037;,, S)IFII IFII \".S (( ) ) (( ) of Theorem5.16proves(4). 0: E A (T).An application 25.4showshow the standardmeasures can (4) => (t): TheproofofCorollary be recoveredfrom the finite dimensional of the likelimarginal distributions hoodprocesses. Hence,the assertion. 0) \037
!i'
!i'
\037
\037\037
n E N, convergesweakly 60.4CoroUary.A sequence ofexperiments E 8(1;.), issatisfied: iff any of the followingconditions (1) (u\".J\"c; N convergesweakly for every a E A (T). (2) N converges pointwiseon Safor every 0:E A (T). 1. (3) (ME\" for every
(H(En.a\302\273\"E
(4)
II
dP,.,r !I' ( ((dp,.,$ )lEa p. )) \".S
II
\037
cunvergesweakly fur every
SEa ami every
\"EI\\I
aE A(T).)
of R(a),a E A (1'),with Theorem60.2and the ProofCombinethe compactness assertions of 53.8,53.10 and 57.11. 0) uniqueness
60.5Corollary.Assume En E
8(1',.), 8(1',.), I F,. E
n E
E FE E,
that
N,
E\"
weakly and
\037(T).Let r.
\037
F,.
O. If E\"
-:SFf',.for
weakly where every n E N
then E;2F. Although 8(T)/-- is a compact spacefor the weak topologyit neednotbe To obtain we need additional compact. sequentially sequential compactness
on the parameter set 1'. assumptions topological Let us beginwith a particularcase.)
60.6Lemma. If T is a countableset then En E
every
sequenceof experiments
8(7;.),n E N, containsa weakly convergentsubsequence.)
Proof Apply that S(o:)is sequentially compactfor every 0:E A (T) and use a standarddiagonalargument. 0) Assume that (T,d) is a metric space,The experiment E 60.7Definition.
is)))
304
Chapter10:Asymptotic DccisionTheory)
continuous if t 1-+;:is continuous. The sequence of experiments E\" e 8 n e N, is continuousin the limit if for every > 0 and every t e T there is 0(8,t) > 0 such that lim supd1 (P.w.\" P.w.,)
\302\243
\"CO
!\\j)
Notethat every equicontinuous iscontinuous in the limit.) sequence
60.8Lemma. Let (1',d)bea metricspaceandassumethat En E is continuousiff (En)\"cN is continuousin the limit.)
\037
E weakly.Then
ProofThis isdueto the fact that lim d2 (\037.s, \037.I) = d2 (\037, .P,))
\".. a\">)
for every pair(s,t) E T x T.
0)
60.9Theorem.Let (1',d) bea separable metricspaceanda.fi.fiume that (En)nf! N is a weakly conver\037enl subsequence.) continuousin the limit.Then (E\,,")4;N contains and dense. Thenthere isa subsequence ProofLet 10s; T becountable No s; N such that En,Tor'\\T\" -+ Ero weakly for nE No.We prove that (En)\"f'NO is weakly convergent. By Coronary60.4(3) it issufficientto show that for = {51' SN} E A (T) and every E Jr.'(T)with
if>
..., homogeneous {t ..., (fJ
if>
-
P\302\253
the form cp = f 0 A wheref RIa.l iR. Sincecp is positively satisfies a Lipschitz the same istrue off Let P = b condition and 'P = f 0 PIS'Then 'P E ff(T) and 'P0 PIS = 11'. We have)
and
IN} E A
- J'Pdun.,1
IScpdun.a.
-
- Sf ((d dL.
Ps
- J f ((d dL
dP, d r. PS J f {.I P Ps)IEI2) ((d L. l's)'ell) a.ufJL. .1
P,
s\037cr
P,
d L. P. J f lea.) HZ
)
oS
-
-
\302\253ufJ
crufJ)
- dp,
< LIp(f) L. J = i,;q d \037
-
\037
L P,
dp', d d L. P, L P,
I2v{J)
= 2Lip(f)
N
i
r. -=
1)))
/I p\"
- p',I/.
\302\253ufJ
aup)
d L PS JEll
s,-p)
\302\253ufJ
.
\037
((d L Ps)scfJ)
(To)
61.Contiguous sequencesof c'Iperiments
l05)
Sincep can be chosen The sequence No is a Cauchy sequence. <J tpdo;..p)\"f! close to sequence. 0) arbitrarily
ex
we
obtain that also <J cpdU\".a)\"E
N
is a Cauchy
60.10Corollary.ut (T, d) be a separablemetric space.Any equiconlinuous subsetof8(T)is weakly sequentiallycompact.)
61.Contiguoussequencesof experiments) We
of the preceding keepthe notations paragraph.)
of experiments 61.1Definition.A sequence n E N, is contiguous if E\" E &(T,.), for every pair(s,t) E T x T the sequences of probability measures Nand (P,..S)\"E (P\"..)\"4iN are mutualJy contiguous.)
if the probability 61.2Definition.The experiment E E S(T)is homogeneous
measures t E T, are mutualJy equivalent.) p\"
-E
61.3Theorem.Assume thai E\" iv
weakly.Then E ishomogeneous ifJ(E\,,a\037
contiguous.)
ProofIf E\"
-+ E weaklythen for eachpair(s,I) E
T x T the binary experiments P,.,,),n E convergeto the binary experiment(\037, P,).This provesthe (P,..s' assertion in view of Theorem18.11. 0) \037,
It shouldbe noted that the weak limit of a sequence of homogeneous need not be experiments homogeneous. For contiguous of experiments weak convergence sequences may be describedin terms of oneparticularlikelihood For the followingwe take process. an arbitrary elementof T and keepit fixed.Forconvenience we denote it by O.)
61.4Lemma. If E E 8(T)thenfor every
ex
EA
Po(N) = 0 and
- J n (dP'') dPo rr
H(Ea) (z)
lEa dR0
_
\037
(T)Ihereisa setNEd such Ihat
n [P,(N)]%',E Sa.
lea)
-
Z
1 dQ decomL P, and let dPo ,N bea Lebesgue Proof For E A (T)let Q = ) ( positionof Q with respectto Po'Then,for Z E ex
I ex
I
I.\037
Sa)))
306
10:Asymptotic
Cbapt\037r
H(E.)(z) = f J2
DecisionTheory)
(
\037ci
r dQ
dP, =J lea dPo
%'
%'
dPo+ NJ lEa n dP, dQ.
n( )
(dQ))
The assertion now followsfrom)
dP' :r dQ Jn N tu (dQ)
\037
dp, :r. 0 n dQ s \",a ( dQ) ) .y
61.5Lemma. If the sequenceof experimentsEll E 8(1',.), nE then for every E A (1')andevery Z E Sa) :r . hm H(E\",a)(z)J fl dPn.t d1',..0= O. \"-CI) I(a(dp11.0))
f\\)J,
is cOn/iguou.(j
ex
-
d L P,..tand let .d Q .,Nil be a Lebesgu\037 Proof For (; A (T) letQ\" = \037. tu lexl ( P,..o= )n E to of with Then decompositionQII respect 1',.,0' P,..o(N\") 0, N, implies II
ex
lim /I
'CI)
=0 n P...I(NIIY'
if
ZE
Sa.
I\037a)
followsfrom Lemma 61.4. Now, the assertion
-E
0)
61.6Theorem.If the seqt\037ence of experimentsE\" E 8(1;.),n E N, is contiguous then En
weakly iff for every
!fJ
P..,o ((:plI.r )/(a
-.
ex
EA
(T))
p...o_ .!t'((:p, Po)rea)
)
Po weakly.
)
issatisfiedin view of Theorem60.3 ProofIf E\" E weakly then the condition assumethat for every E A (T)) (4), Conversely, ex
dP,..r p. _ !i' dP, !i'((dp\",o ) ((dPo) )IU 11.0 _
Po weakly.
IE;Q)
)
Let ex E A (T).For simplicitydenote Jln
' )
dP,.,, !/!((d' 1>,..0 )rea
1=
P.s.o
n
EN, and
lim J
n
x\037'
dJlII
=J
n x;'dJl, tEll)))
I=!f'
dP,
((d-Po)
tEa)
Let us show that) II-.CI) lEa
Jl
Z
E
SIJ'
Po).)
61.Contiguous sequencesof expcriments Firstwe notethat for a > 0 and n E J
\037
N)
n x;'dJl,. It'a: L r n x;'dJ1,. L na ( r> xsdJl,.Y' L2 ( r> X,dJln):'. I x,1 x.1 \037
maxlxrl>/I I\"a:
307)
1%,1>/1 $1\"2
I\037a:
\037
t' II SF
I
III
I
t'
I
III)
that Now, Theorem18.9 implies Jim
sup
J
11I.'00 ,.ENm;axlx,j>1II
n x:'dJl,.= O. I\037a
11:a:)
show that) The same inequalities
nX:'dJl= O.
lim /I
-0co mu IJx, I > Q I E II lEa)
Thus,we obtainfor every Z E SII and every continuitypointa > 0
of)
\037(maxlx,1fIx:'Jl) IEII
I Ell)
n
J lEa: X;'dJl
\037
n
x:'dJl+ RI (a) J mulx,I:i/llEa: 11'11)
= lim
J
,.-owmulx.I\037Q
n x:'dJl,.+ R 1(a) '\037a
IFII)
\037
\037
\037
lim inf J 11X;'dJl,.+ \"-':10
R 1 (a)
\"11)
lim supJ fI X;'dJl,. + Rda)
,.-ro
lEa:)
Jim J ,,--romu'x,l\037o
n X:dJln+ R 1(a) + R 2 (a)
rea:
lEa:)
= \037
n
X:'dJl+ RJ (a) + R 2 (a) J lEa: mulxrl;io IEII)
n
J IE2) x:'dJl+ R 1 (a) +
R 2 (a)
where)
= O.) Jim + /1- (IRI(a)! IR2(a)1) tXJ)
This provesthe assertion.
0)
61.7CoroUary.A contiguous n E N, con.fiequenceof experimentsE,.E 8(1;.), vergesweakly iff for every
IX
EA
::., '
!l'((
,.,0 )
lEa:)))
1;,,0 )
(T)the di.'itrihutinns n E
\037,
convergeweakly.
308
Chaptcr 10: Asymplotic DecisionTheory)
Proof If (E\u")N convergesweakly then the finite dimensional marginal of the likelihood distibutions to Theorem processes weakly according converge 60.3.On the otherhand, if thesedistributions convergethen Lemma 61.5 that the Hellinger transforms implies convergefor every E A (T)which proves fJ.
the converse.
0)
62. Convergencein distributionof decisionfunctions) Let nE
of experiments 1;.t T and assumethat (E\nE") is a sequence E\" E 8(1;.), N
N, converging weakly to E E SeT).)
62.1Definition.Let p\" E 1I(E\",D),n E N. The sequence (p\\"O+l") distribution to P E 11(E,D) if) Jim \"-tQ)
converge.fiin
p\"(/,P.w,,) = P(/' P'), IE f{?b(f),t E T.)
It is easy to seethat this definitionis an extension of the corresponding notions. I f is a o f estimates elementary sequence ,,-measurable (\"\\",,") N decisionfunctionsare Q,,(., B) = 1B K\", K,,:Q\" D then the corresponding ne N, Be \037o(D),and hence for converges weakly to \037(\"IP,) t E Tiff in to) every (e\\"e") N converges distribution \037CI
t)
\037
(\037(\"\"IP.w,'\302\273\"\037N
(}:(lll,B) 1-.1 8
,,(m),(m,B) E Q x 9Bo (D). =:lp\", n E N, then in {1}) Similarly,if D = {O,1}and (l\"(., N converges iff n to the functions t E distribution (! N, converge power p\" ,
\302\253(l\\"e")
\037
62.2Definition.An element P e 1I(E,D) is an accumulationpoint(in distriof generalized decision functionsp\" e 1I(E\", D),n eN, if bution)of a sequence for every 7 E A (T) and every finite set G \037b(D) there is a subsequence \037
N
(a.,G)
\302\243;
N
such that
p\"(/, P\".,)= P(/' P,), fe G, tea.. \"'N(<<.G)) Jim
62.3Theorem.(LeCam[1979]).Every sequenceofgeneralized decision functionsp\" E \037(EfI' D),n E N, has accumulationpointsP e \037(E, D).) Proof For every)'e A (T)and n E N let K\", y: Q x
kernelsuch that
IIRK..,\037P,
-
P\",rll
<
L1
(/::\"..\"
E.,)+\037. t e
y.)))
.\037\"
\037
[0,1]be a stochastic
62.Convergencein
distribution
of decisionfunclions
309)
For this,conferRemark 55.6(2),and Theorem55.9.Then) P\",y:(I,Jl) \037
P,,(f,R\"\",yJl), IE \037b(D), Jl E L(E), are elements of rM(E, D).Let /1y be a weak accumulation pointof (/1\".'I)\"\" N and let p be a weak accumulation pointof (P'l)Y4iA(TI (cr.Theorem42.3). Fix 0:E A (T)and a finite setG fib (D).We show that for every 8 > 0 and every no E N there existsnc no suchthat) p\".(/,P...) P(/' P,) < 1:,f E G, t EO:.) \302\243
\037
-
I
I
Firstwe notethat there exists E A (T),r ;20:,such that) \"I
IPy(f,P,)
- P(f,
<
P,)\\
3' IE G, tE ex.) \302\243
Let a:=max {lI/lIu: lEG}and n 1 E N be such that
1 11(E\".'I'Ey ) + < n
Thenwe choosenc
\037
'f
f;
n
I
3a
-
\037
I P\"e
2' IE G, tE
that)
\302\243
ex.)
- P\".,(I,P,) (I, p\"e,')- (f, R,,\"..P,)
1/\037,,'
=
.)
max {no,nd in sucha way
P,) Py(f, P')I< IP\"..'1(f, Since)
nt
\037
I
Y
-
P\"e
y
I
e
< IE G, t E ex,) 11/1111 1'\"\", R\"\"'.'1P'1I \"3'
the assertion is proved.
0)
It is now clearthat a sequence (P\,,e")N which has exactly oneaccumulation in distribution to this accumulation pointconverges point.)
62.4Corollary.Supposethat (T, D.W) is a decisionproblemwith a lower sem;cont;nuous losslunction.Then every accumulationpointP E 14(E,D) of a E P\" rM(E\", D),n E N, satisfies sequence P (J.Y\" P,) lim supP\" (J.Y\" \037
Proof Let V
\037
W
,,-co)
1'\",,),t E T.
be a boundedcontinuous lossfunction,Thenfor fixed(E T
there existsa subsequence N, N such that))) \302\243
310
Chapter
10:Asymptotic
DecisionTheory)
P,) = lim fJn(V\"
P(V\"
nEN
P,.,,) limsupfJ,,(H-;,P,.,,). ,,-co) \037
r
Passingto the supremum over a)) V
W
\037
the assertion follows.
0)
lower boundsof Now, we arrive at the result which is the basisof asymptotic risk functionsfor a large variety ofsituations.)
62.5Theorem.Supposethat (T, D, W) is a decision problemwith a lower lossfunction.Let !M !M(E,D) be the set of allaccumulation semicontinuous \302\243
Pn E pointsof the sequence hull. Thenfor every subset
\302\243I(E\",
\037
\037
lim inf sup
,,-a)
,
E
To
(\"\\
D),n E N, and co itsweakly closedconvex \302\243I
T
1;'p\" ( H-;, P,.,,)
inf
\037
II
(!co91
sup , P ( H-;, P'). CI
To)
a subsequence Proof Let mE STo and = supp(m). Choose fX
\037m
lim infJ p\" (H-;,\037,,)m (dt) = lim J PIle .... :n E \"
\"
J.\302\245\"
\302\243
N such that
P\",,)m (dt).
FIlm)
Let P E !M(E,D)bean accumulation pointof the sequence (P,,)nEN.... Obviously, thisisalsoan accumulation of the It that for follows point originalsequence. lossfunction V W there exisL\037 a subsequence every boundedcontinuous \037
Ny
\037
satisfying)
N\".
= J P(V\" lim J P\"(V,,p,.,,)m(dt)
p')m(dt).
\"fNv)
Thisimplies that = lim J P,,(H-;, lim infJP,,(H-;, p\",,)m(dt) p\",,)m(dt) n-a) \"EN...) \037
Jim
> lim J P,,(v,. p\",,)m(dt) supJ PII(V\" p\".,)m(dt)
,,\037N...
,,\037Nv)
= J P(V\" p')m(dt).)
FromLemma47.2we obtain) lim inf J n-a))
P\"(J.\302\245,,
p\".,)m(dt)J P(\037,p')m(dt). \037
It followsthat)
-inf ,csup P,,(H-;,p,.,,) = Jim inf sup J P,,(\037,p\".,)m(dt) ,,- S1.0
Jim \"
00
To
co
f'\\
'I;')
\".1;
n T\)
62.Convergencein \037
sup inf J IJ(
J.Y\"
meSTo6e.1l) \037
/It;
An A
co.)
of decisionfunctions
31
1)
p')m(d/)
SUp inf J {J(J.Y\" ,\"\037STO
distribution
p')m(dl).
the proof. of Theorem46.3completes application
0)
minimaxboundfOT risk functions.) particularcaseisthe famous asymptotic
62.6Corollary(LeCam[1972; of Theorem 1979]).Assume Ihat Ihe condilion:,' Thenfor every sequence D),11EN, (62.5)are satisfied. PIt E !!J(E\", Jim inf supPIt /:,) inf supfJ P'). ,,-\037 (J.Y\"
\037
t\037T\"
(J.Y\"
fJc9l(E,D)t(T)
Another application dealswith admissibility. Recall,that a function r: T -+ 00,+ 00]is an admissible risk function for E and (T, D, W)
(-14
every /3 E
if
for
(E,D)
{J(U-;,P,)
\037
r(t), tE T, implies {J(Ut;,P,) = r(/), IE T.)
62.7Theorem(LeCam[1972]). SupposethaI (T, D, W) is a decision problem with a lowersemicontinuous lossfunction.Let r: T -+ (-00,+ 00]he an admi'isibleriskfunctionfor E. If a sequence of 13\" E 14(E\",D),n E N, satisfies r(t), tE T,) limsup{J,,(U-;, P\",r) \"-00) \037
then)
lim {J\"(J.Y,, p\",,)= \"-.0())
r(l), lET.)
point Proof From Corollary62.4 it followsthat every accumulation fJ E a (E,D) of N and even every {J E co31satisfies J;):5; r(/), tE T, {3(U-;, which implies that by admissibility = r(t), t E T,) fJ(J.Y\" for every fJ E co91.Now Theorem62.5provesthe assertion. 0) (fJ\1Ifi")
\037)
Another version of thisresultis)
62.8CoroUary.Supposethat (T, D, W) is a decision problemwith a lower semiconlinuous lossfunclion.Let r: T -+ ( 00,+ 00] be an admissiblerisk funclionfor E. If a sequence ofPIt E &i(E\",D),n E N, satisfies)))
-
312
Chapter 10:Asymptotic DecisionTheory)
lim inf
,,-QO)
P\",to)
P,,(J.\302\245,ot
< r(to))
for someto E Tt then there existst JET such that P\".t.>> r(lI)'
limsup ,,-QO)
P,,(J.\302\245,.,
63.Stochastic convergenceof decisionfunctions) isa sequence of experiments E\" E & (T,.)tn E Nt decision weakly to E E tf (T).Let D be a topological space.) converging Let T,.r T, Assumethat (E\\"E") N
63.1DefinitioD.Let a\" E [;it (E\",D)and t\" E [;it (E\",D),n E N, be two sequences
of decisionfunctions. The sequences (a\\"E") Nand (t\,,t,") N are stochastically
equivalentif) lim If
0\"\"
,,.....It'J)
-fr,,1P...t= 0, IE
c'jf,,(D),
t
E
T.
of decision It is immediatethat stochastically functions equivalent sequences
have the same setsof accumulation points.)
63.2Remark. The conceptof stochasticequivalenceis closelyrelatedto in measureof decision functions. convergence be Let a metric (1) (Dt d) compact space.Let A 5:r.tb(D)be the setof all functionssatisfying)
-I(
1 and I/(x) 11/1111
d(x,y), Xt Y D.) it is easy toseethat A is compact and) Then by the Arl:ela-Ascoli-theorem \037
y) I
\037
\302\243
-
sup{If(x) f(y)l:fEA} = d(x.y), x.Y ED.) that) Stochastic equivalence implies
lim
-/,c\"
Ifa\" ,,-coJ sup IFA) which is equivalentto
l
dP\".t=
0,
tE
T,
= Ot tE T. lim JH d(x,Y)l1\"(w,dx)t,,(w,dY)P,,.r(dw)
\"-00)
(2) Let (D,d)
it
followsthat lim
JJ \"-00J \"\)
be a metric spacewhich isseparable From(1) and complete. = 0, d(x,y) a,,(w,dx)t,,(w,dy) P...t(dw)
t
E
T,
63.Stochasticconvergenceof decisionfunctions
313)
that (U )IIEN and (T\IIEN") converge to some K 5:D.Suppose for every compact we find for Since is on every e E [;it (E,D) in distribution. qp' tight !?I(D) may c > 0 and t ETa compact setK D suchthat II
\302\243
< lim supJ 0\",,(., D\\K)dP\",I
,,-< E. lim supJ T\"(,,D\\K)dPII,r \"-00) \302\243,)
co)
Thenstochastic equivalence implies) lim (u\" \"-CO)
@ t,,) P.s,r {(x,y) E
D2:d(x,y) > c} = 0, c > o.
= (3) Let D = {O,t} and denote q>1I followsthat
0\" 11
(
.,{l}),
11'\"
=
TII
(.,{t}),n e N. Thenit
= 0, nE N. lim JIq>II-1p\"'dP\",r
\"-cO)
be a metric spacewhich is separable and complete. Assume -+ + that there are sequences \"II: !lll D, nil:!lll D, n E N, satisfying) B) = 1B o nil' ne N, BE!M(D). ell(\" B) = 1B o \"II' t,,(., (4) Let (D,d)
of (2) Then,under the additional assuptions lim
P\"\"
II-cO)
{de,,\", n\") > t} = 0, t > O.
Supposethat two sequencesof decisionfunctionsU e 91(E\",D) and in distribution to the same limit e E (it (E,D). TII e (it (Ell'D),n e N, converge When is it possible to conclude that the sequences and ('II)IIEI'II are which is stochastically equivalent?The followingis devotedto this question II
(O\"II)\"EN
answeredby Theorem63.6. Let p E (E,D). A distributional of fJ is a neighborhood anothersetof the fonn) \037
set containing
-
< t,fE G, SE a.}) F= {\"IE 91(E,D):Iy(f,Ps) P(f,!'s)1 rc,,(D)is a finite subsetand E > O. The system of distributional of p is a filter which iscoarserthan the filter of neighborhoods weak neighborhoods of p. Let uscallan element if P E (E,D) uniquely determinedby itsdistribution = = p.) y(j,!'s) P(f,Ps) for aUfe\037b(D) and.fET implies}' 63.3Lemma. Supposethat PEal(E,D) is uniquely determinedby its distribution.Then thefilterofdistributional with thefilter neighborhoods offJ coincides where a. E A (T) and G
\302\243
\037
of weakneighborhoods of
p.)))
314
Chapter
10:Asymptotic
DecisionTheory)
of P and denoteby:Fthe filterof ProofLet U bea weakly openneighborhood distributional of Since neighborhoods {3. p is uniquely determinedby its we have) distribution
n
F = {fJ},
F@\037)
where F denotes the weak closure of F\\ U = '/),
}\037
FE !F.It followsthat
n F,')
and
since (E,D) EM
U.
F\037F\037
IS weakly
compact,there existsFE.rF such that
0)
63.4Remark. Let E = (0,.91,{p,:t E T})bea dominatedexperimentwhich is and let D be a locallycompact decision base. complete spacewith countable Thenevery q E 9t(E,D) is uniquely determined To seethis, by itsdistribution. = let P E (E,D) be suchthat p (f, P,) If}P, for aU E beD),t E T. If f}o is a substochastic kernel such that f3(f,Jl)=f(}oJlif fEtGoo(D),JlE L(E),then tE T. It followsthat (}o is a stochastic kernel and f(}oP'=f(}P',fE\037oo(D). = that P (}.) hence,P E fit (E,D).Now, completeness implies
I
\037
63.5Lemma (LeCam [1979]).Supposethat
\037
E weakly and that If (} is non(}\" E 9t(E\",D), n E N, convergesto (} E [it (E,D) in distribution. randomizedanduniquely determinedby itsdistribution, then
- (fe,,)2P,,)= 0, IE
lim (f2(!\"p\"\"
\"-'00)
Proof.Fixfe beD),f\"4= such that) \037
if
P\".r
(fe). U\302\253(},
\037
-+
\037b(D), t E
T.
a number N(e) 0, t E T, and t> O. We win construct
f 2 e\"P\".,(fell)2 < e Let Jl =
E\"
n
\037
N(t).
and define)
e):={PE \037(E, D): 13(f2,.P,) I
t -1 P,I< 4' {J(f, -ff}JlI< 8}') 2
C
(}
Jl)
I
of fl, and in view of Lemma 63.3 U(q,e) is a weak neighborhood Obviously, there are E A (T),a finite subsetG
{pE
\302\243iCE,
-
\037
fJ
D): P(g,\037) g(}\0371< fJ, g E G, I
,<;
E
(X}
Let}'= Gtu{t}and chooseN(f.) such that II(E..,> E,)< min
, g E G} {2mox {II:II,:
411\0371I;
})))
\037
U(f},E).)
63.Stochasticconvergenceof decisionfunctions
315)
and)
if Igell\037.s-ge\037I<\037 sE\037,gEG,) whenevern
> N(r.). For every n
that IIP.\"
\037
N(E) there is a kernel KtI
- RK.P,II< min {2ma.{II:II.:g G}
SE Y.)
II
(2\302\273.
- ge\0371< b,
and thereforeUti E
411\037IIJ
Yt(E, Ell) such
DenotingU :=(]II RKn E {il(E,D) we obtain
(cf.Remark 55.6 Igu,,\037
.
\302\267
E
E
U\302\253(],
E), n
-
SE ex, n
N(E),)
\037
N(E). This implies by
\037
definitionof
V\302\253(],
E))
t
If2 UIIP' f 2 eP'1< 4' Ifu,.(fe'P,)
-
e
fe\302\253(fe>'
P')I< 8')
Sincee is non-randomized it follows from Theorem42.9that
-fe)2P, f2 - (JU)2P, -f2 - 2(full -f (})(f . .P,) = f2 c-c
o Sf 2 u P' ([U)2 P, lI
II
ulIP,
\037
U\"
+ (full
II
P,
fl
P,
(}
n
4
followsfrom Now, the assertion
-
f2 (lll P,..I (frlll)2
-(f -[ +2
\037.I
E
2
f2
UII
RKn
= f 2 UIIP,
P,
(
flIt
RK,,)2P, +
2
UII)
\037
E;
\037
\037
t,
,,\037N(t).)
o)
to give a complete solutionof the question Now, we are in a position posed. ConferTheorem23.10.)
63.6Theorem(LeCam [1979]).Let
EE tf(T) be an experiment and D a are equivalent:))) topological space.Forany e E 9t(E,D) the followingassertions
316
Chapter10: Asymptotic DecisionTheory)
(1) Whenevera sequence of experimentsE\" E &(T,.),n E N, convergesweakly to E and 0;,E Yt (E\",D), n E N, and E [;it (E\",D), n E N, converge to e in then (u\,,") Nand distribution, N are stochastically equivalent. anduniquely determined (2) e isnon-randomized by itsdistribution.) 'fIt
\"
(t,,),,\342\202\254
(2):We define an experiment F= (D x D, \037o(D)@ d, Proof (1) {Q,:t e T})by Q,:(f,g) 1-+P'(fe.g), fE g e !R,,(d),leT.) Let us show that E --F. The relation E S Fisclearsincefor every r e .(jf (E,D) there is u: c.o),B).-. -r(m,B), (x,m) E D x D, BEfBo(D),in {il(F, D) \037
!R,,\302\253(fIo(D\302\273,
\302\253x,
satisfying
fuQ,=ftP', te T,fe\037b(D).
-
the mapping L(E) L(F)beingdefinedby Conversely, \037:
,) p(fe'g), fe :l',,(fJo(D\302\273,ge :l',,(.9I) 7iP:(f,g).-. is a transition. This implies E 2 F. Now we definea non-randomized function) decision K: D x D D:(x, m) 1-+x for F. For every fe \037b(D) we obtain
-
fKQ,= J/o KdQ,= fIfo K(X,m)e(m,dx) p'(dm)= feP\" t e T. On the otherhand,(1: m), B) 1-+ e(m,B), (x,m) ED x D, Be &10 (D),is in Yt (F,D) and satisfies \302\253x,
f(1Q,=.f(}p',tET. It followsthat .fuQ/= fKQ\"
tE
T,
-
impliesby condition(1) that Ifu fKIQ,= 0, te1:fe \037'b(D). A standardargument yieldsIfa fKlp == 0 for all p E L(F)and therefore f(1J1 = fKJ1, f ErebeD),J1 E L(F).)
-
which
Putting Jl =
f. Q\" t E T, it followsthat
(fq)2p, = f2qp', t E T,fe \037b(D),) that is non-randomized which implies by Theorem42.9. It remains to show that (! is uniquely determinedby its distribution. If \037
te1:then a directappli-))) e1 e91(E,D)is such thatfelP'=feP',fercb(D),
64. Convergenceof minimum cstimatcs
! !!
317)
-
cationof condition (1)impliesthat Ifl! fl!11 P, = 0,f E \037b(D), t E T, which provesthe assertion. (1):Let U\" = (0'11+ tIt)' n E N. Since(Qn)neN convergesto e in (2) that it foJIowsfrom Lemma (63.5) distribution = 0, fE \037b(D), t E T.) lim (f2(1\"P,.,r (f(1,,)2 P\",r) ,,-IX))
?
Denoting = t (0'\" tIt)' n E N, we have (fcp,J2+ (fe,,)2= [(fC1,,)2+ (ft,,)2] U2 + f2\"(,,]= f 2 e\",)
\037
and therefore)
G\"
.
2 n EN.) f2 - (f(1,,) This provesthe assertion. 0)
(I
2
\037
(1\"
Thereisan interesting of the preceding the assertions consequence concerning
IL should determination of decision functionsby their distribution. be recalled that completeness isnota propertyofequivalenceclasses ofexperiments ascan in general, beseenfrom the caseof exponential it may Therefore, experiments. determines the underlying decision function for one happenthat a distribution However,if equivalentexperiment. experimentwhereasit doesnot for another the underlyingdecision function is non-randomized then uniqueness carries class.ConferalsoTheorem23.9.) over to the whole equivalence
63.7CoroUary.Let (1E 9t(E,D).If there existsan experiment F = (D\"silt{Q,:t E T})'\"E and G E \037(F, D) such that f(1p, = fGQ\" fE 'Cb(D),t E T,) and uniquelydeterminedby itsdistribution, where G isnon-randomized then also(1
isnon-randomizedand uniquely determinedby itsdistribution.)
64. Convergenceof minimumestimates)
Let (T,d) bea metricspaceand letT,. !;;T beopensubsets satisfyingT,. r T. Let sI n be a of a nd stochastic E (0\", p\,") sequence probability spaces consider n EN, whereZ,,(t): 00,+ 00]are sI,,-measurable Q\" processes (Z,,(t\302\273UT\"' functions,n E N. Assume that the stochastic processes (Zn(t\302\273re T\"' n E N, are let a be Moreover, (0,.fII,P) spaceand let T bea probability separable. stochastic where Z(t): (-\037,+ 00]is ..c1-measurable.))) separable process '\"
\037,
- (!l-
(Z(t\302\273r\037
318
Chapler10:Asymplolic DccisionTheory)
64.1Examples.(1) The resultsof the presentsectionmay be appliedto the Let En = (On, followingsituation.
d,
\037r;/\",
{\037.I: t E \037}), n E
E = (0, {.P,: t e T})be experiments. Then we fix n E N, p:;;;; Ps, and define
\037,
and
somesET,put p\":=p\".s'
.
dP,..rtE\037, liE N, and Z,,(t) = -logdP.
-, \",s)
Z(t)= -Iogddp' te T. Ps)
In sections 65-67we shallassumethat Tisa Euclideanspaceand apply the results of the presentsection to the samesituation asunder (1),butdefining (2)
-
Z,,(I)= f W(t s) f;ds), t E 7;, n E N, and Z(t)= J Wet .v)F(ds), t E T,)
-
where (W (.
-
lossfunctionand F\" are posterior distributions for E\" with respect tosomepriormeasures n E 1111' .v\302\273u
T is a
\037.)
For notational convenience we introduce) 1={tE 1;.: B:(w,,) Z,,(t)(w,,) seinf Zn(s) (ro,,)+ e} \037
'f\
and)
Br.(w) := {te T:Z(t)(w)
\037
inf Z(s)(w)+
IE 'f)
c},
which isintroduced in the followingdefinitions is e N, t: O.The tenninology to decision motivated by itsapplications theory.)
n
\037
kernel (1:Q x BI(T)..... 64.2Definition.A stochastic [0,1]is a minimum estiif g(.,EO) = 1 P-a.e.) mate for (Z(t\302\273'ET
-
64.3Definition.A sequenceof stochastickernels(},,:U\" x !J4(\037) [0,1], if for every > 0) of asymptoticminimum estimates n E N, is a sequence \302\243
= lim J l!n(.' T\" \\ B;)dP\" O.
\"-00)
64.4Examples.(1) Considerthe caseof Example 64.1(1).An estimate for every Ps.sET, is a maximum (l E [Jt (E.T) which is a minimum estimate A sequence of estimates likelihood estimate. ell e gp (E,.,7;), n e N, which is a minimumestimates forevery sequence of asymptotic sequence (\037.s)nE N' sET. of asymptoticmaximumlikelihood estimates. is a sequence 64.1 T hen the respective estimates are Consider the case of (2). (2) Example of asymptoticBayesestimates.))) and sequences calledBayesestimates
64. CoO\\'crgcnccof minimum estimates
319)
Now we statethoseconditions which are usually neededfor a satisfactory theory of minimumestimates.)
64.5Conditions for T' setK T such that) (1) For every E>0 there is a compact P{ inf Z(x) inf L(y) +\037} < t. (Z(t\302\273'E
\037
\037
'5T)
zf;T'K
pathsP-a.e.) (2) The process (Z(t\302\273,c T hascontinuous
64.6Conditions for (Zn(t\302\273,t.T..' nE\037. (1) For every e> 0 there is a compactsetK T such that lim supp\" { inf ZII(X) inf 2,.(y)+ I;}< 1;. n--co XE T..\\K ,eT..) and are (2) The processes (Z,.(t\302\273,a.., n E N, have continuous pathsp,,-a.e. and boundedon compacts in (\037)-probability.) equicontinuous of existence of minimumestimates.) the question First,we consider \037
\037
Then 64.5(1) and (2).are satisfied. 64.7Theorem.Suppose,that conditions
there existminimum estimates for (Z(t\302\273'F
T')
followsfrom Proof.If we can prove that P{Bo(w) 0}= 1 then the assertion For the existence theoremfor measurable Theorem 6.10. selections, every n E N setsuch that let Kn c T be a compact =t=
P Since)
inf
Z(x)
\037
{xeT,K\"
inf Z(y) +
)leT
inf Z(X) > inf Z(y) +
{xfT\\K.. it
)lI'T
\037
n
\037
n
< . \037
}
}
\037
n)
{BO =t=
0} J:r-a.e.
followsthat P{BO 0} 1-\037, n EN, which provesthe assertion. =t=
\037
o)
n)
64.8Theorem.Supposethat conditions Then 64.6(1) and (2) are satisfied. there existsequences whicheven satisfy ofasymptoticminimumestimates lim
,.-ro)J
en(.'Bno)dP\" = 1.
Proof For every k E
let KJc
\037
setsuch that T be a compact
Z,.(x):$inf {x( Tn'KIc
lim sup1>,. /I-\302\253)
\037
inf
)IE
Tn)))
- < -1 . l
ZII(y) + k
}
k
Chapter 10: Asymptotic DecisionTheory)
320
Let N be suchthat n Ic
P\"
inf
{xeT\"\\Kk
\037
N\"
Z,,(x)
\037
implies1',. KIc and \037
inf YET\"
-
l Z,,(y)+ k < k)
\037.
}
Since) inf
{.:c.T\"\\K,,
Z,,(X)> inf
Z,,(y)+ k.l
}
y.T\"
\037
=t=
{B\"O
-1
0} \037-a.e.,
follows that P,.{B\"O * 0} 1 if n N\", which impliesthat Jim p\" of measurable * 0}= 1.FromTheorem6.10followsthe existence \"....00 such that \",,(w,,)E B\"o(w,,). Define\"\" arbitrary on {!to maps\",,:{B\"O * 0} 1',. = 0}and e\"(w,,,B):=18 BE14(1',.). 0) WISE !l\", it
\037
{B\037
\037
-.
(\",,(w,,\302\273,
64.9Remark. The readerwill notethat the proofof Theorem64.7is a literal copyof the secondpart of the proof of Theorem37.8.which provesthe of generalized In a similarway we shallprovethe existence Bayesestimates. For of maximum existence ofasymptotic estimates. Bayes provingthe existence estimates ourTheorems likelihood 64.7and 64.8are lessuseful sincein thiscase the stochastic processes Tn' n E N, dependon SET whereas the of sET.) maximumlikelihood estimates must be constructed independently (2,.(I\302\273'e
64,10Lemma. Suppose that condition 64.6( J), issatisfied. Thenforevery e > 0 setK T such that) there existsa compact lim \037{!tc K} > 1 t, ne N.) \037
-
\302\243;
\"
.00)
Proof Let E > O. ChooseK lim supP,,{ inf
,,-co
XFT,,\\\"
\037
T such that
Z,,(x)
\037
inf )'fiT\
Z,,(y)+ e} < 6.
Since)
{ inf ;rl:T,,\\\"
inf Z,,(x)> YfiT\
the assertion is proved.
ZII(Y)+
\302\243;
\302\243}
{\037
\302\243;
K}
0)
64.11Definition.A sequenceof stochastickernels ell:!1x .'M(1\037)-.[0,1], 11
nE
N, isuniformly tight if for every I:> 0 there isa compactsetK
\302\243;
Tsuchthat)
Iimsupp\"{en(.,K)< l-e}
64.12Theorem.Supposethat condition64.6 (J). is satisfied.Then
every)))
64. Convcrgenceof minimum cslimalcs
sequence of asymptoticminimum estimates for
(ZII(t\302\273,e
:nt)
T..'n e N, is uniformly
tight.)
Proof Let t; > O. ChooseK
ell(.'T,,\\K) n
\037
T accordingto Lemma64.10. Then)
\037
ell(\" T,.\\B:)+ ell(\" B;,\\K),)
e N. This implies that) by definition lim supJ QII(\"
11-00
T,.\\K)dp\"
lim supJ QII(\" B:\\K)dP\" <
\037
\342\202\254
11-00)
smce) =F 0}< p,.{Q,,(., \037\\K) > O} \037{B;\\K \037
for sufficiently large n E N.
\342\202\254)
0)
This tightness assertion hasimportantconsequences.)
64.13CoroUary.Assume that T is a locallycompact space.Supposethat conditions kernels 64.6(1) and (2),are satisfied. ,f a sequenceof stochastic 0',,: (211x \037(T,,)
\037
n EN, is stochastically [0,1]. equivalentto a sequence of
asymptoticminimumestimatesfor (ZII(t\302\273UT\"'
of asymptoticminimumestimates for
(ZII(t\302\273,C
nE
N, then (0'11)isalsoa sequence
T,,'n E N.)
of asymptotic minimumestimates for ProofLet (Qn) bea sequence T..' n e N. which isstochastically a compact equivalentto (un).Let > 0 and choose (Z,,(t\302\273re
setK c T such that for n
\302\243
\037
N(e)) f.
J QII(\" T,.\\K)dP\"< 4') e
-.
T,.\\K)dp\"< 4) JO',,(.,
64.6(2),there exists > 0 such that the sets By condition \037
A II
:=
-
IZn(x) Zn(y)1 {4(x.)')<6.x.Yf!\
satisfy P\"(A\037) < X,YE K,WE (2\",)
SUP
\037 \037
}
as soonas K c 7;.. Hence,PII(An)
\037
1
\037
xeB:/2(w),d(x,y)<\037 imply yEB:(w), neN. It fo))ows that for n
\037
N(E))))
-\037,
and for an
Chaptcr 10: Asymptolic DecisionTheory)
322
J (!n(',8:12) a;,(
t
.,
T,.\\ B:)d\037
K n B:/2) Un (.,K\\B:)dP + 2 J QII(\"
\037
n)
t + J Qn(.'Kn B:,/2)U (., K\\B:)dp\" II
\037
An)
tI(ellx.)')II6. x,)'(; (In(''dx)u(., n
\037
dy\302\273dP\".
K)
\037
Stochastic equivalence implies Jim II
.supI QII(\" B:/2)U (.,T,. B:)dP\" t:. \037
isa sequence of asymptotic minimumestimates it followsthat
Since
\302\253(111)
lim supJ un n'\"
(.,
T,,\\Jr,.)d\037
8.
\037
<;0)
This provesthat (Zn(t\302\273,c
\\
II
<;0)
Tn' n
of asymptotic minimumestimates isalsoa sequence for
(0'11)
EN.
0)
64.14CoroUary.Suppose that condition 64.5and64.6aresatisfied. Assumethat for every rl E A (T) =2 lim P), weakly.) 9'\302\253ZIl(t\302\273'EcII\037)
\302\253Z(t\302\273,u
I
n\"'QC)
Thenfor every compactK Jim
\302\243;
T
supp,.{B:n K =*' 0} supp,. n K =*' 0} lim lim n.'<;O P {BOn K * 0}
n-<;o
{BIIO
\037
C
\302\260
\037
.)
\037
Proof Let K c T be compact.The first inequality is trivial. Moreover,we observethat) = P{BOnK:f:0} JimP{JrnK=t=0} C
\302\260) \037
sincecompactness of K implies) n {BCnK* 0}= {Bor\",K * 0} P-a.e. c> 0) lienee,it sufficesto show that) Jimsupp\"{B:nK 0} P{BCnK =*' 0}+ 26) 11'<;0) =t=
\037
for sufficiently small > O. For every sufficiently small f. > 0 and n setKc c: T, Kc ;2K, such that))) have a compact F.
\037
N(t) we
64. Convcrgcnccof minimum estimates
-
s; Kc}> t \037{a:
and P{RCc:Kc}> 1
[;
-
323)
r..
This followsfrom Lemma 64.10. Since)
B:n K =*' 0 wheneverB:
\037
BE
\037
K
Kc and Kc
n K 4= 0
inf2\" +
inf Z,.
iff
\037
and since)
1;\"
inf Z
iff
F.
Kc)
inf Z + t
\037
K
Kr.)
jf B s; Kc, and sincefrom condition 64.6(2) jt followsthat) weakly on \037(Kc) C
-.
9'\302\253(Z(t\302\273'f,K.IP)
\037\302\253Z,.(t\302\273hK
we obtain)
lim supp,.{B;n K =F 0}) \"....
00)
\037
\037
t + lim supP,. {infZ\" \"....00
\342\202\254
\037
K
K.)
+ P{infZ infZ+ \037
K
Hencethe assertion.
inf Z\"
\342\202\254}
\037
2\302\243
K.)
+ t} + P{Bt(\"\\K* 0}.
0)
of resultconcerning in distribution Now, we arrive at the desired convergence minimumestimates.) that conditions 64.15Theorem.Suppose 64.5and 64.6are satisfied. Assumethat
for every (l E A (T)
lim 9'\302\253Z,,(t\302\273'E::IIP,,) =
\"-00)
weakly.)
9'\302\253Z(t\302\273'<.\302\253IP)
itsinfimum in exactlyonepoilltP-a.e.. process If (he stochastic T attains then every sequence of asymptoticminimum estimates for T\", fl E to the minimum estimate convergesin distribution for (Z(t\302\273,c
(Z,.(t\302\273,.,
\037,
(Z(t\302\273'\037T')
of asymptoticminimum estimates for (Q,.)is a sequence n EN. Let K If > 0 and K T be compact. then we have)
ProofAssume that (Z,,(t\302\273f\037T\"'
\037
\302\243
\302\243;
7;\"
T..\\B:)dP\". SQ,.(.,K)dp\" \037SQ,,(.,KnB:)dp,.+ SQ,.(., Thisimplies that limsupJ (},,(., K n B:>> O} K)dp\" lim supPIt {e(., \037
1I\037c()
II\037(X))
\037
Jim \"
supp,.{Kn B,.L + 0}. .00)))
324
Chapter
to:Asymptotic
UecisionTheory)
Corollary64.14yields limsupJ (lll(\" K)dp\" P{BOn K II -4 \037
=*=
0}.
<XI)
Let (l be the minimum estimate for (Z(t\302\273tE T' Sincecard BO(w) = 1 P-a.e.it followsthat (!is non-randomized and) K) = 1}= J e(.,K)dP.) P{BOnK+ 0}= P{(l(., This implies limsupJ (!II(\" K)dP\" J (!(., K)dP. 11\037
\302\253>)
the proofof the assertion. Now, uniform tightness completes
0)
the previousresults In the subsequent sections are appliedto asymptotic Baycs estimation sincetherebyit ispossible toguaranteeconditions 64.5and 64.6in a estimation is way. The problemof maximum likelihood statistically plausible
discussed below.)
64.5and 64.6are satisfiedwith the notations 64.16Remark. If conditions of 64.1 for then the of the to r esults Example (1) every sET, application previous
maximum likelihood in theory is a matter of routine.The problemconsists the validity of thoseconditions. of experiments Weak convergence establishing doesnotimply the secondcondition and thus,for an invarianccprinciple for the likelihood have additional to be made. W e processes usually assumptions shallreturn to thisin Section84.)
65.Uniformlyintegrableexperiments) A Borel Let (T, d) be a metric spacewhich is locallycompact and separable. K 5;; T. measure is a measureJ1 E aCT)such that J1(K)< 00for every compact a Let 1',. of opensubsets such that 1',. T. Consider 5;; T, n e N, be a sequence = of experiments I E \037}), n E N. Moreover, let(1l1I) (1JII \"s;f\"' sequence Ell on .tM(T). of Borelmeasures be a fixedsequence of thissectionis concerned The main assumption with the separation of of their neighborhoods by tests.Recall Definition pointsfrom complements
{l;'.,:
i
36.1.)
65.1Definition.Suppose that the experiments Then Ell'n E N, are measurable. the sequence if for every SETand every (; > 0 (En) is uniformly (JllI)-integrable there are a compactset K T and a sequenceof criticalfunctions))) \302\243;
65.Unifonnly q>\"
E
\037
integrableexperiments
325)
(D..,.!:II ,,), n E N, such that P,.,;rq>\"
-
>1
and
t:
J
Tn'\
P\"\"
q>,,/-t,,(dt)< e
assoonasSET\"and K T,.. \037
84,1n. The is satisfiedseeDiscussion For an example where this condition isjustifiedby the followingtheorem.) tenninology
65.2Theorem.Supposethat the experiments E\", n E N, are separableand Then the sequence measurable. (E,,) isuniformly (/-t,,)-integrable iff for every sET and > 0 there isa compactsetK s T such that) F.
dP,..,
/-t,,(dt)> e } < e J T\"\\K dP.
p\",s {
II,S)
assoonassET\"and K
\037
T\".)
Let sET and Proof (1)Assume that (E,,) is unifonnty (\037,,)-integrable. a compactset K T and a sequence of tests E :F(D\",dII)' o < t: < 1.Choose
\302\243;
nE
N, in sucha way P.\".s(ft\"\" > 1
that)
82 ---
and
2)
J
\037,,(dt)
P\",,\302\253(P,,)
Tn\\K)
< (.2
2
assoonasS E 1;.and K 1;..Oenote) \037
.dP\"., - /-t..(dl)> l:}, nE N.
B\"I={ J
_
.I'\"\\K
dp\",s)
Thenit followsassoonas sET,.and K
-
) = (1 P..,s(B 1.. II
\037
+J q>,,)dP\",s
P\",Al
\037
q>\"dp\",s
B\
q>,,)
+
]
II
q>\"
\302\243.Tn\\K
\302\2432
\037\"2
T,. that)
:
dp\", .
j1,,(dt)dP\".., d1',..$)
r.
+ < e.) 2
set K (2) Assumeconverselythat there is a compact
\302\243;
T such that)
dp\"\"
J:,.s{ J -) T..\\K dI\".s)
\037,,(dt\302\273\302\243}<\302\243
> 0, assoonassET\"and K Note, that there are sets M\" E
for a given
\302\243
\302\243
\037r#\"
T\",
such that
P\",s(M,,)=
1
and)))
326
Chapter
10:Asymptotic
DecisionTheory)
tE T\", n EN. (Thepointisthat Mil isindependent \037.II\037..(lM.. \037.sl\037\"..(lM..t of t E 1'\" which is possible in view of separability.) Let) \037
< t}, Cn :=M..(I { J dP..., Il..(dt) Tn\\K dP...s) Then we obtainthat P,..s(C..) >1 J Tn\\K
-
t;
EN.
11
and)
dP\"., p..(dt)= Tn\\\".n P\".,(C..) J ), dP. d\037.$p..(dt) u
\".s)
-
= J J dP\".11l..(dl)dP\".s < Cn Tn\\K dP. \".s)
assoonassET,.and K
\302\243;
E;
0)
1\037.
of uniform integrability for posterior considerthe consequences distributions. w e need a lemma.) First, Next, we
lIs:
65,3Lemma. Suppose thaI (Ell)iseqlliconlinuous and Ihalp,,(U) > 0 ifU Tis open,n E N. Thenfor every
P,..s{ J B(r,6) a.\037
dp,.
of
E
T and eL'ery t > 0 there is
\302\243;
<5
0 > 0 such thut)
p..(dt)< 2 p..(B(St } < t 0\302\273
...s)
dP.\037-
soonasSET,.andif 0 < < f10') f1
36.12.ChooseSET
and let N (s) it follows N: S E 1',,}. Fromequicontinuity that for every t > 0 there > exists(jo 0 such that)
Proof Let
us apply Lemma
-= rnin {tlE
II
P,..s-P,..,<
if
II
tE
1\037
and
des,I) < ho.)
\037
choose0 > 0 suflicientlysmall to guarantee that B(s.f1o) provesthe assertion. 0) We may
<'5
\037
Tf'f(s)' This
A sequence of estimates 65.4Definition. Qn E .\037(E..,T)t n E\037, is uniformly setK T such that tight if for every sETand every 6 > 0 there is a compact < t K) < 1 P,.,s{e..(.
-
assoonas SE and K S 1\037
\037
r.)
\302\243}
1\037.
of Borelmeasures Throughoutthe followingwe assumethat the sequence K T.))) (Il..)converges strong1yto a BorelmeasurepifM (T) on every compact \037
65.Uniformly
327)
integrablecxperimcnts
andthat J.L(U) > 0 if U T 65.5Theorem.Supposethat (EJ isequic:onlinuous i!;open.Then (E,.)is uniformly (J.L,.)-inregrable of posterior iff the sequence \037
to distributions F,. with respect
J.L\",
n E
is uniformly
\037,
tight.)
Let sETand > O. By Proof(1) Assume that (E,.)is uniformly integrable. we may find 11> 0 such that Lemma (64.3) \302\243
\037.s{J
dP,.\"
/1n(dt)<,,}< 2 \302\243
dP.
\",s)
for sufficiently large n E N. By Theorem(65.2)choosea compactset K <;;;.T
such that
dP,..r P...s{ J dP. J.L,.(dt\302\273\302\243,,}<2) T,,\\K n.s \302\243
that) assoonassET,.and K 1',.. Then it followswith Theorem36.14 > c} P\".${F,.(T,.\\K) dP...1 = P...s{ dP...r J.L,,(dt)> J.L,,(dt)} \037
J dP. ,..s
\302\243
T,,\\\"
J
d
dP...rJ.L,.(dt\302\273c < 11 =P...s{S T,,\\\"
P...s)
e
}+2
dP.,..s)
forsufficiently large n E N. Thisprovesonepart of the assertion sinceuniform for a finite subset o f N is trivial. tightness distributions are uniformly tight.It is suffi(2) Assumethat the posterior cientto provethe condition of Theorem65.2for sufficiently large n E N. Let subsetsatisfying) Ko T be a compact \037
<1 \037.s{F,.(Ko)
-
\302\243}
:3)
and)
Jl,.(Ko) a> 0,) \037
for sufficiently large n E N. tebysev'sinequalityyields) 3 p.\".s. J dP,.,rJ.L,.(dt ) > Jln(Ko> < . c } 3) {\"0 dP.,.,oS \302\243
Choosea compact setK S T, Ko S K, such that) p.\",5 E,.( T,.\\K) >
{
\302\2432(1-
t)
3a }
<
\302\243
3)))
328
Chapter 10:Asymplotic DecisionTheory)
assoonass E
J\037,
K
Then it followsthat)
T\".
\037
Pn(dt) > t P...sTn\\K J . \037;./ } n,s
{
-
\302\243
\037
3
+ p.II.S
{
J
Ko
f.
F.t
+ P,..s
{
\"'\"'\"\"3
E \037
23
dP. \037 Pn(dt)
J Tn\\K
>
d\037\037
dP\"
.,Pn(dt)
2 f.
}
3 PII(Ko)
dp,.,$)
(1',.K) >
g
\\
f\037Ko)
2
3p..(K o) }
+ ?t.s F,.(T,. \\ K) >
{
r.
\03723+P,.,s {
F,.(T,.\\K\302\273
E2
-
(1 r.) 3J1(Ko) } [.2
o)
and that J1(U)> 0 if U T 65.6Corollary.Supposethat (En) isequicontinuous ;s a .'M(T)wheref T -+ is open.If (En) is uniformly (Iflp..)-inregrable measurahlefunction,thenfor every SETand t; > 0 there isa compactsetK s T \037
\037m
such that
P .S { J T II
If(t)!F.:(dt\302\273\302\243}<\302\243
..\\K)
assoonassET,.,K
\037
T,..)
ProofCopypart (1) of the proofof Theorem65.5.
0)
in the sequel.) importance The followingtwo lemmasare of technical
and uniformly (tln)-integrabl that (En) isequ;cont;,molls 65.7Lemma. Suppose Assumefurther that p(U) > 0 if U T isopen.Thenfor every sET,t > 0, and > 0 such thut) U ofs there exists\" every neighborhood lim P,.,.s{f\037(U) >,,}> 1 t.) \302\243
n--
-
\302\253J)
Proof Let sETand > O. Chooseb > 0 in such a \302\243
that for sufficiently large n
p..., s
B(s,b)
\037
U and
N)
dP,..,J.t..(dt ) < Pn(B(s,[))} < . 3)
J 8(s,6)
{
E
way that
dp,.,s
\037
2
This is possible by Lemma 65.3.In view of Theorem65.2we
may find
a)))
65.Uniformly
integrableexperimenls
329)
compactsetK s; T such that) { P,..$
e
dP\".,
f
T\"\\K
f:
> < dP. Jl.,.(dt) j } 3
SE
if
1'\",
K
\037
1'\".)
11..1)
Moreover, tebysev'sinequalityimplies) P,..$kr
dp\"
r
Jl,,(dt)
{ dP,.:s
We
- e-
> 3 Jl.,.(K)< B
obtainfor sufficiently large11E P.,..sF. ,.( U) < Jl.II(B(!i,
2
{
3'
}
\"
11
N)
/(
O\302\273
.
SE 1 , K S; III')
\037
3
+
3
Jl.n(K\302\273
)}
\037
dP,..t (dt) < Jl.,.(B(s, + - p. {f dP. 2 } <5\302\273
\037
JIll
\"..<
U
\"..1)
Choosing . . f J,l,,(B(s, < 1mm O < t1 = 2 <5\302\273
\\
\"\"'00
provesthe seertion.
_
/(3 \302\243
+
3
2\037
3
< 1:.
Jl.,,(K\302\273
\302\243)
)
0)
and uniformly (Jltt)-integrable. 65.8Lemma. Supposethat (E,,) isequicontinuous Assumefurther that p.(U) > 0 if U S;T is open.Thenfor every sETand every t > 0 there is C(r.)< 00such that
>
of posteriordistributions is Proof Let sET and > O. Sincethe sequence uniformlytight it issufficienttoshow that for every compactK T there exists C(e)< 00such that F.
\302\243
:::
p..,{F.(K\"{ > C(e)})>.}
P.\".$
{f
f. dP,.r ---=dt) < <( dP.\" 3 Jl.
11..1)))
<5
}
330
10:Asymptotic
Chapter
DecisionTheory)
for sufficiently largen EN. Let Bit..., of K and Ii e B. 1 i B,. be a partition such that k, \037
\037
-
IE B} < ' 1 i k.) sup{IIP,.\" p\"\".II: 6J111(K) \037f,2
\037
j
\037
The existence of sucha partitionfollowsfrom equicontinuity, from compact-
nessof K and from J111(I()J1(K)> O. Let C < 00be an arbitrary constant. \037
Since)
dF
J1\"
-\037.
{d
1 >C < = -.
nE N
} C'
J111)
,
it followsthat for sufficiently largen E N p,.,$
dP\".. J111(d/) > \037
f
dP,..s {Kn {\037;.c} ,,\"
< P\",s =
dP\".t
J
{Kn {d,,\"
}
4\302\243..>C
dP.\",S
dP,..I , + C 1=1dP,..s 2 } !\037
\037
.\037
> J111(dt)
dP 1-1dP.II,S J1 ( \"
II .\"
.\037
-1=1
p\".s
{
>c}dp,.,$
K\"{\037\037
- { -1 J :$:r.L. D II,S
t 2
\037
\302\2430
8'''{\037\037.>C}
dP\".t,
dp\",t
1-1
r
-
IIP\",t
\037
\037
t_ 6
Eo)
\037.
This implies)
(d )
f,\037
2} dP\".s\" 2 . oe2 = _e ' P\".,.IIJ1,,(dt)
dp\",$ 8.,,{\037:->C}
Ie
dp,.,$
---/11>-
\037
i
3
::
> K() P...{F.( { c})>.} d > $ 3t + P,.,s J dP. J1,,(dl) P,.\"
{Kn{ >c}
f,\037
II,S
}
\037\037:
\037
_23c +
p\",s
.. \037
\037e
3
+
dp,.r I:O >_ dP. C 2} 11 \",S { 1
A:
\"
.\037
i Ceo. \037_
i= I
J
J1\"
\037P\",'j
- 3 + l:!!_.
dP.\",S <
dP\".s
2\302\243
C\302\243o)
.
d t f we Ch oose Th e assert10n IS prove
C(\302\243)
> 6k 2.t' F.
0)
})
}
dP. d) .---!!:!. J1,,(dl)>-
dP. \037 J1,,(dl) L Ie
\037
d
{
I;O
>C + 2\"
df\037
\037('\\
o)))
2}
66.Uniform
of generalizedBayescstimalCS
tightness
331)
66. Uniformtightnessof generalizedBayesestimates) the casewhere(T, . I) isa finite dimensional In the followingwe consider linear = measureon T. In addition, the situation isthe spaceHnd 11. A.T, the Lebesgue the in section. sameas the preceding Call (E,.)unifonnly (Jl,.)sequence if for g:11-+ o f order it is ItIP. Let p uniformly (g Jl,.)-integrable integrable W (0) = 0, be a lossfunction.) W: T -+ [0,00), I
66.1Remark. In the followingwe denote
-
2,.(1):= J Wet x)f;.(dx),t E T,..Il EN. We
64and 65.) shallpermanentlyrefer to Sections
66.2Definition.A sequence ofestimates of (!n E fJi (En' T,.),n E N, isa sequence if it is a of estimates Wand (for asymptoticBayes sequence asymptotic n EN, for every sequence for minimumestimates (P,..s)' T.) (Jln\302\273
.\\\"E
(Zn(t\302\273IE:T\",
66.3Lemma. Assume that (E,,) is equicontinuous and uniformly (Jl,.)-inregrabl W 1. is order and order level of p of separatingthen for every If p, compact h < supW such that (; > 0 andevery SETthere exists \037
P,.,s{Zn(s) > b}< f: assoonas SE
1\037.)
Proof Let sET and > O. First,we assume that W is not bounded.From Corollary65.6it followsthat there isa compactset K STand > 0 such that) E;
P,..s{J
W(s
-
\037
> c5} < I; I) F;.(dt)
T\"\\K)
assoonass E 1',.. K f.;; 1;..Now the assertion followssinceW(s on K.
- .)isbounded
that W is boundedand separating. Next assume Let c < supW be such that = U {sE T: W(s t) c} isa neighborhood ofs.From Lemma65.7we obtain > 0 such that)
-
\037
Yl
< 0i}< I; Pn.s{F,;(U) assoonas U S W.l.g,'1> 0 can bechosenin such a way c < supW V;;,From
-
J W(s
7\037.
that
- t) F,.(dt) supW. F,,(1',.U) + (sup - V;;) = supW(lsupW. F;,(U)- 0iF;,(U) \037
W
\\
- V;;.
F;,(U\302\273
= supW
+
f\037(U))))
\037(U)
332 it
Chapter 10: Asymptotic DecisionTheory)
followsthat) \037.s{JW(s
- t)F;.(dt)> sup W
t1}
\037
< 01}< t. p\".s{F;.(U)
0)
66.4Theorem.Assumethat (E,,) isequicontinuous and uniformly (jJ,,)-integrab/e > W 1. that order is order of p As.\\\"Umefurther of p,levelcompactand separating. Thenfor every SETand every e > 0 there existsa compactsetKt T such Ihal) { inf Z,,(x) inf Z,,(y)+ e} < e P...s \037
\037
:;crT\"'''\"
assoonasS E 1;.,Kc
yrT\
1;..)
\037
Proof Let SET,t: > O. By Lemma 66.3there existsb o E (0,supW) such that c t) F,,(dt) fb} 1 P,..s (J W(.\037
-
\037
\037
-
2)
assoonasS E Reducinge> 0, if necessary, we achieve and such that (bo + t:)/(1 t:) < supW. Let C s;; T be compact e \037.s {J\037(C) < 1 e} <
-
1\037.
-
2)
assoonass E 1',.,C 1',.. This is possible by Theorem65.5. \037
Let
-
-
> 1 e}.) {F;.(C) > 1 e. Now we choosea compactsetKc s;; T in such a way Then P\".s(A\".c) A\".t:={IW(s
-
that)
W(x
-
t) >
t)
_
bo +e 1
-t:)
F;.(d/)
\037
bo} (\\
whenevertEe and x Kc. \037
sinceW is levelcompact. If x f Kt then we obtainon AII.t This is possible J W(x \037
As a
-
t)}\037(dl)
J W(s
-
\037
cJ W(x
t).Fn(dl)+ e
-
\037
t)}\037(dt)
> bo \037 F;.(C) bo + e
inf J W(y
)lET\
I-t:
-
t)
f\037(d/)
\037
+ e.
0
of asymptoticBayes we obtainexistence and tightness consequence
estimates.)
66.5Corollary.A\037'sume that the conditions of Theorem66.4are satisfied. for Wand (jJ,,). (1) Thereexi.'It.\037equencesof asymptoticBayesestimates (2) Every sequence of asymptoticBayese\037.timatesfor Wand (Jl,,)isuniformly tight.)))
67.Convcrgenceof generalizedBayesestimates
333)
Proof (1) Apply Theorem64.8. (2) Apply Theorem64.t 2.
0)
67. Convergenceof generalizedBayesestimates) the generalcasewhere(T,d)isa metricspace.Let usconsider First,we consider the situationof Section65 but assumeadditionallythat the sequence of experiments (E,.)converges weakly to an experimentE c cr (T). a is basicapproximation lemma.) Our first assertion
67.1(.emma.Supposethat
-.
Let K <; T be compactand (Ell) is equicontinuous.
and bounded.I1Jenlor every t > 0 there are a (T)-measurable measurablepartitionB\\, B2\"'\"BAI of K, poi1J/SIj E B, 1 $;i M, and E IR, 1 i M, such Ihat)
f
K
IR'\"
f?4
\037
j
\037
<Xj
\037
P,..sicf(l)dP p,.(dr). d\037.r
r
II
wheneverSET\"and K
s
\037
M rJ.
dP\".r j
j\037l
J\037,.(Bj)
dP,..s
<
t;)
T\".)
Let t > 0 and K T ProofThe proofisalmostthe same asfor Lemma36.16. =. = It isclear t hat n <00. I fC E C 0 orIIfll..= 0 then compact. sup{p\"(K): N} the assertion is trivial. Assumethat C>0 and 11/11.. > O. Fromequicontinuity of (E,.)it followsthat there existsb > 0 such that) \302\243
\037,s
dP.,..s ,..s dP'-
dP\".I' dP\".I:<
t .- I.f d(t..( 2) < a, 1\\,1 2 E K, 2CII/I'..) \037
whenever sET\"and K T\". Sincef is boundedthere M = and g L 1B. such thatllgll.. 11/11.. \302\243;
(Xj
i-I)
is a step function
\037
-
M
L sup l(t) j\"'l rEI(
rJ.
j
18
_.' r.
;(1) < 2C)
i M, may be chosenin sucha way that diam B; < b, 1 i followsby M,and that {Bj : 1 i M}isa partitionofK.Nowthe assertion the chain of inequalities in the proofof 36.16. 0) already considered
The setsB;,1 \037
\037
\037
\037
\037
-
\037
67.2l..emma. Supposethat
f
K
Rm
-
Let K (E,,) is equicontinuous. fJI (T)-measurable andboullded. If E,. E the\)
\037
T be compactand
Chapter 10:Asymptotic DccisionTheory)
334
dp\"
f
,
JlII(dl)IP\",\037)
ff'(t
-+
dp'
f
ff'(i\\f(l)dl'sJl(dl)I\037))
weakly,)
for every SET.) I.el) Proof By Lemma60.7the limit experimentE iscontinuous. lfJII
dP,..t (dI), (I) If d P. I(
=
-
JlII
n E
N,
and)
\".S
CPo
= J f(l) dp'Jl(dt). d Ie Ps)
for a given e > 0 choose a partition81, 82 \"\",8M of K, pointsIi E B Moreover, 1 i M, accordingto Lemmas67.1and 36.16. and Denote) \037iE\037,
\"1'\"
\037
\037
M
=
dP,..t, \"Jll! (\037),
.L dP 1-1 \"
CXj
nE
t\\J,
and
\".S)
dp,j = 11'0 ,-1 dP. J1(Bi)' \037
CX
.\037
j
oS)
Thenwe have) -+ !I'(lpol\037)weakly, !I'(Ip,,/P\".s) < assoonas s E 1',.,K S 1',.,) lip\" P,..\037
-
lfJ\"
I
E:
and) \03711PO
-lfJol< e.)
Since > 0 is arbitrary it followsby easy arguments that) 1-;
\037(cpIIIJ\037.s)
-+ \037(CPoll\037) weakly.
0)
are the basicfacts for limit theoremson Bayes The followingtwo assertions
estimates.)
that (E,,) isequicon/inuous anduniformly (JlII)-integrable 67.3Lemma. Suppose Then E is Jl-integrable.)
Proof By Corollary36.4it is sufficientto show that = 0, SE T.) p. fJ(dt) = {J :;()} Let sET,c > O. Choose a compact setKt S T satisfying))) \037\037
67.Convergenceof generalizedBayescstimates
-
dP,..,
335)
f.
J dP. p,,(dt)> & < 2 n.s) } {T\"'K
p\".s
assoonasS E Tn, it
Kt
\037
1\037.
Thisispossible FromLemma67.2 by Theorem65.2.
followsthat for every compactK ;?Kr) dp'
f>s
&
p(dt)> e < 2 {K\\K.J dP. } 5)
that) which implies
dp'
= . p(dt)> & < J dP. } 2 {T\\Ke s)
\037
\302\243.
Hencewe obtain)
!,(dl)> C + 2
P,
{I
-
\037\037
\037
Ps
!'(:.) }
dp'
J d p(dt)> T\\K. Ps {
e
e
<2+2<&'
\302\243
-
2--
+ Ps J tip, p(dt)> Jl(KrJ } {K. dPs } f;)
0)
-
that (En) i.'i equicontinuous and uniformly (Pn)-integrahle 67.4Lemma. Suppose oforderp 1.Assumefurther that Jt (U)> 0 if U s; T isopen.Iff T !Rill isof orderp then) \037
-+ .sf<Jf(t)F(dt)I\037) weakly) if<Jf(t)\037(dt)IP\".s)
for every sET.) Proof Let seT. It followsfrom Lemma 67.2that for every compactK with SE /()
-
weakly.) if<JJ(/)\037(dllK)I'p\",s) !l'(Sf(t)F(dtlK)I\037) Since)
-
IJf(t)\037(dtlK) Jf(t)\037(dt)1 + J r;,(\037\\K)JJ(t)\037(dtlK) \037
-
and)
Iff(t)F(dtlK) Jf(t)F(dt) + F(T\\K)ff(t)F(d/lK) \037
f(/)F,.(d/)
T\"\\K)
I
f T\\K)))
f(t)F(d/)
\037
T
Chapter 10:Asymplotic DecisionTheory)
336
followsfrom Theorem65.5and Corollary65.6that for every t > 0 there isa compactsetKe S T suchthat) it
-
Prt,sjJf(t)f;.(dtIKt)J/(t)\037(dt)1< t)
assoonasS E 7;.,K( S T,., and)
- J/(I)F(dt) < 1:.)
\037IJ/(t)F(dtIK\302\243)
I
Now) a similarargument as in the proof of Corollary67.2 provesthe assertion. 0)
-
Fromnow, we assumethat (T) I. I) is a finite dimensional linearspaceand = Let W: b e a T [0,00) lossfunction.) Jl AT' and uniformly (p,,)(E,,) ;s equ;conl;nuous 1.Let W beof orderp. Then Ihe slochastic processes)
67.5Theorem.Supposethut integrableof orderp
\037
tE\037,nEN,) Z,,(t):=JW(t-x)\037(dx), in (P\".s)-probability areequ;continuous for every seT.)
ProofWe have to show that for every sET,x E T, and every > 0 there exists E:
0> 0 such that)
-
> t} sup IZn(Y) Z,,(x)1 P....s{ B(.'(.6))
<\037:
YE
assoonassET\",B(x)\037) c ChoosesET,x E T and t> 0 arbitrarily.Let .5) > 0 be such that W(y t) ;:;;C3 tiP + C4 ifY E B(x,01) and t E T.Thenwe may find a compactset K 5 T
-
1\037.
1
such that)
P\".s
sup
J W(y T..\\IC
{YEB()I;.61)
as soon as SET\" and M21=sup{W(y - t):
Y
E
K
s
- I) F,,(dt)> 4e
}
T.w.
E
<3 .)
This is clear by Corollary65.6.Let
B(x)\037)) t E K}. By Lemma 65.8there is M) < 00
satisfying) E;
f.
> Mt > 4M < 3) 2} { {dJl\" } assoonassET\".Let O2 > 0 be suchthat) P...sF\"
sup )'\037
df\037
f B(x.(h)k
I
W(y
_
-I)- W(x - t)\\AT(dt) < 8 e
M))))
67.Convergenceof generalizedBayesestimates which implies
sup kf W(y ,..B(x.62) I
- - - t)IJl,,(dt)< t)
W(x
1;
4 J\\1
I)
b = min {<5h 2} then for sufficiently large n E N. If we choose
it
<5
P\"., sup
{
Yf'
\037
}
sup kf IW(y-t)-W(X-t)lf\037(dt\302\273\037 P,..syeB(x,6) -
}
{
{
W(Y-/)\037(d/\302\273\037
}
>
F. P...{M2' {:\037 M,} + Mt' sup YF
\037
followsthat)
IZ\"(Y)-Z\"(X)I>E:
B(x.6)
sup J +2P\".J)'eB(x.6) T..\\K \037
337)
B(x.6)k
P...sM2 . F;,
{
f
I
W(y
- /) - W(x - t)IJl,,(dt)> 2t-
}
dF,. > M
i;
> < } 4 } +\"4
_2t2
2\037
1
{dll\"
+
E;)
for every for sufficiently large n EN. Since(Z,,(X\302\273xeT..iscontinuous P\".s.a.e. follows. 0) fixedn c by Lemma 37.7,the assertion \037
and uniformly (p,,)(E,,) is equicontilluous 1.Assumefurther that W is\037'eparating, level-compactand
67.6Corollary.Supposethat integrable oforderp of orderp.
\037
11EN, isstochastically (1) If a sequence ofestimatesE !it(E\",1;,), equivalent isalso with a sequence then a estimates of asymptoticBayes (0',,) sequence of (f\"
asymptotic:BayesesLimates.
-
If the stochastic processZ(t):=J W (t x) F(dx),t E T, attaillsits insET,then every sequence fimum in at mosta single of asymptotic point\037-a.e.. to the generalized Bayesestimates convergesin distribution of E.) Bayesestimate (2)
Proof(1) Apply Corollary64.13. (2) Apply Theorem64.15. 0)
67.7 Corollary.Assumethat the conditions 67.6(2),are satisfied. orCorollary
Then every sequence which convergesin distribution to the geneof estimates ralizedBayes estimate of E is a sequence All of asymptolicBayes estimates. estimates are of asymptoticBayes sequellces stochastically equivalent.)))
338
Chapter
10:Asymptolic
DocisionTheory)
it Moreover, Bayesestimate ofE isnon-randomized. ProofThe generalized itsdistribution since the distribution is uniquely detcrmined determines) by \037
M:A
1-+J II W(x
-
t)e\302\253(JJ,
dx) P'(d(JJ)p(dt),
A
E.G',
A)
and)
-
dM = JJ W(x t)Q(.,dx) F(dt).) dP(\037j
lienee,Theorem 63.6impliesthat all estimateswhich convergeto e in are stochastically In particular, distribution equivalent. they arc stochastically of asymptoticBayesestimates which existsby equivalent to a given sequence 67.6 66.5. the assertion f ollows. Corollary By Corollary 0)))
11:GaussianShiftson HilbertSpaces)
Chapter
In the precedingchapterswe have met Gaussianshifts several times.The rolewhich theseexperiments distinguished play in the classical theory isdueto
the fact that they admit a particularly simplestatistical analysis.Roughly However,in speaking,their theory can be presentedin terms of linear algebra. and came the that m ore more statisticians to conclusion the lastdecades data almostnever originatefrom Gaussianshift experipracticalstatistical ments.Nevertheless, of this type are stillof centraltheoretical experiments The reasonare the resultsof asymptotic statistics. significance. The asymptotictheory of statisticsdealswith localapproximations of for large samplesizesby simplerexperiments. Therebyit turns out experiments conditions limit experiments are usually Gaussian that under mild smoothness
shifts.
Forthe application of the asymptotic decision theory of Chapter10tocases where the limit is a Gaussianshift we need a complete statistical theory of In caseof finite dimensional Gaussianshift experiments. parameter spacesthe are results scattered o ver the pertaining precedingparts of this book (cf. sections28.30,34, 38).The asymptotictheory of non-parametric methods which are Gaussianshiftswith infinite leads,however,to limit experiments is dimensional This the reasonwhy we take up the theory of parameter spaces. Gaussianshiftsagain and presentit general enoughto coveralsothe infinite
dimensional case. The caseof infinite dimensional parameter spacesrequiressometechnical in Section with linear processes toolswhich are collected 68.It is concerned and Section of a Gaussian 69dealswith the generalconcept cylindersetmeasures. shift experimenton a Hilbertspace. We characterize the likelihood of a process Gaussianshift and show that linearprocesses define generalized decision functionsin a canonical way. Gaussianshift experiments are invariant under the translation groupof the I n is of Hilbert case the Hilbert finite dimension the space. underlying space Gaussianshift can be represented asfull shift on this finite dimensional Hilbert But if the Hilbertspaceis space,generatedby the standardnormal distribution. notof finite dimension then sucha representation isimpossible sincethere isno standardnormal distribution on such a Hilbertspace.If the Hilbertspacecan becompleted in a suitable it becomes to possible way (Abstract Wiener spaces), the Gaussian shift an lion of the Hilbert generate experimentthrough opera on the T hese space samplespace. questions, togetherwith someimportant are discussed in Section 70.))) examples,
340
68.Linear stochastic processesand cylindcr set measures)
In Sections 71 and 72 we presentthe theory of testingand estimation for with Gaussian shifts. S ection 7 3 deals the c ase of Abstract general particular Wiener spacerepresentation which admitssomesimplification. The contentof this chapteris well-known.The first presentation in the from in can literature our viewpoint be found the thesisof Moussatat[1976] which waswrittenunderthe guidance ofLeCam.Fromthere we take the proof of Theorem71.14. The thesisof Moussatatalsocontains a particularcaseof the minimaxboundgiven in Theorem73.6. Thistheoremisformulatedfor the to first time by Millar L1979] resultsof LeCam.The referring unpublished shouldbe general minimax boundof Theorem72.7, althoughunpublished, welJ-known,too. A
of the subjectwith a similarintentionas oursis Millar presentation
[1983] .)
and cylinderset measures) 68.Linearstochastic processes
...
Let (H,< ., .
H
be a separableHilbert space.Any linear functionZ: 1i'(0,d),where (0,d) is a samplespace,iscalledlinearprocess.) \302\273
H be a measurable 68,1Example.Let K: Q... mapping.Then
Jz E H, defines a linear process, Z (h):=(h,K To exhibitanotherimportantexamplewe needsomebasicfacts onGaussian (.\302\273,
processes.) Let T:f= 0 be an arbitrary set.A stochastic 68.2Discussion. process X = (0,d, P,(X')'ET) is a Gaussianprocess if all finite dimensional marginal of X arc Gaussiandistributions. The function) distributions
/).-J (Xs - P(Xs
K:(s,
\302\273
(X,
-
P(X,\302\273dP,
s,t E T,)
i.e.) the covariance isthe covariance of X. Obviously, K ispositivesemidefinite, It
It
i-Ij-l) \037
\037
(Xj(Xj
K(tj,t j) 0 \037
...,
..,
for every choice (t 1> /,JE T', . (X,,) E R'. As a matter of fact, every positive semidefinite symmetricfunctionK on T2 is the covariance of a Gaussianprocess. To seethis,let Q = \037T, = (R)T and definefor every (Xc A(T) a probability measure \037I\037(\037)Q! by) \302\253XI'
d
J exp
; (L ) xrJ\037
'\037Q!
\037(dy)
= exp
(
.-1 L 2
S\037Q!
L x(x,K(s,n , ItHI)))
)
\037
68.Linear slochaslicprQCe$SC$and cylindcr set measurcs
341)
(x,),uE W. Then it is easy to see that (\037)a.A(T) is a projectivesystem of limitand measures. DefinePI\037(Rlto be the projective Gaussianprobability ta ke X,
= PH
tE
T.
in ariseswhen T itselfis a Hilbertspace The casewe are interested
In this case<.,. ) is positivedefiniteand thereforethe covariance ., function of a Gaussian (H,< .
\302\273.
process.)
68.3Definition.A GaussianprocessX = (0,d.P.
(X(h\302\273,,<;
Gaussianprocess (for H) if
H)
is a standard
(1) JX(h)dP=O, hEH, (2) J X(h l ) X(h 2)dP =
...
68.4Theorem.Let X: H L2 (0,,f;/, P).ThenX isa standardGaw;sianprocess hE H.) \037(X(h)1P) = iff X islinear andsatisfies \\'o,lIhll\037'
Let (al Proof (1) Supposethat X is a standardGaussianprocess.
and (hi'\"h,,)E H\",n E N. Then f
(.t.a,
dP =
X (h,\302\273)'
,t,
i,ala
Jf
\"
\"
\037
\037
...a,,)
E
R\"
X(h,) X(hJ)dP)
..
= L... L...a.a.
=
II
2: C(j h i=
i Il
2
.
I)
\"
\"
Hence,L ajh = 0 implicsL \037jX(hj) = 0 P-a.c. 1=1 i=l that X islinearand (X(h) I P) = (2) Assumeconversely, C(,,) E Ifi\" and (hi'\"hI!)E H\",11E N. Then we have) (C(t j
9'
...
=
fexP(i.t
a.X(h.\302\273)dP
1\" - 2: -. 2 (
=exp
11
\037JchJc112
Jc=1
\302\253h\",
hE I/. Let .I,,\"1,
fexP(iX(.t,a.h.))dP)
)
=exp
... h,)}
Thisimplies that the vector (X(h l ) and covariance matrix
vo
I \037a.'
X(hll
\037\
(
\302\273
-.21\"\" L L C1.JcC1.,
)
isjointlyGaussianwith mean zero o)
68.5Example.IfH isof finite dimension then any standardGaussian process 2 X: H... L (Q,d, P) can bewritten asX(II) (h.K).h E H, whereK: 0 His))) \037
-
342
Chapter
11:GaussianShifts on Hilbcrl Spaces)
a random variable.E.g.take an orthonormal base(el II K 1= X(ej)ej .
'\"
ell)
of H and define
\037
i= 1
-
HilbertspaceH a It will turn outbelowthat in caseofan infinite dimensional be derivedfrom a can never standardGaussianprocess mappingK: Q H.)
68.6Definition.Let!fbe the system of finite dimensional linear subspaces L s:H. A family ofBorelmeasures \037LI\037(L), L E 2',isa cylindersetmeasureif it isprojective, i.e.if Ll L2 impliesJILl = !f(PL\\1ilL).) \037
68.7Examples.(1)The system of standardGaussianmeasures N,.,L E :.e, It a iscalled the standard set measure Gaussian defines cylinder Nil:(N')'.f!:/.\"
-
distribution on H.
linearprocessZ:II 9'(0,d) definesa cylinder set measure to the followingconstruction: Forevery L E !f'let ZL: Q L besuch according that 2(h)=
\037
\037L
Z under P(=df(ZIP\302\273.
isthe ofa standardGaussianprocess It iseasy to seethat the distribution standardGaussiandistribution.) (3)
68,8Discussion, In a separable Hilbertspacethe Borel-cJ-fields of the weak and the strongtopologycoincide sincethe closedballsare alsoweakly closed. of H (=dl(lf\302\273. Therefore,it is not ambiguousto speakof the Borel-a-field A Borel set B II is a cylinder set if there existsL E!I!such that B = (BnL)
\037
\037,
1
\037
\037L
Borelmeasure.)
68.9Theorem.If H is of infinite dimensionthen Nil cannot be ex/ended/0 a Borelmeasureall .91(11).)
ProofWe shallshow that the outermeasureofevery balliszero.If Nil couldbe to extendedto a Borelmeasurethen the measure of the ballsshouldincrease radii tend to infinity. oneas their baseof /I. Define Let r> 0 and (ej)je an orthonormal n 8 = {IIE H: L <\",ej>2 r 2},n E N. Then BII!B(O,r) as\" 00.It will turn j= out that Nil(B ) t 0 asn 00.))) 1\\1
11
\037
I
II
\037
-
69.Gaussianshift
cxpcrimcnls
...
The set Bn is a cylinder setwhose baseis the ballof radiusr dimensional subspace Ln = span{e1 e,,}.Therefore,) N lI (Bn) =
the finite
n) NLJB\"r.L 1)
J
n
(2n)2) \037
in
343)
\037
... J)
x\037
\037
exp
\
(-\037 it,X?)dX , ...dx o)
,-\\)
1 , A.,,(B(O, =. - ----,.20\"r (2,,)2 r\302\273
--.-.
r\"
(1+
\037))
which tendsto zeroas n -+ 00.The assertion follows.
0)
68.10Corollary.If H isofinfinite dimensionthen a standardGaussianprocess cannOIberepresented asX(h) = Horelmeasurablefunction.)
(X(h\302\273hH
(h,K(. hE H, whereK: Q -+ H isa \302\273,
a standardGaussianprocess, Proof.If there existeda mappingK generating m easure itsdistribution a Borel on 9f(H)with the would be coinciding standardGaussiandistribution. By Theorem68.9,such a Borelmeasure does not exist. 0) then
69.Gaussianshift experiments)
(.,
is a Euclideanspace.The standardGaussianshift on Supposethat (H, is the experiment(; = (H,rJI(H),{NH * Ch : h E H}).) (H,
(.,
.\302\273
.\302\273
is an (., experiment t!(H)which is equivalent to the standardGaussianshift on (H . . a separable Hilbertspace. extendthe In the following(II,(., denotes definition.)
69.1Definition.A Gaussianshift on a Euclideanspace(H,
.\302\273
in
t
< ,
\302\273.
We
.\302\273
preceding
A Gaussianshift on (H,< . , . is an experiment E E G (H) 69.2Definition. such that for every finite dimensional L subspace H the experimentEL is a \302\273
\302\243
Gaussianshift.)
69.3Lemma. (1) Any equivalent.)))
Iwo Gau.5sianshifts on the same
Hilbertspaceare
344
Chapter 1I:GaussianShifts on Hilbert Spaces)
(2) Every experimentwhichisequivalent toa Gaussian shift isitselfa Gaussian shift.)
are true for Euclidean Proo}:It is obviousby definitionthat the assertions the followsby considering the restrictions to spaces.Hence, general assertion finite dimensional 0) subspaces.
otherwords,the propertyof beinga Gaussianshift is a propertyof and there isexactlyoneequivalence ofexperiments classfor equivalenceclasses eachHilbertspacewhich containsGaussianshifts. Thereis a strongrelation betweenGaussianshift experiments and standard processes.) In
69.4Theorem.An experimentE = (D,d, {Ph:hE H})isa Gaussianshift iff the stochastic (X process \" which isdefinedby) (h\302\273hI!
-
dP = 1 hE H,) exp(X(h) 211h1l2), dP; under Po.) isa standllrdGaussianprocess and thereforeits equivalence classis Proo}:In any caseE is homogeneous determined of show that for a by the distribution H'Easycomputations c finite dimensional L H subspacc (X(h\302\273,,(
d(NL * f.h) = dNL
.
exp\302\253h,
- 2\"hll), he L. 1
IdL.)
2
observethat the distributions of I.underNL and (X iff isa standardGaussianprocess. 0) Po coincide
We
\302\253h,
idL\302\273he
(h\302\273he
I.under
(X(h\302\273hell
It shouldbe notedthat on a given probabilityspace(D,.rd,P) there exist which are differentfrom eachother.On several standardGaussianprocesses with a particular the otherhand the standardprocesswhich is associated P-a.e.) Gaussianshift is uniquely determined
69.5Definition.Let E be a Gaussianshift.The standardGaussianprocess of E.) associated with E iscalledthe central process
69.6Corollary.For every standardprocess(X(h)ho;H there existsa Gaussian is central.) shift Efor which (X(h\302\273h4iH
of (D, Proof If (X(h\302\273hH is a standardprocess and Phld',hE H,
\0379/,
by)))
P) then we define Po1=P
69.Gaussian shift
experiments
345)
1
dP\"
2 hEH. dRo) =exp(X(h)-2I1hll),
It iseasy to seethat (D,.>I,{P,,:hE H})is an experiment.
U)
can bewritten If(H,(.,.\302\273isa Euclidean spacethen a linear process asX(h) = (h,X),hE H, where X: Q 1/isa random variable.If H is central for a Gaussian shift E then letus callthe randomvariableX central for
-.
(X(h\302\2731IE1I
(X(h\302\273'H:
E, too.)
(.,
69.7Examples.(1) Let (H, . be a Euclideanspaceand E = (H,fB(H), {P,,:Jz E H})the standardGaussianshift on H. Then X = id H iscentralfor E. (2) Let H = IRk and (x,y):=x'fy where f is a positivedefinite (k x k)matrix. Then Nil = r I. Since) dv\" .I'= dvO.r-1)(x) exp(h' I x 2 h' f h), h E H, x E H, \302\273
--
I
the experimentE = (IR,\\ X = id H iscentral. (3) Let l/ = Ilk and
Thenwe have)
-d n.,.
dV
-
,1 -
\\10,
{vlt
\037\",
.r
,:hEIR\"})is a standardGaussianshift and
(.,.as before
but let
>
x- 21hI r h),
(x) = exp (hI
\\10,1')
E = (IR\\ 14k, {\\In.r: hE R k }).
hE H, x E H,
-,
which implies that E is a Gaussianshift and X = r idll iscentral. Note,that E is not a standardGaussianshift on (H,( . (4) Let (H,( . be an arbitrary Hilbertspaceand L H a finite dimensionalspace.Let E = (Q,.>I, {P,,:hE H})be a Gaussianshift on Hand EL its If ofE and if k\" restriction to L. H is the central process e,.}isan
.,
.,
\302\273.
\037
\302\273
...,
(X(h\302\273\"E
orthonormal basisof L then) XL =
,. j-L,) X(ej)e j
isthe centralrandom variableof E,..Thisisdueto the fact that for each hE L) X(h) = X(
,.
L, (h,ei)ej)
i\"\"
=
/I
,.
L (h, = (h,1:=') L X(el)e/). ;=1X(el) e)
If Lit L2 are finite dimensional and L, L2 then it isclearfrom subspaces abovethat XLI = PLIoXL:' then))) (5) If (/I,( . is a EucJideanspaceand L H is a ]inearsubspace, \302\243
.,
\302\243
\302\273
346
Chapter
11:GaussianShifts on Hilbert Spaces)
E = (H,91(H), {Nil.e,,:hE L})isa Gaussianshift and PL isitscentralrandom variable.This is the caseconsidered in Section 30.)
...,
69.8Remarks, (1) Let us computethe Hellingertransforms of a Gaussian shift.For (ZI, z,,)E S\" and h E H, 1 i n. we have) \037
j
\037
.; = 1 L(hj) - 211hdl2 dPo) dPo J exp J j1)1 )) ( ( ( ) 1 1\" 2 = cxp - - L zill h dl 2 + -- L zjh 21=1 ) ( 2 j =1 2 1 . J cxp L _ zjh j ,r. :i:. ) 2 1-1) ) dPo ( (,-I dP
\"
\"
Zj
i\037l
d\037
\"
j
Zjhj
= exp
1 ( 2
\"
2 zjllhl 1l +
t\037l
1
i
n
zlh
l)
I\037l
})
Conferthe result of Example5.11. betweentwo measures of a Gaussianshift is) (2) The Hdlingerdistance 1 1 2 2 h dl2 +lIh2 2 d2(PItI,Pltl)=I-exp -4(lI )+Sllhl+h211
(
=
t-exp
)
11
(-\037lIhl-h,II')'h,.h,EH.)
It followsthat any Gaussianshift is uniformlycontinuous. invariant. This followsby easy (3) Every Gaussianshift is translation transfonnsgiven in (1).) from the Hellinger computations and 69.9Lemma. Supposethat process 11 is a stochastic = E (Q,d, {P,,:hE H})an experiment.Then E isa Gaussianshift and (X itscentralprocess iff (X(h\302\273,,\037
(h\302\2731tE
II
(1) the distribution logdP\037 , X(h 2) Po is Gau.'isianwith expectation !\302\243
(
-
II \037
h
(
dPo
)
,II', andcovariance(h\" h,),hi E H, i = 1,2,andeither
0)
(2) E isa Gaussianshift or under Po.) isa standardprocess (3) (X(h\302\273IaEH
Assumeconversely, ProofIt isclearthat conditions (1)and (2) are necessary. that (1)issatisfied. Then it followsthat the variance of)))
69.Gaussianshift
dP;
dP log
X(h) +
347)
experiments
1
2 11hll2
under Po iszerofor every hE H. Hence, (2) and (3) imply each other.
69.10Theorem.A stocha.'ttic process
(X(h\302\273\"\"
E = (D,SII,{P,,:hE H})iff)
0)
i'icentral for the Gaussianshift
II
2'(X(h1)IP\"2)= V(I...II2>.II\"IW for a/l
hi
E
H,112 E H.)
is central.Let hi E H, h 2 E H. Then we Proof (1)Supposethat invariancethat) obtainfrom translation (X(h\302\273\",,\"
= !t' 2'(X(h,JlPo)
(lOg\037:
+ IIh,lI' Po) \037
= 2' + 11\",11' (lOgd\037\037::, PA') 1 2 = Y X(h 1 1 + 11hz + +h 2 2 2\" 111 PA')) 211h1 ( \037
)-
\"
11
21h2
)-(h ,h )IP ).)
= Y(X(h1 h2 E
1
2
It2
that .5f(X(h1)IP\"2)= \\'<\"I.II2).1I1t1l12 for all hi (2) Converselyassume H. A standardargument showsthat for every hElland C E R)
-1
1( \037
1:\037\037>uP(c-illltl'2)}
(X(I.\302\273c)
-dP\"
)(dPo
-21 (
exp c
11
E
H,
> -a e )) = 0 0\
h II 2
Po
the Po-expectations of theseexpressions vanish. It followsthat Moreover, -l{x(h\302\273cd Poll{los +illllIl2>c}
= 0)
\037;\037
for every h E
Handc E R. Thisimplies that)
1 dP\" _ 2 10gdP -X(h)-21Ihll Po-a.e.,hEH.) o
o)
(.,
69.11Corollary.SupposeIhal (H, is a Euclideanspace.A random variableX: Q H ;s central Ihe Gaussian for shiflE = (Q,d, {Pit:hE H})iff
-.
\037(XIPh)= eh *
-
Nil'hE H.
.\302\273
If H is a finite dimensional Hilbertspacethen the centralrandomvariable H is a non-randomized decisionfunction in :?t(E,I/).In general, docsnot definea decision function in at(E,II).))) however,the centralprocess X: Q
Chapter
34M
11:GaussianShifts on Hilbert Spaces)
functionf H Iij iscalleda cylinderfunction if there is L E !Rsuch that Such a process ( H be an arbitrary linearprocess. fOPL = f Let Z = (Z definesa systemof randomvariables(ZL)Le\037which are connected by) A
\037
(h\302\273\"
ZLI = PLI oZLz if
LI S L2
.)
Z isof the form Z(I1)= (h,K), 69.12Remark. Supposethat the linearprocess hE 1/,where K: Q -+ 1/is a Borelmeasurable function.Then for every L E !f 0 = we have Z,. P,. K. If P E :lI(E, functionwhich isdefinedby H) is the decision K
then)
P(f,Jl) = ffo Kd}l= ffo PL 0 KdJl = ffo Z\"dJl)
iff E f/ b (If) is a cylinder function satisfying f 0 P,.= f for someL E !R and if Jl E L(E).For every L E !f' the random variableP,.C K =:Z\" isalsoa decision functio