This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
0 and therefore 0) < H(t).This implies S(w) < t and
PsoCP> PSz
o = 1
cP\302\253
ii>\302\253
-
p\037
_\302\253
Fromt 1.7we obtainthat) d Y H o Psipy dO' 9=&0)
iscontinuous on [0,(x].It followsthat there issomej E [0,(X] such that d
_
=
Ps
d8
9=90)
p,
Thisprovesthe assertion.
0)
:
for all (x, p E [0,1] there donotexisttests 13.2Remark. Let 90 E e.Ingeneral, = p. Hencethere neednot existtwocP E Y such that PsoCP= (x, dO' PalfJ 9
sidedS-testssatisfying IX
E
.. theseconditions. But for
d.
90
[0,1]there are two-sidedS-tests
Pc.= 90q>
IX,
d8P9q>)
cp* E
every
fF satisfying
80 E e and every
=0.)
8=80)
lemma choosing This followsfrom the preceding qJ =:
(X.)
13.3Lemma. Supposethat E isan exponential experimentofrank I. Let8 90
- 00< + k Sf ' (h ) a.e.
S-testsatisfying and letq>* be a two-sided k 1 e R, k2 e litsuch that
..-
cP
_
I if fa > k 1fao 2 90 {o if fa < k 1f90 + k 2 Sfao')
t1
\037
=+=
t2
<00.Then there are
v _
ProofWe have to show that there are kj E n, k 2 En, such that I if t > k'iebS + \"2SeIlS, _ (Ilv )_a.e. q> {0 if 1 < \"Ie\"s+ k'2 Se\"5, where b = .90 9 * O. the system of linearequations))) Case1:Assumethat '1< 2 , Consider
-
'
13.Two.sidedtesting for exponential experiments:Part 2
ebll + It ebll = I, It.eb/2 + k;12 ebl2 = 1 The system has a uniquesolution since) e'\" t 1 ebll ..12)(t 2 = el'111 D.= evIl) .. k't
65)
k\037
.)
-
I
12ebrz
t
I)
) > O.
The solution is)
- ( e - II') e ) I-D 1 .1t - -D(e e '2) 1
k'
t 2 bl2
bll
bl
2
lo
.)
we notethat sgn b sgnk\037. Then,there is no intervalwhere Moreover, g:t 1-+k'li\" + t eb, , I E R, \037
k\037
isconstant; in addition, the property) g'(t)(s t) 0 if g(l) g(s) that g isa quasi-concave function. Thisimplies that {g::::: of implies I}consists = at mosttwo elements and {g>1} (/ 1,/2) since{II'12} {g= 1}.Hence 1> k'ieb, + k\037tebl impliest < II or t> 12 , and 1
-
\037
\037
\037
The system has a uniquesolution since)
Ie .- be (:1 D'0 + t)e blo
CblO
blo
b10)
is) The solution
-1
(blo + l)eb'O, k'i ::::: D) k'z
=
1
be'\"\302\260.
D)))
= e2b'O(bto +1
-bl > o)
o.)
Chaptcr2:Elementary Theory of TestingHypotheses)
66
notethat sgn b * sgn and argue as in case1 : h'. t k'I e'\" + k' ((I' t E IR ,) isa quasi-concave Thusa function,and there isno intervalwhereh isconstant. localextremumof h isa strictglobalmaximum,and from h(to) = 1,h'(1o)= 0 and h\" (/ 0) t' 0 we get that 1> k 1 e + kiteblis equivalent to t * 10'This provesthe assertion. 0)
2'
We
k\037
\037
bl
13.4Theorem.Supposethat E is an exponential experimentof rank I. Let 90 e andcp .'F.If cp* E.'Fis a two-sided S-testsatisfying \342\202\254
\342\202\254
= Psocp. d p, * do sCP I}'
PSolp,)
-
= d
df:)
9\03790
P\037cP
9\03790')
then)
Pscp.2:PsCP if 9 * 90 , classof testsfor (H,K).) S-Iestsisa complete Thus, the setof alltwo-sided
'
S-testwith t l = -00.Letfo.=fso ProofCase1:Assumethat cp* isa two-sided and fl'= Sj\037. We observe,that lp* isan upperS-testand from Theorems(11.6) i t follows that is a N P-test the for (!0'!1) in senseof cp* generalized (10.4) The same is true of cp, and cp* and cp coincide Definition(10.2). (hv)-a,e,on {ft * t 2fo}.Hencecp is(hv)-equivalent to an upperS-testand from (9.9)(2) it followsthat the power functionsof cp and cp* coincide. is provedsimilarly. Case2:If t 2 + 00,then the assertion there Case3:Assumethat 00< t l t 2 <00.Let 9 * .90 , By Lemma(13.3) are k I E !R, k 2 E IR, suchthat
::
* cp
1 {o
=
if if
-
\037
fs>klfso+k2 Sfso. (hv)-a.e. fs < ktfSo+ k 2 Sfso'
we obtain From(10.3)
Pscp.
\037
PsCP if
9 =F 90 ,
0)
13.5CoroUary.Supposethat E isan exponential experimento.frankl.lfvalue
at a pointof e S-testscoincide andslopeof the powerfunctionsof two two-sided on S-testsare admissible then they coincide Moreover,two-sided for (H,K).)
e.
is an exponential experimentof rank I. Let = Assume that cp.E '\037a(H, K) {90},K {.9:[) .90}.
13.6Corollary.Supposethat (1.E
[0,1],80 E
\342\202\254o),
and H =
is a two-sided S-test.Then)))
E
=1=
13.Two-sidedlesling for exponcnlial cxperiments:Part 2 P\037tp*
= sup{Pscp: cpE .rp;a.(H, K)} if 9 =f 90
67)
,)
= 0 sincethe power K) satisfies !Fa.(H. PsoCP= (I.\037 Ps'P 3=80) has a localminimumat 90 , 0
ProofEvery
cp E
13.7Corollary.SupposethaI E is an exponential experiment01rank I. LeI (X
E
[0,1],90 E e, and H = {.90},K = {.9:9 .90}.If a lestcp E !Fa.(H,K) i.'i =4=
K) at a single optimalin S;a.(H, poill/.9 =t: .90 then it is uniformly optimalin \037a.(H, K).)
the assertion istrivial in case = 0 or = 1.Let ProofSinceE ishomogeneous E (0,1), Further let cp E !F,AH, K) and 9 E K be the pointwhere cp is optimal. a two-sided i .e. E be S-test, cp. :Fa.(H, K) I jf S < t 1 or S> t 2' v-a.e. . cp = {o If t 1 < S < t 2 . (X
(X
(X
.
-
'
(0,1),Lemma 9.7 implies 00< 'I :::::; 2 < + oc. lienee,Lemma 13.3 k l E 1R, k 2 E !R such that) givesconstants As
(X
E
* cp
- {0 _ I
This shows that
if
I> k1/80+ k 2 S/80
if f< kd8.0+ k 2 Sfso ' tp*
is a generalizedN P-testfor
(j\037o'
Sf90;
j\037).
From the
with tp* optimalityof cp it followsby Theorems10.4and 13.4that cp coincides v-a.e.on {Is k1fso + k2Sj\037}.Accordingto the proofof Lemma (13.3) the and coincide v-a.e. where t S t teststp S 1 and tp* 2' Thus,tp isa two-sided S-testand thereforeuniformlyoptimalin (H,K). :f=
=+=
=+=
0)))
!F\302\253
Chapter3: Binary Experiments)
The power and simplicityof the lemma of Neyman and Pearsongives riseto the development This theory is a of a general theory of binary experiments.
caseofthe generaldecision seemsto simple special theory,but itsisolation in severalrespects. beadvantageous Generaldecision o n relies theory powerful In contrast,the theory of binary experiinstruments of functionalanalysis. ments can be basedexclusivelyon the lemma of Neyman and Pearson.A betweenexperiments, the so-caHed decision-theoreticaHy meaningfuldistance deficiency,is easy to handlefor binary experiments. In Section 14we definethe errorfunction of a binary experiment. Section 15 containsthe discussion of the natural semi-ordering of the classof binary In particular. we show it can be expressed experiments. by the pointwise The topological structureis treatedsimilarly, orderingof errorfunctions. and convergence of binary experiments Finally,it is provedthat semiordering can alsobe expressed functions o f by power optimallevel-oc-tests. Sections 16and 17 arc concerned with equivalence classesof binary experiments and their characterization. It turns out that besides the errorfunction boththe distribution of the likelihood ratioand itsMellintransform determine A the equivalence classes iscompactness of the spaceof uniquely. consequence binary experiments. It is convenient to introduceat this point the conceptof contiguous of probabilitymeasures. theseare sequences of binary sequences Essentially, The whoseaccumulation experiments. experiments pointsare homogeneous essential in Section 18. facts are presented the of 3 is Theconcept ofcontiguity is whole well-known. Practically chapter very
dueto LeCam[1960].)
14.The error function) Consider a binary experimentE = (D,.9I, (P,
Q\302\273.)
14.1Definition.The errorfunctionof the binary experimentE is the function g:[0,1] [0,t] which is definedby g(oc)= inf [(1 oc)Pcp+ ocQ(1 cp)]t oc E COt 1].
-.
-
-
.F\037
00 we definefor reasons In the followingassertion ofconvenience: = 1.The t +00)))
14.The error function
69)
to Q and of Q with respect to Pare ofP with respect Lebesgue decompositions denotedby and M) (\037; . N).) (\037\037.
14.2Theorem.Let g be the errorfunction of E. if 0 :5k \037.
areequivalent: e:F then the followingassertions
(1)
\037.
(2)
g(IX) = (1
E
'\302\245J.;(P,
-
Q) for
\037
IX, 0
\037
IX
\037
- - 1.
= 1
k
-
<X
+ exQ(l
ex) p\037.
\037.).)
Proof (1) (2):If k = 0 then Q\037* = 1andtherefore \037
+
C 0) Pcp\" C 0) Q(I \037
-
= o.)
rp\")
\037
On the otherhand we have)
g
C ) \037o
=g(l)
Q(I-cp)
\037
O.)
\037
:\037\037
If k = 00then p\037. = 0 and therefore
C:'00)
Pcp\"+
C:00) Q(I-rp*)=
O.)
On the otherhand we have)
1 g 1+ 00 = g(O) = inf ( )
-.
P\037
= O.
tPE\037)
Let 0 < k ex
<00and =
1
l+k For any iF we have dQ dQ J dp dP+ M'r-.N' J (l-IX)-IXdP 1 [ IX
\037dP
(1-
+ ex! = (1
-
\037)dQ
IX)
The infimum isattainedfor
1 if \037.
\037E
=
0 if
+
(1- L ex)
P
Denote 11'= {Ps:9 E e}.Let QoE C(1I')besuch that Q() '\" We (1) (3): = Ll (a,do,Qo)' Let may chooseL(E)= Ll (n,.9I,Qo) and L(EI..#o) M1: L,(0.,do,Qo) -+ L,(0.,d, Qo) be the operatorsuch that for every classf E L.(n.sfo.Qo) the value M1f is the corresponding Qo.equivalence class Qo).Note that the functionsin MIl are doequivalence in Lt (Q,.PI, where 0 denotes the Qo-completion measurable of 0 in It isobviousthat . . to sh ow t h at M dPsl.PIo = dPsl.PI . h It remams MIlSa stocastlc operator. 1
e.
9.
\037rd
\037\037
See.Thisfollowsfrom 20.6where for every measurable e. 0 for every
/1 E
\037rd.
(dQoldo )
it
isshownthat
dQol\037rd
is.ii0-
:PsII\037: Qo)
24.14Remark. Suppose that E = (Q,.rd,9')isa dominated experimentand let il be the 9-completion of .PI.Define E = (Q,.ii, Then it is obviousthat 9). of E.On the otherhand we notethat EI.9I= E and henceE isa randomization d isE-suflicient. HenceTheorem24.13 that E isa randomization of E.) implies
24.15Theorem.Supposethat E = (n,.91,{Ps:.9 E e})is a dominated experiment and T:(a,.PI)-+ (at>.PI.)is a measurablemapping. Then the following assertions areequivalent: (1) T is E-sufficient. (2) (3)
T.E. E?: E IS a randomization
017;.E.)
The further argumentsrely ProofTheequivalence (1) (2) followsfrom 22.4. 1 on Lemma 23.5which impliesthat '4E and EIT- (.PI.)are mutual randomiwe obtainthat (1)isequivalentwith zationsofeach other.From Theorem24.13 I of EI T- (dI)' Sincethe relation of beinga the fact that E is a randomization of T.E. is transitive it followsthat E is a randomization randomization that a E randomization anotherapplication of transitivity shows is Conversely, of T.E.This implies of EIT-1 (d1) if it is a randomization (1). 0) \037
to hasan immediategeneralization Theimplication (3) => (2) ofTheorem24.13 arbitrary pairsofdominated experiments.)))
25.Slatistical isomorphism
109)
24.16Theorem.If a dominated experime\"tFe 8(e)is a randomizationof a dominated Ee then E f F. experiment 8(e) Proof.DenoteE= (DI\"SIIt., {\037: ge e})and F= (Q2\"\0372' {Qs:Be e}).Let If M.:L(E) L(F) be a transitionsuch that MI = Qs,9 E d let the be Let M2 be the classin M fP e :IF (Q2' 2) corresponding Markov operator which isdual to Mt.Then for tp e M2 ip, 0 tp 1,we have = QsCP= QsfP,ge 0) Pst#' = Ps(M 2cp) = (MIPs)(cp)
-.
e.
\037
\037
(\302\243).
e.
\037
\037
of the preceding isnotvalid.The orderstructure which Theconverse assertion
of' doesnotcoincide with the is definedby the relation\"is a randomization of the former order orderstructure A decisiontheoretic interpretation in Chapter9. However, both order structureis possibleand is established relation. Thisis provedfordominated structures inducethe same equivalence \037.
in the followingsection.) experiments
24.17Remark. Finally, we notethat Theorem(23.9)remains valid if the that Fisa randomizof exhaustivityisreplacedby the assumption assumption ationofE.Forprovingthis,repeatthe proofof Theorem(23.9)usinga pairof insteadof the kernel K.) operators
25.Statisticalisomorphism) 15.1DefinitiOD. Two dominatedexperiments in See)are statistically isomor-
of the other.) phicif each is a randomization
Example.If E = (D,.91, {P,9:9 e e})is an experimentand .91 0 d is a sub-a-field then E and EI.91 iff E 0 are statistically EI..9I isomorphic o.Thisis a consequence of Theorem24.13. 2\037.2
\037
'\"'\"
2)
shallprovein thissectionthat two dominatedexperiments E and Fare iff E.-F. This resultis a particularcaseof the statistically isomorphic 2 randomization criterion which is provedin full generalityin section 55.First, we consider finite experiments.) We
25.3Definition(Blackwell[1951]). Let e be a finite setand = E (D,..91, Then) {Ps:9 E 8})an experiment. 2: p. Pa )Sf9 a.9 ) ((d!: d\037
a\302\243:=!R
fJ
fJE6)
is the standardmeasureof
E.)))
1to
Chapter4: Sufficiency.Exhaustivity
and Randomizations)
25.4Theorem(Blackwell[1951]). l.ete beafmitesetand E, FE Bee).Then F isa randomizationofE iff thereexistsa stochastic kernelK:58x \037(58)-+ [0,1] satisfying thefollowingconditions: (1) JA(y)K(x,dY)=A(X),xeS8,9Ee. (2)
tJF.(A)
= J K(x,A)tJF(dx),
A
E \037(S9).)
9 E e}) is a randomizationof Proof Assume that F = (U2 , d2'{Qs.: = E (Ul , d'l' {P\037:.9E e}).For E and F we define the so-called standard experiments)
1;.=(S9'\037(S9)'{AUF.: 9 E e}), and 1;'=(S8'91(59), {AUF:9 E e}),) Let)
d
Ps. Tt:Wtl---t. d
(
7\"2:
W2 I---t
.
)h8,
-(Wt)
L P(I (1.9) Q9
(d I:Q(I .\037
(IE 9)
(W 2
\302\273
)h9,
Wt E
W2 E
1;
{It,) U2 .
1;.. 1;..
ThenTt isE-sufficicnt, isF-sufficicntand Es = F.It follows E, = E F is a that is a randomization of Ej. By assumption from Theorem24.15 randomizationE, and by Example 23.4(3) and Theorem 24.11,f; is a of F. It followsthat F's is a randomization of Es and Theorem randomization f\037
24.12implies, that Es iseven exhaustivefor F's.Hence, there existsa stochastic kernel M:S9x \037(S9)-+ [0,1]suchthat J [JsduF = J M(.,A)PsduE , A E \037(S9)' 9 E
e.
A)
be the probability measure which is definedby Let JlI6V(S9)x !M(S9) p(A t xA 2)= J
;=1,2.
M(x,A2)uE(dx),AiE\037(S8)'
AI)
kernel suchthat Let K:S9x \037(S8)-+ [0,1]be a stochastic p(A t x A 2) = J
K(x,AI) p(dx.S9)
.042)
= J K(x,At)uF(dx), AjE \037(S9)' i =
1,2.
A2)
It
is obviousthat
K satisfies we have) condition (2).Moreover,
JAduF = J M(..A)AduE = AJ \037dp
A
= JJ A(Y) K(.,dy)dut\" A)))
25.Statistical isomorphism
111)
for 8 e e and every A 2 E \037(S9)' Eventually improvingK ona setof\",-mcasure zeroyieldsa kernel satisfying condition (1). in t his reversed order yieldsthe proofof the converse. 0) Repeating proof
of potentialtheory we have obtainedthat F is a Usingthe terminology randomization ofE iff is a dilationof U\"\
(1\302\243
25.5Corollary.Let
e be a finiteset and E.FE 8(e).Then E and Fare
isomorphic iJJ = UF') statistically UE;
Let us apply Theorem isomorphic. ProofAssumethat E and F are statistically For Z E S'rthe function (5.19). (X')'ET 1-+ n x:',(x,),\037TEST, 'f!T)
'1'::
isconcave. Since is a dilationof UF (1\302\243
we have)
H(UE) :\":' J tp:dUf = JJ tp:(y)K(x,dy) uF(dx)
Jtpz{JyK(x, (1p(dx) = J tp:du,.. = H(uF). Sincealso(1F is a dilationof (1E we have \037
=
dy\302\273
H\302\253(1F)
\037
H(UE) which
implies
H\302\253(1E)
H\302\253(1,..).
Then with the notationof the proofof Assume conversely, that uE = (1,... = and F and are Theorem25.4we have Es Sinceboth E and it followsthat E and F are statistically statistically isomorphic isomorphic, f\037.
too.
l:.\037,
f\037
0)
can be expressed in terms of the likelihood Statistical isomorphism processes. Let E=(Ql,.w'l,{p e}).Then dPa s Ee}) and F= (Q2,.w'2,{Qs:8e
(dP.S) dQa and are the likelihood of E and F with base9. processes (dQs)aE9)
a\0379
25.6Corollary.Let e be a finite set and E, FE 8(e).Then E and Fare
statistically isomorphic iff)
dQ !1' Ps - !1' ((dPs)af!9 ) ((dQs) dPa
a
Sf!9)
Qs
)
for every 9 E e.) Thenwe have UE = (IF' ProofAssumethat E and Farestatistically isomorphic. Let .9 E e and Z E S8such that Zs * O. Easy computations yield)))
112
Chapter 4: Sufficiency,E'l:hauSlivity Zl>
d\037
J I}s
( ) d\037
and Randomizations)
dQ dPs= H(uF.)(Z) = H(uF) (Z) = J }Js
Zl>
(dQ:)
dQs.)
of Theorem5.13provesonepart of the assertion. For the other application part we notethat)
An
dPtI dPs
dP(1
d L.
1+ L '4-a
,.s)
\037
and)
dPs d L. fE9)
Ps-a.e.if
(1 9,) \037
d\037
dPs)
-
\037
1 1+ L dPr ,*sdPs)
\037-a.e.)
Sincea similarrelationholdswith P replacedby Q, it followsfrom the that) assumption,
dQtI _ 1'. !I'((d dP Qs) S - !I' tlE9 :L )tlE9 ) d-:L ) (( Q, ) t69) u9) for all 9 e the assertion = for every 9 e. Adding theseequalities tI
Pr
E
follows.
E
(1\302\243
UF
0)
Now, we are in the experiments.)
criterionfor finite positionto provethe randomization
25.7Theorem.Let e be a finite .<jet and E,FE 8(e). Then E and Fare F. statistically isomorphic iff E ..... 2)
is a consequence of Theorem24.16. Assume ProofOnepart of the assertion that) converselythat E 2' F. Let mE 59'Theorem22.5implies a
I
for all
8
I
a,bE lit Let
Po
= 1 I
=
-
L Ptl + b <1e9) L m tl Ptl ae9
e
_181L a
QtI + b L mtlQtI <168 aE8)
Pa. PI = I a\0379
m(1 Pt/' a\0379
Qo=
1
fer
a\0379
Q\",Ql
ofTheorem(22.5) Another application yields(Po,PI),... 2 (Qo,QI)' a69m\" Q\". FromCorollary16.5we obtain) \037
dQl !I'(dPt Po =!I' dPo ) (dQo ) Qo)))
25.Statistical isomorphism which implies
fi'
(
=
-
m..-.!!!:\037
\037
8T9 d L a,,9) \037
\037
I
\037
I
\037
a\0379
113)
p.
)
f1
1 !I'(Se9 L m9 dQs lel Lf;9 Qa . ) 'd!:Qa f1
aE9)
Thisis valid forevery mE Sf) and thereforeit followseasily that
25.5. by Corollary provesthe assertion
u\302\243
=
UF'
This
0)
is a littlebitmoreinvolved.) The caseof arbitrary dominated experiments Then E 25.8Theorem.Supposethat E.FESee)are dominated experiments.
andF are statistically isomorphic iff E '\"F. 2)
is a consequence of Theorem24.t 6. Proof Again) onepart of the assertion '\" Assumingthat E 2 F it followsfrom Theorem22.5that the mappingPs H Qs definesa linearis0metry betweenspan{P8: 9 E e}and span{Qs:9 E e}.It can thereforebeextendedto a linear isometrybetween\037{Ps:9 E e} and \037 {Qs:9 E e}.Let Po E C(&')such that Po\"'\" {P8: 9 E e}and let Qobe the image of Po under the isometrydefinedabove.Another application of Theorem22.5 that) implies
(QI>\037\"I>
We
{P8: 9E ev{Q}}) 2'(02,.912, {Qs:9E ev{Q}}).)
obtainfrom Corollary25.6that)
dQs dPs !I'((dPo !I' P.0 _ )Su ) ((dQo)Sea Q ) on for every (8) and henceeven for = e. Denotethisdistribution 0)
C(E A
(X
In it followsthat Qo\"'\"{Qs: 9 E 8}.Now, we define @ Se9oSi(IR+)by /1. particular
the experiment)
G=
(\037!, @ &fI(R...),{AJl:Be
lh9)
e}).
Let)
TI : WI H
dP8
(d)10(WI)) . dQs. (dQo (llJ2))868),
WI E
QI')
(1)2E
O2
S\0379
T2:llJ2
\037
,)))
114
Chapler4: Sufficiency.Exhauslivity
and Randomizalions)
7i..!:: ..1-'.
= (j = 72 Thenwe have Moreover. by Lemma20.6the mappings Tt = E and and T2 are E- and F-sufficient. Theorem 24.15 E are G Hence,by TI.. = and F and F G are also statistically statistically isomorphic. isomorphic. T2,. Thisprovesthat E and F are statistically isomorphic. n)
25.9Corollary.Two dominated experimentsE= (Ql..!31., {Ps:ge e})and F = (.Q2'.!31 9 E e})arestatistically 2 . {Qs: isomorphic iff)
)...p.) !R(( =
!R((
.
\037\0371
\037\037
Q.),
ge e)
Then Corollary25.6 Proof.Assumethat E and F are statistically isomorphic. that implies
!R((
L.p.)
\037
\037\037
!R((
\037\037:
L.Q.} 9E e,) e.
for every ex e A (8) and hence even for ex = If converselythe condition is satisfiedthen it followsfrom Corollary25.6 A glance at Theorems are statistically that all finite subexperiments isomorphic. 25.8and 22.5showsthat in thiscase\037ven E and F are isomorphic. 0)
to dominatedexperiThe methodof the proofof Theorem25.8isrestricted a mentssinceit restson a particular construction of \"standardexperiment\"on For arbitrary experiments thisconstruction is not possibleand rather to provesimilarresults. differentmethodshave to bedeveloped Thisisdonein Chapters8 and IR\037.
9.)))
Chapter5: ExponentialExperiments)
The objectof the presentchapteris to providethe most importantfacts
the testingof linear functionson Gaussianshift experiments. concerning to give elementary Admittedly. for one-sided testingproblemsit is possible proofsof the basicfacts,usingonly the Neyman-Pearsonlemma. But for two-
sidedtestingproblemsthe mostconvenientway runs over the theory of conditional testsfor exponenlial However,we discuss experiments. exponential for to that our extent asis indispensable purposes. experiments only Section26 containsthe basicfacts on exponential such as experiments with respectto a one-dimensiona reductionby sufficiencyand decomposition into a of of rank 1.In Section 27 mixture experiments parameter exponential is usedto developsometheory of conditional tests.The thisdecomposition resultsobtainedthere are appliedto Gaussianshift experiments in Section 28. The contentof Chapter5 isa purely classical oneand itspresentation goes A much moredetailed discussion of the back to the bookof Lehmann [1959]. is offeredby Wilting [1985].) testsand itsapplications theory of conditional
26.Basicfacts) 26.1Definition.Let (D,d, v) be a (i-finitemeasure spaceand let (-0) R be openand convex.An experimentE = (D,d, {P\037I..w': [) E e}) is an exponential II
\037
experimentif)
dP. =
dt C(9)hexp
where h:Q -+ IR and S:Q
\302\2539,
-
S\302\273,
R\"
[) e e,)
are .\037-measurable functions.)
From Theorem By definition every exponential experimentis homogeneous. 20.9it followsthat the mappingS isE-sufficient. Hence,by Corollary22.4,we
E '\"E. For the analysisof testingproblems is thereforesufficientto S. consider E insteadof have
it
S;
E.)
26.2Remark. Supposethat E = (Q, .9 E (:o)}) is an exponential experiment.Let ,tV 0 s:; bethe (i-fieldwhich is inducedby S.Since..c{ is E-sufficient 0 there existsa Borelfunction hi:R k -+ such .\037,
{P..\037:
.\037
\037
that)))
116
Chaptcr 5:Exponential Experiments) dP91,\037o=
dvldo
C(8)hl oSexp\302\2538,S\302\273, 8E e.)
If we denoteJl :=hi'V 0 S-I then we obtain) d\037(SI P.) 9 (t) = d/l
C(8)exp\302\2538,t\302\273,
te
IR\\
8e
e.
HenceS.E = (IRk, ar, {.P(SI P,9):8 E e})is an exponential experiment.
26.3Definition.An exponentialexperiment E = (IRk,
ducedif)
dP9 dv
(1) =
C(8)exp
\302\2538,
t\302\273,
t
E
IRJt,
\037k,
8E {P,9:
e}) IS re-
8 E e.)
26.4Theorem.A reducedexpo1lemial 9 E e})is comexperimem(IRk, fH\", {P,9: plete,i.e.: lfg: -+ iii isa Borelfunction satisfying Psigi<00and Psg= Ofor = every 8e e.then g 0 v-a.e. IR\"
followsfrom Theorem5.7. ProofThe assertion
0)
26.SCoronary.If E isan arbitrary exponential experimemof the form of 26.1 which on S is uniquely then every non-randomizedcritical function only depends determinedby itspowerfunction.) and thereforeevery critical Proof The reducedexperimentS.E is complete E is its determined uniquely by power function.FromTheorem S. 24.15it followsthat E is a randomi7.ationof S.E. If tp = qJ 0 SE fF (Q,d) is and has the same non-randomized then q> e (IRk.\037)is non-randomized power function for S.E as tp has for E. By Remark 24.17tp is uniquely determined by itspower function. 0) functionfor
\037
of which involve onesinglecoordinate Sincewe consider only testingproblems 9 E e and treat the othercoordinates as nuisanceparametersit will be of e tochangeournotation.In the followingwe write the elements convenient [} of as pairs(8,t') where 8 E IR and t' E \037A-t. Let us callthe first coordinate l' the nuisance (9.t) E e the structural parameter and the secondcoordinate With of parameter. thisnotationthe densities a reducedexponential experiment can be written as)
d::.(x. t
y) =
C(8,t') e9 e
IRA:
-1,(8,t) E
e.
The following is the basic decomposition theoremfor exponential experiments.)))
26.Basicfacts
117)
26.6Theorem.Supposethat E = (IRk, fMI', {P9.r : (8,t) E e}) is a reducedexin the (8,t) E e} can be decomposed ponentialexperiment. Then &' = {p$.r:
followingway:
Thereare Markov kernels &I\037 x \037Ic-I, 8E pr 1 (e),and probability measures
G$,rl\302\243r
-1,(9,t) E e,satisfying the relation P$,r(A)= J f9(A);y) Gs,r(dy),
for every A E \037, (8,r) E a, such that the experiment(!ii,&f, (1)for every YE {&(.,y):9E prl(e)})is a re\037Ie-I
ducedexponential experimentof rank 1,and 8 (2) for every E e the experiment(IR\" I,\037\" I, {GS.r : experiment.) exponential
-
!e E
9
})is a reduced
function Proof There is a probabilitymeasure PI\037k and a \037-measurable
h:R x IRk-I
IR+
Ps.r(A) =
such that)
C(9,1:)J eS.-.:+(r.y) hex,y) P(dx,dy) A)
forA E !!lie, (9,t) E
e.Takefor examplesome(80 to) e and definep:=PSo.ro' ,
E
easure PI\037k can be decomposed The probabilitym by a regular conditional k -1
probabilityQIR x.tJIsuch that
peA) = J Q(y,AY) Po (dy),
A E
8Ik
,)
where Po = Plar-t x IR. It followsthat for every A f9,r(A) =
E
rJr and every (.9,'t) E
e
Sx C(9,'t) Rle-. J J e e(r.) hex,y) Q(y,dx)Po(dy) . y
A\"
Let us define k
(y, 8):=J eS}Ihex,y) Q(y,dx),(y, .9)E IRk -I X pr.(8). R)
It isobviousthat for every 9 E prde) we have Po{ye (Rk -I:k (y, 9) = oo}= O. Thisimplies that A{.ge prl (8):key,9) = oo}= 0 for Po-almost every ye R\"-I. is an However, prl(e)is an open interval and {.gepr.(e):k(y,9)<00} -} interval for every y E HI.-I.This implies that for Po-almost every y E [Rk the set{.9E pr}(8):key,9) = oo}= 0. We may thereforechangethe kernel Q in such a way that k (y, .9)< 00 for every Y E -I and every .9 e pr.(e). Let yE \037k-l be fixedand assume that key,91) = 0 for some91 E pr1(e).It follows that h(.,y)= 0 Q(y,.)-a.e.and thereforek(y,9) = 0 for every 9E pr.ce).Hencewe obtainthat for every YE IRk-I the function 9.-k(y,9) eitheris positiveor vanishesidentically. Let M= {yelRk-l:k(y,.)=O}. It is clearthat Mefif\"-I.We define for BEprl (8)and Be IR\"
iN)))
118
Chapter 5:Exponential Experiments)
Q(y,dx), y;M, k(;,8)!eSXh(x,y)
\037(B,y):=
and F;I!\302\245
&(B,y) = &(B,Yo) for some fixed Yo;M if ye M. Then x IRk-I,8 E prl (8), satisfies (1).For (8,t) E e and Be \037-Iwe define G9,,(B)= C(8,t) f e:be
Obviously, (2). Gs.,I\302\243lt-l, (8,t) E e, satisfies followsfrom) The remainingassertion P9.r (A) = C(9,t) f eb e
=
C(9,t) R\"-I,t>' J J e9xe
=
C(9,t) M')J F;(A',y)k(y,9)e
=
C(9,t) f J<$(AY, y) key,9)e
=
C(9,t) f f;(A',y) f eS h(x,y) Q(y,dx)e
R)
=
C(9,t) J \037(A', y) eSx e
=f
f;(A'.y) G9,r(dy), A E !JIk,(9,t) E
e.
0)
It is importantto note that the measures Gs.r , (9,t) E e, are necessarily pairwiseequivalent. of the kind provedin Theorem26.6it is immediate Given a decomposition = X that Gs,rl\037-l P9,rl R \037k-l and that y 1--+&(A',y), ye IRk-I,isa versionof that this conthe conditional probabilityof Ps,r given Pi -I. It is essential of the nuisance ditionalprobabilityis independent parameter t.)
26.7Remark. Supposethat E = (D,.9I, (.9,t) E en is an exponential {Ps.r: a If we denoteS..E Then E reduced experiment. experiment. S. is exponential = (HI:, for every A E 1r 1r,{Q9.,: (8,t) E e})then Theorem26.6implies Qs.r(A)= f f;(A',y) Gs,r(dy),(9,t) E e, which means in terms of E that) A} = \037,r{SE
It followsthat
)
I Fj(A',y)2'(SIPs (dx,dy).
Ale
,...,
...,
s\\ (82 , Ps.r(S A 182 Sk}= Ps(ASz..... which doesnotdependon the nuisance parameter \342\202\254
t.)))
S.\302\273
27.Conditional tcsts
119)
26.8Lemma. Supposethat E is an exponentialexperiment and let cP E:F(a,d). Then E is continuousand for every t E prz(e) the function 9 PS.rCPisdifferentiableon 9'with derivative t---+
i)
=
:19PS.rCP9=90)
U
p9o.r (S.cp)
- PSo.r(S\\)p . (cp). 9o r
followsfrom Lemmas 5.6and 11.5. ProofThe assertion
0)
26.9Corollary.Supposethat E is a reducedexponentialexperiment and on e80 then letip E F (R\".aI\.")If t !',go. r cP is constant t---+
=
!o
U.:1Ps\"cP9=90
J:
9 fS(cp,y) 9=90)Gso,,(dy),
Proof Let \037o.rCP = ex, t E elJo. Then it followsthat F9o(cply) = ex, G9o.r-a.e., (90 , t) E We obtainthat
e.
a J U 9-.&(cp,y) \037
Gso.,(dy)
-
S=lJo
= J F9o(cp id,y) G9o.r(dy) ex J FlJo(id,y) G9o,,(dy) = P9o,,(cpid) exP 9o.,(id).
-
Thisprovesthe assertion.
0)
27.Conditionaltests) Supposethat E = (A\", experi{PS.r:(9,T) E e})is a reducedexponential ment.Let 90 E prt (9).We will consider two kindsof testingproblems: Onesidt:dtestingproblems, which are of the fonn) H = {(9,T) E 9 90, T arbitrary), K = {(9,t) E 8:8> 80't arbitrary},) \037\",
e:
\037
and two-sided testingproblemsof the fonn H = {(9,t) E 9 = 90't arbitrary},
e: K = {(8,T) e e:8
=+=
27.1Definition.A testcp E 1 if
80't arbitrary}.) is a conditional (H\",\037) upper testjf
\037
x,Y ) = 0 I.f x>t(y), x
120
Chapter 5:Exponential Experiments)
As we shallsee,conditional for optimal uppertestsare propercandidates testsof one-sided testingproblems(H,K).)
27.2Lemma. Supposethat (H,K) isa onesidedtestingproblem.A conditional K) respectivelyin K) iff it satisfies upper testlp E !F(R , &r) is in = a Gso.r-a.e., rE eSo .F;o(lp(.,y),y) Ie
:F\302\253(H.
!F\302\253$(H.
.)
:F:(H,10.Then P80.rlp= a,'t' E eso. Then onepart of foUowsfrom Theorems the assertion 26.6and 26.4.Assumingconverselythat w e the condition issatisfied choose1p(x,y) = a,(x,y) E R i. ThenCorollary9.8 Proof.Assumethat
E
lp
that implies .&(lp(.,y),y)
if 9 80 , a .e.,t E eso,) .F;(lp(.,y),y)a if 9> 90 ,) GSo,r\037
IX
\037
\037
which provesthe assertion.
0)
that (H,K) isa one-sidR.dtestin\037 problem. 27.3Lemma. Suppose Thenfor every E [0,1] there is a conditional E lest upper lp Yo.(1/,10.) IX
ofTheorem26.6.Let (t\\"E") Proof.Recallthe decomposition
R
\037
R
bedenseand
define)
00),y) a, n EN},Y ERie 1. It is obviousthat I: R\" -1-+ A is a Borelfunction.Moreover,it is easily seen t(y):=inf{t\":
.F;o\302\253(t\",
that
&o\302\253t(y),oo),y)
\037
showsthat the test)
IX
\037
and F90(Lt(y),\037),y) \037
IX.
Now standardreasoning
I if x> t(y), (x,y)eR, {0 if x
<'P\302\253(x,y)=
27.4Theorem.Supposethat (H,K) is a one-sidedtestingproblem.If cp.E fFa.(H,K) is a conditional upper test then lp* is uniformly optimalin K),) :Fo.$(ll, Proof.Let lp E fF:(H,10be arbitrary. Then P90,rlp = a for every (90'r) E Theorems26.4and 26.6we obtain Combining
e
e.
t E 90. J lp(x,y) F80(dx l y) = ex Gso,r-a.c., The same assertion holdsfor lp*.Hencethere is M R\"-1such that G90.r (M) = 1,'t' E So, and)))
e
\037
27.Conditional tests
121)
= J q>(X,Y)I:\037(dx,y)= ct on M. J ip*(X'Y)\037o(dx,y) Sinceq>*(.,y)is an upperid-testfor every fixed YE IRJ.-lit followsfrom Corollary9.8and Remark 9.9that J q>* (x,y) l\037(dx, y)
J q>(x,y) Fs(dx,y)
\037
Thisimplies PS,fcp.
\037
PS,fq>
if
9\037
90 , (9,t)E
9 90,
if
\037
e.
y EM.
0)
27.5Corollary,Supposethat (ll,K) is a one-sidedtestingproblem.If E :Fg.$(H, in :Fa,(H, K) isoptimal K)forat leastonepoilll(B1, t l ) E e,B1 * 90, then q> is uniformly optimalin !F:(H,K).) q>
Theorems K) be a conditional Proof Let ip* E .'Fa,(H, uppertest.Combining 26.4,26.6and 27.4we obtain = J q>*(x,y)Fso(dx,y) = ct J q>(x,y)Fso(dx,y) the optimality for every (8,T) E and employing of q> at (BI , T I ) we Gs.r-a.e.
e,
obtain)
J (Jq> (x,y) &,(dx,y)
-J
q>.(x, y) Fs,(dx,
y\302\273
G91, tl (dy)
\037
O.
Let M be the setof y where the first equations hold.For y E M consider the , K), where experiments Ey:= {&(.,y): 9 e pr.(e)}and the testingproblem(11 R = {9e prl (8):B Bo}and K:=prl (e)\\R.By the proofof Theorem(27.4), if 91 < 90 in :Fa$(n, K) for E\" y E M. Therefore, q>* (., y) is unifonnly optimal \037
we have)
J cP(x,y) 1-9 (dx,y) \\
\037
J cp* (x,y) &\\ (dx,y), y EM,
and if BI > 90 J q> (x,y) &.(dx,y)
\037
J q>* (x,y) Fs\\ (dx,y),
In any case,we obtain) J cp(x,y)\037. (dx,y) = J q>* (x,y) 1-81(dx,y)) and hence,GS.f-a.e. for every (9,t) E GS\\'fl-a.e.
Y
EM.
e.Now, restrictM to those
pointsy where the last equationholds.Sincethen for y E M the power functionsof (.,y) and q>* (.,y) coincide at Bo and 91, Corollary11.4shows that cp(., i n is K). It followsthat y) unifonnly optimal F;(il, = J q>*(x,Y)Fs(dx,y), Beprl(e), Jq>(x,y).F;(dx,y) and hencethe assertion. 0) q>
As
anotherconsequence we observethat the test q>* of Theorem27.4
is)))
122
Chapter 5:Exponential Expcrimcnts)
admissible. In fact, any bettertestis in \037a.(H, K) and thereforeshouldbe i n with lp+.) optima] !Jia.(H, K).Hence,it coincides 27.6Definition.A testcp E g (HA:, tr) is a conditionallwo-sided leslif 1 if X
(x
( 11;11 ).
- 11;11
-
seethat the preceding isnotdiflicultto assertion isalsovalid if W = 1 1ro.c), l'> O. The charm of the argument liesin the fact that it restsonly on the It
Neyman-Pearson lemma in the version of Lemma 28.1.The proof of the in its fulJ generalityrequires assertion someelementaryBayesiantechniques.
Thereforewe postpone it toSection 40.
resultsto a slightly moregeneral situation. Finally we extend the preceding = For tEL and <1 > 0 let Pr.u !f(<1id+ tl Nil)' In the followingwe consider the = teL.<1> O}). experimentF (H,fR(H). {.P,.u: For every <1 > 0 the experiment (H,fII(H),{P,.C1: teL})is of the same type as the experiment E if the vector spaceH is endowedwith the inner 1 1= <., It is natural to calJ an estimate() e .1t(F, product<.,.)\" \037
f\037
.). median-unbiased for F ismedian-unbiased for every R(e,t, (1)= II W(lx -f(t) I) e(',dx)dP'.C1'teL, i
\037)
(]
iff it
f\037,
<1
(1
>
> O. We denote) O.)))
166
Chapter 7: Theory of
Estimation)
34.6'Ibeorem.Every medianunbiased estimate (!off satisfie.'i for every tEL, (J > 0,)
-
[/(1) <,f(I)+ d]) P'..\037(., and)
R(q,t,
0\")
\037
:;; (,,:,, ) <1>
<1>
Clf\037
)'
< > 0,d> 0,
v O,1S1 (ds).) J W(lIfll'ls!)
Proof.Apply Lemma 34,2and Theorem34.3to F;,(J > O.
0)
34.7CoroUary.The estimate K. = foPL isa medianunbiased estimate off and estimates optimalamongallmedianunbiased off)
35.Mean unbiasedestimation) that e 0 is an arbitrary setand E = (fl,.flI,{Ps:.9 E e})an experiSuppose ment in S(8).Denote = {Ps: 8 E 8). For the purposeof thissectionthe of n !R2 (Q,d,Ps) are calledestimates. elements =*=
(fjJ
91'9)
35.1Definition.A
e
admitsan unbiasedestimateK if K = 9 E e. The set of all (mean)-unbiased estimates off is satisfies f(9) Ps(K), denotedby H(f).) function f
\037!R
An esti35.2Definition.Supposethat f e -+ R admitsunbiasedestimates. mate K* E H(f) iscalledan unbiased estimate ofWliformly minimalvariance
-
f(9\302\2732)
PS\302\253K.
= inf \"E
Ps\302\253K
if)
-
f(9\302\2732),
ll(f))
From Steiner'sformula it followsthat for K2 E HCn,and every 8 E
e
-
f(8\302\2732)
\037\302\253Kl
\037
PS\302\253K2
35.3Lemma. If K and K2 K = K2 Ps-a.e., 8E e.) 1
in
-
f(8\302\2732)
iff
9E
e.
estimatesK 1 E H(!),
any two
Ps(KD
\037
Ps(K\037).)
H(f) are of uniformly minimalvariance then
1
ProofSince (K1 + K2) E H(f) and) \037
J
K1
(
1 2 1 2 + K2 2 K dP 9E e,))) dPs KldPS+ J 2 J 2 s, 2 ) 2 \037
35.Mean unbiasedcslimation it
167)
followsthat this inequalityis even an equality and that
-
j(KI K2)2dPs= 0, ge
e.
0)
35.4l.emma.Supposethat .9I .91is E-sufficient.Forany K e H(f) let ge E(Kldl ) = Ps(Kldl)Ps.-a.e., Then E(KldI) e J/(j) and t
\302\243;
e.
2 ge e.) t )2) $ Ps(K ), Pa(E(KI.9I
of Jensen's inequality. ProofThis is an immediateconsequence
0)
if for every estimate K An experimentE iscalled 2-complete
e.
9E 0, ,9 E e impliesK = 0, Ps-a.e., Thisisa slightly weakercondition than completeness, definedin Theorem16.4.) Ps K =
and Lehmannand Scheffe[1950]). 35.5Theorem(Blackwell[1947]. Suppose
d)
is2-complete. that .r.() .r.(isE-sufficientand that EI If H(f) i.'t an unbia.'ted estimate 0/ uniformly minimalvarianceK*, and K* \037
K*
=
* 0 then there satisfies
Ps(Kldl ) Ps-a.e.,ge e,KE H(f).)
of H(f).Since Proof Let KI and K 2 be elements if gee, f(E(Klldl)-E(K2Id))dPs=0 of d l implies that E(Ktldl ) = E(K2 Idl ) Ps-a.e., ge completeness there is a function K* E H(f) satisfying
e.Hence
= E(KI..r.f 9E e, whenever KE H(f).) t) Ps-a.e., to the preceding lemma,K* is of uniformlyminimalvariance, According K*
0)
estimateK is an unbiasedestimateof In otherwords,every .9I1-measurable unifonnly minimalvarianceof itsmeanf: 91-+PS(K),9 E e.) Then 35.6Corollary.Supposethat the conditions are .';atisfied. of Theorem35.5 the unbiased estimateof minimalvarianceK* e H (f) satisfies)
s J W(K)dPs, 8 E J W(K*)dP \037
for every convexfunctionW: R
-
IR
H(f),) which is bounded frombelow.)
Proof The assertionfollows from inequality. 0)))
e,
K*
K
=
E
Ps(Kldt ) Ps-a.e.,and Jensen's
168
Chapler7: Theory of Eslimalion)
35.7Example.Let (H,( ., . bea Euclideanspace,L H a linearsubspace. \037
\302\273
sufficient mapping for the experiment E PL is a = (H,(JI(H),{NH*tQ:aEL}). Let us show that 9= {NIl*cQ:aEL} is 1 0 = = for the a-field.r41 pi Porthis,let g h PL besuch that 2-complete = for all L. This E that) 0 a \037g implies
Then
(\037(H\302\273.
o=
\037g
II h(x)e -re .-yIIxII 2
11/2
=
(21t)
=
C(a)f. hex)e<x,Q)e
dim
IIYII
2
e.
IIQIF
e<:r:.a)
2
dxdy
-\037 2 dx)
for every a E L. FromTheorem5.7we obtainthat) h+ (x)
e-u.;u:= h - (x) e-
l!!1I: 2
AI.-a.e.)
a E L. Henceevery function/0PL is an unbiased provesg = 0 \037.a.e., estimateof unifonnly minimal variance of its mean.If is linear then fo PL H(f).) which
/
\342\202\254
35.8Example.Keepingthe notationof the precedingexample,considerthe = id+ alNil)'It experimentF = (H,P.4(H),{\037.cJ:a E L, (1 > O})where x EN, is sufficientfor is again easy to show that S:x.-(pdx),!lxPI.(x)II), = a E (J > for and that is S-1 Therefore, F, L, O} 2.completc {\037.a:
-
\037
\037.cJ
.9\"\"'\302\253(1
(\302\243B(H\302\273.
estimates of unifonnly minimalvarianceare exactlythe estimates the unbiased K which dependon x E II through S. For short,Jet us callany unbiasedestimate of uniformlyminimalvariance Theorem35.5statesthat under the conditions imposedthere simply optimal. the setofall optimalestimates isa linearspace,which isgeneratedby a q-field. of Theorem35.5are necessary. It is our aim to show that the assumptions we information on the structure of the setof want to obtain some Moreover, in such caseswhen the assumptions of Theorem35.5are not optimalestimates satisfied.)
K* 35.9I.emma(Rao[1952]).An estimate optimaliff Ps(K*v) = 0for every i.\"
9 e e and every ve H(0).)
Proof An estimateK* is optimalifT for every 8 E e and every v E H(O) the function A.'+ ).V)2), e R, attainsits minimumat = O. This is the A.
\037\302\253K*
caseiff Ps(K*v) = O.
A.
0)
K such that there exists an optimalunbiased Let D be the setof all estimates H be a If K Diet 1t(K) version of the optimal estimateof 9 Ps(K), 9 \342\202\254
e.
estimate of 91--+Ps(K), .9 E e.)))
\342\202\254
35.Mean unbiased estimation
169)
35.10Lemma. ThesetD isa /inearspaceandn isa linearidempotentmapping. is a /inear space.too.) The.'letxeD)of alloptimalestimates in D and let i.., Ji E 'R. It followsfrom Lemma Proof Let \"1'\"2 be estimates that is an + optimal estimate.This proves the (35.9) ;,X(\"I) JlX(\"2)
assertion.
0) .rtf, n !{'2(!1,
In the followingwe topologize the spaceof estimates family
of semi-nonns K t-+ Ps(K2)1/2, 9E
e.
I\037)
by the
\037t:8)
35.11Lemma. The /inear spacex(D)isclosedand x: D -+ x(D);s uniformly
continuous.)
Proof FromLemma 35.9it isclearthat n(D)isclosed.Thus we need only Let \"I'K2 be arbitrary clements of D. provethat x is unifonnly continuous.
Then)
\037\302\253n(KI)
- n(K
2
\302\2732)
=
P9\302\25371(K
\037
P9\302\253K
which provesthe assertion.
1
1
-
K2\302\2732)
K2)2),
9 E e,)
0)
the class..w Now we introduce 0:={A E.9I:l A E n(D)}.It followsfrom Lemma
35.9that
A
E
.vI0 iff J vdPs =
0 whenever
vE
H(O), 9 E
e.
A)
35.12Lemma. (1).910 isa (I-field. (2) (3) (4)
e.\037timate is in 7l(D). Every.91 o-mea.\037urahle EI.9I0 2-complete. If .911 is E-sufficientthen .910 .91 for er;ery9 E e.) i.\037
\037
J
(P\037)
if A E .vI0 Proof (1) It is easy to seethat sf0 is a Dynkin system. Moreover, and a (I-field. .s.I0 ist1-stable then 1A V E H(0) if v E H(0).Therefore, usingLemma 35.9. (2) This is provedby induction, If P9 (K) = 0 for every 9 E e then estimate. (3) Let K be an .91 o-measurable K
E
H(O) and hence
J KdP9 = 0 whenever
A
E
.910 , 9E
e.
A)
e.
It followsthat\"= 0 P9-a.e.,9 E This meansthat EIsf0 is 2-complete. If A E.9I0 then 104 E n(D)and (4) Assumethat .911 isan E-sufficient(I-field. from Lemma 35.4we obtainthat alsoE(1A1.91 1) E xeD).lieneewe have I...= E(1AI.s.I 1)PS-a.e.,9E e.)))
170
Chapter 7: Theory of
Estimation)
Ifwe denoteB= {E(IA I.Iit.)= 1},then BE .Iit.and A = B
(J\037\037),
e.
.9E
0)
shows that the mapping1t is almosta conditional The followingassertion
expectation.)
35.13Theorem(Bahadur[1957]).Themappingn:D -+ n(D)has thefal/owing
properties: then Kh E D and 1t(Kh) (1) If KEDand if h is.91o-measurableand bounded. = 1t(K)h. (2)
If
KE D and
8E e.)
then n(K) = if n(K) is do-measurable,
P\037(KI.9Io)P\037-a.e.,
and bounded. Firstwe notethat) Proof(1) Let\"ED and h bedo-measurable s = 0, 8 E e.) J (Kh 1t(K)h)dPs= J (K
-
-
sinceK
- (,,) H(0).It follows 1t
E
g:9...... Ps(Kh). If v E
1t(K\302\273hdP
that n (Ie). h H(O) then hv E R(O) and
is a mean-unbiased estimate of therefore
0, 8 E e.) Thisimplies that n (K). h is optimal. s= J n(K)hvdP
(2) The propertyjustprovedimpliesthat for KEDand
A E
s = J n(K) l...dPs = J n(\" l...)dP s ;;;;; J \"dPs, 9 E J 1t(K)dP
A
.910 we have
e.
A)
it is a version of Hence,if n (K) is o-measurable, .QSf
\037
(K I.QSf 0)'
0)
The second assertion of Theorem35.13 leadsto the question, whethera given
is .91o-measurable.) optimalestimate
then it is 35.14Theorem(Bahadur[1957]).If an optimalestimateisbounded,
.r;Io-measurable.)
ThenKV E H(O)for every v E H(O). ProofLet K bea boundedoptimalestimate. we obtainthat K isoptimalfor all n E N. Ifp isa polynomial on By induction R then po\"is optimal. SinceK(D) iscompactthe Stone-Weierstrass theorem If U iii isopen impliesthat cp 0 \" isoptimalwhenevercp:!R -+ IR is continuous. 0 then it followsthat lu K is optimaland therefore{K E U}E .Iito. HenceK is .91o-measurable. 0) ft
\037
converseof Theorem35.5.) Now we arrive at the desired
35.15Theorem (Bahadur[1957]).Assume that E is dominated.If for every)))
35.Mean unbiasedcstimalion function which
admit.v
171)
a bountkdunbiasedestimatethere (\037xivt.v an optimal
estimate then ..cI 0 is E-sufficient.)
Proof We prove that .910 is pairwise sufficientfor E. The rest followsfrom Theorem (20.4).Let PI and Pz be arbitrary elements of r!J= {Ps:9 e 8}. such that Q:= L 2-1 Completethe pair(PI'P2) to a sequence (P,J\"\"N d is equivalentwith Then the densitiesfi , i = 1,2,can be assumedto be dQ bounded. \037
\037
\037
l\342\202\254N
\037.
We have
i=
Let h,= n , j = I, 2, are d.-measurable.
to show that \037ci
1,2.Then Ps(hiv) = 0
if
8 E e,
ve H(O).On the otherhand we have v E H(0).This implies \037
Q((h.-
\037ci
But hi
0,
VE
\037ci
).
\037(v)
= 0 and hence Q
v
dR
( dQ)
= 0,
H(O).
- :ciisitself
an elementof H(0) which implies)
Q((h,-
= O.
\037ci
''IT vye
)V)
(
ve H(O),and thereforeQ(hjv) = 0,
. 0btam
t h at
),)
. dfi.IS optima and I
measurable. 0) dQ
.
m view 0f ItS b ound ed ness sI0-
35.16Corollary.Assume that E is dominated. Iffor allA E sI the functions
9.-.P,,(A) admit optimalestimatesthen do isE-suffcient.)
to 35.15 i s estimate off Since\" bounded - -0 ProofLetf e By
R
Theorem
such that /lK.. co
K/lu
-
L /I\"..
,,=
and such that there is a boundedunbiased K off estimate we have show that there existsan optimalunbiased we may find a sequence of stepfunctions(K,J and (eventually by choosing a subsequence))
\"\"+l/1u
<00.
1)
By
for every Kn there is an optimalestimateK: of [) assumption,
9 E 8,n E N. It followsthat for every 9 E e /I
-
-
Ie: \":+1112,9 /I \".. \037
\037
and thus) t;I)
L 11=1)))
-\"..
\"\"+1/12,.9II \"..
.9Ee. /lK:-\":.llb,9<00,
+
ll1u,
n
EN,
\037
Ps (\"..),
172
Chapter 7: Theory of
Let Po E
Estimation)
C(9)such that 96 --Po.Then,obviously,) 00
L IIK\037-K\037+llb.Po
and it followsthat (,,\037) has a Po-a.e. limit \"*.Now, it
,,*is an optimalunbiasedestimate off
isnotdifficult toseethat
0)
36.Estimationby desintegration)
that (1',d) is a metric spacewhich is locallycompact and separable, Suppose the we a Let &f(T) be its BorelO'-field. For following, choose Borelmeasure < if K Tiscompact, and keepit fixed.Let us t hat means 00 p(K) plfj(T), = an experimentE (a,d, {p,: t E T})in 4 (1'). consider TheexperimentE iscalledf1I(T)-mea.\037urab/eif \037(A) is.1J(T)-measurab for every A E d. If E is .\037(T)-measurable then we may define the measure pep):A...... J P, (A) p(dt),A E ,rd.TheexperimentEiscalledweakly continuousif I H P'(A) iscontinuous for every A E d.Sincein this caseE is separable, the v with set9 = {p,:1E T} isdominatedand for every O'-finitemeasurev 1.91 the function (w, t) H (w), (w, I) E a x 1', can be chosens1x \037(T)\037
t.-
\037
\037
\037;
measurable (Lemma4.6).) Then E is p-inlegrable if for 36.1Definition.Supposethat E is measurable. function qJ E (Q,.91)such that every SETand every f; > 0 there is a critical PsCP< f,
-
and J P,(1 T)
!/
cp) p(tlt)<00.
if every testingproblem({s}, In otherwords,E is p-integrable sET, 1'\\{s}), admitsa testof arbitrary smalllevelwhosepower functiondiffersfrom oneby for every hounded a p-integrable function.It is clearthat E is p-integrahle
measureplat(T).)
definition 36.2Remark. ThecritiC<11functionof the preceding may bereplaced
criticalregion.Indeed.choosea criticalfunction cp such that < f.2 and P'(1 p(ti/)<00. ThenA = {q> > f,} satisfies and J P,(A')p(dt)<00. \037(A) < f, T by a
\037q>
-q\302\273
\037
to checkp-integrability criterion Thefollowingassertion isthe mostconvenient of
E.)))
36.ESlimation
36.3Theorem.A isa-jinile.)
by
desintc:gration
weakly continuousexperimentE isJ.l-integrable iff
t 73)
P(J.l)I,r# is
and let seT. From 36.2it followsthat Proof Assumethat E is J.l-integrable .s.Isuch that \037(AII) 11 and P(J.l)(An) < ex; for there is a sequence (AII)lIc N ,....,1,such that P(J.l) sf\" n is n N. Hence there a set isCTDJ E .YI, every E finite.Let S T be a countable and densesubset. Then D = U Ds issuch that seS SinceE is weakly continuous it \037(D) = t jf S E SandP(Jl)I.r#n D is a-finite. followsthat P'(D)= 1 for every t E T. HenceP(J.l)(D')= 0 which provesthe \302\243
I
\037(\037)
\037
\302\243
assertion.
Assume converselythat PC/L)I.ctis u-finite.Then there existsa sequence such that A\" T Q and P{Jl)(All) < 00, n EN. This provesthe (AII)\"\037N \302\243;.91
assertion.
0)
36.4Corollary.A
weakly continuousexperimentE isJ.l-integrable iff any
isfulfilled: followingconditions dp' (1)J dP. J.l(dt)<00\037-a.e.for every SE T. J
of the
dP.
f (2) Thereis a dominatingmeasurevl.ttsuch that J dv d\037
(3) J
tl(dt)< 00 v-a.e.
. .
J.l(dt)< 00v-a.e. .forevery dommatmKmeasurevl.r#.)
d:
Then assertions (3) and (2) follow Proof Assume that E is J.l-intcgrablc. immediatelyfrom) J
dp'
d; (d/) = J.l
-
dP(tl) dv
v-a.e.)
sinceP(J.l)isCT-finite.Similarlyit isshown that (1)isvalid.Nowassumethat (2) d issatisfied. To provethat E is J.l-integrablelet C,,:={J P'.J.l(dt) n},n E N, dv n v < n N. Then A :=CIIn DII' and let D,.i fl, DII E .ct, E N, such that (Dn) 00, E < 00,n E N, and v(Q\\ U A,,) = O. HenceP(J.l)is CTn E N, satisfy P(J.l)(A,,) \037
II
liE '1/
finite. be a probabilitymeasure which is that (1)holds.Let \\' 1.91 Finally, assume -k t E T}and isof the form v = L 2 Ps\" where(SjJkfN isdense equivalentto {p,: te in T.Then we have
-
N)
-
dp, k dp' J dv J.l(dJ) 2 J - J.l(dt)< 00 \037
dPs\
kEN.))) \037Ic-a.e.,
174
Chapter7: Theory of Estimation)
It followsthat N = {J p(dt)= oo}satisfies PSIc(N)= 0 for every kEN and \037; hencev(N) = O. 0) in subsequent sections let Although much moregeneralcaseswill bediscussed us illustrate theseconcepts some by typicalsimple examples.)
.
36.5Examples.Suppose that (H.< . . is a Euclidean space,L H a linear \302\243;
\302\273
and n = dim H.k = dim L.The Lebesgue measureofH isdenoted subspace by A.
H
.
.
(1) Let P )'H be a probabilitymeasureand denoteh = d; . Consider dAB the shift E = (II,f!J(H),{\037: a E L}),where = P.Choosep = L. Then) \037
\037
\302\243\"
-
dP(A.J(x) = dA. 1I I hex a)A.dda) = hex pdx)+ a\302\273).dda), I
-
A.
XE H.)
Since)
J f hey LJ.i) it
+ a) A.dda) L1(dy) = P(R)= A.
1
followsthat dP()'L)/dA. H < 00 )'H-a.e. This provesthat E is L-integrable. In the specialcasewhere L = H we A.
obtaineven P().II)= ).11' For laterpurposeswe notethat a t-+\037. a E L. is continuous for the variational distance in any case.This is a consequence of Remark 3.1.
Now.assumethat dimL < dim H and considerthe shift with scale a E L.a > O}). where P\".rJ = !R(aid+ alP). parameter F = (H.PA(H).{Pa.o: 1 a E L. a> O. Choose @ da.Then p(da,da) = ).,.(da) (J) (2)
-
\037\037(Jl)
(x) =
dAH
f
R+
JL -i-rr a
h
X
(
= J f }+1h x R+La
(
-a a
-
) Adda)da
PL(X) + a a)
dP(p) = 1__ (x) IIx - pdx)II\"-k
Now, we put y =
\037,
ex
=
\037
a)
(J
IIx PL(X)II.which leadsto
dA.H
k
-
f f '!II\"'cx\"
1
X
h(cx
) )'L(da)du, x Ell.
- pdx) +Y)AL(dy)dcx,xeH.)))
llx_pdx)1I
36.ESlimation
by
175)
dcsintegration
on S. We want Let S = {zE H:IIzll= I} and let t be the uniform distribution we that < for this show and to show that dP(J.l)/dA.H 00 lH-a.e.
l j.i
ex
n
- -1h(exz + y) k
A.
L (dy) dexr:(dz)
= 1.)
is almostimmediatesince(ex, z, y) 1--+ n k h(exz+ y) is a version of It followsfrom Remark (3.1) @ t @ AL)' HenceF is J.l-integrable. dP/d(J.'R' that for shifts with scaleparameter the mapping (a,0-)1--+\037.(1' (a,0-)E L x +, is for the variationaldistance.) continuous But this
I
ex
\037
36.6Definition.A measurablefunction f T --. is (E,J.l)-integrab/eif E is (If J.l)-integrable. IR
I
I
then every boundedfunction is (E,J.l)-integrable.) If E is J.l-integrable.
36.7Remark. Let f T IR be a measurablefunction.Theorem 36.3and are equivalent: Corollary36.4imply that the followingassertions functionf T --.IR is (E,jl)-integrablc. (1)The measurable \037
isa-finite. A E.>1, A 1--+J P'(A)I/(I)IJ.l(dt), (2) The measure P(I/IJ.l): (3) S I/(t)I
dp' J.l(dt)< 00\037-a.e.for every sET. dP. s dp'
measurevld suchthat J 11(t) J.l(dt)< 00 v-a.c. (4) Thereisa dominating dv 1
(5) J 1/(t)I \037; J.l(dt)<:x>v-a.e.for every
and f T If E isJ.l-integrable
.- dP(J.l)) dP(J.l))
then the RN-derivative) is (E,p)-integrable
----
\037
dP(IJ.l) dP(f+J.l)
measure vi d.) dominating
\037
dPCIJ.l) dP(J.l))
ofIcouldbe iswell defined.Theideaof using this RN-derivativeasan estimate to the measure with respect calledestimation by desinteRration J.l1\037.)
36.8Examples.(1) Considerthe shift of Example 36.5(1).Let p> 0 and that P(II.I/')< co.Then) suppose
-
-
AL) dP(II.I/' (x) =
+ Ilia pdx)I/'h(x p,,(x) dAH 2'lUx pdx)+ all'hex PL(X)+ a) A.dda) + 2'I/xll'J hex pdx) + a)A.dda),x E
\037
-
-
a\302\273).,,(da)
-
H.)))
Chapter 7:Theory or Estimalion)
176
Thesecond term isfinite AII-a.e. (1).Thefirst term isalsofinite by Example36.5 smce)
J L\037
f
II
< 00, + all\"h(y + a) Adda)).v. (dy) = P(II.II\")
y
I.)
Hence11.11\" is (E.AL)-integrablc.
(2) Assumethat dimL < dim 1/.For the shift
is valid.Let p > 0 and J IIx
\037\037r;:.\037nasscrtion
\037P
F with scaleparameterthe IIxll' P(dx)< 00, Then we _ PL(X)II\"
Q!.!I'Jl) (x) dlH X 1 pdx) + a . S J n+l-lix-pdx)+all\"h IR-L(J ( (1 ) A.L(da)d(J
-
\0372\"
x t 2\" R.Lu\" J J \037 IIxll'h (
_
PL(X) + a (T
) lL(da)d(J
- +yl/\" J San-k-1-\"lItXPL(X)I! IIx - pdx)lIn-k 2'
x .PL(x)
A+
-
\\Ix
-- -
lx pdx)
II
2'I\\xl\\'
+. 11
XE
L
x PL(X) . + Y)A.ddy)d(X _
h\302\253(Xi
We
-
/i\"=k
PL (X ) 11
X
J R\"
I
(X
n-k-11 I(
X
(X
II
X
- pdx) + ) . (d ) d - ) PL ( X
y
A.
L
Y
(X,
II
II.)
know from Example36.5(2) that the secondterm is finite i.H-a,e. For the
first term thisisalsovalid since) S f J (Xn SR+ L)
=
It-l--,
\\ltXZ
+ yll'It(tXZ + y)lddy)d(X!(dz)
xII' P(dx)<00. J--x--------pdx) II
1\\')
\\I
Hence11.11' is (F,Jl)-integrable. (If we only assumethat POI.II')< 00 then followsthat (a.a)
..... aII)'.(a.a) II
E
(3)
If we specialize P=
We have))) (E..A.,,)-integrable.
N H in
is (F.I')-integrable.) every linearfunctionf L -t iii is
L x R+
(\037
(1) then
it
36.Estimation \037P(fAL)
dP(A,J
dP(fl.J
=
-
dA H
-
--
desintcgralion
117)
dP{A.J
/
dAll
- llix-a1l )
ff(a) ,. exp(
by
1
2
--.
AL(da)
J
J,.)exp( 2 11x aW) l,.(da)
-
and from IIx all2 = IIx
- p,Jx)
11
2
-
+ IIpdx) all2 it followsthat
-
dP(fA.,) f o PL') dP(A,J If we specialize P = Nn in (2) then it is not difficult to seethat the conditionis satisficdiff n k > p. If k > 1 then every linearfunction and we obtainin a similar f L IR if (F,p)-integrable way as for (3)
-
(4)
-.
,,-
dP(fp)
dP(P)=foPL')
The basicideaisas follows. The distributions. Now, let us introduce posterior
!
mixture)
A
x B\037 P'(A) p(dt),
A
E.9I,Be &I(T),
x isdecomposed along(0,sI) by a kernel F:!1
\302\243i(T)
JB P'(A) }l(dt)= J F(w, B) P(p)(dw), A
- [0,1]
such that
E.\",BE(}I(T).
A)
Any kernel with this
distribution.) propertywi1l be calledposterior
-
A 36.9Definition.Supposethat E is weakly continuous and jl-intcgrablc. x lI(T) [0,1]iscalleda posterior stochastic kernel F:!1 distribution (with
respectto p)
if)
. dP(1BP) F(B):=F(.,B) E dP(p) , BEfJ (T).) 71lenthere that E isweakly continuousand JJ.-inteKrable. 36.10Lemma. Suppose
existposterior distributiolls.)
Proof Let v d be a dominatingmeasure and let Q114be an arbitrary measure. Let) probability I
N=
{
f
;
l'(dOj
(O,W)})))
178
Chapter 7:Theory of
Then P(j.t)(N) =
Estimation)
O. For every BE(JI define)
F(B)= JB
/
dp,
j.t(dt) dv-
I Q(B) Thenwe have for every A
E
dp' J dV\" II(dt) on Q\\ N, on
N.)
sI with P(j.t)(A) < 00and every BE14(T))
d = J r P' JI(dt)dv JA F(B)dP(j.t) dV A\\N' = = J P,(A\\N)p(dt)
P(1BP)(A).
B)
Thisprovesthe assertion.
0)
The precedinglemma also gives us a methodfor computingposterior
distributions.)
36.11Examples.(1) Consider the shift of Example 36.5(1).It is easy to see that a
distribution is) posterior
_dP.
JB d
A.
- a)
(x
-
H
F(x,B) = dP J
(x
AL(da)
(x,B)E H x .\037(L).
,
a) )'L(da))
dA.H
If we specialize P = Nil then we obtainthat F(x,.)= Epr.(JC) * Nt\" x E H. the shift with scaleparameter of Example(36.5) (2) Now, consider (2).Then
a
distribution is) posterior
-
1 dP X a d(J L (da) H nrr d B (J I\\H ( (J ) , (x,B) E H x rill (L x F(x,B) _ x a H (J}+1 d H ( ) (/(JJ..,.(da)
-
\037
A.
\037
+ \037
).
d\037
(J)
A.
to P = NH doesnot leadto a considerable Specializing simplification. which doesnot rely on a distributions We needa representation of posterior in Theorem36.14 after somepreparations.) measure. It isobtained dominating
36.12Lemma. Supposethat E is weakly continuous. Then) \037
JB
{
dp'
I p.(dt)< 2 j.t(B)
II
\037
d\037
}
4\037\037\037
\037
- P'II, BE
tM,
sET.)))
36.Estimation
by desintegralion
179)
is trivial if J.L(B)= n. If Il(B)> 0 then we have) ProofThe assertion Ps
JB
{
=
I Jl(dt)< 2 Jl(B)
dp' dPs
}
dp' dp, { dp' {, dPs
1 2
J.L(dr)> J.L(B)
J.L(B) r
\037
\037
\037
\037
\037
r
\037
Recall,that
1
J.L(dr)> J.L(B)
t
, --
d\037J.L(dt)
d\037
0
P, II.
4 supII P,
}
\"2
2 :r 1 dp' J Jl(B) uB)
}
if r....... definitionE is continuous for the p, is continuous
by
variationaldistance.)
36.13Corollary.Supposethat E is continuousand Jl(U) > 0 for every U c T, U 0. Then) dp' [ dp,J.L(dt)> 0 \037-a.e.
open
=*=
fo.revery S E U and every openU c T, U:+:0.) Let U T be open, S E U, and B(s,c):={tE T:II P,II < c} U.Then)
Proo!
\302\243;
\037
P,
{!
\037;,
\037
\037
-
choose t > 0 such
that
\302\243;
I'(dl)=
o}
dp' < _1 J.L(B(s, 4c. J.L(dt) 2 {B(s,e) dP. } S
e\302\273
\037
o)
oS)
36.14Tbeorem.Supposethat E is continuous, Jl-integrah/eand that J.L(U) > 0 distribution F satisfies) for every open U c T, U * 0. Then every posterior dp'
_
. J - .J.L(dt)
p.s-a.e.,B E /A (T), S E T. F(B)= !,_d\037 dP, J dP.J.L(dt) s)))
\037
J
80
Chaptcr7: Thcory of Estimation)
ofa posterior distribution we have for every ProofLel BE91(T). By definition
Ae.91)
= P(IB Jl)(A) = J P,(A)Jl(dt). J F(B)dP(Jl) B
A
dp' P'(N) 0 and P,(A\\N) = AJ d - dp', t e T, The P, existence of such a setN followsfrom weak continuityof E. Let t E T. From >0 Thus we obtain it follows that J Corollary(36.13) dP,) Jl(dt) p'-a.e. Let
Ned be such that
o
'\"\"'
\037l}
J
A\\N
!.
dP'
=J J Jl(dt)d\037 F(B)dP(Jl) 8A\\N dP. s = J
J8
A\\N
f
dp'
dp'Jl(dt) dp,Jl(dt)dp,. dp' dt J dp' Jl ( )
dP. s)
-
. followssince . dP(Jl) dp' --- - on Q\\ N. The asserlton Jl (dt) = J dp.s)
dP,
o)
36.15Corollary.Supposethat the conditions aresatisfied. A of Theorem36.14
measurablefunctionf T -+
\037
is (E,Jl)-integrable iff
SE T.) flf(t)1F(dt)< 00 p'-a.e.,
In this case)
dP(hl)= dP(Jl))
sET. SfdF f's-a.e.,
and 36.14. combine36.7(3).36.13 The second ProofFor the first assertion
followsfrom Lemma 36.10.
0)
lemma.) Finally,we prove a useful approximation
36.16Lemma. Supposethat E is continuous.Let
...,
K
\037
T be compact and
K -+ R !?I(T)-measurable and bounded.Then for every t > 0 there are a measurablepartitionBt, BM ofK,pointst i e Bi . 1 i M.and <Xi E lit 1 i
f
\037
M, such that) \037
-
dp'Jl(dt) lcf(t)dp' f
\037
M (J.j j\037l
\037
dp' Jl(Bj) < E;, SET.))) dP;
\037
37.GeneralizedBayesestimates
un)
If Jl(K) = 0 or IIfllu:=supIf! = 0 then Proof Let c> 0 and K s; T compact. K the assertion is trivial.Therefore,we may assumethat Jl(K) > 0 and IIfllu > O. SinceE iscontinuous there exists{) > 0 such that) dp\" \037
dPs
- - -P. 1< whenever Itt - < dP,2 dPs
\037
II P\"
/2
I
I;
2JL(K)IIfllu)
t21
f>,
t..
t2E
K, SE T.)
!of
Sincefisboundedthere isa stepfunctiong = i=L !Xi lBI such that IIgll.. IIfllu and IIf gllu < ' The sets Bi , 1 I. M, may be chosensuch that 2Jl(K) diam BI < f>, 1 ; M, and {HI:1 ; M} is a partitionof K. We obtain) \037
-
I
\037
\037
-
\037
\037
dp'
(t) d
K
=
-
\037
-
M
\037
dP'.Jl(BI) Jl(dt) ,-1 i .L d Ps
If
\037
\037
ex
\037
dp, (f(t)Jl(dt)+ I}If(t) g(t)ldp'Jl(dt) r k
\037
M
d\037
i\037t
M
\037
+
i\0371
\037
\037
. Jl(K) +
2Jl(K)
\037
.
2Jl(K)
M
dp' ( ) Jl(dt) dP. dP. Ig(t)1 dPsJl(dt)) g(t) ki r
get)\037
\037\037
Jl(B,)< E;.)
dP,
dPs
d\037
d\037
d\037
o)
i\0371
37.GeneralizedBayesestimates) and assumptions 36. of Section keepthe notations In the followingwe considera loss function W = (U0/ET' given by IE T, suchthat (s,t)...... (s,I)e Tx T, is \037(T)x \037(T)J-Y,: T\037 [0,(0), measurable and W(O) = O. If E Jl(E,T) isan estimate then the risk of (!with respectto W is given by We
\302\273-;(s),
\037
tE T.) U(ro,ds)P'(dro), H the notations) Forconvenience let us introduce \302\273-;UP,:=
\302\273-;(!:
\302\273-;(S)
w...... J
\302\273-;(s)
(!(w,ds),
WE
fl, t E T,)))
182
Chapter 7:Theory of
and) l?
ESlimation)
p,:Bt-+ f (!(w, B)
\037
(dw),
BE94, t E T.
discussfirst Bayes estimates assumingthat IIIgJ(T) is a finite measure. Generalized are the extension of this conceptto non-finite Bayes estimates t he Recall that F denotes measures w.r.t.Jl.) distribution 1l1\037(T). posterior
We
37.1Definition.Supposethat JlIEl(T)is finite
is Jl-intcgrable. An to Jl and W) if estimate e*E fJl(E,T) iscalleda Bayesestimate (with respect P'Jl(dt) for a1l e Yt.(E, T).) J J.v,q*P'Jl(dt) J \037
J.v,\037
and
W
\037
37.2Theorem.Supposethat E is weakly continuous, JlI5l(T)isfiniteand W is thenfor every SET) (!*e {Jt (E,T) isa Bayesestimate If an estimate Jl-integrable. J
(J\302\245,U*)
(w) F(w, dt)
\037
J J.v,(s)F(w, dt) P(Jl)-a.e.)
Proof Assumethat there existsSoE T and A E .91with P(Jl)(A) > 0 such that f
Definean
< f(W;(!*)(w) F(w,dt) J.v,(so)F(w,dt)
if
WEA.)
estimate C!E fJt(E,T) by
(!(w, .) =
(!.(w, .)
if
w rf A,
I
ClJ
'f
{
\302\243\"0
EA.)
Thenit is easy to seethat
<J
P'Jl(dt). 0) If E is weakly continuousand Jl(U)> 0 if U c:T is open, then the exceptional setis assertion P(Jl)--{p,:t E T}.In thiscasein the preceding J J\302\245,e\037Jl(dt)
J\302\245,(!*
of \037-measurezerofor every t E T. The assertion of Theorem37.2is the startingpoint for the definitionof generalized Bayesestimates.) if for every SETthe 37.3Definition.A lossfunction(U0u;Tis(E,Jl)-integrable
function / t-+ J.v,(s),/ E T, is (E,Jl)-integrable.)
and (Jt;)'ETis(E,Jl)-integrable. 37.4Definition. that E isJl-integrable Suppose to Jl and An estimate (!*e fJt(E,T) isa generalized (with respect Bayesestimate W) if)
(w) F(w,dt) J (J.v,(!.) for all .'IET and rE T.)))
::;J
J\302\245,(s)
F(w, dt)
\037-a.e.
37. Generalized Bayesestimates
183)
37.5Remark. Let usmake somecomments on the preceding definition. First we notethat the inequality may
be written as)
O.(w, ds')
f (f J-v,(s')F(w,
\037
dt\302\273
J
F(lt),dt)
J\302\245,(.\037)
r E T, sET.) p\"-a.e.,
Assumefor the moment that
W'-'inf J J-v,(s)F(w,d/),
Q,
WE
SE T)
isd-me:asurable. Letting) B(w) 1={s'E T:J J-v,(s')F(w, dt) = inf J '/fiT) we seethat
F(w, d/)},
J\302\245,(s)
WE
Q,
0.isa generalizedBayesestimateiff
o.(w,
B(cd\302\273
=
p'-a.e.,t E T.)
1
The preceding of generalized remark showsthat the existence Bayesestimates
depends on regularity properties of the posterior risks the:mostgeneralset J J-v,(s) F(w.dt),sET.w E Q,We: donotintend to state: of generalized of conditions the existence since which implies Bayesestimates, for most specialcasesexistence followsjust from a moment'sreflection. of which is useful in many cases. we will work a out set conditions However,
s.-.
First,letus consider examples.)
37.6Examples.(1) Let us consideragain the
shift on a Euclideanspace beenintroducedin Example 36.5(1).Define the loss function by \037(b)= lib a1l2, a,bEL, and assumethat P possesses finite momentsup to the order2.Then)
(H,< ., .
\302\273
which has
-
\".(x) = J aF(x,da) =
- a) (x - a)
:; dP
-T
Ja
J
(x
d)..H
is a generalized Bayesestimate. of (1)but let (2) Keepthe situation I if b a > C. = W a (b)
if
(da)
,
xE
H,
Adda))
a e L.
IIb-all
{o
-
AI.
II
The pertaininggeneralized are calledMaximum Probability Bayes estimates
Estimates.
Nil'Then it is easy to see that for bothtypes of lossfunctions,,*= PL is a generalized Bayesestimate. the caseunder (1)to P = (3) Now we specialize
It)))
184
Chapler7:Theory of Estimation)
will turn
outin Section38that this iseven vaJid for a considerably larger class
of lossfunctions.
a Gaussian shift with scaleparameter and define (4) Finally let usconsider
-
\037
as in Example 36.5(2).Definethe Jossfunction by , a(b,t) = (J2 I1b aUZ, a,bE H, (1, > O. Then it is not difficult to seethat K. = PL is a generalized Bayesestimate. existence theorem. Now we approachthe announced In the followingJet (H,< ., . bea Euclidean spaceand assumethat T = H. = W(s t) where W: T....., Moreoverlet Jl = All and [0,(0)is a measu= rablefunctionwith W(O) O. An experimentE E 8(T)is\037-integrableoforder p 0 if it isJ\037-integrableforf t r+ t liP, t E T. If W isoforderp and E isAIIintegrable ojorderp then (Jt;)u;H is (E,AH )-integrable.) \037
'\037
or
-
\302\273
J\302\245,(s)
II
\037
37.7 Lemma. Suppose that E;scontinuous and ;,w;ntegrable oforderp.If W is riskfunctions of orderp then the posterior
st-+fW(s-t)F(dt),sEH, arecontinuousPr-a.e. Jorevery r E H.)
-
Proof ChooseS E Hand r E H. Let B(s,b l ) H be such that W(u t) CJ \" IliP + C4 if u E B(s,01)and IEH. SinceE is AII-integrab]eof orderp it \037
\037
followsthat
JiltjlPF(dt)<00 J:,-a.e.,rEH.) of r E H. Now Jet > 0 Theexceptional setN E .91can bechosenindependently arbitrary and fix somew N. Thereexistsat 0 such that) \302\243
\037
\037
II tll f IJrJIP>a.
P
- -. I;
. F(w, dt) < 3C
J)
Define u-(c),=min{W, CJa + C4 }.We may find bE (O,bl ) such that) C
II u-(t)(U
-
t)\302\243(w,
dt)
-
- J u-(t)(s-
I)F\302\253(I),
dt)1 <
\037
3)
Hencewe wheneverII u s\" < O. Thisfollowsfrom continuityof convolutions. obtainfor every II E B(s,b)) I
J W(u \037
- t) F(w, dt) - J W(s -
-
+ J (W \302\243
\037-
3
-
\037t\302\273(u
F(w,dt)1
-I - t)F(w, - F(w, + J - -
If \037C)(u t)F(w,dt) t)
t)
dt)
(W
+2CJ f II/JlPF(w,dt)\037\302\243. 11'11' >41.)))
dt)1
\037C)(s
\037t)(s t) F(w, dt)
0
37.Generalized Bayescslimates
HS5)
37.8Theorem.Supposethat E i...continuousand l,,-integrable of orderp. there Assumefurther that W is separating, level-compactand of orderp. 111en existgeneralized Bayesestimates.)
of Remark 37.5issettledby Lemma37.7.If ProofThemeasurabilityquestion we can provethat B(w)* 0 on a set A E d with \037(A) = 1,S E H, then the assertionfollowsfrom the existencetheorem on measurable selections,
Theorem6.10.
1SI step:Let.'iE Handt > O. Firstwe notethat there existsa b < supW such
that) \037{J W(s
- t) F(dt)> b}<
To provethiswe
\342\202\254.
setK integrablethere existsa compact r
\037{
is not bounded.SinceW is (E,AH)Hand l> > 0 such that
first assume that W
W(s
-
\302\243
t) F(dt)> l>} < t.
H\\K
is boundedon K. -.) isboundedand separating. Let c< sup
The assertion followssinceW(s
W besuch that that W Now, assume = B {tET: W(s-t) c} is a neighbourhood of s. From Lemma 36.12we obtainthat there isan 11> 0 such that
::;
< 11} < t. \037{F(B) The number 11can be chosensuchthat c< supW
-
t) F(dt) = supW' (1
f W(s
-
\037
= supW
-
We
ohtainthat)
supW' F(B')+ (supW '7)F(B) + supW' F(8) '7F(B)
-
F(8\302\273
11F(B).
Henceit followsthat)
-
- 11.
W(s
t) F(dt)> supW
-
2 '7 }
{F(B)< 11}< E. 2nd step:Let.rE Hand > O. We showthat thereexista compact setK Hand a number 0 < l> < such that) inf f W(x t)F(dt)> inf J W(y t)F(dt)+ l>} > 1 c. \037(f
\037
\037
E;
\302\243)
\037{
\037
-
-
XE II\\K
)I\037
-
H)
Since > 0 isarbitrary thisimplies to the first that B(w) * 0 \037-a.e.According W a b < such there exists that) step sup \302\243
\037(J
W(s -t)F(dt)
\037
b} 1 \037
- 2'l: -
Let 0 < l>
C c H satisfying)))
Choosea compactset
186
Chapter7:Theory or Estimation)
-o}< 2 - t)F(dt) b}rI{F(C) 1-o}.)
<1 Ps {F(C) and let)
A,.::::; {f W(s
\302\243
')
\037
Then we have \037(Ac) > 1
\037
- t. Since
is level-compact there existsa compact setKc H such that tEe and x rt Kc imply W(x t) > (b + 0)/(1 0).Then we obtainfor every x rt Kc that on the setAI:)
-
W
\302\243;
J W(x
- t) F(dt)
\037
> b+b F(C)
1-=b.
\037
\037
cJ W(x b
-
t)
-
F(dt)
-
+ 0 J W(s t) F(dt)+ 0 \037
inffW(y-t)F(dt)+o. D
yeH)
proofof the precedingtheoremshowsthere existgeneralized Bayes which are even non-randomized estimates estimates.) As the
38.Fun shift experimentsand the convolutiontheorem)
l,
Let G be a locallycompact groupwith countablygeneratedtopology.Let = ).tif and At be a pairof right and left invariant measures such that Z(O'IJ.,)
u:x.-.X-I, X E G. The Borel-u-field of G isdenotedby &I(G).The modulus function of G isdenoted by A.)
. .
38.1Definition.Let PltI(G)be a probabilitymeasure onG.Thenthe experiP:t e G})iscalledthefullshift experimentgenerated ment E = (G,{M(G), {\302\243,
by
P. For shortwe denotePc:=
\302\243,
P, t E G.)
A measurable 38.2Remark. The followingfonnulasare immediate. function
is P-integrable iff it is Pc-integrable and JfdP,= Jf(lx)P(dx), (E G. we have Pc(A) = P(t-I A), t E G, A E 14.If P )'1then In particular, \037
tE G, and)
-
dp' _ dP -I (t x) (x) dJ..;
A.r a \"
.c.,
t
E
Pc
\037
I'b
G.)
d)'\037
38.3Lemma. (Torgersen[1972]).Supposethat E is the full shift experiment
areequivalent:))) generated by P. Then the followingassertions
3M. Full shift
experimentsand the convolution theorem
187)
(1) P l\" t E G} is dominated, (2) {p,: IE G} l\" (3) {p,: (4) E is weakly continuous, (5) E is continuous.) \037
\"\"
Proof'(1) (2):Let A e \037(G).Then (I:,+ P)(A) = J 1A (Ix) P(dx) = P(t 1 A), \037
t
E
G.
If AtCA) = 0 then AAt -t A) = 0 and hence(t,* P)(A) O. t e G}.If measuresuch that Q {p,: (2) => (3):Let QIEM(G)bea probability 1 A E &iJ(G) and t E G then Q(t- A) = 0 implies \037(t.-I A) = 0 for every s E G. = Hencewe obtainP'\037(A) 0 for all s E G and there Q(A) = O. It followsthat Q(A) = 0 iff
-
\037
0= J Q(t-lA) A(dt)
= J I lA (IX)Q(dx)A,(dt) = I A,(Ax-l) Q(dx)= l,(A)jL1(x I) Q(dx),)
where L1
is the modulusfunction of G. Since > 0 we obtainQ L1
(3) => (1):Obvious.
(5) \037 (4) \037 (2):Obvious. (3) =- (5):It is a well known fact that
limII/(x)
'-e)
-/(t- x)1).tCdx)= O.
.
Since)
for every IE
dee, - P)
- l(.
!e(G,.'!I(G), It) 1
1
dP_ (I 1 x),
=
d;.,--(x) d).,
the assertion follows.
XE G, Ie G,)
0)
that G isAbelianand let G be the charactergroupof G,i.e.the group Suppose of all homomorphisms x: G {zE c:Izi = 1}.The Fouriertransform of a measure PIBI(G) is probability
-.
P:
X
t-+ J XdP.
X
E
G+.
It iswell known (seee.g.Parthasarathy [1967]) that two probability measures iff their Fouriertransforms on fM(G) coincide coincide.)
38.4Lemma. Supposethat G is Abelian.If p(x)* 0,X E G*,then the fullshift measures.) experimentE consists 01pairwisedifferent probability ProofAssume that
\037
=
\037,
s,lEG.Thenit
foHowsthat
for every X E G*)))
188
Chapler7: Theory of Estimalion)
=
f x(x)P'(dx) f
(x) \037(dx) and thereforeXes) = X(t) for every X E G*.This provess = X
t.
0
Estimationfor full shift experiments dependson the methodsdevelopedin 36and 37.) Sections
38.5Lemma. Every fullshift experimentE is A,-integrahle.) ProofLet usshow that J P,A,(dt) = l,.If A E ll(G)then we have = f P'(A) A., (dt) ffl A (tx) P(dx)A, (dt) = HIA (tx)A.,(dt)P(dx) = A,(A). a)
38.6Example.Themain examplein the presentsectionwill bea full shift ona
Euclideanspace(H.<'. For thiscasethe precedingassertioncan be improved to the followingfact:If P has finite p-th momentsthen E is J.H integrableof orderp. This is the particularcaseof Example 36.8(1),where '\302\273.
H=L.
Throughoutthe followingwe supposethat E is a dominatedfuJi shift measure experimentgeneratedby the probability Let us compute the posterior distribution given A,. A particularcaseof the PI\302\243f(G).
in Example36.11 followinglemma iscontained (1).)
38.7Lemma. The posterior with re.'IpeCl to A, is F(x,.)= ex.P, distribution = XE G, where P .!f(uIP).) Proof Let A E \037(G).Thenwe have
!
dP
dP
d)'\037
d)'\037
(t-Ix) A., (dt) = J 1,4(/-1) dP =A(x)J 1A (xt -1) d).
(IX)At (dl)
\037(t\302\273).(dt)
= A (x>J1,4(xr ) P(dt)= L1 (x) 'p(x-I A). Henceit foJlowsfrom the proofof Lemma 36.10 that 1
F(x,A) = J(x)
A) =
:(\037-I
P(x-I A).
0)
thispointthe readershouldrecallDefinition32.1where amenablegroups of certain areintroduced. Amenabilityis the reasonof the optimum properties We have beenconcerned beforein Section 32. situation with a similar estimates. distributions.))) w e statesome lemmas for First, approximation posterior At
38. Full
shift
cxpcrimcnlsand
convolulion lheorem
the
189)
38.8Lemma. Supposethat G isamenable.Theilfor every probability measure (K\ QI\037(G)and every summingsequence Jim
*
At) , J (ex
\"-00
Q) (K\037)
K\
K\"
I.,(dx)= o.
followsfrom) ProofThe assertion
i (ex
*
Ar (\037II)
Q) (K\037)
1,(dx)
\"
1
= A,
1 (1 J.r(dx) (K ) J KJx) J -lKJxt\302\273 Q(dt) II
1
= J Q(dt)
J OKJX)
A,(K,,)
-l Jx) l K
KJxt\302\273
A,
(dx)
1
\037JQ(dt)A
,(K) J 11,,(x)-l,,(xt)IA,(dx).0 K
K
K\
II
Let F bethe posterior distribution 38.9Lemma. Supposethat G isamenable. of E given J.,.Thenfor every summingsequence) lim \"....00
l/,
KII)
= O. HKJIF-F(.IK\lIdP,A,(dt)") )
-
ProofSince F F(.IK ) = F(K\037). n E N, and sinceF(x,.)= II
II
II
only show that)
lim
1 \037 JJ \"-00\"', ( \"K\ )
(\302\243.1
\302\267
P)
(K\037)
P)(K\037) J (ex.P)(K\037) (e, P)(dx)= (E, and thereforewe may apply Lemma (38.8) with Q = Every
.
P we need
;.,(dt)= o.
.P.
.
We have)
P, (dx)
\302\243;(
P.P.
0)
measure QI9I(G)definesan estimate probability e e \037(E, G) by (l(x, Q. XE G. .)I=\302\243X.
Such estimates satisfy e p, =
P, *
Q, t E G.)
38.10DefinitiOD. An estimatee E Yl(E,G) is a convolution kernel if e(x,.) = tx * Q.x E G, for someprobability measure Q E 91(G).) 38.11Remarks. (1) An estimate(l E gr(Et G) is a convolutionkernel
iff)))
190
Chapler7:Theory of Estimation)
the condition is necee(tx,tA) = e(x,A), X e G, t e G, A E gI(G).Obviously, we define which To show ssary. t sufficiency, Q:=e(e,.) impliese(x,A) = Q(x- A) = (ex.Q) (A), x e G, A E \037(G). of a convolution kernel satisfy P, t E G, the distributions (2) Sincep, = = t E latter G. The an invariancecondition, eP, namelyeP, propertygives riseto the followingdefinition.)
.
\302\243,
\302\243,.
38.12Definition.An estimate e E gr(E,G) isequivarianrif eP,=
\302\243,.
eP, t E G.)
38.13Remarks. (1) An estimatee E 9t(E,G) is equivariant iff (eP,)(tA) = (eP)(A), t e G, A E the special casewhereG = (H,< ., . isa Euclidean (2) Consider spaceand K: H P = NH . In this casea non-randomized estimate Hisequivariantiffit is of the form K(X) = x + a P-a.e.for somea E H. To see this note that \302\2438(G).
\302\273
equivarianceimplies SI(K(x\302\273P,(dx)
= SI(K(s+ x)
-.
- s)P'(dx)
for all .'1,tE H and every IE ljjb(H).Sincethe Gaussian shift experiment is = it follows that K(S+ x) K(X) + s P-a.e., .'Ie H. An application of complete Fubini'stheorem yields p2{(.'I, x):K(S+ x) = .'I + K(X)}= o.
Hencethere existsXo E H such that K(S+ xo) = S + K(Xo) for P-almost every S E H. Now, taking a = K(XO) Xo provesthe assertion. The last rcmark showsthat sometimes non-randomized equivariant estikernels.Can a similarassertion mates are almostconvolution be provedfor
-
arbitrary equivariant estimates? In the followingtheoremit is shown that even the distributions of nonare \"in the mean\"convolutions.) equivariant estimates
38.14Theorem.Supposethat G isamenableand let(K,,)bea summingsequence. measures Qn I\037(G),n E N, e E 91(E,G) there existprobability
Then lor every .'iuchthat)
lim II ,,\037oo
,(
A
r
\302\267
) \"in)
\037
\302\243,-1
Proof Recall that for every
h:G2
-.
IR
-
eP'J.,(dt) P.Q\"II = O. boundedfunction
.\037(G) @ .\037(G).measurable
we have)
= H hex,t)F(dtIK,,)(x)Ps(dx)A,(ds),) hex,t)P,(dx)A,(dt) LJ L n E N. We obtainfor every A E 84(G))))
Ful\\ shift
3\037.
x,(
experimentsand the convolution thcorcm
191)
(aP')(tA)i.,(dt)
f
) \"K\"
\037
=
J If e(x,tA)F(dtIK,,)(x)\037(dx)i.,(ds)
) ( \"K\"
I\037 r \037
=
l)K,,)L If e(x,lA)
* (\302\243;1;
P) (dt) \037(dx);',(ds)+ R,,(A))
where) IR,,(A)I
\037
l,(\037,,)
By Lemma 38.9it
LJ
IIF- F(.IK,,)/IdJ:;',(dt).)
followsthat
lim sup IR,,(A)I = O. ,,-\302\253t
If
A..aI(G))
we denote)
Q,,:A
1
\037
l,(ds), J e(x,xA) l's(dx) l,(X\"K\ ) f .
A
E \037(G),n
EN,
then we obtain)
1
A
,(K) inr (eP')(tA)l,(dt) \"
= R,,(A) + 1 A
xtA) P(dt)f>s(dx);',(ds)
,(K) Ie\f If e(x, \"
= R,,(A) + J Q,,(tA) fl(dl)= R,,(A) + which provesthe assertion. A
first
(P.Q,,)(A),)
0)
is the convolution theorem.) importantconsequence
38.15Theorem.(Boll[1955]).Supposethat G is amenable.Then for every eE equivariantestimate that Qp, = p, * Q, t e G.)
91(\302\243,
a probability measureQ19I(G) such G) thereexists
of Proof SinceQ is equivariant we obtainfrom Theorem38.14a sequence measures probability Q\"I&f(G),n E N, such that
- P.
lim lIeP
Q\"II=
O.
\"\"'<10)
Let QI\037(G)be a vague limit pointof
(Q,,),,\302\253:\037.
Then for every fe \037oo(G)
we)))
192
Chapter7: Theory of Estimation)
have)
foP = ffd(P+ Q).)
Since(}pis a probabilitymeasurethe same is true for P.Q and Q.
0)
38.16CoroUary.Supposethat G is amenable.Then for every equivariant estimate there existsa convolutionkernel with the same distribution. Let W: G [0,(0)be a lower semicontinuous function and definethe loss function by \037(x):=W(t- x), x G, t e G. Then the risk of an estimate G) is given by o
-
I
\342\202\254
\342\202\254
91(\302\243,
\037(}p,
= Sf W(t-Iy ) e(x,dY)P'(dx),tE G.
kerneldefinedby If (!isa convolution
QI\302\243\302\245(G)
then
tEG. \037(!P,=SWd(P.Q), theoremforamenablegroupsthe risk of any cquivariant By the convolution
estimate isalsoof thisform.The maximalrisk of arbitrary estimates can be boundedfrom below by the risksofconvolution kernels.)
38.17Theorem.Supposethat G is amenable.Then for QE
!Jt(\302\243,
.
such that measureQI.1J(G) G) there is a probability
\037(! P, sup IE (i)
\037
f W d(P Q)
functionW: for every level-compact
G
every
estimate
- [0,(0).)
of probabilitymeasures Proof FromTheorem38.14we obtaina sequence
11E N, such that Q\"I\037(G),
lim supJ fd(P* Q,,),IE \037,,(G). sup \"6f(}P' \"-'00) \037
If the right hand sideis zerofor all fE \037oo(G)then it followsthat sup \037QP,= W(oo) 'fiG) and the assertion is valid with arbitrary Q.Otherwise,let Qol\037(G)bea vague limit point of (Q,,),,( N, Qo O. Then Qo is a vague limit point of =t=
p.
(P.Q\UN.")Let Q = Qo1(G). Qo.Then QI\037(G)is a probabilitymeasure.If fE fCt (G) then we have lim inf f fd(P.Q,.)
\037
\"\"'00)
f
fd(P.Qo)
\037
f
fd(P.Q)
and thereforefor every levelcompact functionf E fCb (G)we have (since)))
38.fuJl
shift
experimentsand thc convolution thcorcm
193)
supf-IE'Go
(G\302\273
lim SUpJ fd(P* Q,,) J fd(P* Q)
,,-
\037
co)
We
obtainthat)
supf(?P' J fd(P* Q) I( for every levelcompactfunctionfE\037,,(G) and the proofis finishedby Lemma \037
G)
6.4.
0)
and suppose that 38.18CoroUary.Let W: G -+ [Ot 00)belowersemicomillUOUS there is a pointYo E G such that)
J W(x'yo)P(dx) J W(x.y)P(dx) if yE G.) Then the non-randomized estimateK: x \037 X. YOt x E Gt \037
ha.\037
the following
properties: (1) K is optimal(for W) amongallconvolutionkernels. (2) If (i isamenablethen K isoptimal(for W) amongallequivariantestimates. (for (3) If G isamenableand W islevel-compactthen K isa minimaxestimate W).)
and (3) followsfrom Proof (1) is obvious,(2) followsfrom Theorem38.15
Theorem38.17.
0)
Now we turn to the casewhere G isa Euclidean space,denotedby (H,< ., .
\302\273.)
38.19Example.Let W(x) = IIx1l2, X E H. Thenthe assumption of the preced-
ing corollaryis fulfilled with Yo = J xP(dx),providedthe first moment exists. Hence,K: x \037 x + Yo' x E H, is a minimax estimatewhatever PIPA(H) looks
like.If PIEI(H)iscentrallysymmetricthen the first moment iszeroif it exists and hence,the identity is minimaxfor quadraticloss.This easy fact can be extendedvery far as we shalldo below.) 38.20Lemma. Let C, D 5 H be convexsetswhich are centrallysymmetric. Then) l\302\253C+
y)r-.D) l(Cr-.D),YE H.) \037
It is that C and D arc closed. Proof In view of Theorem(6.1)we may assume clearthat we may even assumethat C and D are compact.For any convext compactsetA II let SeA) A + ( A). Then it is easy to see that \302\243
-
I\"\"\"
\037
Cr-.D,using that S\302\253C+y)r-.D)!;;
\037
C and D are convex and centrally sym-)))
194
Chapter 7: Theory of
Estimation)
it issufficientto show that metric.Hence,forthe assertion ).(A) for convexsetA 5 H. But this isan easy consequence of the Brunnany compact, MinkowskiTheorem6.2.For this,put k:=dim H and notethat ).(S(A\302\273
1 ).i\"
=
(S(A\302\273
11
).1(2A + 21(
-
1
A\302\273
\037
11-
1
2).i(A) + 2 ).k(
38.21I\037mma. (Anderson[1955]).LetP <( ,t.Supposethat W:
H -+ [0,(0)aresubconvex.Then) f WdP
\037
f W(x + y)
A)
=
\037
1 ).i\"
(A).
0
- dP di and
P(dx), ye H.)
ProofLet C 5 H beconvexand centrallysymmetric.Thenfrom Lemma38.20 it
foltowsthat for every ex
P(C+ y) =
\037
0,y E H)
1.t(C+ y)n {
>
\037\037
.(
;:;;I Cn Putting C = {W
=
{\037\037
>a})da
P},P 0, we obtain P{W>P} P{W(.+ y) > P}, \037
da
a})
P(C), ye H.
\037
\037
{J
\037
0,ye H,)
and therefore) QO
QO
P{W>{J}d{J f0 P{W(.+ y) > {J}d{J =fW(.+y)dP, yeH. 0) At this point we couldapply Corollary38.18 to obtainthat under the of Lemma 38.21 the identity is the optimalequivariant estimate. assumptions J WdP = fo
\037
However,for the verificationof the minimaxpropertywithout assuminglevelof Wwe have to go back to Theorem38.14.) compactness
- dP
38.22Theorem.LRt P <{)..Supposethat di..and W: H -+ [0,(0) are subconvex.Then Ie:x....... x, X E H, is a minimaxestimate (for W).) Let QE .\037(E, H) we may assume that Wis lower semicontinuous. ProofW.l.g. of Then from Theorem(38.14) we obtaina sequence be an arbitrary estimate. measures n E N, suchthat probability Q\"I\302\243\302\245(H),
sup H'\"e IIEH
\037
;;;; lim
supJ (W (\\ a) d(P* Qn)
n-QO)))
38.Full for every
a> O. Sincewith
W
experimentsand
shift
the
convolution theorem
195)
and also W r.a, is lower semicontinuous
that it follows subconvex, w,.eP\" J (W r.a)dP, sup heR) \037
a> O.
From)
J WdP
--=
sup J (Wr.a)dP 0>0)
the assertion follows.
0)
Thequestion whetherthe identityisthe only equivariantestimate with minimal risk is answeredby the followingcorollary.)
38.23Corollary.Let P
\037
A
and p(z)
=+=
0,Z E H.
.\\\037uppose
that
W:
H -+ [0,oc)is
andsuch Ihat) level-compact J W dP < J W(x + y) P(dx) if y * O. Then an equivariantestimale (l E H) has minimalrisk (for W) 91.(\302\243,
= 1,4(x) P-a.e.,A
E
(ff
(l(x,A)
&f(H).)
measureQI\037(H)it followsthat Proof.Since(lP= P * Q forsomeprobability J WdP = J Wd(P*Q). First,we show that Q = f:o.For C( 0 let DII = {xe H:!Ix C(}.Define A(.I= inf J W(y + z)P(dy). \037
II
\037
z.D.)
Theseinfimaare attainedsincelevel-compactness implies Jim z..\037)
J W(y + z) P(dy)= supW.
HenceMil > J W dP for every > O. We have \037
J Wd(P.Q) = H W(y + z)P(dy)Q(dz)
+ Q(D(J)Mo + Q(D(J)Mo.) It followsthat Q(DIl)= 1 for every C(> 0 and thereforeQ = to. that for every t E If and Thus,we obtain(!p= P. Equivarianceof (!implies \037
=
A
E
fM(H)
\037
(1-
Q(D(J\302\273Mil
-
P(dx)= peA t). II be compact. SinceshiftsareA-integrable J P(K t) )'(dt)< 00,))) J (l(x+ I, A)
Let K
Q(D\037)Ma
-
196
Chapter 7: Theory of
Estimation)
and hence)
-I(z,r -x) J (}(I,K) P(dx) (dx) = J e-I(z,,) ZE H. P(K
Je
-
A.
t\302\273)'(dt),
Fubini'stheorem thisyields p(z)J e-i(z,r)e(t,K)/.(dl) = p(z>Je-i(z,r) which provesthe assertion. 0) By
lK(t\302\273)'(dl))
The idea of the followingproof has been used also by Ibragimovand
Has'minskii [1981].)
38.24Discussion. Lemma). If P = Nil then (Equality in Anderson's J WdP = J W(.+ y)dP impliesy = 0,) for every subconvexfunction W: H -+ [0,(0)which islevel-compact, separating and of finite order.It shouldbenotedthat noof theselatterconditions can be omittedwithout compensation. Lemma 38.21. In To provethis,let y 0 and recallthe proofof Anderson's W view ofthisproofit issufficientto show that there issome < such that) Po sup =t=
P{W>P}< P{W(.+ y) > P} for all P Po. ChoosePo in such a \037
way that {W
\037
0 as inner Po} contains
sinceW is separating. Let Po P < supWand denote point.This is possible = C {W P}.Then we have \037
\037
P(Il\\C) < P(H\\(C iff)
J.
-y)n
((C
>
{
\037\037
-
y\302\273)
_})
d (cn{
\037\037
>_}))
for someex > O. This is the inequalitywe shallprove below. Let
_.=sup
C S;
{Y:
{
>
\037\037
Then
r}}.
it
followsthat C s;;
{
\037\037
Moreover,we show that
oCn
{
\037\037
For thislet
=
,,}
\037
0.
b.!0 and choosex.
E
Cn
{
\037\037
<\"+
n E
b.}.
N. Then)))
;<;
,,}.
3M. Full shift
(X
(X,,)< a
\037
+ b\",
nE
experimentsand the convolution lhcorem
N, and there is an
197)
accumulation point Xo of (x,,)
\037\037
a. Since C is dosed we
(xo) =
satisfying
have
Xo E
C, and since
\037\037
t
we have Xo E ac.Thus,we obtain
>
\037
\037},
{\037\037
XOEacn{
\037\037
=+
Next, Jet us show that \037
a}).)
\037 Xo\037s\302\253c+y)n{
Assumingthe contrary,namely)
y)n
XOE S\302\253CI
\037 {\037\037
dP
1
=2(C+Y)11 {d)'
\037(X
dP 1 + 2(C-Y)fl d)' it
{
would followthat) Xo
a}) }
\037
(X
}
-
,
= 1 (Xt + y) + 1 (X2 y) = 2 2
where x,.X2 E C,x, + Y E
1
;;;
{
a}.x,
\037\037
(X2 + X2),
\"2
-
YE
{
SinceC
\037
\037\037
\037
\037}.
and sinceXo isa boundarypointof the strictlyconvexset
;;; {we obtain a} that \037\037
Xo
= necessarily XI
+Y E
\037
{
\037\037
a}
{\037\037
X2 = Xo. But
and
Xo
-
Y
E
{
\037
\037\037
\037
a}
+
isa contradiction to the fact that Xo is a boundarypointof the strictlyconvex dP set dl 2.a .
{
-}
.)
It followsthat for somet > 0
>
B<Xo,8)\037
(s\302\253c+y),,{ a}))'.))) \037\037
198
Chaptcr7: Theory of Estimation)
On the otherhand Xo E acf'I
=
{
\037\037
Therefore
a}.
xoetf'l{ > a}.
(1-\037)
\037\037
This is due to the fact that 0 E (; and 0 E
(1 n
separate
>
{
\037\037
Xo
onecould
a}.(Otherwise
a contradiction to Xo E from (;andproduce
clearthat the openset U,\037 B
((1 i)xo.n satisfies =0. >a})
Uf'lS(C+Y)f'I{
\037\037
C).Now. it is
but
uus(c+Y)f'I{>a})\302\243cf'l{>a}.) \037\037
\037\037
This implies)
>
'(S(C+Y)f'I{ >a})d(cf'l{ a})).) \037\037
\037\037
Another application of the Brunn-Minowski theoremasin the proofof Lemma 38.20provesthe asserted inequality.)
solutionof the estimation Now, we are in a positionto presenta complete with linearrestrictions. problemfor Gaussianshift experiments and consider the experimentE = (H,1I(H), Let L Hbe a linearsubspace {NH 6,:tEL}),This is not a full shift experimentif L 4= /I. We dealwith the L IRk wherewe assume a linear function! problemofestimating w.l.g.thatf issurjective. Thismeans, that the decision spaceisD = and the lossfunction is of the fonn W; (y):=W (y [0,(0).We y E IRk, tEL. where W: iii\" estimateK = f 0 PL is the best possible shallprovethat the non-randomized
.
\302\243;
-
-
\037k
f(1\302\273.
-
choice.)
we shallfrequently 38.25Remark. In the proofsof the followingassertions
usea reductionof the problem. Let F = (H, {Nil.6,:t L kerf}) and letG:=K.f: To describe the experimentG we endowR k \302\243f(H),
with the inner
product) \302\253Yl'Y2\302\273:=(X
.X2)
1
jf
y,=f(xi),xi.lker.f.;=1,2.)))
38.Full With
shift
experimentsand the convolution theorem
199)
thisinner productwe have \037(KINH.6,) = NGlIc.6/(,), t
1.kerf,
and thus
G = (RJ:,EI\", {NRIe.6/(1):t 1.kerf}).) The pointis that G is a full shift experimentwhere the theory of the present sinceK = f 0 P,.isa suffcientmapping,G and can beapplied. section Moreover, F are statistically whether E and G are (Thequestion isomorphic experiments. makes no sensein ourframeworksince and G have isomorphic statistically \302\243
differentparameterspaces.)
Forconvenience Our first resultisconcerned with the convolution theorem. kerf) Nil. IE L, Q,= NRIl.Cf(')'1.1
we denote.P, =
C\"
38.26Theorem.Let e E g,(E,IRk) be an estimate H..hichis \"equivariantfor the in the estimation off\" followingsense: * (ePo), tE L. eP,= Then there existsa probability measureR such that e.P,= (f 0 Pd'p')* R, tEL.) \302\243/(t)
I \037k.
!i'
ProofSincee E fJt(E,IR\") and 9t(F,IRk) = 9t(J::,[Rk) we have e E !1i(F,IRk). The F and G are statistically and thereforethere exists experiments isomorphic t E 9t(G,IR\"} such that tQ,= eP\" 1.1 there existsRial\" kerf By Theorem38.15 such that
eP,= tQ,= Q,.R, t.lkerf, or in otherwords) (ePo)= NAIe. * R. t 1.kerf. SinceIE L occurson bothsidesonly throughf the equationis valid even for \302\243/(1).
every
\302\243/(')
IEL.
0)
Next, we show that
K
= f 0 P,.is a minimaxestimate.)
38.27Theorem.Assume that inf \"F\037(F..R\")
Proof Let e E Then)))
W
i.v
lowersemicontinuous and subconvex.Then)
sup JYreP,= J W(foPL)dN\". ,pl.)
91(\302\243,
H\")
and
choose't E 9t(G,IR\"} as in the precedingproof.
200
Chapter 7: Theory of
sup \037e\037 r L
\037
Estimation)
SUp
r.l ker f
E
= SUp
\037e\037
r 1 ker f)
\037tQ,
= J WdQo = J W(f\302\260PL)dN H
This provesthe assertion.
.)
0)
the uniqueness FinaJly,we consider problem.)
-
38.28Theorem.Assume that
the subconvexfunction W: IRk [0,(0)is leve/compact,separatingand of finite order.Then every equivariant estimate (!E 91(E,1R ) (in the senseof Theorem(28.26))with 1t
\037eP,= J
W(fopJdNll , tE L,)
satisfies U (x,B) = 1B (f 0
p,.(x\302\273
A.-a.e.,BE fill<
.)
as in the proofof Theorem38.26. ThenCorollary 38.23and Discussion 38.24imply that Proof.Chooset E 9t(G,
\0371)
'[(.,B)=1 8 Qo-a.e.,BE\037k, or in otherwords) J (1(.,B)dNH = J IB KdNH O
A
A)
for all BE gr, A c:K-1 (81\.")If we put
(1(.,B)dNll = NH(K-
=
A
1
f 1(--1(81)
(B\302\273,
,,-1(8) then it followsthat BEgr,
and hence)
e(x,B)= 1
if
fopdx)EB
A.H-a.e.
0)
carriesover to the casewith an It is clearthat the precedingdiscussion
scaleparameter (J > O. additional
since If P NH then the reductionusedin Remark 38.25is not possible This isoneof the reasonsfor the \" = f 0 P\" neednotbe a sufficientmapping. of the theory of the next section.) development =*=
39.The structuremodel) In thisparagraphwe consider which generalizes the caseof full shift a situation 38.))) considered in Section experiments
39. Thc slructurc model
39.1Example.Let P
201)
and considerthe n-fold product E\" IE n E {(P t,)\": !R}), N, of the full shift generatedby P. The on the samplespacen = Ii\" by translation groupG = IR operates gw = (WI + g,W2 + g. w\" + g), g E G, wE Q. ofQ. Each orbitisisomorphic Theorbitsof the groupactionform a partition can we with the operatinggroup.Therefore, identify the pointsWEn by the Let M = {wE L Wi = O}.This is a crosssectionof followingprocedure: i-I each orbit in exactly onepoint.Each WEn can be orbits,i.e,it intersects = (R\",\037\",
\037
AI\302\243itl
*
...,
!1: /I
in the followingway:) decomposed W
=
-1\" .L n
+
Wi
1=1
Denoting
-1 i-L
(
WI
- 1\" .... - -1\" .r ) . .1: 1-1 .-1 WI'
n
w\"
WI
n
\"
T(w) =
Sew) = we obtainW
\037
n
and
Wi>
t)
- nl=1 .L 1
(
WI
- nl=l) -1 L /I
/I
Wi>\"\"
W\"
Wi
),
T(w) + Sew), where
T(g + w) = g + T(w),
w
\037
n, g E G,)
and)
S(g+ w) = Sew). we n, ge G.) In otherwordsTisequivariant and S is invariant. Let (n,\037)be a measurable spaceand let G bea groupoperatingon Q.We assumethat G is a locallycompact groupwith countablygeneratedtopology and the operation Let ).,. @ ..
(\302\243I(G)
q:g 1-+g 1,g E G.)
The followinganalysisis basedon onefundamentalassumption.)
39.2Assumption.There is an (.>1.
\302\243I(G\302\273-measurable
satisfies T(gw) = gT(w) wheneverg E G and WE Q. The followingexamplegeneralizes Example39.1.)
map T:Q
- G,
which
39.3Example.Supposethat Q = H where(H,< ., . isa Euclidean spaceand \302\273
let L H be a linear subspace. Then G = L operates on H by the operation xw :=X + w, x E G, WE H. It isclearthat the orthogonal T = PL projection satisfies the condition of Assumption 39.2.))) \037
Chapter 7: Theory of
202
Estimation)
39.4Example.As beforelet a = HandL s;;; H bea linearsubspace satisfying = x dimL < dim H. But now let the semidirect G IR. L product operateon H + by (a,s)w:=aw + s, (0',s) E R x L, WE H. If we define = idH P,. and (.)II, 1'2)satisfiesthe conditionof Assumption39.2for 1'2= PL then l'= (1111 WE H\\ L. To seethiswe have to show that) 1\037
T(O'w+ s) =
-
(0',s) T(w)
= (O',s)(lll1 (w)lI, T2 = (0'11(w) II, a T2 (w) + s).) (w\302\273
\037
But thisis immediate since)
111; (uw + s)1I= lIu11 (w) + 11(s)1I= ulll1(w)1I) and)
1i(uw+ s) = uT2 (w) + T2 (s) = uT2 (w) + S.) ForW E L the function Tdoesnotmap into1R+ x L However,for ourpurposes it doesnot matter to repJace Q = H by Q = H\\ L sinceL is a subsetof show that Lebesguemeasure zero.Denotek = dim L. Easy computations da @ ).L(ds) is a left-invariantmeasure on G and ).,.(du, A(da,ds)= ds) = da@ ).L(ds) is right-invariant.
!
ak\0371
0')
39.5Lemma. The mapping S:WH (1'W)-IW , under the operation ojG, i.e.it satisfies) (1) Sew)= S(gw), (2)
WE
WE
a, is maximal invariant
a, gE G,
S(wl)= S(W2) implies Wt = gW2 Jorsome g E G.)
Proof (1)This followsfrom
-1(gw) = -1(gw) = (Tw) -1g-Igw = (Tw) -1W = S(w).
S(gw)= (2) WI
(gT(w\302\273
(T(gw\302\273
If 8(wl ) = 8(w2) then (Tw1f WI = (Tw2)\"
= Tw I
I
W2 . '(T0>2)-I
I
W2
and therefore
0)
each orbit in exactly onepoint.Every 39.6Lemma. The set im S intersects (J) = g . S where g E G and S E im S,namely admitsa unique decomposition WE !1
g = T(w),S = Sew).)
Proof The definitionof S showsthat Sew) and ware on the same orbitfor that im S intersects eachorbitat mostat one))) every WE a.Property(1)implies
39.The structurc
model
203)
of the asserted point,i.e.im S is a crosssectionof orbits.The existence = w Q. To follows from W E let w T(w) Sew), proveuniqueness decomposition = gs,g E G, se im S.Thisimpliesthat Sew) = S(gs)= S(s)= s and therefore T(w)s= gs.Applying T we obtain) T(w) T(s)=
T\302\253Tw)s)
and thereforeT(ro)= g.
=
T(gs)= gT(s))
0)
39.7Corollary.The setimSisin.PIsinceimS= T-1 {e}.) T(w) = e. If conversely Proof If s = Sew) = T-1 (w)w then T(s)= T-1 (w). .1 . = = sET-I{e}then T(s) e and thereforeS(s) T (s) S = s. 0)
39.8Examples.In the caseof Example39.3the maximalinvariant is S:x x x PL(x),X E 1/.For Example39.4we have S:x........ PL (x) -'x E /I. In the x PL(X)) secondcaseim S = L1 {xE H:!Ix = I}.
-
(\\
_-II
II
\037
11
39.9Remarks. At
the first glanceAssumption 39.2might seem to be very Morefamiliarare thc followingtwo assumptions: restrictivc. (1)Thereis a crosssectionM of the orbitsof G onQ. This means that for every wED the setM (]{gw:g E G}contains exactlyoneelement. The of all WE Q coincide. (2) (Theisotropic isotropic subgroups subgroupof WE D is (gEG:gw= w}). Let If G bethe commonisotropic SinceH isa normal subgroup subgroup. we may replaceG by the factorgroup G/ /I. Then the commonisotropic subgroupof every WE Q is H = {e}. It isclearthat our Assumption 39.2implies conditions (1)and (2) with M = im Sand H = {e}.However, apart from measurabilityquestions alsothe converseis true sinceeachro e Q admitsa uniquedecomposition as w = g . s with g E G and s EM, Defining T(w) = g iff ro = g . s yieldsa mapping T:Q -+ G which satisfies T(gw)= gT(w),g e G,we o.) \037
39.10Definition,Let PI.saf be a probabilitymeasure.For p(g-IA), A E d. Then the experiment E = (D, let\037: A
every
gE G
g E (i}) is calledstructuremodelgeneratedby P if the groupoperation satisfies condition \037
.\037, {\037:
39.2.
The casesconsidered in Examples39.1, 39.3and 39.4are structuremodels. In the followingwe denotePs:=\037(SIP).It is clearthat Ps = \037(SIPg),
g E G, (by
(39.5\302\273.)))
204
Chaptcr 7:Thcory of
Estimation)
39.11Remark. We definethe measures) A.t * Ps:A H H l (gs) ;.t(dg) Ps(ds), A E.\037, and A, * Ps:A H H 1...(gs) ).,(dg)Ps(ds), A E .91.) A
are O'-finite sincel(and A, are O'-finite and P::;is It isclearthat bothmeasures finite.Moreover, we observethat) (l,* Ps)(A)= J t1 0 T del,* Ps), A E .tI. ...)
whereA isthe modulusfunction of the groupG.Sinceim * Ps.) that At * Ps '\"1..,
A
\302\243
(0,00)it follows
39.12Lemma. Every structure modelisJ.,-integrable sinceP(A,) = i.,* Ps.) Proof If A E .91and g E G then we have A) J p(g-1
A,
(dg) =
;.,(dg)P(dw) IS 1...(gw) = JJ 1... (gT(w).r) A., (dg) Fs(ds) = JJ 1... (gs)i.,(dg) P.r;(ds)= (A., * Ps)(A).
0)
39.13Lemma. Themeasurei.(* Ps is invariant, i.e. (A,.Ps)(gA) = (A.t * Ps)(A), g E G, A Ed.) Proof For gl E G and A E.c{we have
Ps)(g1A) = H l\"...(gs)A. t(dg)Ps(ds).) Since1\",... and sinceAI. is left invariant on G the assertion (g.\\') = 1...(gj'\"lgS) (A., *
follows.
0)
39.14Theorem.For
every structure
equivalent:
(1) P 4.A,. Ps, (2) (3) (4) (5)
.\037
i.\\'
dominated,
9 l, * P.r;, \037
E is weakly continuous, E is continuous.)
Proof (1)<:>(3):Let A E
d.
Then)))
model the following assertion.\\' are
39.The struclurc model = Peg-1A) = \037(A)
(hs) II 19-IA
205)
.
dP deA, Ps) (hs) A,(dh) PS(ds)
-
- (hs)i.t
d(I., Ps) dP -1hs\302\273).,(h)Ps(ds). = SI1A (hs) d .(g * P d(A, s))
'.
Thisimplies \037
\037
i'l * Ps.
-
Qld
measuresuch that Q {P,:gE G}.We be a probability (2) => (5):Let claim that Q Q(A,), Indeed,if Q(A,)(A) = 0 then there existsagEG such that Q(g-1 A) = 0 for every hE G and therefore A) = O. This impliesp,,(g-1 = PlICA) = 0 for every hE G.HenceQ(A) O. By Lemma 39.12we have Q(I.,)= A, * Qs and Remark 39.11implies * '\" * therefore At * Qs. A( * Qs A, Qs. It follows that Q i'l Qs and that * Qsis an invariant measureand therefore Lemma 39.13 implies \037
9
\037
\037
).\"
(w) =
d\037
d(\037(-\"Q)
dP
d().,* Q)
(g
. Q-a.e.,g 1.(.
-1w)
showsthat g\" The Lemma of Riemann-Lebesgue
.
hm \"\"'00
I
.
.dP .dPQs)(g-1hs) - d(A(.
G.)
-g implies
=0 hs) ;.,(dh) (g;1 Q.(})
-dCA,
E
for Qs-almost every S E im S.Hence
.
I1m \"... (0
. Qs) _ (A,d()., Qs) d(i., Qs) dPg
d\037\"
*
*
lemma (Remark and Scheffe's
proves(5).
(3.1\302\273
(5) \037 (4):Obvious. (4) \037 (3):Weak continuity implies P(A,). Since follows. PC),,)= A, - Ps'\")., Ps the assertion (3) ::>0(2):Obvious. 0) \037
\037
\302\267
we see immediately that condition(3) may be consequence to 9 A., * Ps sinceA, * Ps = P(A.,) f!J and A., * Ps\"'\"A, Ps.) strengthened 39.15Examples.(1) When we are dealingwith a situationlike Example39.3 dP P )'H'Let h = - -;-.It followsthat then we usually assume
-
As an easy
\037
\037
peA) = J h(PL(X) + x
- PI.
d\"'H
(x\302\273
A)
A.H(dx)
!
= J J hey + z) i.ddy) L (dz) = I hd(AL.;'L!) A A.
A)))
\302\267
206
Chapter 7: Theory of Estimation)
for all A
\342\202\254
we have \037(G).Therefore
dPs (y) = hex + y)J.ddx), YELl.. I
dIL\037
with Example39.4. Herewe observethat (2) Thingsare morecomplicated peA) =
=
!
-
x
-
h(lIx p,Jx)H _ PL(X) + Ilx PL(X)
)'H(dx)
pdx\302\273
II
z
+ J!h(lIzlI'lI\037
Y)
),L(dy)ALJ.(dz),
A
e 14.)
Let t be the uniformdistribution on {ze [): II z /I = I}.Usingtransformation to coordinates onesees that is with polar 1'-density ,91(II.III),L\037) a measure q...... C q\"-lc-I, q> 0, where n = dim Handk = dim L. Thenit followsthat) peA)
1 = JJ J CrL'h(as + y) k+T dat(ds)),L(dy), A
A E
oc)
a,
and therefore)
dPs
dt
(s) = SSCoc
1
\" h(oc.\037
+ y) (rc +l doc),r.(dy),) sEimS.)
39.16Lemma. Supposethat G is Abelian.If .!t>(TIP)(x) * 0, X E G*,then the SlrucluremodelconsislS mea!1'Ure.r.) 01pairwisedifferenlprobahi/ily Proof.Assumethat
\037I
=
\0371
for someKI'K2 E G.Thenit followsthat for every
XE G*) IX(T(w\302\273\037I(dw)
=
fX(T\302\253(I)\0372(d(l)))
and thereforeX(gl) = X(g2)for every X E G+.Thisprovesgl = g2'
0)
39.17Examples.Theassertion of the preceding lemmacan be verifieddirectly in particularcases.Let Ie(10 (H).If in Example 39.3there existsan aE L, a * 0, such that
= P then PM = P for all n EN which implies JI(x + na) P(dx)= JI(x)P(dx), \" EN. \037
-
-.
SinceI(x + na) 0,x E H, as n 00,it followsthat P = O. leadto) considerations As for Example39.4similar
u\"-1
II(a\"x + q _ .-a)P(dx)= f/(x) P(dx), n EN,) which admitsonly
P = 0, or P = 1;\"/(1 - a) if q *
1.)))
39.Thc structurc
modcl
207)
39.18Lemma. The invariant a-jie/d
d = {A Ed:gA = j
A
if g E G}
isgeneratedby the mappingS,i.e.A c .f1f, iff A = S-1(A).) Proof Followsfrom Lemma 39.5.
0)
which is provedin the followingtheoremisbasicfor the The decomposition
modelisa mixtureof full It isshownthat a structure rest of the presentsection.
shifts.)
structure 39.19Theorem.Supposethat E = (0,.>1, {\037:g E G})is a dominated a im exists there model.Thenfor every s E S fullshift = (G, that: E such {P;:g GJ) \302\243S
\302\243I(G),
(1)
.91i'
under distribution is a version of the conditional
\037
of T given
A, = {hE
G:hs E A},
(J) 1-4!'i(tIJ)
(2) Pg(A) = J J>;(AJPs(ds), g E S E im S.)
G,
A
E.>I, where
Proof First,we notethat for every CE 5B (imS))
-1dP
= P{SEC}= Ps(C). (hs)).Adh)Ps(ds) H C d(Af * Ps))
Thisimplies that
.1.(dPP ) (hs) ).Adh) = 1
J d(
\302\267
s)
Ps-a.e.
For thoses E im S for which the equality holdsdefine)
P;(B)=
!
(g-Ihs) ).,(dh), BE!M(G), g E G.)
d()':\037 Ps)
For the restdefine {P\":g E G} to be any shift experiment. It is then clearthat for every s E im S the experiments) ES:=(G,fM(G), {J>;: g E G}))
areshift experiments. and Ce:\037(im S) (1)We have to show that for every BE \037{TEB,SEC}= J P;(B)Ps(ds),gE G. C) Thisis easily seenfrom))) \302\243I(G)
208
Chapler7: Thcory of Eslimalion)
dP -1 (g hs) ).,(dh) t dO..,... Ps) = \037{h.\\': he B,.\\'EC}= Pg{TEB,Se C}. needonly be provedfor setsA e .91of the form assertion (2) The second A = {hs: hE B,sEC}, Be \037(G),CE\037(imS).
Ps(ds)= cJ J>;(B)
r r \037
But thishasbeendonealready under (1).
D
39.20Discussion (Fisher'sfiducial distribution). (1) Let us first discussthe caseof a full shift.HavingobservedWE Q onemight beinterested in particular . the covering of the randomset(J) B where Be 8d(G).It isobvious probability a moment's reftectionthat by = \037{WE Q:ge w. B} PCB), Be ii,ge G. Theinteresting feature of thisequation is the fact that the right hand doesnot dependon g e G. Therefore,PCB)can be interpretedas the probabilitythat w. B coversthe unknown parameterg E G. (2) Now, we turn
a similar to structuremodelsand attempt to establish
relation. The preceding Theorem39.19 showsthat
P:{hE G:g E h . B} PIl(B), B E
g E G,S E im S. Again, the right hand doesnot dependon g E G and thereforethe number of the random setT(w) . B,given Sew).We PS(WI(B)isthe covering probability into may put the covering probability =\"
PS(WI(B)=
\037\037:'I
\037,
(T(w). B).
has beencalled It is by R.A.Fisher. fiducialdistribution of the mostimportantcasein statistics wherea posterior an analysis experiment is possible without using a priordistribution. of fiducialdistributions and posterior distri(3) Thereis a strongrelation butions.Let us provethat,independentlyof the choiceof SandT the fiducial with the posterior distribution for Ar. distribution coincides = Choosew h.'iand B E (G).Then
Thedistribution
\037\037:\037
\037
J B
dP
d{;.,-.i>s)
-1
(g w) ).r(dg)= J s
= J I B (g -I)
dP
d().(.A)
-1
(g hs) )'r(dg)
dP
d(;',,,,Ps) (ghs) A(dg) dP
= L1(h)J 1s(hg1) (gs) A.,(dg) d(A,... Ps) =
.1
(1'(w\302\273
J ls(T(w)'g-I)
d(A\037=
Ps)
(g'S(w\302\273;'(dg).)))
39.Thc structure
model
209)
F for ).,is) distribution Hence,the posterior dP r w ).,.(dg) (g * f's) d(J.( F(w,B) = dP -I ' (g w ) 1_,.(dg) J
, -_.
I
\302\273
d(J.;-:-P.<;)
-
J 1B(T(w)g-1) d()'\037\037-P\037
_.-.-
J
d(
)'\037\037
(g'
(g.
S(w\302\273
S(w\302\273
A(dg)
).(dg)
\037<;)
= J 1B (T(w)g t)PS(I4)(dg) =
P\037\037c:J)(B),)
Be a. This leadsto a simplerformula for the covermg probabilities: p.<;(6))(B)= F(w, T(w) . B), WE !l, BE forA,. satisfy an invariancecondition, distributions namely (4) The posterior where we fl,
a.
. gB)= {>S(w) (Tw' B) = F(w, B), we fl, Be \037(G), ge G.
F(gw,gB) =
pStgW)
I
I
\302\253T(gw\302\273'
of the coveringprobabilities: This leadsto a further modification PS(C/)) (B)= F(S(w),B), wED, B E \037.
It must be stressedthat the fiducial methodis only a computational T satisfying Assumpestimator methodand doesnotdistinguish any particular choiceof T shouldbesuch that the coveringprobabilities tion39.2.A suitable T are large in somesense.E.g. if for someB E \037(G) there existsan estimator (5)
and satisfyingAssumption(39.2) F(w, T(w) . B) = supF(w, g . B), leG)
wED,
then the confidence regionl'(w).B hasmaximalcoveringprobability. of remarkscan beextendedliterally to the computation (6) The preceding risksgiven S(w).Since(4) implies conditional 1 J1B(h-g)P;(dh)= J1B(h)F(s,dh),SE imS, Be \037(G), ge G, risk with respect to a lossfunction W: G [0,00)is the conditional
-
I)F(s,dh),seimS, geG. fW(g-lh)PgS(dh)=JW(h In otherwords,given Sew) the conditional risk of T(w) is = J W(h- I . T(w\302\273F(w,dh).))) J W(h-I)F(S(w),dh)
210
Chapter7: Theory or Estimalion)
But the latterisnothingelsethan the posterior risk of T with respect to A., and lossfunction (W(g-l.\302\273gI!G' Thus,in caseof structuremodels,we have
the
risk as a conditional risk without of the posterior obtainedan interpretation In this sense, with Bayesestimates any needof a priordistribution. generalized conditional risk functions. can viewed be asestimates respectto A., minimizing In the followingit is our aim to find estimators which are optimalwith risk functions. It will turn outthat thisisthe same to the unconditional respect risks.) classwhich minimizesthe conditional
of estimates). Remark (Decomposition We keep the notationof (39.21) Then we have for every bounded,\037-measurablefunction Theorem39.19. fQ-+1R \037(fldi)= Jf(hS)\037s(dh), gE G.
another Consider an estimate e E (E,G).Forevery SE im S we may construct s x estimate e':(h.B) e(hs.B).(h, B)e G a.It isclearthat (le fJl (E G),and
.
\037
\037
we have)
ePg = Je'P;Ps(ds),ge G.
Thisfollowsfrom)
'
I';
= JJ e(hs,B) (dh) Ps(ds) (B)Ps(ds) J (e 1':> = J e(en,B)Pg(dw) = (e\037) (B), BE91.)
.
39.22Definition.An estimate e E (jt (E,G) is equivarianriff e\037 = eg (eP)for every g E G.)
39.23Definition.An estimate e E [Jt (E,G) isa convolutionkernelif e(gen. gA) =
\037(en,
A)
for all en eO,g e G, A e (G).) \037
-
It is clearthat every convolutionkernel is equivariant.A non-randomized estimate K: 0 G is a convolution kernel iff K(gen)= g K(en),g E G, WE Q.)
39.24Remark (Decomposition of convolutionkernels).If e E at (E,G) is a
convolution kernel then we may write e(en,B) = e(T(w)Sew),B) = e(S(w),T-1 B), (l) E Q, BEtM(G). If we denote Q'(B):= u(s,B). SE imS, Be \037(G) then we have e(hs,B) = Q'(h-1 B), hE G, se imS, Be a (G).It followsthat in this case(with the notationof Theorem39.19).) \302\253(1)
(B)= JJ e(hs,B) J>;(dh)Ps(ds)
(e\037)
\037
HQ\037(h-lB)J>;(dh)Ps(ds)
=J
(I';.Q')(B)Ps(ds).)))
39.The slruclurc model
211)
Thisexplains the term \"convolution kerne)\".Conversely, eachstochastic kernel defines convolution k ernel a to .\037EimS, .n-+Q$I\037(G), according e(w,B) 1 =
QS(W)
{T- (w).B}.
The followingis an extension of the convolutiontheorem to structure
models.)
39.25Theorem.Suppo.\037e Ihal G i.'I amenable.Then for every equivariallle.'Ilikernel s Ho QSIfJI (G),s E im S,such mate e E 9t (E,G) there existsa stochaslic that) Q\037
=
J(p;.QS)Ps(ds),gE G.)
Proof For Q e (Jt (E,G),se im S,define Q\037:
BHo A.
1
Be a(G), ne\037, ,(K,,) H e(hs,hB)P;(dh\302\273).,(dg), K...)
where (K,,) is a summingsequence. Then it followsfrom Theorem38.14 that)
.
hm \"-o\037
1
sup
1
BI!!I(G)A, (Kit )
j (e$P;)(gB)
A.,
..)
(dg)
- (ps..Q:)(B) = 0,
S E im S. Let IE
denote
AltU'.
q\302\273
= \037 J II ) \"1
A,(K
and)
\"..)
S
[JJ/(g-lh) (e P;)(dh)cp(s)Ps(ds)];',(dg)
(F.Q:)(dh) q>(s)Ps(ds).)
B,.U:cp) = JJf(h)
of bilinear. functionsand by By weak compactness
Theorem6.11there is an
ne N, accumulation pointQ:sHoQ$I\037(G)of the sequence QII:sH>Q\037I&f(G), t x on ttoo(G) L (imS,Ps) and a for the topologyof pointwise convergence such that) s; N subsequence No lim A,,(f,cp) = lim B,.(f,cp)
\"eNo
liE
=
1\\:10)
H/(h)(PS.QJ (dh) q>(S)Ps(ds))
for allIE ttoo(G)and q> E V (imS,Ps).Sincee is equivariant we have P, fE\037oo(G},nEN,) A,,(f,1)=le and therefore) P = Hld(PS * QI)Ps(ds), fE '6'00 (G).)))
le
212
Chaptcr 7: Theory of Estimalion)
Since)
1= 1q P = JJ 1d(?* QS)Ps(ds) it foHows that Q:s Q\" can be adjustedPs.a.e.to be a non-degenerate kernel.This provesthe assertion. 0) stochastic \037
39.26Corollary.Assume that G is amenable.Then for every equivariamestimate there existsa convolutionkernel with the same distribution. Let W: G -+ [Ot (0)be a lower semicontinuous functiont let the lossfunction be (W(g-l. and definethe risk of an estimate l!E fJl (E,G) by \302\273g6G'
W(g-ly) e(wt dy) \037(dw), gE G.) It isclearthat the risk of an equivariantestimate isindependent ofg E G.If e is S a convolution kerneldefinedby Q:s\037 Q I.1l(G) then \037e\037:=H
\037e\037
= II W(g-ly)(\037 * QS)(dy) Ps(ds) = H f W(g- 1xy) E>;(dx)Q\"(dy)Ps(ds) = f If W(xy) PS(dx)QS(dy)Ps(ds) = IS W(y)(ps QS)(dy) Ps(ds).)
.
39.27CoroUary.Supposethat G is amenable.Then for
every estimate
kernel Q:s QS (G) such that e E (jt (Et G) there isa stochastic supU-;e\037 JJ W(y) (PS * QS)(dy) Ps(ds) \037
I \037
\037
gEG)
fUllctionw:) for every level-compact
copythe first part of the proofof Theorem39.24.Then we obtaina possiblydegenerate S E im 5t satisfying) kernel s lim A,,(J: 1)= Hfd(}>\"* Ps(d.\037), fE rrJoo(G).
ProofWe
may
\037
Q\037t
Q\037)
\"t;
No)
Let QS:=
Q1\037G)
measure Qbt whereQb(G)> Ot and QS an arbitrary probability
on \037(G),elsewhere. Then it followsthat inf fe\037
\037
gEG
for every
lim A,,(f,1)
\037
\"eNo)
ffd(ps
fe fto(G)+.Hence,we
fE \037b(G) supf()\037 g.G)
\037
.
QS)Ps(ds)
have for every
Jfd(ps.QS)Ps(ds).
The proofis finishedby Lemma 6.4.
0)))
function level-compact
39.The structure:
model
213)
of the lossfunction W. Note that Q is independent in as Section 3 8 we conclude that in caseof an amenable groupan Similarly We have to discuss the optimalequivariant estimateis a minimax estimate. how to find uniformlyoptimalequivariant estimates.) question
39.28Lemma, Supposethat there is a measurablefunction tp: im S -+ G such that for every y E G
J W(xrp(s\302\273 PS(dx) J W(xy) PS(dx) \037
Then for the convolution kernel K:
m.-T(m)
Ps-a.e.) tp
(S(w\302\273,
WE
fl, the following
assertions holdtrue: K is (1) uniformly optimalamongallconvolutionkernels. (2) If G is amenable then K is uniformly optimalamong all equivariant estimates andisa minimaxestimate.) Proof (1)The risk of K is J
W(K(W\302\273
P(dw)= II
W(Xlp(S\302\273
PS(dx)Ps(ds).)
From the assumption it followsthat K minimizesthe risk over all convolution
kernels.
of (1)and of Corollaries 39.26and 39.27. (2) This is a consequence
0)
It is rather obviousthat the estimates definedin Theorem39.24are (\037)!HGunbiasedif \037:=W(g-I.),g E G.) The final assertion of thissectionwill be that uniformly optimalequivariant estimates can be obtained as generalized for A,. In view of Bayes estimates Discussion 39.20(4) and (6),theseare just the estimatesminimizing the conditional risks.)
39.29Theorem.If a non-randomized generalized BayesestimateK with respect to ..t,is a convolutionkernelthen it is uniformly optimalamong allconvolution kernels.Hence,if G is amenable then K is uniformly optimalamong all and minimax.) equivarianrestimates
we obtainthat a dominated structure modelis ).,Proof FromLemma 39.12 = Discussion 39.20 shows that F integrable. \037. Now, let\"bea non-randomized, generalized Bayesestimate. By assumption, K is even a convolution kernel.We notethat for all WE and hE G)
!l
f
W(gT-t (w)h)
(dg) = fW(g-th)Pf\037\037(dg) = J W(g-th)F(w,dg).))) pS\302\253(O)
214
Chapter 7:Theory of
Estimation)
The defining propertyof generalized impliesthat for every Bayes estimates
heG)
PS(tAJ) (dg) J W(g TJ W(gh) p\"\037(tAJ)(dg) &'-a.e. SinceK is a convolutionkernel,we have T- 1(w) K(W) = where II' the condition of Lemma 39.28. satisfies This provesthe assertion. 0) t
\037
K(\302\243O\302\273
\302\253(0)
Ip(S(\302\243O\302\273
Let us consider someexamples. the caseof Example39.3. 39.30Example.Consider DenoteII = dd:. 11
2 = 11.11 and that F satisfies appropriateintegrability Thenit followsfrom Example37.6(1),that conditions.
(1)Supposethat K(X)=
W
- a) J.dda) xEH,)
f ah(x
fh(x\037a)l.dda)-' isa generalized and a convolution kernel.Thereforeit isoptimal Bayesestimate in the senseof Theorem39.29. If W is notboundedthen assume that F (2) Let P = NHand W subconvex. satisfies conditions. In view of Example 36.11 appropriateintegrability (1),a estimate isobtained generalized Bayes by minimizing) z'
W(z' - a) e-'2llpd.x)-aIiZ A.L(da). z' 1
L. we seethat for every x E H. From Anderson's Lemma (38.21) \037
f
E
is a a i t estimate and convolution kernel. Therefore is Bayes optimalin generalized the senseof Theorem39.29. K
= PL
39.31Example.Now we consider . the caseof Example39.4.Let II = d d\037
2 and (1)Supposethat W = 11.11 conditions. Thenit followsthat)
=---( - a ) JIh
F satisfies appropriateintegrability
X a ---1 \037d(1).,Jda)
----------.
fJ ah
K2(X)
I.H
that
(J
X
-
0-\"+3
1
XE
H,
(--(1 ) (;n+.jd(1).dda))
and a convolution is a generalized kernel.thereforeoptimalin Bayesestimate the senseof Theorem39.29. If W isnotboundedthen assumethat (2) Let P = NHandW besubconvex. Then a generali7.edBayes F satisfiesappropriateintegrability conditions.
isobtainedby minimizing))) estimate
40. Admissibility Z'
-a
1
of estimators
215)
1
2 Z'H H W ---;;--e-a}IIP1.(Z)-1I11 a,,+1ldda)dG, z'eL, ( )
for every x E H. From Lemma 38.21we see that the integralover L is minimized by of (1 > O. Hence\"2 = PI. is a generalizedHayes z' = PI.(x), independently and thereforeoptimalin the senseof Theorem39.29.) estimate
40.Admissibility of estimators) In this sectionwe discussbriefly somewell-knownfacts concerning admissibility of estimates. Let E = (Q,.r:I,{Pc:tEL})bea dominated structure modelwherethe operatwe denotethe a Euclidean is For convenience ing group space(L,<., of L on by + \".Let W: L [Ot 00) be a subconvex function.) operation
!l ..
.\302\273.
\037
40.1DefinitioD.A generalized for (W (t Bayesestimate a Pitman estimate for W.)
-.
\302\273,
ELandA L iscalled
40.2Examples.The followingis a particularcaseof Example 39.29.Let Q = and L = IR. Definethe operation by \037\"
t
+ W = (t + Wit
..., t
+ w,,),
t E IR,
WE
R\".)
is) (I) If W(t) = t 2, t E R, then the Pitman estimate
K(W)
=
Jt
If
(ll)
(jAH
dP
S
- I)dl
-, -dP
d).H)
(w
wEQ, tE IR.
t)dt
=
Itl, tE R, then any Pitman estimateIS a median of the hencesatisfying) distribution, posterior (2)
Wet)
a(w)
dP
(w
JI)di;, + 0() dP
- I)dt
-f -(w-t)dt d).H)
1
2'
wEQ.)
0()
-
If Wet) = 1 1(0.()(I), IE IRt then the Pitman estimates are calledmaximum probability estimates sincethey satisfy))) (3)
216
-
Chapter 7: Theory of
J
dlH
x\302\253(J))-c
(4)
IfP =
- t)dt= uR J-CJ dl'dP,- - t)dt, !1. s+c
dP
X((J)+(
Estimation)
\302\253(J)
SUp
-
!
(J) E
\302\253(1)
H)
N R \" then in any
caseK(W) = n
i:.
i-t)
Wi>
WE
a, isa Pitman estimate.
40.3Remark. Extending the precedingexample,supposethat Q = H, If P = NH then (H,< ., a Euclideanspaceand L H is linearsubspace. of K = PL within the classof equivariant Theorem38.28provesadmissibility estimates and (takef= idL).Thisresultcouldbeextendedto P * NH if \037
.\302\273
-
:\037
of Corollary38.23,usingthe decomposition of assumptions But evenfor the Gaussian caseP = NH it isknown that K = PL Theorem39.19. isinadmissible assoonasdimL > 2,(Jamesand within the classofallestimates W
satisfy the
Stein [1960]). We are interested in this sectionin admissibility within the classof all estimates. In view of the precedingremark we confineourselvesto the case = dimL 1,i.e.L = lit)
40.4Definition. An estimate if for any further (l e .t1t (E,R) isalmo.rtadmissible (1E rJi (E,IR) the inequalities) estimate J.v,0'J; J.v,eJ;, tE!R,) \037
imply)
=
J.v,(1p,
all t E R.) 0P, for ).-almost
J.v,
40.5Lemma. Let W(t) = t 2, t E R. Supposethat (l E fJl (E, isalmostadmissibleand satisfies H (x t)2e(W, dx) P'(dw)< 00, t E IR. and uniquely n01lrandomized Then it followsthat (l E 9t (E,R), (l is admissible, determinedby itsriskfunction.) \037)
-
-
momentsare finite it followsthat e(.,{+ 00, co})= 0 ProofSincethe second (E R. Hence(!E 91(E,IR). Then p'-a.e.,
-
J (Jlx \037
e(w,
P'(dw) H (x 1)2e(w,dx) (w) < 00, tl
dX\302\2732
P,
that implies Jlxlu(w,dx)< co p'-a.e.,IE IR.) estimator))) Hence,the non-randomized
t E IR,
40. Admissibility
of tstimators
217)
,,=Jxe{.,dx) is well-defined. yield Easy computations -- W; K .p, = SUX2(l(W,dx) (f X(l(w,
-
W; Q.p,
P'(dw), t E\037,)
dX\302\2732J
we arrive at) and,since(l isalmostadmissible
p'-a.e.,tEfR.) Jx2Q{.,dx)-<JxQ(.,dx\302\2732=O This means,that Q is non-randomized sinceit coincides with K. Now, let (1E .rJt (E, be an estimator satisfying) \037)
W;O'.P,\037W;K'p'. tEIR.)
Then a is almostadmissible, too,and can be replacedby a nonrandomized estimator \"1:Q R. It followsthat
-
-
I(K1 -/)2d'p'= J
J
K1
(
+K 2
-I)
-.-!-.f
2
dPr + J
=
K
(
I
2 f (\"t
K
2 I
) dp'
- I) d'p'+ 2 s 2
I
(K
t)
2
dp',)
IEI1i.)
This implies of K and K l' that by admissibility S (K 1
whenceK 1 =
-,,)2dp,= 0, ).-a.e.,) K
Ie IR. 'p'-a.e.,
0)
The followingassertion containsthe basicideaof Blyth'smethodfor proving
-
Let h(t) = s(lt I).IE IR, be a positiveand continuous admissibility. probability such that s: i s w e Later, shallput density [0,00) (0,00) decreasing. h(t)
\037
n(1
\037
/2)
' IE R,
and h(/) =
vk
exr
( ), \037
40.6Theorem.Lei(!be the Pitmanestimatefor W If lim (J J.v,eP,' h(8t)dt inf J J.v,a P,'h(a)dt)= O. ,-0
-
tI (\037(E,R))
then Q is a/most admissible.)))
IE\037.)
21H
Chapter 7: Theory of
Estimation)
ProofAssumethat (1E tit (E,A) satisfies)
tER.
J.Y,G.p,\037 J.Y,u.P\"
If (!, denotesthe Bayes estimatefor Wand the prior distribution Jl\" : t 1-+th (tI), 1E R, then we have
\037'
J J.Y,eePrJl.(dt)J J\302\245,a.P,Jl.(dt) J J.Y,e\037Jl.(dt). \037
\037
the sets) Consider Ma.h =
{III< a, J.Y,eP,> J.Y,u P, + b},
Then)
a> 0, b > O.)
-
J J.Y,qP,JlI(d/) J J.Y,C!tPrJle(d/) \037
J.Y,(1.P,)Jle(dl) f(J.Y,e.P,-
\037
bJ..(MII,b)
\302\243h(\302\243a).
The assumption then implies that
l(Ma.b) = O. Hencethe assertion.
0)
Let us consider 40.7 Discussion the caseof Wet) = 12, tE lit If (Stein[1959]). and has a finite second is) moment,then the Pitman estimate PI\037l iscentered dP
K(x)=Jtdl (x-t)dt=x-JzP(dz)=x,xEIR.)
The Bayesestimates K. for the priorsJl.are given by) KC(X) =
J If;(x,dt), x E
IR,)
the posterior of E and /J.e' Thisgives) where f; denotes distribution J J\302\245,KPrh(\302\243/)dt- J J.Y,Kc.P,h(et)dt
- (dx)Jl. - -t1 JI (x) - (dx) 1 -2t(x= - H (x t 1 - 2/(x- f;(x,dt)Ps(dx)/J.,(ds) = - HI (x - \":(X) t 1 - - (x = ./J..(ds) t II (x = -1H (x P'(dx)/J.c(ds). = 1 JJ (x t
-
1)2.P,
2
(dr)
K;(X)
(K.
t)2 .P,
Ke(x\302\273)'p'(dx)/J.l(dt)
2
K.(X\302\273)
2
K;(X) 2K,(X)
Ke
(X\302\273
t)))
2
K,(X\302\273
\037(dx)
/J.,(dt)
of estimators
40. Admissibility
Since)
219)
..
-I)dP d).(x I)h(el)dl X-K\302\243(X)=J(X-I)\037(x,dl)= ---iP----.J (x -I)h(el)dl) J (x
.
d).
we obtain
J I-\302\245,KP'h(tl)dl-J I-\302\245,K,P,h(et)dl =H
- t) dP (x - l)h(t/)dl dP dJ. -------n. ) Jl (x - t)h(tt)dtdx dj>- J (x t)h(r.I)dt
J (x
2
-
(
fiX
h(t(z-
= J ($zh(e(z f
p(dZ\302\2732
x\302\273
P(dz))
x\302\273
dx.
in a suitable choiceof h such that the last Thus,an admissibility proofconsists tendsto zerofor t O.) expression
-.
Let Wet) = t 2, t E [R, and PI\037I such that 40.8Theorem(Stein [1959]). J z P(dz)= 0, J Z2 P(dz)< 00, Then the Pitman eSlimateK = idR is admissible and uniquely delermined by itsriskfunction.) the preceding discussion and notethat) Proof We continue
-
- -
= (Jz(h(e(z h(tx\302\273 (J zh(t(z h(tX\302\2732 P(dz).) J Z2 P(dz)'J (h(t(z x\302\273
p(dZ\302\2732
\037
- -
x\302\273
p(dZ\302\2732
x\302\273
for t Moreover, J
h(\302\243(z
\037
-
1 we have)
x\302\273
P(dz) P( \037
-1,+ 1).)
-I
h(tx+y).
inf \037)I'\037
1)
Now, we specifyh(x):=\037.' _l_-i'XE lit Since 1t
1+x
1+ (z = 1, 1+ +21)2 z) z-\037 lim
there issomec> 0 such that) inf
h(z+y) c'h(z), ZEUi. \037
-1\037)';:;1)
Hence,we may
estimate)))
220
Chapter 7: Theory of Estimation)
-
(J zh(r.(z dx J P(dz) J Const.J Z2 P(dz). JJ (h(t(z x)
-
h(\302\243(z
p(dZ\302\2732
x\302\273
- -
x\302\273
\037
h(tx\302\2732
/ h (tx)dxP(dz).)
we have) Moreover,
- -
J (h(e(z x) = Jh 2
h(ex\302\2732
(\302\243(z
h
x\302\273j
/ h(ex)dx
2 (ex) dx ----J
= J \"--
h(r.(x+
-
-
h(f;x)dx 2J h(f;(Z
x\302\273dx
+ J h(f;X)dx
h(\302\243x)dx
z\302\273
1 1 1 1+ e2 (x + Z)2 2 2 =it (1--t-e)y.dx-\037S-f+\037xidx 1 1 xz + Z2 dx = =1 dx = J 2 \"2-') Z2,t (1+ e2 2)2 n (1+ X )2 \"x
;
\302\2432
2\302\2432
\302\2432
\302\2432
Thus,we obtain) J
(J
zh(\302\243(z
-
h(e(z_
x\302\273
x\302\273
z.)
dx = e.Const.(J z 2 P(d
p(dZ\302\2732
P(dz)
Thisprovesthe assertion. we obtaina As a consequence
\302\243Z2
2
\302\273
0)
model.) generalresultfor an arbitrary structure
40.9Theorem(Stein[1959]).Let E = (fl,d, {p,:t E R}) be a structure model convolution kernel 10:Q R, and assume that there existsa non-randomized for W (t) = t 2 . t C IR, whichhas afinile fourth moment.Then the Pitmanestimate and uniquely determinedby itsriskfunction.) is admissible
-
Proof Let So be the maximal invariant basedon and F the posterior is (by Discussion for Then the Pitman estimate distribution dz) F(So(oo), dz) = J (z + T{oo)= J zF{oo, = 10(00) Po (lOld) (00). = J z\037o(W)(dz) + 10(00)
l.
\037
(39.20\302\273)
-
1O{oo\302\273
j
It followsthat T has a finite fourth moment,too. Let S be the maximalinvariant basedon T and denoteby p's,t E IR, s E im S, of T given S = s. We have to show that) distribution the conditional
-
-
lim (H(T 1)2dP,h(tt)dt H (K,
,-.0)
-
= t)2 dP,h(tt)dt)
for the prior#l\" where K, is the Bayesestimator
0,
d;.!(x) = eh(ex),x e IR.
We)))
40. Admissibility have)
-
of cstimalors
221)
-
H (T 1)2dp,h(8t)dl= J H (z t)2P'$(dz)h(t;t)dtPs(ds),) and)
H (Kt
-
= J H (Kt(Z + S) t)2 dP,h(r.t)dt
-
t)2 P,s(dz)h (tt)dtPs(ds).)
z H Ke(Z + .\\'), z E R is the It isclearthat for every fixeds E im S the estimator for the conditional t E R}) and Pl' experimentEs = (R,\037, {P'$: Bayesestimator 40.7to each of theseexperiments, it remainsto show Thus,applyingDiscussion that)
dx Ps(ds)- 0. PS(dz) J h(t;(z 1 1 and considering the proofof Theorem Specifying (x) = . l+x2' lim JJ
\037J
\037
-=
\037\037(\037
l\"'O
\037\302\273
\037$\037d\037\302\2732
x\302\273
h
X
E
IR,
7t
(40.8)it sufficesto provethat Ps(ds)< 00.) J (J Z2 But this is an immediateconsequence of 4 H Z4 PS(dz)Ps(ds)= J T dP < 00. P'(dl\302\2732
0)
casewhere Q = andP = Vo,,' In this case,the Now, we turn to the special result of Theorem40.8can beextendedto a considerably larger classof loss \037
functions.)
40.10Theorem(Blyth [1951]). LetE = (R, a, {VI.,: t E \037}),f= that W is separating and offinite order.Then the identity uniquely determinedby itsriskfunction.)
andassume is admissible and id\037
ProofThe proofis dividedintothree parts. of Theorem40.6is (1) In the first part we shallshow that the assumption 2 . = satisfiedfor h (x) 1/V2n exp ( x /2),x E Sincein the prescntcasethe we have to show that) identity is the Pitman estimator
-
\037.
-
= OCt). inf J U';Qp, v o.1!c(dt) J W(ltl)vo,ddt)(IE 91(F.,/II))
To compute inf
Vo , IfL (dt) J U';e.P,
(l'\037(E.A))
we consider the mixtures) Pr(A x B):=J B)))
V\"
I (A) Vo. \037_
(dt),
A
E
g4(R),
BE91(R),
Chapter 7: Theory of &timation)
222 F.
> O. The marginal distribution for the first coordinate is Jlt(A x R) = VO,I-t/t(A),
and desintegration of with J..lc
J..lc(A x
A
E
\302\243i(\037),)
yields) respectto the first coordinate
B) = J \037(x, B) J..lc(dx, R),
A E
.sf(R), BE .sf(\037),
A)
is given by) wherethe posterior distribution
\037(x,
B) =
JB f
dV,.1
d)' (x) Vo.t/l(dt) = v
dVr 1
di
it
X
E
IR,
(x) VO.1/c(dt))
\302\243i(IR).
(1+c).I/(1 +c)(B),
_
x.-.1x+t) , x E
The estimates Ke:
BE
xf
-
IR, t;
> 0, minimize
SI-+J W(ls tl)f;(x,de), SE R,) for every x E R and therefore inf
J J.\302\245,l!p'Vo,t/c(dt)
QE \037(E,A))
= J W;Kc1;vo.1It(dt)
= II
I: w( ) .0.1
=J
W
I
(dx).0.1{,(dl)
\302\243
x
( }I1+t)) vo..
Integrationby partsyields
-J ( = 2 J (4)( + - + 2q/l + - 1) J a'
VO. 1 (dt) J W(111)
f. 1 V\037.
W
!X)
4>(
+
) Vo,t
a}l1+t\302\273
W(d!X)
[O.a\302\273)
\037
f.
(O.a>
cp(+a) W(da) =
))
-
sincex> y > 0 impliesIcl>(x) identity isalmostadmissible.)))
cl>(y)j
\037
Ix
-
YI
O(e)
cp(y). This provesthat the
40. Admissibility
of estimators
223)
In the be such that supJY,eoP,\037JW(ltl)vo,,(dt). C?oE9t(E,R) fER = 0 A-a.e. secondpart of the proofwe shallshow that eo(x,{oo}) Thecasewhere W isunbounded isobvious. that W isbounded. Let Suppose = = 0> 0 be arbitrary and defineA {Oo(.,{oo})> c5}.We provethat A(A) O. For every x E A we obtain) (2) Let
JJ W (Iy ;;;;
-II)
+c),l/U+cl (dt) Qo(x,dy)
v x/U
.IW(co)+ (I
-
D)f
-
c5)J W
= 0 W(oo) + (1
w(
i:-.-I )
(dt) V'/lI+.\"1/1\"')
III
(V1+E) vo,ddl) ,)
which implies, again for x E A,)
III
JW HW(ly-ll)vx/U+C),l/U+c)(dl)Qo(x,dy)(VI .t't:) vo..