A Bayesian Alternative to Parametric Hypothesis Testing

61

Test (1992) Vol. 1, No. 1, pp. 61-67

A Bayesian Alternative to Parametric Hypothesis Testing RAUL RUEDA

IIMAS, UNAM Apdo. 20726-20, 01000 Mexico DE Mexico SUMMARY A unified approach to parametric hypothesis testing from a decision-theoretical viewpoint is proposed. A measure of the discrepancy between two models is incorporated as part of the utility limction. Specifically, the Kullback logarithmic divergence is considered as such a measure. Under certain conditions, when using this measure of discrepancy there is correspondence with the classical solutions.

Keywords:

HYPOTHESIS TESTING; LOGARIqlIMIC DIVERGENCE; EXPONENTIAL FAMILIES.

1. INTRODUCTION Let X be a random variable with density function p(z[0, ~), where the values of 0 and ~ are unknown. Suppose there exists a parametric family 7 = {p(xlO,~o) : ~, c ,~,0 c c-),~ c f~} containing the true distribution of X. Let Z = {X1, X 2 , . . . , X,~} be a random sample from this distibution of X and suppose the following hypotheses are to be tested:

H0 : 0 c (-)0 vs. H~ : 0 c (5)1, ((-)0 n 691 = 0).

(1)

This problem has been widely studied in the Bayesian framework. The proposed solutions can be roughly divided into two categories: Received April 92; Revised July 92.

Raul Rueda

62

those based on pure inferential arguments, using probability intervals of odds between the hypotheses' posterior probabilities (e.g., Lindley, 1965, 1972; Dickey and Lientz, 1970; Berger and Selke, 1987; Johnson and Traux, 1978, 1987; Berger and Delampady, 1987, among others) and those using a decision-theoretical approach (e.g., Bernardo, 1985; DeGroot, 1970; Zellner, 1971). We note that the majority of these solutions depend upon the dimensionality of each hypothesis. In Section 2, an alternative setting for hypothesis testing, that allows a unified treatment, is given by a decision problem and the solution obtained. Following Bernardo (1985), the Kullback-Leibler logarithmic divergence measure (Kullback and Leibler, 1951) is incorporated as part of the utility t\mction, with a slight modification. In Section 3, it is shown that, for exponential families, this procedure, in some sense, reproduces the classical solutions. Finally, in Section 4, some additional comments are given and possible extensions are suggested. 2. A GENERAL PROCEDURE

2.1. The Problem From a decision-theoretical point of view, the situation stated in (1) may be described as a decision problem, where the decision space D = {do, d1} contains only two actions, namely do (accept/7o) and dl (reject H0), and where the consequences of either action depend upon the tree, but unknown, value of the parameters (0, co). On the other hand, choosing the action do means that the true distribution of X is approximated by a member of ?0 = {P(zlOo, co) C .5" : 00 C O0}, with a similar interpretation for dl. Hence, it is natural to propose, as part of the utility function, a measure of the discrepancy between the two models. Denote by u(d,:, O, co) the utility l'unction for di and define

O, co) =

co;Od +

(2)

with A C ~+, B C ~, and 5(0, co; 0i) a measure of the discrepancy between the true model and a model corresponding to the hypothesis H/(i = O, 1). If Oi(i = O, 1) contains only one element, the maximization of the expected utility leads to the decision to reject tI0 if and only if

Bayesian Hypotheses Testing

63

Eo,~lz[5(O, co;01)-5(O, co;Oo)] <

B1 - B0

A

(3)

When the hypothesis is composite, the usual procedure is to estimate 0i in the con'esponding subset (-)i defined by the hypothesis Hi and use this estimator, say 0i, instead of Oi in (3). Because the final goal is to decide between the two hypotheses in (1), it is convenient to assume the utility function for the estimation problem to be u(0,co,0) = - 5 ( 0 , co;0), (4) so the estimator of 0 restricted to (-)i, say 0", is such that =

co;bd)

c

(5)

In this way, a general decision rule for (1) rejects H0 if and only if

Eo,,~lz(5(O, co. 0~) - 5(0,co. 0~)] <

B1 - B0 A '

(6)

where 0*(i = 0, 1) are the solutions of (5). The use of 5(0, co; Oi) as part of the utility t\mction for testing a sharp hypothesis was proposed by Bernardo (1985), based on work by Fem~ndiz (see FerrSndiz, 1985). In the present approach, (2) is used as the utility function which may be compared with Bemardo's proposal for using 5(0, co; Oi) as a function of the difference in utilities. As a final comment, note that it is not assumed that 6)o tO 601 = 6). The problem was stated as one of deciding whether 0 belongs either to 6)o or to 6)1, but the uncertainty is still represented by 6) (and f~, if the existence of a nuisance parameter is allowed) and not only by 6)o tO 6)1. This situation seems to be morn realistic and, of course, the case when the union is 6) can be handled by this approach. 2.2. Logarithmic divergence as discrepancy function As mentioned above, the problem of paramenic hypothesis testing can be viewed as one in which it is desired to select the family that contains the best approximation to the true model. There are many references

64

RaM Rueda

in the literature proposing a specific fO1Tn for the discrepancy function (e.g., Akaike, 1980; BayaM, 1984; Bernardo, 1979, 1985, 1987; Kullback, 1959; Stone, 1959). Following Bernardo (1985), let

p(zlO, co)

,s(o,~o;0*)=

p(a.10,~o)logp(xl0.,co) d:r for

O*

C O.

(7)

It is readily seen that, using this discrepancy function, the decision rule obtained in (6) is to reject _rio if and only if Eo,wl z

[i

p(zlO, co) iog p(a:lb;'c~ dz

I

<

(8)

with 0* the coGesponding solution of (5), (i = 0, 1). This decision rule does not coincide with that given by Bernardo (1985). 3. EXPONENTIAL FAMILIES As an important application, the situation in which S" is a regular exponential family (e.g., Barndorff-Nielsen, 1978) is now considered. Suppose them is no nuisance parameter and that the prior distribution belongs to the corresponding conjugate l'amily. Let X be a random variable whose density belongs to

~-= {p(zl0)

= a(O)b(z) exp{Ot(z)} : 0 E 0 } ,

where O = {0 c ~ : a(O) C N+} is a non empty open subset in ~. Suppose that

p(o) ~ a(0)"o exl:,{0t0}. Let X1, X 2 , . . . , Xn be a random sample from the distribution of X and let the hypotheses be /to : 0 C (-)o vs. H 1 : 0 C 01,

(10)

where Oo and 01 are disjoint subsets of O. It can be shown (Guti6rl'ez, 1992) that

E~176176

-

0 logH(nl,tl) a,~--7 1

(11)

Bayesian Hypotheses Testing

65

where H(n, t) is the proportionality constant in (9), nz = no + n and tl = to + Y~/i~=lt(Xi). So (8) can be written as _ , -, tl log a(O~) + ( ~ _ 0 ~ ) - - <

a(O;)

nl

B1

-

Bo

A

(12) '

with 0~, 0~ being the respective solutions of (5) and, as call easily be shown, also being the corresponding solutions to the equations

---~-~loga(Oi) 00i

= t--2- (i = O,Z) 7%1

(13)

in the sets Oo and (-)1 respectively. So the solutions of (5) in each subset of O specified by the hypotheses fI0 and H1 are, if they exist, the modes of the posterior distributions restricted to @i, i = 0, 1. That is, the 0* that maximizes (11) is the mode of the posterior restricted to Oi (if one exists, the other exists too). If (10) is tested using the generalized likelihood ratio criterion, it is easy to see that the critical region is of the form

a(01) log a(O0)

+

- Oo)g*

i=1

t(xi) <

(,4)

where the 0i are the maximum likelihood estimators restricted to Oi(i = 0, 1) and K is a constant. Observe that 0* =/)i if no = to = 0; that is, when a limiting, possibly 'non-infon-native' prior for 0 is used (e.g., DeGroot, 1970). Note also that the decision rule (12) is similar to the classical critical region given in (14). The extension to higher dimensions is straightforward. 4. CONCLUDING REMARKS The procedure presented in Section 2.1 provides a unified treatment of parametric hypothesis testing that reproduces, as particular cases, most solutions existing in the literature. Moreover, it is straightforward to perfo~Tn a reference analysis in the particular case of exponential families with this approach (see Section 3). An interesting feature is that, regardless of the form of the hypotheses being tested, the relevant uncertaimy is @ and not only (-)0 U (:-)~. This approach can easily be

66

Raul Rueda

extended to the case in which there are more than two hypotheses. It is sufficient to evaluate the expected discrepancy between the "representative model" within each hypothesis and choose the one whose discrepancy with respect to the true model is smallest. The similarity with the classical results, when the logarithmic divergence is used, is appealing. Of course, other discrepancy l\mctions may be used, including different functions for each one of the decision problems that have been considered here: estirnation and hypothesis testing. It is important to emphasize that, even though an estimation problem is mentioned in (5), the goal of the proposed procedure is to solve the hypothesis testing problem (1) and that (5) is addressed mainly through necessity. ACKNOWLEDGEMENTS The author wishes to thank E. Gutid~Tez for several discussions which resulted in an improved version of the manuscript, and the referees for their valuable comments. REFERENCES Akaike, H. (1980). The interpretation of improper prior distribution as limits of data dependent proper prior distribution. ,L Roy. Statist. Soc. 42, 46-52. Barndorff-Nielsen, O. (1978). Information and Exponential Families in Statistical Theory. Chichester: Wiley. Bayarri, M. J. (1984). Contraste Bayesiano de Modelos Probabil[sticos. Ph.D. Thesis, Universidad de Valencia. Berger, J. O. and Delampady, M. (1987). Testing precise hypotheses. Statist. Sci. 2, 317-352, (with discussion). Berger, J. O. and Sellke, T. (1987). Testing a point null hypothesis: the irreconcilability of ?) values and evidence../. Amel: Statist. Assoc. 82, 112-122, (with discussion). BernarClo, J. M. (1979). Expected inlbrmation as expected utility. Ann. Math. Statist. 7, 686-690. Bernardo, J. M. (1985). An(flisis bayesiano de los contrastes de hip6tesis param6tricos. Trab. Estadist. 36, 45-54. Bernardo, J. M. (1987). Approximations in statistics from a decision-theoretical viewpoint. Probability and Bayesian Statistics (R. Viertl, ed.), London: Plenum Press, 53--60. DeGroot, M. II. (1970). Optimal Statistical Decisions. New York: McGraw-Hill.

B a y e s i a n H y p o t h e s e s Testing

67

Dickey, J. M. and Lientz, B. R (1970). The weighted likelihood ratio, sh,'u'p hypotheses about chances, the order of a markov chain. Ann. Math. Statist. 41, 214-226. Ferr~indiz, J. M. (1985). Bayesian inference on Mahalanobis distance. Bayesian Statistics 2 (J. M. Bernardo, M. H. DeGroot, D. V. Lindley and A. F. M. Smith, eds.), Amsterdam: North-Holland, 645-654. Gutidrrez, E. (1992). Expected logarithmic divergence for exponential families. Bayesian Statistics 4 (J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith, eds.), Oxford: University Press, 669-674. Johnson, B. R. and Traux, D. R. (1978). Asymptotic behavior of Bayes procedures for testing simple hypotheses in multiparameter exponential families. Ann. StatisL 6, 346-361. ! Johnson, B. R. and Traux,/\D. R. (1987). Asymptotic properties of Bayes risk for one-sided tests. Canadian ,L Statist. 15, 53-61. Kullback, S. (1959). Information Theol 3, and Statistics. New York: Wiley. Kullback, S. and Leibler, R. A. (1951). On information and sufficiency. Ann. Math. Statist. 22, 79-86. Lindley, D. V. (1965). Introduction to Probability and Statistics. Part 2: Inference. Cambridge: University Press. Lindley, D. V. (1972). Bayesian Statistics: A Review. Philadelphia, PA: SIAM. Stone, M. (1959). Application of a measure of information to the design and comparison of regression experiments. Ann. Math. Statist. 30, 55-70. Zellner, A. (1971). An Introduction to Bayesian Inference in Econometrics. New York: Wiley.