Joint Statistical Papers of Akahira and Takeuchi

OINI^ STATISTI C AL RS AKAH IRK and

TAKEUCHI

This page is intentionally left blank

5281/tp/path (Converted)-2 7/9/08 3:38 PM Page 1

Composite

Published by World Scientific Publishing Co. Pte. Ltd. 5 Toh Tuck Link , Singapore 596224 USA office: Suite 202, 1060 Main Street, River Edge, NJ 07661 UK office: 57 Shelton Street, Covent Garden, London WC2H 9HE

British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library.

The editors and publisher would like to thank the following organizations and publishers of the various journals and books for their assistance and permission to reproduce the selected reprints found in this volume: Institute of Mathematical Statistics The University of Electro-Communications Springer-Verlag The Institute of Statistical Mathematics Union of Japanese Scientists and Engineers Blackwell Publishing Asia Oldenbourg Wissenschaftsverlag GmbH Physica-Verlag

Universit6 Pierre et Marie Curie University degli Studi di Roma VSP International Science Publishers Blackwell Science Ltd Marcel Dekker

While every effort has been made to contact the publishers of reprinted papers prior to publication , we have not been successful in some cases . Where we could not contact the publishers, we have acknowledged the source of the material. Proper credit will be accorded to these publications in future editions of this work after permission is granted.

JOINT STATISTICAL PAPERS OF AKAHIRA AND TAKEUCHI Copyright 0 2003 by World Scientific Publishing Co. Pte. Ltd. All rights reserved This book, or parts thereof may not be reproduced in any form or by any means, electronic or mechanical, including photocopying, recording or any information storage and retrieval system now known or to be invented, without written permission from the Publisher.

For photocopying of material in this volume , please pay a copying fee through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers , MA 01923 , USA. In this case permission to photocopy is not required from the publisher.

ISBN 981 -238-377-8

Printed in Singapore.

V

Foreword Collaborative work, virtually inevitable in experimental and observational work, is increasingly common in the mathematical sciences too. There can, however, be few collaborations that have lasted so long and produced such an impressive and coherent body of work as that of Professors Akahira and Takeuchi celebrated in the present volume. In the frequentist approach to statistical inference problems with an "exact" solution are important but essentially restricted to certain exponential family questions and to issues involving a transformational family, such as the scale and location model. This means that in more complex and realistic cases approximations cannot be avoided. The approach almost universally adopted is to appeal to asymptotic arguments. It is supposed that the information available is large enough to justify, for example, local linearization of nonlinear dependencies and the invoking of the classical limit laws of probability theory. In more formal terms, the problem under study is imbedded in a sequence of problems in which the sample size, or more generally a measure of information, tends to infinity. Theorems based on the laws of large numbers and, typically, the Central Limit Theorem can then be established about the limiting behaviour of estimation, testing and prediction procedures. These results justify, in some sense , many widely used techniques such as maximum likelihood estimators and associated testing procedures, such as likelihood ratio tests. There are broadly two difficulties with this approach. The passage to the limit is a fiction. Detailed results have to be applied for one or more specific sample sizes. Secondly, there are many different procedures that are equivalent in the limiting sense. While the different procedures will often give virtually the same answer in applications, this is not necessarily so and in any case some basis of choice between such alternative procedures is clearly desirable. The first point is in principle best addressed by finding bounds on the approximations involved, broadly analogous to the Berry-Esseen bound associated with the Central Limit Theorem. While there is some fascinating work of this kind, stemming, for example, from the use of Stein's method, useful results are extremely limited in scope. Instead higher-order terms in asymptotic expansions can be exploited in some generality and this is the approach taken in the present work. Of course the issue still remains of the sample sizes for which the results give good approximations. The second issue of the choice between estimators and test statistics equivalent to the first order of asymptotic theory is subtler and needs very careful formulation. Professors Akahira and Takeuchi have made important contributions to both aspects; I find their results on the second aspect particularly striking. Their broad approach stems from the Neyman-Pearson viewpoint emphasizing properties achieved in hypothetical repeated sampling. As such it is to be contrasted with studies proceeding from what might be called the Fisherian standpoint with its explicit emphasis on likelihood, sufficiency and conditioning. A detailed comparison of these approaches, and of both with Bayesian discussions , would probably be fruitful. This is not the place to discuss or single out individual papers in the volume, the authors having in any case provided a valuable summary and introduction. The careful discussion of nonregulax problems is, however, particularly to be noted.

Vi

Let me conclude by congratulating Professors Akahira and Takeuchi on their achievements; long may their joint work continue!

D. R. Cox Nuffield College Oxford

UK January 2003

vii

Introduction This volume consists of 44 joint papers on statistical inference for about a quarter of a century from 1975 to 2001. In the latter half of 1970's, the higher order asymptotics was extensively developed by many people including Prof. J. Pfanzagl's group, Prof. J. K. Ghosh's one and ourselves. In particular, the second and third order asymptotic efficiencies of estimators are discussed using the Edgeworth expansion under suitable regularity conditions in the joint papers. In cases when the regularity conditions do not hold, there are less well known "surprising" results such as the existence of zero variance unbiasdd estimators based on a sample of fixed size. The order of consistency and asymptotic efficiency of estimators are also strongly influenced from the conditions of nonregularity. Various non-regular cases are discussed in the joint papers. The papers on both of the higher order asymptotics and non-regular estimation formed two monographs of Akahira and Takeuchi (1981, 1995). Other papers are related to the problem of prediction sufficiency, the Edgeworth type expansions and miscellaneous topics. More precisely following subjects and their implications are studied in the joint papers in this volume.

1. Prediction sufficiency The concept of prediction sufficiency (adequacy) was defined and discussed by, among others, Skibinsky (1967), [1], and Torgersen (1977). It was shown by Bahadur (1955) that sufficiency as defined in terms of conditional expectations, under regularity conditions, implies "real sufficiency," i.e. sufficiency in terms of risk functions. In the paper [1], the converse is shown to hold provided the loss function depends on the unknown parameter, but no longer true if the loss function is independent of the unknown parameter. In the latter case, conditional independence still holds but ordinary sufficiency is not required. In the paper [15], we show the relation between prediction sufficiency and the sufficiency in terms of conditional expectations.

2. Edgeworth type expansions In the papers [3] and [6], we give the Edgeworth type expansions of the distribution of the sum of independent and identically distributed (i.i.d.) random variables without higher order moments. For special cases of t-type density, we obtain the exact formulas for the asymptotic expansion of the density, and the results are generalized to the case when the density is approximated by a rational function. In [4], it is also shown that the asymptotic distribution for cases without finite variance are stable laws with fractional characteristic exponents,

viii

and the asymptotic expansion of the density for some case is given . In the paper [ 7], the above expansion is extended to the multidimensional case.

3. Second order asymptotic efficiency In the papers [2] and [5], the second order asymptotic efficiency of asymptotically efficient estimator was discussed for one parameter and multiparameter cases. Under suitable regularity conditions, the maximum likelihood estimator (MLE) adjusted to be second order asymptotically median unbiased (AMU) is shown to be second order asymptotically efficient. And it is shown that the first order efficiency implies the second order efficiency , which phenomenon was also found by Pfanzagl ( 1979). The papers [9] and [13] demonstrate the second order efficiency of the generalized Bayes estimator ( GBE). In [12], we define the discretized likelihood estimator (DLE) as a solution of the discretized likelihood equation, and show that the MLE as the limit of DLE is second order asymptotically efficient. Thus the structure of the higher order asymptotics is clarified by considering the DLE. In the paper [28], from the viewpoint of the decision -theoretic approach, we get the second order asymptotic completeness of the MLE, and, in [10], discuss the problem of the second order asymptotic efficiency of the confidence intervals, that is, consider the asymptotic power of confidence interval up to the order n-1/2 in the neighborhood of the true value of the parameter, where n is a size of sample. In the papers [27] and [34], we obtain the Bhattacharyya type bound for the asymptotic variance of sequential estimation procedures under suitable regularity conditions , and show that the modified sequential maximum likelihood estimation procedure attains the bound if the stopping rule is properly determined , which can not be uniformly attained if the size of sample is fixed.

4. Third order asymptotic efficiency Already three-quater century ago, R. A. Fisher (1925) suggested that the MLE was asymptotically best in the class of all asymptotically efficient estimators. He guessed that the MLE was asymptotically best in the sense that the MLE had the asymptotically minimum loss of information in the class, and calculated the asymptotic value of the loss of information of the MLE and minimum chi-square estimator in the multinomial distribution . Later, C. R. Rao (1961 ) corrected Fisher 's calculation and defined the second order efficiency of the MLE, which corresponds to the third order asymptotic efficiency, in our terminology. However , it was not clear what the minimum of loss of information meant, and comparison of asymptotic variance or mean squared error up to the second order failed to establish uniform superiority of the MLE or any other estimators. This impasse was broken through by introducing the median-bias

ix

correction and considering the concentration probability of the estimator around the true value up to the order n-1 (see also [5], [8], Pfanzagl and Wefelmeyer (1985), Ghosh (1994)). In the paper [8], it is shown that the MLE adjusted to be third order AMU is third order asymptotically efficient for a multiparameter exponential family of distributions. Similar results were obtained by Pfanzagl and Wefelmeyer (1978) and Ghosh, Sinha and Wieand (1980). In the papers [9] and [13] we show that, for any symmetric loss function, the GBE is third order asymptotically efficient in a restricted class D of third order AMU estimators for general one-parameter and multiparameter cases. In [16] and [28] it is seen that, for any class of estimators which admit Edgeworth expansions but are not necessarily AMU, we get the third order asymptotic completeness (or sufficiency) of the MLE together with the second order derivative of the log-likelihood function evaluated at the MLE. In [17], the concept of the asymptotic deficiency of Hodges and Lehmann (1970) is extended to the case when a common parameter is estimated from m sets of independent samples of each size n, and the asymptotic deficiencies of some asymptotically efficient estimators relative to the MLE based on the pooled sample are discussed in the presence of nuisance parameters (see also Akahira (1986)). Further, under suitable regularity conditions, the third order asymptotic bound for the distribution of the sequential estimation procedures is obtained in [30]. And it is shown that the bias-adjusted maximum likelihood estimation procedure combined with appropriate stopping rule is uniformly third order asymptotically efficient in the sense that its asymptotic distribution attains the bound uniformly in stopping rules up to the third order, thus the asymptotic deficiency being zero. For the curved exponential family of distributions, the second order efficiency from the Fisher-Rao approach is discussed by Ghosh and Subramanyam (1974), and also by Efron (1975) and Amari (1985) from the viewpoint of the differential geometry. Amari's approach attracted much attention in that he could successfully connect the curvature of differential manifold with the asymptotic deficiency of the estimator.

5. Non- regular unbiased estimation For the lower bound for the variance of unbiased estimators, most famous is the so-called Cramer-Rao bound. But the Cramer-Rao bound and its Bhattacharyya extension assume a set of regularity conditions. Chapman and Robbins (1951), Kiefer (1952), and Fraser and Guttman (1951) obtained bounds with less stringent assumptions , but they still require the independence of the support of the parameter 0 or almost equivalently that the distribution with 0 Bo is absolutely continuous with respect to that with 0 = 0o when 0o is the specified parameter value at which the variance is evaluated. In the paper [20], the Bhattacharyya bound is generalized to the non-regular case when the support of the density depends on the parameter 0 while it

X

is k-times differentiable with respect to 0 within the support, and the bound is also shown to be sharp, which fails in the regular case (see also Takeuchi (1962)). In [26], we introduce the concept of one-directionality which includes both cases of location (and scale) parameter and selection parameter and other cases, and show that the bound for the variance of unbiased estimators is sharp in the sense that the actual infimum of the variance of unbiased estimators coincides with the bound for a specified 00, for this class of distributions, using the result to minimize the variance under the conditions of unbiasedness, linearly independence of functions, etc. in [21]. We also establish that for a wide class of non-regular distributions the infimum of the variance of unbiased estimators can be zero when the size of sample is not smaller than 2. In [25], we give the exact forms of locally minimum variance unbiased estimators and their variances in the case of a discontinuous density function. In the non-regular case when the amount of Fisher information is infinity, the paper [19] shows that the infimum of variance of unbiased estimators is equal to zero, and gives some examples. It is obvious to observe that for usual "regular" case, any parametric function which has unbiased estimators must be continuous or differentiable in the original parameter (e.g. see Zacks (1971) and Lehmann (1983)). However, in a sequential case or a randomized sample size case when the size of sample is not bounded, the continuity of estimable function does not necessarily follow. In the case of sequential Bernoulli trials, the paper [37] gives a sufficient condition for a parametric function to be unbiasedly estimable and shows the existence of a discontinuous unbiasedly estimable function using non-randomized sample size procedures.

6. Non- regular asymptotic estimation and test In the asymptotic theory of estimation, we try to compare the regular versus non-regular situation to clarify the significance and implication of each of the regularity conditions in [14]. Ibragimov and Has'minskii (1981) also proposed similar considerations for non-regularity. As an example, in the case of estimation of a location parameter in the symmetrically truncated normal distribution, it is shown in [11] that the maximum probability estimator of Weiss and Wolfowitz (1974) is asymptotically inadmissible and has smaller concentration probability than the midrange which is asymptotically efficient. For a family of uniform distributions, it is shown in [44] that for any small e > 0 the average mean squared error (MSE) of any estimator in the interval of 0 values of length e and centered at 00 can not be smaller than that of the midrange up to the order o(n-2) as the size n of sample tends to infinity. And the asymptotic lower bound for the average MSE is also shown to be sharp. In the paper [18], we consider the problem to estimate a common parameter for the pooled sample from the uniform distributions with scales as nuisance parameters. And we compare the MLE with others and show that the MLE based on the pooled sample is not (asymptotically) efficient.

Xi

Fisher ( 1934) calculated the loss of information of the MLE of the location parameter in the double exponential distribution , and showed that the loss is of order V/n-, unlike of constant order in the regular case, hence the MLE is not second order asymptotically efficienct. The results are caused from the fact that the density admits the first order differentiability with respect to the parameter, but not the second order . Hence the first order asymptotic theory of regular estimation can be applied, but, in the second order the situation is non-regular. The paper [32] extends these results by obtaining the (asymptotic) losses of information of order statistics and related estimators, and by comparing them via their asymptotic distributions up to the second order . In [39] , we obtain the Bhattacharyya type bound for the variance of unbiased estimators of a location parameter of the double exponential distribution and the loss of information of the MLE. In [29], we consider the estimation problem of a location parameter on a sample of size n from the two-sided Weibull type density f (x - 0) = C(a) exp(- Ix 01') for x E R', 0 E R1 and 1 < a < 3/2, where C(a) = a/{21'(1/a)}. Then we obtain the bound for the distribution of AMU estimators up to the 2a-th order, i.e. n (2 «- 1)/2, and calculate the asymptotic distribution of the MLE up to the same order. And we show that the MLE is not 2a-th order asymptotically efficient, and give the amount of the loss of asymptotic information of the MLE . In [35], we consider the estimation of a location parameter 0 of the density function with a support of a finite interval and contact of the power a - 1 at both endpoints , where 1 < a < 2. Then we obtain the bound for the asymptotic distribution of AMU estimators of 0 based on a sample from the density. It is also shown that the bias-adjusted MLE is not asymptotically efficient in the sense that its asymptotic distribution does not uniformly attain the bound. In the regular case, it is known that the order of consistency, i.e. the order of convergence of consistent estimators is equal to V/n-, but in the non-regular case with i . i.d. sample it is not always so, and could be nl/" (0 < a < 2), (n log n) 1/2 etc., but are usually independent of the unknown parameter (see Akahira ( 1975a, 1975b)). However, in the non i . i.d. sample case , the order of consistency may depend on the parameter . In [23], we consider the first order autoregressive ( AR) process with a parameter 0, and obtain the asymptotic means and variances of the log-likelihood ratio test statistic LT under the null and alternative hypotheses in the case when 101 > 1. We also discuss the asymptotic distribution of LT under both of the hypothesis . For the case of a two-sided Gamma type distribution , the paper [ 42] shows that the largest order of consistency is n2 and there exists a test with under n2 of consistency, and we obtain the asymptotic power function of the test. In [36] , we propose an amount of information between two distributions which is always well defined , symmetric and additive for independent sample and information contained in a statistic is always not greater than that in the whole sample and the equality holds if and only if the statistics is sufficient. And we discuss the relative (asymptotic) efficiency of a statistic (or an estimator) by the ratio of the amounts of information contained in the statistic and in the

Xii

sample in a systematic and unified way both in regular and non-regular cases.

7. Others For the process of continuous time observations are usually made on a finite number of discrete time points, information thus provided depends on the choice of observation points. In the paper [22], we assume that X(7-) is a continuous time simple Markov process with a parameter 0, and consider the problem choosing observation points TO < r1 < • • • < TT which provide with the maximum possible information on 0. If the observation points are equally spaced, that is, for t = 1, . . . , T, rt - Tt_1 = s is constant, we get the optimum value for s. In the asymptotic theory of estimation, the concept of asymptotic expectation is widely used, and it is usually remarked that it can be different from the asymptotic value of expectation. But, the concept itself is not sufficiently accurately defined in the literature, especially when the asymptotic distribution does not exist. The paper [24] gives a rigorous definition of the asymptotic expectation, and shows its properties, e.g. its linearity and a Markov type inequality. And we obtain the necessary and sufficient conditions for the convergences in probability and distribution. In the regular case when the dimension of the parameter is finite, asymptotic loss of information is constant order, but, for semiparametric models, it is shown in [31] that, under fairly regularity conditions, the asymptotic deficiency of the MLE or any regular best asymptotically normal estimator is infinity. In [33], we consider the sampling properties of the bootstrap process, that is, the empirical process obtained from a sample of size n (with replacement) of a fixed sample of size n of a continuous distribution. And we give the cumulants of the bootstrap process up to the order n-1, and discuss their unbiased estimation. We also demonstrate that the bootstrap process has an asymptotic minimax property for some class of distributions up to the order n-1/2, and further we suggest the bootstrap method can be improved in the next order by taking a sample of size n - 1 instead of n. Usually confidence interval is defined as an interval with preassigned confidence level 1 - a for all the values of parameters. In [38], generalizing the concept, we consider interval estimation procedures with confidence coefficient varying according to the value of the unknown parameter, and associated procedure to estimate the actual level. Such a consideration also leads to more general procedures including conditional procedures when the ancillary statistic is involved. For discrete distributions it is usually impossible to obtain a non-randomized test or confidence interval with exact given size, and an actual size is often quite different from the prescribed level. But a randomized procedure, which is quite nice in theory, is not easily acceptable to practitioners. Still, there is something to promote randomized procedures in practical applications. In the paper [41],

for a family of one-parameter discrete exponential family of distributions, we derive the higher order approximation of randomized confidence intervals from the optimum test. Indeed, it is shown that they can be asymptotically constructed by means of the Edgeworth expansion. The usefulness is seen from the numerical results in the case of Poisson and binomial distributions. The distribution of the sum of not identically but independently distributed random variables are difficult to calculate exactly, and the normal and Edgeworth type approximations can be applied when the number of independent random variables is not too small. But it is not always sufficiently accurate especially for the tail part. In such cases, large-deviation approximations were proposed in order to give better approximations especially for tails. In the paper [43], we obtain large-deviation approximations for the distribution of the sum of discrete random variables and show that they give sufficiently accurate results in various cases. It is noted that the large-deviation approximation connected with the saddlepoint approximation is widely discussed by Barndorff-Nielsen and Cox (1989) and Jensen (1995). In sampling from the finite population of size N, there is a problem how to construct a design in which the inclusing probability of k-unit in the population is equal to prescribed Irk and F_'-1 irk = n, and it is a problem whether there exists a sample design of size n which attains the condition. The problem is solved by the Minkowski-Farkas theorem.

References Akahira, M. (1975a). Asymptotic theory for estimation of location in nonregular cases, I: Order of convergence of consistent estimators. Rep. Stat. Appl. Res., JUSE 22, 8-26. Akahira, M. (1975b). Asymptotic theory for estimation of location in nonregular cases, II: Bounds of asymptotic distributions of consistent estimators. Rep. Stat. Appl. Res., JUSE 22, 99-115. Akahira, M. (1986). The Structure of Asymptotic Deficiency of Estimators. Queen's Papers in Pure and Applied Mathematics 75, Queen's University Press, Kingston, Canada. Akahira, M. and Takeuchi, K. (1981). Asymptotic Efficiency of Statistical Estimators: Concepts and Higher Order Asymptotic Efficiency. Lecture Notes in Statistics 7, Springer, New York. Akahira, M. and Takeuchi, K. (1995). Non-Regular Statistical Estimation. Lecture Notes in Statistics 107, Springer, New York. Amari, S. (1985). Differential- Geometrical Methods in Statistics. Notes in Statistics 28, Springer, Berlin.

Lecture

Xiv

Bahadur, R. R. (1955). A characterization of sufficiency. Ann. Math. Statist. 26, 286-293. Barndorff-Nielsen, O. E. and Cox, D. R. (1989). Asymptotic Techniques for Use in Statistics. Chapman and Hall, London. Chapman, D. G. and Robbins, H. (1951). Minimum variance estimation without regularity assumptions. Ann. Math. Statist. 22, 581-586. Efron, B. (1975). Defining the curvature of a statistical problem (with applications to second order efficiency). Ann. Statist. 3, 1189-1242. Fisher, R. A. (1925). Theory of statistical estimation. Proc. Camb. Phil. Soc. 22, 700-725. Fisher, R. A. (1934). Two new properties of mathematical likelihood. Proc. Roy. Soc. (London) Ser. A 144, 285-307. Fraser , D. A. S. and Guttman, I. (1951). Bhattacharyya bounds without regularity assumptions . Ann. Math. Statist. 23, 629-632. Ghosh, J. K. (1994). Higher Order Asymptotics. NSF-CBMS Regional Conference Series Probab. and Statist., 4, Inst . of Math. Statist., Hayward, California. Ghosh, J. K., Sinha, B. K. and Wieand, H. S. (1980). Second order efficiency of the mle with respect to any bounded bowl-shaped loss function. Ann. Statist. 8, 506-521. Ghosh, J. K. and Subramanyam , K. (1974). Second order efficiency of maximum likelihood estimators . Sankhya Ser. A 36, 325-358. Hodges, J. L. and Lehmann, E. L. (1970). Deficiency. Ann. Math. Statist. 41, 783-801. Ibragimov, I. A. and Has'minskii, R. Z. (1981). Statistical Estimation: Asymptotic Theory. Springer, New York. Jensen, J. L. (1995). Saddlepoint Approximations. Clarendon Press, Oxford. Kiefer, J. (1952). On minimum variance estimators. Ann. Math. Statist. 23, 627-629. Lehmann, E. L. (1983). Theory of Point Estimation. Wiley, New York. Pfanzagl, J. (1979). First order efficiency implies second order efficiency. In: Contributions to Statistics. Jaroslav Hdjek Memorial Volume. (J. Jureckova, ed.), 167-196, Academia, Prague. Pfanzagl, J. and Wefelmeyer, W. (1978). A third order optimum property of the maximum likelihood estimator. J. Multivariate Anal. 8, 1-29.

Xv

Pfanzagl, J. and Wefelmeyer, W. (1985). Asymptotic Expansions for General Statistical Models. Lecture Notes in Statistics 31, Springer, Berlin. Rao, C. R. (1961). Asymptotic efficiency and limiting information. Fourth Berkeley Symp. on Math. Statist. and Prob. 1, 531-545.

Proc.

Skibinsky, M. (1967). Adequate subfields and sufficiency. Ann. Math. Statist. 38, 155-161. Takeuchi, K. (1962). On a fallacy of Gunnar Blom's theorem. Rep. Stat. Appl. Res., JUSE 9, 34-35. Torgersen, E. N. (1977). Prediction sufficiency when the loss function does not depend on the unknown parameter. Ann. Statist. 5, 155-163. Weiss, L. and Wolfowitz, J. (1974). Maximum Probability Estimators and Related Topics. Lecture Notes in Math. 424, Springer, Berlin. Zacks, S. (1971). The Theory of Statistical Inference. Wiley, New York.

This page is intentionally left blank

Xvii

Contents

Foreword Introduction

v vii

1975 [1] Characterizations of Prediction Sufficiency (Adequacy) in terms of Risk Functions 1 Ann. Statist. 3(4) 1018-1024. 1976 [2] On the Second Order Asymptotic Efficiency of Estimators in Multiparameter Cases 8 Rep. Univ. Electro-Commun. 26(2) 261-269. [3] On Gram-Charlier-Edgeworth Type Expansion of the Sums of Random Variables (I) 17 Rep. Univ. Electro-Commun. 27(1) 95-115. [4] On Gram-Charlier-Edgeworth Type Expansion of the Sums of Random Variables (II) 38 Rep. Univ. Electro-Commun. 27(1) 117-123. [5] On the Second Order Asymptotic Efficiencies of Estimators 45 Lecture Notes in Mathematics, 550, Springer-Verlag, Berlin, 604-638.

1977 [6] Extension of Edgeworth Type Expansion of the Distribution of the Sums of I.I.D. Random Variables in Non-Regular Cases

80

Ann. Inst. Statist. Math. 29(3) 397-406. 1978 [7] On Gram-Charlier-Edgeworth Type Expansion of the Sums of Random Variables (III) Multivariate Cases Rep. Univ. Electro-Commun. 28(2) 259-269. [8] Third Order Asymptotic Efficiency of Maximum Likelihood Estimator for Multiparameter Exponential Case Rep. Univ. Electro-Commun. 28(2) 271-293. [9] Asymptotic Optimality of the Generalized Bayes Estimator Rep. Univ. Electro-Commun. 29(1) 37-45.

90

101 124

Xviii

1979 [10] On the Second Order Asymptotic Efficiency of Unbiased Confidence Intervals 133 Rep. Stat. Appl. Res. JUSE 26(3) 99-110.

[11] Remarks on the Asymptotic Efficiency and Inefficiency of Maximum Probability Estimators 145 Rep. Stat. Appl. Res. JUSE 26(4) 132-138. [12] Discretized Likelihood Methods - Asymptotic Properties of Discretized Likelihood Estimators (DLE's) 152 Ann. Inst. Statist. Math. 31(1), Part A, 39-56. [13] Asymptotic Optimality of the Generalized Bayes Estimator

in Multiparameter Cases 170 Ann. Inst. Statist. Math. 31(3), Part A, 403-415. [14] Note on Non-Regular Asymptotic Estimation - What "Non-Regularity" Implies 183 Rep. Univ. Electro-Commun. 30(1) 63-66. 1980 [15] A Note on Prediction Sufficiency (Adequacy) and Sufficiency 187 Austral. J. Statist. 22(3) 332-335. [16] Third Order Asymptotic Efficiency and Asymptotic Completeness of Estimators 191 Rep. Univ. Electro-Commun. 31(1) 89-96.

1982 [17] On Asymptotic Deficiency of Estimators in Pooled Samples in the Presence of Nuisance Parameters Statistics & Decisions 1(1) 17-38.

199

1985 [18] Estimation of a Common Parameter for Pooled Samples from the Uniform Distributions Ann. Inst. Statist. Math. 37(1), Part A , 17-26. [19] A Note on the Minimum Variance Unbiased Estimation When the

221

231

Fisher Information is Infinity

Rep. Stat . Appl. Res. JUSE 32(3) 17-22. 1986 [20] Bhattacharyya Bound of Variances of Unbiased Estimators in Non-Regular Cases (with Madan L. Puri)

Ann. Inst. Statist. Math. 38(1), Part A, 35-44.

237

XiX

[21] A Note on Minimum Variance Metrika 33(2) 85-91. [22] A Note on Optimum Spacing of Observations from a Continuous Time Simple Markov Process Metrika 33(3/4) 217-222. [23] On the Bound of the Asymptotic Distribution of Estimators when the Maximum Order of Consistency Depends on the Parameter Publ. Inst. Stat. Univ. Paris 31(1) 1-16.

247

254

260

1987 [24] On the Definition of Asymptotic Expectation Foundations of Statistical Inference, Advances in the Statistical Sciences 2, D. Reidel Publishing Company, Dordrecht-Holland, 199-208. [25] Locally Minimum Variance Unbiased Estimator in a Discontinuous Density Function Metrika 34(1) 1-15 [26] The Lower Bound for the Variance of Unbiased Estimators for One-Directional Family of Distributions Ann. Inst. Statist. Math. 39(3), Part A, 593-610.

276

286

301

1988 [27] Second Order Asymptotic Efficiency in terms of Asymptotic Variances of the Sequential Maximum Likelihood Estimation Procedures Statistical Theory and Data Analysis II (Elsevier Science Publishers B. V. North-Holland) 191-196. [28] Second and Third Order Asymptotic Completeness of the Class of Estimators (with F. Hirakawa) Probability Theory and Mathematical Statistics. Lecture Notes in Mathematics 1299 , Springer-Verlag, Berlin, 11-27.

319

325

1989 [29] Higher Order Asymptotics in Estimation for Two-sided Weibull Type Distributions Ann. Inst. Statist. Math. 41(4) 725-752. [30] Third Order Asymptotic Efficiency of the Sequential Maximum Likelihood Estimation Procedure Sequential Analysis 8(4) 333-359.

342

370

XX

1990 [31] First Order Asymptotic Efficiency in Semiparametric Models Implies Infinite Asymptotic Deficiency

397

Publ. Inst. Stat. Univ. Paris 35(1) 3-9. [32] Loss of Information Associated with the Order Statistics and Related Estimators in the Double Exponential Distribution Case

404

Austral. J. Statist. 32(3) 281-291. 1991 [33] Bootstrap Method and Empirical Process 415 Ann. Inst. Statist. Math. 43(2) 297-310.

[34] Second Order Asymptotic Efficiency in Terms of the Asymptotic Variance of Sequential Estimation Procedures in the Presence of Nuisance Parameters 429 Sequential Analysis 10(1&2 ) 27-43.

[ 35] Asymptotic Efficiency of Estimators for a Location Parameter Family of Densities with the Bounded Support 446 Rep. Stat. Appl. Res., JUSE 38(1) 1-9. [36] A Definition of Information Amount Applicable to Non-regular Cases 455 J. Comput. Information 2(1) 71-92. 1992

[37] Unbiased Estimation in Sequential Binomial Sampling (with K. Koike) 477 Rep. Stat. Appl. Res. JUSE 39(4) 1-13. [38] Interval Estimation with Varying Confidence Levels 490 Metron 50(3-4) 3-18. 1993 [39] Second Order Asymptotic Bound for the Variance of Estimators for the Double Exponential Distribution 506 Stat. Sci. 4 Data Anal., VSP Internat. Sci. Publ., Zeist (Netherlands) 375-382. [40] On the Application of the Minkowski-Farkas Theorem to Sampling Designs 514 Statistica Neerlandica 47(3) 221-223. 1997

[41] Randomized Confidence Intervals of a Parameter for a Family of Discrete Exponential Type Distributions (with K. Takahashi) 517 Commun. Statist . - Simula. 26(3), 1103-1128.

Xxi

[42] The Existence of a Test with the Largest Order of Consistency in the Case of a Two-sided Gamma Type Distribution

543

Metron 55(1-2) 93-107. 1999 [43] The Higher Order Large-Deviation Approximation for the Distribution of the Sum of Independent Discrete Random Variables (with K. Takahashi) 558 Commun. Statist. - Theory and Methods 28(3&4) 705-726. 2001 [44] Information Inequalities in a Family of Uniform Distributions 580 Ann. Inst. Statist. Math. 53(3) 427-435. Permission

589

1 The Annals of Statistics 1975, Vol. 3, No. 4, 1018-1024

CHARACTERIZATIONS OF PREDICTION SUFFICIENCY (ADEQUACY) IN TERMS OF RISK FUNCTIONS BY KEI TAKEUCHI AND MASAFUMI AKAHIRA University of Tokyo and University of Electro- Communications Prediction sufficiency (adequacy ), as it is usually defined in terms of conditional expectations, does imply "real" prediction sufficiency ; i.e. sufficiency in terms of risk functions . The converse holds provided we permit the loss to depend on the unknown parameter . This is no longer true if we insist on loss functions which do not involve the unknown parameter. Conditional independence still holds but ordinary sufficiency may fail. If, however, we require equivalence of risk functions , then ordinary sufficiency and, consequently , prediction sufficiency follows.

1. Introduction . It has been shown by Bahadur [2 ] that sufficiency as defined in terms of conditional expectations , under regularity conditions , implies "real sufficiency " i.e. sufficiency in terms of risk functions . Furthermore it follows from Theorem 11.3 in Bahadur ' s paper [ 1 ] that prediction sufficiency is equivalent to ordinary sufficiency w.r.t. a larger class of probability measures. One may therefore expect similar results to hold for prediction sufficiency (adequacy) as well . In the case of prediction problems it may be of interest to consider loss functions which depend only on the decision to be made and the quantity to be predicted . If we insist on this restriction, then prediction sufficiency in terms of risk functions no longer implies prediction sufficiency as it is defined in terms of conditional expectations . Conditional independence holds but ordinary sufficiency may fail . It will, however , be shown that equivalence of risk functions implies ordinary sufficiency and consequently prediction sufficiency. (One of the authors proved this in an earlier work [7].) We will , essentially , use the framework of Skibinsky [ 6]. The notion of adequacy in Skibinsky ' s paper is, however , replaced by the notion of prediction sufficiency. 2. Theorems . We shall assume that we are given a model consisting of a sample space (^2', Ste) and a family {P0 : 0 E 9 ) of probability measures on .S^l. A sub a-algebra . of _V summarizes what can and what can not be observed. Similarily , a sub a-algebra ' of ,S describes what we are interested in predicting . Finally, we are given a sub a-algebra M. of . and our problem is to decide if anything is lost by basing our predictions on rather than . The prediction problem is assumed to be completely described by a decision space ( T, .9 ), i.e. a measurable space and a loss function L from 9 x 2' x T to [0, oo[. It will always be assumed that L as a function on ^C'' x T for given Received January 1973; revised August 1974. AMS 1970 subject classifications. Primary 62805; Secondary 62007. Key words and phrases . Prediction sufficiency, conditional independence , equality of risk functions. 1018

2

SUFFICIENCY IN TERMS OF RISK FUNCTIONS 1019

9 e 6 is ' x 9 measurable . This implies that the loss does not depend on all of x e 1', only on the part of x which is to be predicted. A decision rule 6 will here be defined as a Markov kernel 8 (S I x) : S e .I, x C,2' which is .O' measurable when S is fixed and a probability measure on -9'- when x is fixed. If S is a decision rule then its performance characteristic Pa(. 0); 0 e 6 may be defined by defining - for each 0 - Pa(. B) as the probability measure on x .9'" defined by

Pa(C x S 9) = S c 8(S I x)Pe(dx) The risk function ra(0); 0 e 8 of a decision rule 6 is given by: 0e8. ra(B) = S [ S L0(x, t)8(dt I x)]Pe(dx) , The risk function is determined by the loss function and the performance characteristic through ra(0) = S Le dpa (. 10) , 9 e 8. A decision rule 6 will be called measurable if 8(S ^ .) is _'50. measurable for each S. DEFINITION 1. .eV and 25' are conditionally independent given

if

(i) Pe1(C .9') = P0'(C I _e,) a.e. [P,,] for all C e ' and for all 0 e 8. It is shown in Loeve [5], page 351 that (i) and (ii) are equivalent: (ii) pe."(B n C I ) = P9'(B I _,^F.)P0 "(C I A) a. e. [Pe] for all Be.,Wand all C e9andfor all 0e8. We define prediction sufficiency (adequacy ) and prediction sufficiency in the wide sense as follows: DEFINITION 2. .moo is prediction sufficient for M w.r.t. le . iffy is sufficient for M and M and ' are conditionally independent given .moo. DEFINITION 3. .-Wo is prediction sufficient in the wide sense for ' w.r.t. o and (b) there exist M. if (a) M and le' are conditionally independent given measurable sets B, and B, so that P0(B1 u B,) = 1 for all 0 e 0 and Pe (• ^ ^) is independent of 0 if x e B, and P "B; s(. I .moo) is independent of 0 if x e B,.

In the following example we shall show that ., is prediction sufficient in the wide sense for.' w.r.t. 'e but not prediction sufficient for M w.r.t. '^6'. We assume that X„ X„ • • • , X. and Y are random variables such that X„ X„ • • • , X,, are independently and identically distributed as N(6, 1) while the conditional distribution of Y given X„ X„ • • • , X. is N(0, 1) or N(0, 1) as u, Xi > a or be the a-algebras induced by, respectively, (X„ E, Xi < a. Let , A and X„ • • • , X.), min (a, E i Xi) and Y. Then o is prediction sufficient in the wide sense for W w.r.t. ;%' but not sufficient for

That sufficiency alone is insufficient in prediction problems may be seen by considering , for example , the situation where Pe does not depend on 0 and

3 1020 KEI TAKEUCHI AND MASAFUMI AKAHIRA

,V, = It is then fairly obvious that prediction of a which is not independent of ^% should not, in general , be based on .moo. It follows, as has been pointed out by Skibinsky [6], from Theorem 11.3 in Bahadur [1] that , is prediction sufficient for . w.r.t. ' if and only if is sufficient for w.r.t. all probability measures on ' of the form: B n-+ Pe(B C) ,

0 E 8 and P0(C) > 0. where C E In analogy with Theorem 10.2 in Bahadur [1] we get: THEOREM 1. Suppose A is prediction sufficient for .,W w.r.t. 1 'and there exists a regular conditional probability P°(• I ✓ ^a) of given which does not depend on B. Let 6 be any decision rule from (', _e-Wfl to (T, ,l) and put 6(S ^ x) _ S h(S I x')Pz-4 (dx' I S E -9'-, x E Then $ is .tea measurable and it has the same performance characteristic as 3. In particular & and 3 have the same risk functions. PROOF.

ue(C x S I S)

= S c a(S I x)P6(dx) = S c E' (S( S I •) I M.) dPe =ScE`''(3(S I .) I M., `e)dPe=Sc6(SI•)dPe=u0(Cx S b).

REMARK. As is immediately seen from above, we need to allow randomized decision rules. This is not always necessary for the subsequent discussions. Consequences of "risk prediction sufficiency" for various classes of loss functions. In order to show prediction sufficiency of .M:,o we must establish conditional independence and ordinary sufficiency. We will assume that we are given a certain class of loss functions and that to any loss function within that class and to any decision rule 3 corresponds a decision rule A which is M. measurable and has uniformly smaller risk than 6. The problem is to decide whether this suffices to establish conditional independence or ordinary sufficiency. It is clear that conditional independence cannot, in general, be established by only considering loss functions which do not depend on x. Similarly loss functions which do not involve 0 will, in general, be insufficient to establish ordinary sufficiency. Conditional independence may, however, be established by considering only loss functions which do not depend on 0. Similarly, and this follows from corresponding facts for sufficiency (see Bahadur [2], Blackwell [3] and Le Cam [4]), sufficiency of M. may be established by considering loss functions which do not depend on x.

Conditional independence may be established by considering the two decision problem with loss functions not depending on 0 as follows: THEOREM 2. Consider the decision space T = {0, 1} and the set of all loss functions, L, of the form L0(x, 0) = II(x) , x E , 0E8, L6(x, 1) = pIc.(x) , where p E ]0, 1[ and C E 'Y.

x E L" ; 0 E 8 ,

4 SUFFICIENCY IN TERMS OF RISK FUNCTIONS 1021

Suppose that to each decision rule 3 and to each loss function L, of the above form, there corresponds a o measurable decision rule fi so that r;(9) < ra (B)

,

0 E0.

Then and '' are conditionally independent given .moo.

Before proving the theorem a few remarks may be in order. REMARK 1. It follows from the proofs that we may restrict attention to nonrandomized decision rules. REMARK 2 . The proofs imply also that much smaller sets of loss functions will do . We may, for example , restrict C to a s-system generating 'f. REMARK 3 . The parameter space 0 does not play any role in this theorem. We may-and shall-in the proof assume that 0 consists of a single point. Conditional independence is, in this situation , equivalent to prediction sufficiency. PROOF OF THE THEOREM. We may, by Remark 3, omit the subscript 0. Furthermore a decision rule 3 may be identified with the critical function x n-^ 8(1 I x). The risk may then be written: r(d) = S L(•, 0) dP + S [L(•, 1) - L(•, 0)]S dP

= SL(•,0)dP+(p+ 1)S(p+ 1 - J)odP = S L(., 0) dP + (p + 1) S [p + 1 - P-(C I .,W)1 3 dP

S L(., 0) dP - (p + 1) S [P(cl M) P +

11

dP ,

where is obtained iff a = 0 a.e. or 3 = I a.e. as PI(C I .') < p/(p + 1) or P"(CI --51) > PAP + 1) The same argument applied to M. implies , by the assumption of the theorem, that the minimizing 6 may be chosen M. -measurable and such that 3 = 0 a.e. or 8 = 1 a . e. as P'(C I .moo) < pl (p + 1) or P""(C I A) > Pl ( P + 1). It follows that the event [P '(C I ) < p/(p + 1)] and the event [P "'(C I M^) < Pl (P + 1)] are equivalent provided P'(CI. ) * pl(p + 1) a .e. and PE'(CI.2'o) * pl(p + 1) a.e. This implies that the random variables PI (C I M) and P '(C I . .) have the same distribution . Hence, since P& (C I M.) = E° (P(C I P`'(C I M.) = ' are conditionally independent given P{'(CI ) a. e. It follows that is' and o•

REMARK . This form of the proof was suggested by one of the referees. A criterion based on least squares prediction theory is , as has been pointed out by one of the referees , even simpler to establish . Consider a sufficiently large class of square integrable and 7 =measurable random variables g. To a

1022 KEI TAKEUCHI AND MASAFUMI AKAHIRA

given g we associate the loss function L(x, t) = (g(x) - 1)2 , x E 2', t E ] - oo, co[ . Then a predictor d minimizes the risk if and only if it is a version of EI(g ^ M). If .-W, is assumed to be just as good in this situation then E' (g I ) = El'(g ^ moo) a.e. This establishes conditional independence if, for example, we admit all functions g = I, where C runs through a s-system generating '. For the Lemma and Theorem 3 we assme that {P8 : 0 E e} is dominated. Then we get the following lemma. measurable critical function cp there exists ao measurable LEMMA . If for any -,qcr.. ical function (P such that E,((P) = E,((p) for all 0 E 8 , then .,_--e. is sufficient for _-R The proof of the lemma is essentially the same as in Bahadur [2 ]. The outline is as follows: Let 0, and B, be any two points of 8. Let ¢ be a most powerful test for 0 against B,. Then for some k

fi(x) = 1 if

PP' > k 1

=0 if

dP"
and the set {x : (dPes/dP01)(x) < k} is Vie-measurable . Further for every c (including oo), the set {x : (dP92/dPB1)(x) < c} is M.-measurable . Hence .mo o is pairwise sufficient for M. Since {P0 : 0 E 8} is dominated , _,W. is sufficient for The following proposition is an immediate consequence of the lemma: Suppose that the decision space T = [ 0, 1] and the loss function L satisfies L(x, t) = t, for 0 < t < 1. If for any . 'measurable decision rule 6, there exists a .moo measurable decision rule A such that ri (0) = r8(0) for all 0 E 8, then M. is sufficient for M.

From the above , we get the following theorem. ' -measTHEOREM 3 . If for any loss function L not depending on 0 and for any cam urable decision rule 6 , there exists a A-measurable decision rule S such that ri(0) _ o is prediction sufficient for .' w.r.t. r,(B) for all 0 E 8, then Various criteria for prediction sufficiency may be obtained from these results by considering, in addition to the loss functions described above , loss functions which depend on 0. If we insist on considering only loss functions which do not depend on 0 then we will, in general, not be able to conclude prediction , ' ' is sufficiency . This follows by considering the case where ancillary and independent of .' and {Po : 0 E 81 is finite. Let, in this situation , ( T, %) be any decision space and L any loss function to (T, -). which does not depend on 0 and 6 any decision rule from Choose a to E T such that S L(x, ta)P0(dx ) < S [5 L(x, t)P0(dx)]v,, (dt) , 0 E 0,

6 SUFFICIENCY IN TERMS OF RISK FUNCTIONS 1023

where vo(S) = S 8(S I x)P0(dx). Then ro(t•)
BE®.

The necessary condition for prediction sufficiency in the wide sense is obtained as follows: THEOREM 4 . Suppose that , for each 6, there are regular conditional probabilities PO(B I geo) : B E .Gif and P,(C I Via) : C E ' '. Suppose further that :isa is prediction w.r.t. sufficient in the wide sense for Let L be a loss function not depending on 0 and assume that there is a . •-measurable function r on B, and an s > 0 so that S L(x', r)P(dx' I .%5o) < S L(x', t)P(dx' I :0) + e , t E T on B,. Then there corresponds to any decision rule 3 a ..-measurable decision rule S so that re(S) < ro(a) + e , 0 E may be defined as r on B, and as E(d(• i •) I A) on B,. REMARK. There exist , for each e > 0, a moo-measurable r on B, satisfying the desired inequality provided (i) There exists a countable subset It,, t„ • .. , t,,, • • • } of T such that for all t E T and for all x E B, =` (dx' I .c^'•) S S L(x', t)PP'*(dx' I ,^^' o) , inf. S L(x', tn)P and (ii) For every pair i, j and for any e > 0 the set M{j(s) = [x: S L(x', t,)P;" (dx' I .moo) < S L(x', tt)P.- (dx' I •) + s) is measurable. PROOF OF THE THEOREM. Put, for each 0,

5 e *(SI •) = S 3(SI x')Pe(dx'I.^) By the proof of Theorem 1:

e + re(a) = s + S [S L(x, t)be*(dtI x)]Po(dx) = S B, [ S L(x, t)S(dt I x)]P0(dx) + s + SB, {S [S L(x', t)P.(dx' I . •)]8B*(dt I x)}Po(dx) SB, [S L(x, t)S(dt I x)]Po(dx) + S B, { S L(x', b(x))P0.z(dx' I .)}Pe(dx) = S B, [ S L(x, t)5(dt I x)]P0(dx) + S B, L(x, d(x))Pe(dx)

re(b) . Acknowledgments . The authors wish to thank Mr. M. Takahashi of Osaka University for valuable suggestions and the referees of the Annals for co-operation in completing the final version.

7

1024 KEI TAKEUCHI AND MASAFUMI AKAHIRA

REFERENCES [ 1] BAHADUR , R. R. (1954). Sufficiency and statistical decision functions . Ann. Math. Statist. 25 423-462. [2] BAHADUR , R. R. (1955). A characterization of sufficiency . Ann. Math. Statist . 26 286-293. [3] BLACKWELL , D. (1953). Equivalent comparisions of experiments . Ann. Math . Statist. 24 265-272. [4] LE CAM , L. (1964). Sufficiency and approximate sufficiency . Ann. Math. Statist. 35 14191455. [5] LOEVE , M. (1963). Probability Theory, (3rd. ed .). Van Nostrand , Princeton. [6] SKIBINSKY , M. (1967). Adequate subfields and sufficiency. Ann. Math . Statist . 38 155-161. [7] TAKEUCHI , K. (1966). On some statistical prediction procedures . ( In Japanese). Keizaigaku Ronshu 32 23-31. FACULTY OF ECONOMICS DEPARTMENT OF MATHEMATICS UNIVERSITY OF TOKYO UNIVERSITY OF ELECTRO -COMMUNICATIONS HONGO, BUNKYO-KU CHOFUGAOKA , CHOFU-SHI TOKYO , JAPAN TOKYO, JAPAN

8 Rep. Univ. Electro-Comm. 26-2, (Sci. & Tech. Sect.), pp. 261-269 February, 1976 261

On the Second Order Asymptotic Efficiency of Estimators in Multiparameter Cases* Masafumi AKAHIRA** and Kei TAKEUCHI***

Abstract Suppose that Xi's(i=1, 2,..., n) are independently and identically distributed with the density f (x, 8, ^), where 8 is a real valued parameter and a is a real (vector) valued parameter. We consider a (sequence of) estimator(s) which is k-th order asymptotically median unbiased, and define k-th order asymptotic efficiency. We have a formula for the distribution of the second order asymptotically efficient estimator and show that a modified maximum likelihood estimator is second order asymptotically efficient.

1. Introduction Second order asymptotic efficiency of estimators has been discussed by Pfanzagl [2], Takeuchi and Akahira [3], Takeuchi [4] and Akahira [ 1] for one dimensional case. In this paper we extend a similar approach to multiparameter cases and obtain a straight -forward generalization of the one-parameter case. Let Xi, X2,. - -, X., ...

be a sequence of independent identically distributed random

variables with the density f (x, 0, ^), where 8 is a real valued parameter and . is a real (vector) valued parameter. We assume that ^ is a nuisance parameter . We shall define an estimator of 0 to be k -th order asymptotically efficient if the k-th order asymptotic distribution of it attains the bound of the k-th order asymptotic distributions of k-th order asymptotically median unbiased ( AMU) estimators of 0. We shall obtain the bound of the second order asymptotic distributions of second order AMU estimators and show that a modified maximum likelihood estimator is second order asymptotically efficient.

2. Notations and definitions Let X be an abstract sample space whose generic point is denoted by x, B a a-field of subsets of X and let ® and 5 be parameter spaces which are assumed to be open sets in R' and R° respectively. (We denote by R➢ a Euclidean p-space with a norm II • II.) We assume that 1 =(ci, ... , l p) (ES) is a nuisance parameter. We consider a sequence of classes of probability measures {PB, t, i : (0, 6) E®x 5} (i=1, 2, ...) each defined over (Z B).

We shall

denote by (X (" ), B"`)) the n-fold direct products of (1, B) and the corresponding product measures by Ps, e"" = PB, t, 1 x ... x Pe, e, ,,.

An estimator of 0 is defined to be a sequence {On}

* Received on December 10, 1975 ' Statistical Laboratory, University of Electro-Communications *** University of Tokyo

9

262

Masfumi AKAHIRA and Kei TAKEUCHI

of On)-

measurable functions Bn on Xin) into (9 (n=1, 2, ... ).

Definition 1.

For each k=1, 2, ... , {0n} is k-th order asymptotically median unbiased

(or k-th order AMU) estimator if for any 8o=(Oo, eo )E®XE, there exists a positive number S such that k-1

1

1

lim sup n 2 Ps(n){n2(en-8)50} - 2 =0; n-.00 9o8x8 :

Isk -sl<6

lim sup nk21lPs(n) {n2(Bn-0)>0} - 1 I =0. n-ooBeex8 : Iso-sl
Definition 2.

_ 1 k-1 8, 6)+n TG1(t, 8, e)+... +n - 2 Gk_1(t, 8, ^) is called to be the k-th order asymptotic distribution of it if

PB, E (n) { n2 (O n -0)
lim n- M

n

2l k

-n- 2Gk -1(t, 0,) =0. 1

Consider the problem of testing hypothesis H+ : 8=Bo+tn-2(t>0),

against K : 0=0o,

We shall denote by pn(t, an, Bo, ^) the power function of the most powerful level an test (00 \\ sup

k-1 limn-_ 2 {an(t, an, Bo, o)-Ho+(t, Bo, 6o)

flax) : a n = (1/2)+o(n- 'k-"/')} n-. oo

1 k-1 -n THi+( t, Bo, co )-... -n - -2 Hk- 1+(t, 0o,

o)} =0,

then we have Go(t, Bo, o)
G1(t, Bo, o)=Hi+(t, Bo, eo)

then

G1( t, Bo, eo )
(i=0, ... , j-1),

Consider next the problem of testing hypothesis H- : 0=0o+tn 2(t<0),

against K : 0

=00, ^_^. If for ^=^o and for each t<0 k-1 inf limn-2 {19n(t, an, Bo, o) -Ho-(t, Bo, ^o) Ilan} : an=(1/2)+o(n-a-1)/')} n-'oo -1 k-1 -n 2H1-(t, Bo, co)-... -n - -2 Hk-1-(t, Bo, ^o)} =0,

then we have Go(t, Bo, o) >Ho-(t, Bo, Sao) ;

Definition 3.

if

Gi(t, Bo, o)=Hi-(t, Bo, . o)

then

G1(t, Bo, eo)>Hf-(t, Bo, ^o) (j1,. .. , k).

(i=0, ... , j-1),

Suppose that {0n} is k-th order asymptically median unbiased, it is

called to be k-th order asymptotically efficient if for each OE19 and each ^EE

+(t, 8, ) for t>0, G1(t, B, ^)==-(t, e, e) for t<0, i=0,...,k-1. In the subsequent discussions we shall deal only with the second order asymptotic efficiency,

but the line of discussions can be extended to general cases.

10 February, 1976 On the Second Order Asymptotic Efficiency of Estimators in Multiparameter Cases 263

3. Second order asymptotic efficiency in multiparameter cases. We assume that 8 is an open set of R' because in the subsequent discussions there arises no substantial difference between R1 and R°(p>2). Let X1, X2,. - -, Xn,... be a sequence of independently and identically distributed random variables having a density function f (x, 0, ^). First we shall obtain the bound of the power functions. Consider the problem of testing _ 1

1

hypothesis H+: 0 =6o +tn 2(t>0), ^= ^o +un -2, against K : 0 =6o, ^ =eo, where u is an 1

1

arbitrary but fixed constant . Putting 61=6o+tn-2 and ^1=eo +un -2, we define Zin as follows: Z tn=log

f (X1, 60, e0) f (Xi, 61, 1)'

The most powerful test is given by the following rejection region n E Zin>C, i=1

where c is some constant. For each i =1, 2, ... , Zin is expanded as follows :

Zin=-

60> aBlogf(X1,

1

o)(tn Z)-F log f(X1, 6o, ^o)( un 2)

02

02

t2n-1 ) - 1 2 a62 log f (X i, 60, eo)( Bad log

1 6

2 log f (Xi, Bo, eo)( u2n-1)

f (Xi, Bo, ^o)( tun'1)

a3

a631og f (X 1, 6o, ^o)(

63 3 t3n -3 -f )- 6 a 3 log .f (X i, 60, ^o)(u3n Z )

- 2 6020 log f (X 60, ^o)(t2un

2) - 2 a6aelog

f (Xi, eo, eo)( tu2n- z )

3 +o(n 2).

Hence E Zi n is asymptotically normal. If 0=Bo and e_ ^o, then the asymptotic mean µo and i=1

n the asymptotic variance 602 of Zin are given by i=1

_1 [lo = 2 (Ioot2 + Iuu2+2loitu)+o(n Z); _1 Qo2=loot2 -}-I11u2+2Ioitu+o(n 2),

where

a

loo=Eeo. E.{86 log f(Xi, 60, eo)}2=-Eeo,eo{ a62 a 2log f(Xi, 60, eo)} Io1= Eeo. co{aB log f (Xi, 60, o)^ log f (Xi, 60, eo)} z =-Eeo,E°{aBa-c

log f(Xi,Go, o)

a

2

a2

Iu=Eeo.eo{a log f(Xi,60, o)} Eeo, Eo{a 2logf(Xi,60, o)}. If 6 = 61 ^ = ei, then if follows that

11 264

Masfumi AKAHIRA and Kei TAKEUCHI

Ee,.e,i

lo

log .f(X1, 60, o)}

) f f (Xi, 01, el = Ee.. e. {aB log f(X Bo, o

(X 1, 6o, ^o)J

=Ee..E.{aB logf(X1, 90, ^o )}Z( tn Z)

+Eo,.e.{a6 log f(X+, 6o, o)a log f(Xi,6o, Co)}(un 2)+ o(n Z) Ee,. e, {a- log f (X r, 60, o)} Ea.. e.

{aB log f (X 6o, o)

+E0,,e.

log Ax " 6o, 6)} (tn z )

.

log f(X;, Bo, eo)} (un- Z)+o(n- ).

{a

n For 6=61 and 6=E1, the asymptotic mean ft1 and the asymptotic variance a12 of E Z;n are t_1

given by 1 ,u1= - 2 (Ioot2+I11u1+2Iiotu)+ o(n z) ; 1 C18=0 + o(n Z). n

Hence the asymptotic power of the most powerful test of E Z, , is obtained as follows : i=1 O(,uo+/s1) = o( Ioot$+I11u°+2lo1tu) Oro

(3. 1)

_ n2 where fi(x) =J e a du. Since u can take arbitrary values, then the power function of the tests of the (composite) hypothesis is not larger than the infimum of (3. 1) with respect to u. A u minimizing Ioot°+ I11u°+2loltu is given by uo=-(101/111)t, and it follows that O( Loot I,luo2+2loltuo)=1,/j* t,

where I *=loo-(Io1E/I11). n

1

In order to obtain the expansion of E Z; n up to the order of n- z we put a Jooo=Ee..e.

a6'

log f(X+, 60, o)a6log f(X+, 60, o)}

z J001 =Ee.. e.{a6z log f (X 6o, Uo) , log f (X f, Bo, ^o)} z

Jo1o =Ee.. e.

U

aea log f(X 1, 6o, p log f (X

eo, 6o)}

E

Jo11=Ee.. e• aea^ log f(X 0, 6o, eo)0 log f(X,, Go, eo)} J1o =Ee.. e.

a

log f (Xi, 6o, Up log f (X 6o, U}

a J11=Ee.. e.

U 8 log f (X 1, 6o, eo) log f (X B0, o)}

;

12

February, 1976 On the Second Order Asymptotic Efficiency of Estimators in Multiparameter Cases 265

Kooo= Ee,.E,

K 001 = E0, . E, K011 = Ee,.

{a6 L {a6

3

log f(X,, 60, o)} ;

log

E'L^86 log

K 111= Ee,. E,

{bl_^

z

f

(X r, eo, o)}

{

log f (Xi, B0, 0)11

f (X r, 60, ^o)) {a log f (Xi, 60, o)} J 3

log f(Xi, 60, U}

Under suitable regularity conditions the following hold : s Es,. E, a8s log f (Xi, 60, eo) = -3Jooo -Kooo ; s

Ee,. E• a62 log f (Xi, 60, eo)a^ log f (Xi, 60, 60) _ - Joo1- 2Jolo -Koo, Ee,. E,

log

{a

f

E

(X r, 60, ^o)a$ log

f(X

+, Bo, co) = -Juo -2J011-Kou ; I

s

Ee,. E, a^s log f (Xi, 00, eo) = - 3J111-K111. n 1 If B=6o and 6=^o, then the asymptotic mean Ito of Zr„ up to the order of n _2 is given 1=1

as follows : ,1o= 2 (Ioot2+I11u2+21oltu) 1

+ Z

n Z {Jooot2+(Joo1+ 2J olo )t2u+(J11o+2Jo11)tu2+ J lllu3} 1

+

6

_

1

n 2 (Kooot3+3Kooit2u+3Kollt2u +K111u3) +o(n Z).

When u= - (Io 1 /Il i)t, Ito is written as the following form : Ito= 2 I*t2+ 6 n 2 (3J*+K*)ts+o(n 2). n

Similarly the asymptotic variance ao2 and the asymptotic third moment ro of Z;,, are given i=1 _ 1 ao2=l*t2+J*t3n

1 _

2+o(n

2);

9

1 _

1 _

ro=E9,.E,( Zi„-Ito) _K *t3n Z+ o(n z).

If 8=61 and then if follows that logf(Xi, 61, 6) f(X r, 60, eo)

=1+a8 log f (Xi, 60, eo)(tn- 2)+a,- log f (Xi, 6o, ^o)(un z ) +

2

{a 2 2

+28

log f (X r, 60 ,

as

^o)(t2n -1)+

log f (X r, 60,

eo)(u2n -1)

log f (Xi, 60, ^o)( tun 1)

+ 2 {00 log

f (X r, 60, So)} 2t2n

-1+ {a^

log f (X r, 60, eo)}Zu2n-1

13 2ss

Masfumi AKAHIRA and Kei TAKEUCHI

+2j O log f (X,•, 9o, Eo)}lad log f (X r, 9a, ^o)}(tun 1 )^+o(n-1). Hence we have /t1 = /to + ( Ioot2+Iliu2 + 21oitu) I

-n T {Jooot3+(Jooi+2Joio)t2u+(J11o+2Jo11 )tu2+Junu3} I - 2 n T (Kooot3+3Koojt2u+3Ko11tu2+K111u3). If u= -(Ioj/1oo)t, then it follows that

-1 1 1 /a1= - Z I* t2- 6 n T(3J*+2K*)t3+o(n T).

Similarly we obtain _ I a12= ao2 +n

I

z(Kooot3+3Kooit2u+3Koutu2+K111u3)+o(n 2)

1 1 _ =I *t2+n _2(J*+K*)t3+ o(n Z) ; s I I 1 r1= Ee,.e,(Ztn-,u11 =ro+o(n T) = -K *n Tt3+o(n T).

/

If we choose a c such that Pe,.e,(n) ^XZ1n>c 1 +o(n 2), -I 11 2

then c(=ci) is given by * 1 * c1=- 2* t2-3J* 62K n-Tt3+6 *n-1.

Hence we have I'e..f. EZin > c1

t

{

-z ) (3.2) tl^ 37 *+ 2K*t2n_2 +o (n

6VI

where (P (x)0(u)du with O(u)=-Zee2. The bound of the power functions of the problem of testing hypothesis is obtained by (3. 2), that is, Ho+(t, Bo, 6o )=O(t ✓I**)

(3.3)

H1+(t, 60, ^o)=O(t)(3J6 I *)t2

(3.4)

By a similar way as the case t> O it is shown that in the case t
The bound for the distributions of second order asymptotically

median unbiased estimators is given by

Go(t, Bo, Eo)=(P(tIII*) ; G1(t, 60, Eo)=o(t)(3J6 *+2K* )t2, where I*, J* and K* are defined in the context. Let 9(={&„}) and e(= {fin}) be maximum likelihood estimators of 0 and 6, respectively Since

Eaelogf(Xi,e, )=0;

14 February, 1976 On the Second Order Asymptotic Efficiency of Estimators in Multiparameter Cases 267

iE -log f(X1, 0,1)=0, expanding them in the neighborhood of B = Oo and _ ^o respectively , we have log f (Xi, Bo, o) EB +{^E aB2log f(Xi, B0, X0)(9-0o)+fE f (Xi,

1 { i=1ae E a3g log +2E

aBa-

log f(X+, Go, co)(e-eo)}

ag

log f (Xi, 9o, eo)(^ -^0)2 X0)(9 -0o)2+ E =16602

eo,

as

i=10B2a^

log f (Xi, 90, U(0-0 04-6) }

-0; no iE

log f (Xi, 9o, ^o) a3

+ E log i=1 800^

+

{i^ 2

+2E

aB2a a3

i=100 2

f(X,,

a.,

E aaglog f(X,, 90, 60)(x o) ^o)(9 -0o)+ j=1 I

log f(Xi, 90, ^o )(0-Oo)a+ E ag log (X 90, ^o)(^ -^o)2 s=1a^

log f(Xi, B0,

U( 9- 80)( - 0)}

-0. Putting 1 n a Zo=n_2 E - log f (Xi, Bo, U; i =108 1 n a

Zi=n 2 log f(Xi, Go,co); s Zoo=n 2 E { logf(Xi, Bo, o+Ioo} ; i=1 aB f(xi, 0, 0)+I1II Zit=n ZEfj1og a i=1

Z oi=n

2

^

Z, {1og f(xi, B0, ^0)+Iol},

i=1

000

we see that they are asymptotically normal with mean 0. Since O=Zo+(-Ioo+n

1 1 1 _1 Z Zoo)n2(9-9o)+(-Ioi+n _ZZoi)n2(cf-^o)

+ (-3Jooo-Kooo)n2(0 -0o)2+ 2 (-Jiro-2Jo11-Koii)n2( -eo)z +(-Joni-2Jolo-Kooi)n2(9-Oo)(e-^o)+o(n 2); 0=Zi+(-Ioi+n 2Zoi) n2(9-Bo )+(-Iii+n 2Zii)n2(E-e0) + 2 (-Jooi-2J010-Kool )n2(0-0o)2+ 2 (-3Jiii-Kni)n2(e-^0)2 +(-Jiio-2Joni-Koii )n2 (9-Oo)(E-^o)+o(n 2 ),

it follows that

15 268

Masfumi AKAHIRA and Kei TAKEUCHI

(loolii - Io12)n2(6 - 8o) =131Zo-Jo1Z1+n T(111Zoo-Io1Zo1)nT(8-Bo)+n 2(I11Zo1-IoiZii)n2 1 -1

• -Eo)- 2 n T(3Jooolii-Jootlol-2Joiolo1+Kooolll-Koollol) • {n2(B-Bo)}2- 2 n 2(Jiiolii+2Jo11111-3J1111o1+Kou111-K1111o1) 1

1

• {n2(e-^o)}2-n _T(JooiIii-Jiiolo1+2Jo1o111-2Jo111o1+Koo1111 -Ko111o1) {n a (9-Oo)(E-^o)} + o(n 2 ) Putting D=Ioo111-Io12, a=-Iol/Ill, b=-Iol/loo, we have nT(8-8) _ J (Zo+aZ,)+I2n T(Zoo+aZol)(Zo+aZi)+-j _n T(Zoi+aZ11)(Z1+bZo)

1n T(3Jooo+Jola+2Joloa+Kooo+Koola)(Zo+aZ1)2 _ 1111002 11I oo2n-

2D 2 -1112

2(

Jllo+

2 Jo11+ 3 Jiiia+ Koii+ K llla)(Zi+bZo) 2

goon-i(Joo1 +Jiioa+2Jo1o +2Jolla+Koo1 +Kooia)

D

• (Zo+aZi)(ZI+bZo)+o(n- Z ).

Since Ee..E,(Zo2)=1oo, Ee..E.(Z12)=111, Ee,.E.(ZoZJ)=1o1, Ee..E.(ZooZo)=Jooo, Ee.,E.(ZooZ1)=Jool, Ee..E,(Zo1Zo)=Jo1o, Ee,.E,(Zo1Z1)=Jo11, E9,.E,(ZIIZo)=J11o, Ee,.E,(Z11Z1)=Jill,

if follows that 1 J*+K* - -1 Ee,.E.{n2(0-0o)}=- 21*2 n 2+o(n 2), Ve,.E.{n2 (9-8o)} =I*+o(n 2), 2 EB.. E, {n 1 (0-80)} 2= -3J*I 3 K*n- i +,(n -1 -2').

If we define 9* by B*=96n1* then H* is second order AMU and second order asymptotically efficient at O = 8o and ^ _ ^o. Since K* and I* depend on Bo and 6o, we may change 9* into the following form 0**: 9**=B_ K*(8, ^) 6nI9, j) Thus we have Theorem 2.

B** is second order AMU and second order asymptotically efficient.

For any consistent estimator {8n°} if the asymptotic variance of {0n°} is given by

16 February, 1976 On the Second Order Asymptotic Efficiency of Estimators in Multiparameter Cases 269 Ve.e{n2(B„°-B)}=i +o(n z), then we can transform {0n°} into {Bn°#} such that the latter is second order AMU. In a similar way as [3], if there exists the second order asymptotic distribution, it is shown that {9n°#} must be second order asymptotic efficient. We assume that {Bn°} has a second order asymptotic distribution and _ 1 1 _ 1 EB. e {n2(Bn°-6)} =c1(0, ^)n 2 +o(n 2) ; Ve.e{n2(On°-6)} =I*(6, )+o(n z) E°.e{n2(e n°-6)}3=cs (6, s^)n 2+o(n 2).

Let 0 0(= {en °}) and

^ °(= Ono

{^n

°}) be consistent estimators of 0 and ^, respectively. Putting

*=Bno-n

_1

1 C ,( 6 0, ^°)- W11* (0 0, S°)C3(6 °, ^°)},

we see that {&n°*} is second order asymptotically efficient. References [ 1 ] Akahira, M., "A note on the second order asymptotic efficiency of estimators in an autoregessive process ," Rep. Univ. Electro-Comm. 26-1, (1975). [ 2 ] Pfanzagl , J., "On asymptotically complete classes ," Proceedings of the Summer Research Institute on Statistical Inference for Stochastic Processes Vol. 2, (1975).

[ 3 ] Takeuchi K. and Akahira, M., "On the second order asymptotic efficiencies of estimators," to appear in Proceedings of the Third USSR-Japan Symposium on Probability Theory. Lecture Notes in Mathematics. Springer Verlag. [ 4 ] Takeuchi, K., "T6kei-teki suitei no Zenkinriron (Asymptotic Theory of Statistical Estimation)," (in Japanese), Kyoiku-Shuppan, Tokyo, (1974).

17 Rep. Univ. Electro-Comm. 27-1, (Sci. & Tech. Sect.), pp. 95-115 August, 1976

On Gram-Charlier-Edgeworth Type Expansion of the Sums of Random Variables (1)* Kei TAKEUCHI** and Masafumi AKAHIRA***

Abstract Gram-Charlier-Edgeworth type expansion of the sum of i. i. d. random variables without higher order moments are given. Exact formulas for the asymptotic expansion of the density for special cases are given and it is shown that the results are generalized to the case when the density function is approximated by a rational function.

1. Introduction Gram-Charlier-Edgeworth ( G-C-E) expansion is applied to the sum of independently and identically distributed ( i. i. d.) random variables which satisfy some regularity conditions. More precisely suppose that X1, X2, • • •, X,,, • • • are i. i. d. random variables whose characteristic function 0(t) (=E(exp it X;)) satisfies the following :

(i) q5(t) is k-times continuously differentiable at t=0, which implies that moments of X; up to the k-th order exist. (ii) For any p> O

sup I O(t) I <1.

Itl>p (iii) For some m>0,

S cb (t) i mdt<

oo.

Then it can be shown in [4] that the usual Gram-Charlier-Edgeworth expansion ([3]) is ap-

plicable both to the density and the distribution functipns of TX;. 1 The above regularity conditions are not the widest possible [1] but similar necessary conditions exclude various cases which are not altogether trivial. For example the condition (ii) excludes discrete distributions, and (i) does the distributions which have infinite moments. The purpose of this paper is to show that expansions similar to the G-C-E expansion can be obtained for cases where the above regularity conditions do not hold true.

The case when Xi's are integer-valued and have moments was discussed by one of the authers in [4] where it was shown that * Received on April 27, 1976

The results of this paper have first been presented at the Third USSR-Japan Symposium on Probability Theory at Tashkent, August 1975. ** University of Tokyo *** Statistical Laboratory, University of Electro-Communications

18 96 Kei TAKEUCHI and Masafumi AKAHIRA

n

Pr{
z h5(z)-hi(z) +o 24n 2n 24vzn (--) hz(z)+hs(z)+L =O(z)-cP(z) 1 6 V3 -n where p=E(Xt), az=V(Xi), flz=E(Xi-p)3/a3, ,84= {E(Xi-ft)4/v4} -3 and P(z)=3: cp(u)du ith ip(u)=(1l

w { d' e-(z 'iz)

}.

2j)e-<" 2

and hi(z) is the Hermite 's polynomials i.e. hi (z)=(-1)' ez.'zx

Further expansions for this case was also given in [4].

In this paper we mainly deal with the case when moments do not exist. If the second order moments exist , but higher order ones do not, then we may obtain the expansion of the type

Pr{ 1

3(z)+... ng 1/ n a Cz)=O(z)+U n

but gs(z), ... are no longer equal to the derivatives of tp(z).

When even the second order

moments do not exist, but the distribution of X, belongs to the domain of attraction of the stable law of the exponent a, then we may get an expansion n

EXi-bn Pr{ I
2. Continuous cases We first note that if the characteristic function of the distribution is integrable, then it is absolutely continuous, and the density function is obtained by the Fourier transform of the characteristic function.

Example 2 .1. t-distributions with odd degrees of freedom : suppose that Xi's are i. i. d. random variables having the continuous distribution with the density f(x)=-

1

B(2 , p- 2) (1+x2)' where P =l, 2, • • • . This distribution does not have the moments of higher order than (2p-2)-th order. We shall obtain the characteristic function O,(t) of the distribution.

Put lv(•x)=2^c

Since

itl v-ie-iti-u=dt.

19 August, 1976 On Gram-Charlier-Edgeworth Type Expansion of the Sums

lP(x)=2n Jo tP-ie =ZZ,j'(P){(1

1t(l+1 p

- (i+") xdt+ 1 tP- 1e (1-i1)Xdt

+i x)P +(1-ix)P}

(2. 1)

(1-1C2x2 +1C4x4-•••)

where ,,C1 ( i = 2, 4, • • •) afire binomial coefficients , it follows that ./ P(x)=ap, il1 (x)+a,. 212 ( x)+... +ap, P IP(x),

where for each i =1, • • •, p, ap, i is some constant. Hence we have

OP(t)= ( a ,, i+aP, 2 ItI+ ... + a p. PIt I P - 1 ) e- M. Since OP(0) =1, it is seen that a,,1=1. From above we obtain

1090P(t)=- ItI+log(1 +ap,2 Itl +... +a P,PIt IP-'). (I) p>2. Since EP(x)=Sxfp(x)dx=0 (p>2), expanding log O,(t) we have log 'YP(t)=- CP.2t2+24 CP, 4t4+...

CP

+6

. 3 ItI3 .3 ItI3

1 ,5ItI5+...

2

+11 +1 CP

where for each i = 2, 3, • • •, c,, t is some constant. Since there exist the moments of Xi up to (2p-2)-th order, it follows that

,5=...= CP•3=CP

CP.2P-3=0.

Letting ^„ (t)=E[exp(itZ„ )] with Z „=E (Xt-lt)/(1/ n o), we obtain log^.(t)=n log ^P( no) 76 0 2+24 n + 720n2+••• 5

5

+6 n1/2 + 120n312 ... =e-

(012)Il+

r3

6n1t2

i'+ + 144n"2 l t

It1

3+

,4

t4+

L32

t5 +

24n 72n s

1296n5/2 I

t I9 +...

where for each i = 3, 4, • • •, 7t is some constant. For each odd number r we define gr(x) and Gr(x) as follows : g,-(x)

1 ( It I re-(t - 12)e-itmdt. 27C

Gr(x)=c gr(u)du.

Then the density function f„ (z) is given by

75 120n312

I t 15

20 Kei TAKEUCHI and Masafumi AKAHIRA

98

a

n(z)= 1 e-(:-/z)11+24nHe(z)+72nH6(z)+... 7374 +144n312

73 3 + 1296 ns/ag9(x) + ... Further the distribution function of Z. is obtained as follows : z F.(z)=O(z)-tP(z) 24nHa(z)+7 32 H5(z)+...

+ 673/-G3 (z) + 120n3/2Gs(z) + 144n3 /2G7(z)

where Gi(z)=S: g;(x)dx. Since the distribution of Z. is symmetric, it follows that Gl(0)=0. By (4. 4) of Appendix of section 4 we have Gt(x)= e-(x=12)et21 2 x o Hence it is clear that gi(x) is also calculated. Since I t se-(2 /z)-uxdt g3(x)= 1 2icJ_. 1 tz

2Z _.

t I e- (t= /z)-t txdt

z = (-1)dxagl(x) dx3G1(x), it follows from (2. 2) that g3(x) is calculated.

In general we have for each odd number r gr(x)=(-1)(r-1)

/zdxrG

l(x) ;

1

Gr(x)=(- 1)(r- 1)/zd r- G t ( x ) . Futher since

d Gi(

x)=g1(x)= 1 -xGi(x),

repeating differentials we obtain d dxaGl(x) _ -Gi(x)-xgi(x) =(x2-1)Gi(x)- ;

dt.

(2.

2)

21

August, 1976 On Gram-Charlier-Edgeworth Type Expansion of the Sums

99

3

d 3G1 (x)= 2xGi(x)+(x2-1)g=(x)-1 _ - (x'- 3x)Gl(x)+!(X2 -2); 4

d GI(x)-(x'-6x2+3)Gl(x)- IW-5x);

Hence we generally have for each odd number r T

- T 1(x)=(-1)rHr(x)Gi(x)- 4-Kr-i(x), where Kr -i is the polynomial of (r-1)-th degree. For large x, Gi(x) may be expanded as follows : e-<=*iz>ecx-0-i2 du o

Gi(x)=

= 1 e W/2)-ma du It o 2

4

_ (1+ 2 +

8

6 +... ) +48+... , du

(_L+

+...

it For example we have G3(x)-(xs-3x)Gi(x)+ (x2-2)

it (x2+x4+...t. In general Gi(x) and gi(x) have for large x the order of x-r" and that of x-r, respectively.

(II) p=2.

In this case if follows that f2(x)= 2 X(l+xz)z i(x)= 1 7t(1+x2) 1 l2(x) = It 1+x2)2 ( 1- x2). (

Since f 2(x)=11(x )+ 12(x), the

characteristic function is 02(t)=(l + I t j )e-M.

In a similar way as the case when p>2, the characteristic function ^„ (t) of Z. is given by /

an(t)=e-u'/z)(1+

8

4

6

3I1^ n 4n+18n+...

Hence the distribution function Fn(z) of Z. is expanded as follows : Fn(z)-4(z)+3 nG3(z)-co(z ){4nHs(z )- 18nHs(z) I ...j.

(III) p=1.

Since f i(x) is the Cauchy density function , it is easily seen that the distribu-

tion of Z. is also the Cauchy distribution. Example 2.2. (Non-symmetric case) Suppose that Xi's are i. i. d. random variables with the density function f v(x) as follows :

22 100 Kei TAKEUCHI and Masafumi AKAHIRA

P-1 x>0; fP(x)= (1+x)P'

0 , x<0, where p>1. We shall first obtain the characteristic function Op(t) (=E(expitX;)). Since

OP(t)= (p-1)ei, dx, (2.3) o (1+x)P we have for p>2. OP1(t)=

dtO P(t)

_ (p-1)ixe`txdx

J (1+x)P p_21oP- 1(t)-iov(t)• (2.4) On the other hand applying integration by parts to the right- hand side of (2.3) we obtain

(2.5)

0P(t)=1+pit20P-1(t) Substituting (2.5) into (2.4) we have

(2.6)

OP'(t)=(pt 11-i)OP(t)-pt In order to solve the differential equation (2.6) we put gp'(t)=(L-!_ i)(t). If t>0, then it follows that g,(t)=Cite-le-rt

(2.7)

'P(t)=O (t)g,( 0

(2.8)

where Ci is integration constant. Putting we have 'V P'(t

) _ ^P( t )g P'(t ),/+ ^P'(t)g P(t)

=( pt 1

-i)YP(t) +^P'( t )gP(t)

Then it follows that (/1 P'(t

)g P (t) =

-(p- 1)/t.

Hence it is easily seen from (2. 7) that

0P (t)=-(p-1)Ci- 1t- Pert. Considering that

0p(0) =1, we have from (2.7), (2.8)

(2.9)

and (2.9)

` OP(t)=(p-1)(^u-Per°du+C2)tP-Ie-rt r

for t>0

(2.10)

where C2 is integration constant. It is seen from (2.5) that the solution (2. 10) of (2.6) is also applicable to the case when 1
23 August, 1976 On Gram-Charlier-Edgeworth Type Expansion of the Sums

101

Since ttt1 -11 elM

0p(t )=

c°° (p-1)e'tcx+u tom o t °(l+x)° tdx

= lim r(P- 1)e`vdy 11 P

t-roo t

=0 it follows that C2=0. Hence we have for t > 0 ^(t) _ (P-1)(S°°u-Pe'"du)tp- 1e-it. r

(2. 11)

Next we consider the case when t<0. Since 0p(t) is the conjugate complex number of 0,(- t),

it follows that for every t<0 I Op(t)=(P-1)(Sltlu-Pe - iudu) t I P-le-it

(2. 12)

If t is a small and positive number, then Op(t) may be expanded as follows.

Put Ap +iBp=

S

1°°u-

Pe'udu.

Since U2 (iU)P-1 1 u ° e"`-1-iu + 2 +...JS (for integer p) (P-1)! is bounded for 0
r

UE

(iU)P-11

Cp+iD,=Sou- P e'"-1-iu + 2 -}-••.- 1 du (for integer p)

we have Sou Pe'"du =Ap+iBp+ S u-pe'udu r r 1 =Ap+iBp+S u-P er "- l-iu-...- (iu)p-1 du

r (p-1)! J

°_1 1 zu u-P(1+iu +...+ ( ) Jdu t (p-1)! p 1 =ap+ipp- Ku-Pfer"-1 - iu-...-((pu ) 1)! }du

+cr u- p l+iu +•••+ p{p!

(1 Zr)P 1

1

(P -1)!

= «n +i^p-Sou

du

(p+1)! ((p+2)! +...}du

+ u-P+iu1- °+...+ zp 1u 1 du Sr { (p- l)! }

for integer p, where ap=Ap+Cp and ,9p=Bp+Dp. If p(> 1) is integer, then

(2. 13)

24 102

Kei TAKEUCHI and Masafumi AKAHIRA

u

pe`udu = ap+ip,-

Jr

Jo

u-

p{(Z

U )p+

+1

(Z1

+ ( ZU )p +2

p! (p+ _}..2)! i)! (p

+.„}du

P-1 ik 1

+E

-S

k=Ok! r

u k -Pdu

ip t- ip+1 t2+ iv t3+ =av+i^p- -

p! 2(p +1)! 3(p+2)!

p +

- 2 zk

log t

1 ( 1-tk-P+I)- iP-1

k=0 k! k-p+ / (p-1)!

_

P-2 ik 1

iP-1

ip

+1 =(ap+ap' )+i(QP+RP')-kEk!k-p+1tk (p-1)! logt-pit

ip+l

3 2 2(p+l)!t + 3(p+2)!t +...

where ap' and 9p' denote the real part and the imaginary part of p-2 ik 1

= k! k-p+1 respectively. Putting ap*=ap+app'' and ,p*=/p+Rp' we obtain

Jr u

pe'udu=ap*+ifap*

p-2 Zk 1 E

k-ok! k-p+1

_

ip-1 iP tk-P+1_ -t log t-

(p-1)! p!

2

3

2(p+1)!t +3(p+2)!t +... From (2. 11) we have for small t>O OP(t)=(p-1)(, U-"e-'"du)tP-le-tt r /f p-2 iv-1 iP = (p- 1)f 0 Zkaktk-(p-1)I tP-11og I t I +ap*t P-1+iRv#t P- 1- *'tP

+0(1 t I P) }(1 - tt- 2 t2+ 6t9+...) =

/, 11-2 + ..h -l) k Zkaktk-( O

(C

ZP-1

p-2)!

ip

(p

tP

-(p -1) EOik+laktk+l+(p

M

- i(p- 1)a p *tP +(p- 1)Qv *tP+ 6 (p-1) E ik

+

1aktk

..//,^

n

P -l logl tI +(C- 1)ap * tP-1 + i( p- W P* -1

2

z 2)! tp

log I t

(P - 1) F, ika*tk+2

+s+...+ao(^ 21M tv

+o(I t I P)

(2.14)

For t
=a2+iP2 -Sru-21(Z1)2+ (26)2+...}du+

25 August, 1976 On Gram-Charlier-Edgeworth Type Expansion of the Sums 103

+t-1-ilogt =az-1+i/32+ t -i log t+ t + Similarly we have u- 2e"`du= a 2- I+i(32+ 1 +i

iti

I

t

logItI +ItI+....

I

2

From (2. 11) and (2. 12) we obtain

02(t)=1 - it log1 tI + (as -1)ItI +(/32-1)it

+t2logI tI +13 2t2- i(a2 -1 ) tIt I +o(t2). Since a2+i/2 = 5u 2(e'n - 1-iu)du +^u-2eiudu o i

it follows that az=S0u-2(cos u-1)du+cu-z cos udu ; ri (3z= 0u- 2(sin u -u)du+Tu-z sin udu.

J

We can calculate a2 as follows : a2=f'u-2(cos u-1)du+1 0 - sin u du+1 0 o U

= -u-' (cos u-1)

=1-2 Since the value of /32 is not easily obtained, putting /z=r+1 we have

02(t)=1- i tlogI tI -

2 I tI +irt+t2logI tI

+(r+1)t2+i 2 tI t I +o(t2) Letting 0n ( t)=E[exp(i tZn)] with Z„=(EX;-an)' bn we obtain log 0n( t)=n log 02(bn) -ibn

=-ibnt-ibnt(logl tI -logbn)-2b-I tj+ibrt z

z

+bn2 (log I t I -log bn)+(r+1)b -j+=2bn2t I t I -

n t2

2bn2(log I t I - log bn)2 +sbn2(1G2-472)

-Z2b„zt!t1 +o(bn2). If bn= n and an =nr+n log n, then

(2. 15)

26 104

Kei TAKEUCHI and Masafumi AKAHIRA

log an(t)=-it logltl - 2 ItI -lognt2+ 1 t2 log ltl + r+lt2 n n n z

+i Znt

I t I- (lo2n) t2+ on nn t2 log I t

t2(log

Tn

I t I )2+ 8n(Z2-4, 2)t2 -i2n t I t I

+o (1). n

function with the characteristic function expj - 2 It I -it log I t I }. This is the spacial case of a stable law with the characteristic exponent 1([1], [2]). By the expansion of the asymptotic distribution, it follows that the order of the next term is (log n)2/n. Next we consider the case when p=3. ` We calculate a3 and 8s as follows : a3=`ou 3jcosu-1+ 22Idu+ u scosudu

2z(cosu-1+ 2 )

- 2 cos u

I I 2 u-

1- 1

lu-2(sin u-u)du

J 020

2 sin u du

1 j9z- - 7 1 . 4 2 2 4' Qs Sou - 3(sin u -u)du+ U- 3 sin u du I 1 i

z 2

(sin u-u)

o+ 2

u-2 2 sinU ^+2

S

S

ou'2

ou 3(cos u -1)du

cosudu

=2+2=1-4. From (2. 13) we have for small t>0 21u-3ei°du=-r-

z +i(2-" -2cu 3(e"`-1-iu+ 2 du

uz +22tu-3(1+iu- 2 )du =-r- 2+2-2-3 -(1-t1-2i(1-t + log t Hence we have from (2. 11) and (2. 12)

+o(t).

I

27

August, 1976 On Gram-Charlier-Edgeworth Type Expansion of the Sums 105 a

s

03(t)= tE1 -it- 2 +it + X xlt +t:+logIt^-r-2-i2 ^tI-3t+o(ItI)] =1+it+tz log I t I -rtz-i 2 t I t I +o(ta).

(2. 16)

Letting ^„(t)=E[exp(itZ„)] with Z„=^(X;-1)/ ✓nlogn we olttain

log (t)= -4 log nt + n logos(Vn -log n) E

1 gn(logitj-log g no n)-log nt2 _i irt1tI - 1 tz+o 2logn 2logn (FO91 )

1 +loglogn+2r+1 tz logn lognl l

+ log n tz log

It - t

2log

nt t + o (

log n/'

Putting c„z=1+loglog+2r+1 log n log n

we have n(t)=(1+ logn(tzlogjtI - i 2tItI)}exp( ct2t2)

1

l 1 +o (log n

(2. 17)

It is clear that the first term of the right-hand side of (2. 17) corresponds to the normal distribution. In order to obtain the density function of Z. corresponding to the right-hand side

of (2. 17) we need the Fourier transforms of (t2 log I t I )e- /2) and it l i le- ('1/z) will be done in section 4 in detail. Further we consider the case when p=4.

Then we calculate a4 and ^4 as follows : a4=

i1-

uz -4(cos u-1+ 2 )du+S1 u-4cos udu

=-U33(cosu-1+2)l0 3`ou-3(sin u-u)du

-u3scos uIW- 3Tu -5sinudu

The discussion

28 106 Kei TAKEUCHI and Masafumi AKAHIRA

A4

I / us = f ou-4I sin u-u+ 6 ^du+^I u-4 sin u du =-u33lsin u-u+ 6310+ 3Tu 3(cosu-1+

2)du

u-3 - 3 sin u I+ U- 3 cos u du 5 as

18+ 3 7_r 36 6 From (2. 13) we have for small t>0 3^u-4e" `du= 3(a4+ii4 )- 3ru-4 ( ,i-- 1 - iu+ 22+z63 du z o \

+3f u-4(1 + iu- Z -z63'du

= 3(aa +ifi4) -z8

-11- t

)-

3 i (1 - 72

+ Z (1- t,+ 2 logt+o(t). Hence it follows from (2. 11) and (2. 12) that t3 Zt3

04(t)=I1-Zt- - - + -- - + - .. 2 6 •I 1+ 6it - 2t2+ 2tIlogIt^+...1 22t3logItI+•••

(2. 18)

Letting ^ „ ( t)=E[exp(i t Z.)] with Z„=2I E X ; - 2 )'n3we obtain

n\ log 0.(t)=_ 3ynit+n log04

t VK

/ logn t2 _4logn . s z t +0 n 3n1V n . 2 3✓

Hence it follows that an(t)=e-"-/2) 1-4 log nit3+o( log n n ( 3n 3 11-

(2. 19)

Corresponding to the right-hand side of (2. 19) the density function of Z. is expanded as follows : 2S c22i2>^1- 3V3n (z3 -3z) }+oI log J. Example 2.3. (Symmetric case) J

Let X; (i = 1, 2, • • •) be i. i. d. random variables with the density function f p(x) as follows : fp(x)=

p- 1 2(1 +1xl)p'

_oo <x
where p> 1. In order to calculate the characteristic function Op(t) (=E(expitX;)), we have

(2. 20)

29

August, 1976 On Gram-Charlier-Edgeworth Type Expansion of the Sums

107

(p_1)' it ^v(t)= J_ 0o 2( 1 +I x I)pdx 1)eitxdx (p-1)et udx+T (pJ ao 2( 1-x)° o 2(1+x)° = f°° (p-1)e-itxdx+ ( p-l)eiexdx. T . 2(1+x)' o 2(1+x)p Since

c(p- 1)e_itxl {2(1+x)p}dx

is the conjugate complex number of

{2(1+x)p}dx, the characteristic function

To (p-

1)e'txl

0p(t) agrees with the real part of the functions (2. 11)

and (2 . 12) in the case of Example 2. 2. If p is integer, then it follows from (2 . 15), (2. 16) and (2 . 14) that for every t

¢2(t)=1- Z ItI-t2logItI +(r+1)t2+0(t2); 03(t) =1+ t2 logI tI-yt2_

(2.21)

Itl3 +0(ItI3)

(2.22)

Op(t)+ - 1 - p-1 +p-1 t2+o(t2), (p>4). (2.23) { 2 2(p-3) p-2 When p=2, letting On(t)=E[exp(itZn)] with Z„=EXrln we obtain log ¢„(t)=n log cb2 n ) =- 2ItI

` 2

+n(logltl- logn)+

2 r +1 n t,

z _8n t2+o(t2).

Hence it follows that On(t)=a-c "i2 t : ji-log nt2+ n t2logl tI +8(7 8n)-x2t2 +o(t2).

(2.

24)

The distribution function with the characteristic function e-("i2)1tt is the Cauchy distribution having the following density function f (x) : 2 f (x) =tt2+4 x2 . In order to obtain the density function of Zn corresponding to the right-hand side of (2. 24), we need the Fourier transforms of tee-("12)jt l and (t2 log I t I )e-(,/2)Jt!. The discussion will be done in section 4 in detail. n

When p= 3, putting Z„= log n (

i

Xtl

l^ n log n, we have

1 1

2

t)=log n

(logItl- log n -

z rt log n

_z

loglog n

3 t

n (log n)3/2 1 2V -

I +0

n

Hence it follows that 1

1

l

an(t)=a-t'1z 1+lognt2logl t I +2(logn)Zt°(log t)2+... J (2.25)

1

30 108 Kei TAKEUCHI and Masafumi AKAHIRA

Then we note that the expansion of the power of 1/log n is obtained. In section 3 we shall get the density function corresponding to the second term of the right-hand side of (2. 25). But it is difficult to obtain the density function corresponding to the third term of the righthand side of (2. 25).

The asymptotic expansions given as above are also applicable to the other distribution cases. Example 2 .4. Let Xi's (i =1, 2, • • •) be i. i. d. random variables with the density function f (x) as follows : .f W= 2(1+ X 2)3/2

g(x)=f(x)- 1 2(1+IxI)3 we have c1129(71)d1/= 2 log

( +x)- 21/1-Fx2

- 2 log(l+x)-(l+x-1)+{(1+x)2-1{ = 2 log {( +x)/(x+ 1)} - 2 (1+x)2' Hence is follows that

J x2g(x)dx= log 2. Since (x)dx= 2 ; ^_O^xg( x)dx=0,

it is seen that log

eitzg (x)dx=

2t2+0(t2)

2+

By Example 2.3 we obtain 1 T errsf (x)dx=

err: x )sdx+ _We'rsg(x)dx

2 (1+

= 1+ 2 t2 log j t j_ log

Putting Z„=E Xr/

^+

7t2+o(t2).

n logg we see that Z. is asymptotically normal and the next order of

the expansion is given by (log n)-'. Example 2.5. Let Ur (i =1, 2, • • •) be i. i. d. random variables with the uniform distribution on the open interval (0, 1).

Put X, Since

2 (1 Ur

1(

Ur) (1=1, 2, ...).

1 _1 2 1u u

x= -

31 August, 1976

On Gram-Charlier-Edgeworth Type Expansion of the Sums 109

it follows that 1 dx__1 1 du 2 {u2 +( 1-u)2 = 2 ru2(11 u)2 -u(12 u) ^, On the other hand since X

2 (1-2u)2 _ 1 _ 1 u(1-u)'

4u2(l-u)2 4u2(1-u)2

we have

2u(1 - u) -1. Hence dx=2(x2+l +Vx2+1 ). du Therefore the density function f (x) of X; is given by AX) = 1 2(x2+ 1 +Vx2+ 1)'

g(x) __ 2(1

1 +x2)(1+x2+V1+x2)

f(x)=2(1+

2(1+x2)3/2+g(x)•

x2)

f (x)dx=1; Sg(x)dx =1-

2

+l=2-

5

2

;

5x2g(x)dx= {f(x)-g(x)}dx=

Z -1.

It follows by Example 2. 4 that the characteristic function 0(t) of X; is given as follows : fi(t)-1+ 2 e-1°1 - c log It I

tz 2 2 -1 t2 +(t2)

logjtj+(7+ Z)t 2 +o(t2).

=1- 2 jtjZ

Then the asymptotic expansion of the distribution of

X;'n agrees essentially with the case

when the density function is 1/ {2( 1+ I x 1 2)z} The above results obtained may be generalized into more general case. Suppose that Xi's are i. i. d. random variables with a density f (x) such that (i) f(x)=O

for x
(ii) there exists a bounded rational function g(x) such that for some positive integer m Tx*{f(x)-g(x)}dx=MR , exist.

k=0, 1, •••, m

32 110 Kei TAKEUCHI and Masafumi AKAHIRA

Then the characteristic function of Xi is expressed as E(eitx

i)=Jo

et1xg (x)dx+ eiDF{f(x)-g(x)}dx

= (°°ertxg(x)dx 0 +jMo+M1it+•••+ k k(it )k}+o(ItV)• On the other hand g(x) may be decomposed as g(x)=E j kEI(xc'a,)k+E s^ {r;+( xjk Qi)2}s Since g (x) is bounded , ai<0 for all j, hence

c'ketLZ dx can be expanded around t=0, o (x-ai)k

as was done in Example 2. 2., and r°° d;k eits djk eitp; eitx dik eits Jo {7i+(x-(3i)2} sdx=^ (7i+x2)s dx+Jo {ri+(x-^3i)2}sdx

(2. 26)

Then the second term of (2. 26) is expanded as in Example 2. 1 and the third one is easily This establishes the expansion of E(e112) and n we can get the asymptotic expansion of the distribution of E Xi similarly as in Examples 2. 4 1

shown to be expanded into a polynomial of t.

and 2. 5. For more general cases if the density f (x) satisfies the assumption that f +(x) and f -(x) defined by f+(x)- 0 If (x)

for x<0; for x>0;

- _ 0 for x<0; f (x)-{f(-x) for x>0; satisfy the condition (ii), then the above arguments are applicable to f + and f - separately and we get a similar expansion.

3. Lattice cases Let Xi (i=1, 2, • • •) be non- negative integer -valued random variables with P r {X i = x }= Pk ( x )=

(k-1)! x =01 ... k>2 . (x+1)•••(x+k)'

Putting. Ok(B)= E 6XPk(x)

s-o

we have E Bx(k-1)!

dk

aek j0k (0 ) 0k } =(k -1) ! x=0 1-0 Then 4'k(0)

=6 -1

( 0-0 -1 0 1 -rf

dri + x(O),

where x(0) is some polynomial of (k-1 )th degree. Hence the characteristic function of Xi is given by Ok(eit) and

33 111

August, 1976 On Gram-Charlier-Edgeworth Type Expansion of the Sums a

1

x

Pr^E Xi=x = {(Pk(ei:)}"e-ia=dt. 2n'

For example we consider the case when k=2. Then 02(0) =8 -'

r

B _ 2dr7 +a9 +b

=(1-B) 0x(1-B)+ +a9+b. 2

(3. 1)

It follows from (3. 1) that a=b=0. Hence we have 0(t)=02

(eit)=e - 2it(1 - ei`)log ( 1-ei`)+e-n.

Since log(1-eit)=log(-it)+log{(1-e1 )/(-it)} =logItI-i 2itl+log(1+2+ i Z it +o(I tI =login -i2 tl+ it follows that 0(t)=-it {logItj-i

2 ^tI-

it)+ l-it+o(ItI)

=1-itlogltl-2 ItI-it+o(ItI). Putting z=(x-nlogn + n)/n, we have Its-itlogltl-itzldt. PrfExi=x}-^1 exp(i 2ntc 2

In a similar way as the continuous case we may be able to expand up to the higher orders. Similarly we may expand the probability of E Xi for any integer k. Also as was done 1 in section 2, we can expand the characteristic function if Pr {X i = x} = p( x) satisfies the following : (i) p(x)=O

for x
(ii) there are constants a2, a3, •••, am such that

=_o

xk{p(x)_a2p2 (x)-...-a,,,pm(x)} =Mk

for k = 0, 1, •••, I exist and is finite.

4. Appendix In order to get the density functions corresponding to the characteristic functions &W given in sections 2 and 3 we need the Fourier transforms of it i t l e-i'12, tee-(-/2)111 and t2(logItl)e-11 1.

(log I t I )e-D'iz,

For the purpose we shall calculate gi(x)

Hence we have

Tt(cos tx) e-d'i2dt (4. 1)

34 112 Kei TAKEUCHI and Masafumi AKAHIRA

G(x)=^ gi(t)dt= 1 (sin tx)e °''zdt. a Z o

Applying integration by parts to (4. 1) we obtain gi(x)= 1 (-cos tx) e-t'iz o

Tx(sin tx)e-d'Izdt

-xGi(x). Putting Gl(x) = u(x) e-x''z

(4.

2)

we have gi(x)=Glt (x) =u!(x)e - x''2-xu (

x)e-x'/z

=ut (x)e-x='z-xGi(x). Then it follows that (4.3)

U ( X ) 1 cet'izdt+C Ico where C is an integration constant.

If G,(0)= 0, it is seen from (4. 2) and (4.3) that C=O. From (4. 2) and (4. 3) we have Gl(x)= e-x'12 et2t2 dt. (4.4) Next we calculate th Fourier transform of it It I e-' 1/2.

By (4.4) we have

1 `itItle-(0/2)-ttxdt

2j d 1 (I t i e ct=iz^ ttx dt

d x 2^c _ -glt (x) dz dxzGi(x) - x -(xz-1)Gi(x).

(4. 5)

t I )e-t'iz and (t2 log I t I )e-t2/2. Further we shall calculate the Fourier transforms of (log I Put K(x)

2^ (

log I t I )e - (t'iz) -its dt ;

-ttxdt. K*(x)=2^ ( log t)e-W12)

Then it follows that K(x)=K*(x)+K*(-x).

The first and second derivatives of K*(x) are

given as follows : K*t (x)

3

2x o ( 1

K*t'(x)= _

27c o Applying integration by parts we have

t log t ) e-(t =iz) -ttx dt

(t2 log t)e-(1'12)-itxdt.

35

August, 1976 On Gram-Charlier-Edgeworth Type Expansion of the Sums

113

K*"(x)= -2 (1+ log t-ixt log t)e-(`'iz)-t12dt

1 e-(x=12) -K*(x)-xK*(x). 2 ✓ 2n Hence K*(x) is the solution of the second order differential equation 1 y"+xy'+y= - 2 cp(x)' where

cp(x)=

(4.6)

e`/2. 1 12tc

In order to solve (4. 6), we put y"+xy'+y=0• (4.7) Since y"=-(xy'+y)=-(xy)', it follows that y=cie-x ' i2+c2, where cl and Cl are integration constants.

Hence y=cp(x) is the solution of (4. 7).

Putting y=h(x)cp(x), we substitute it into

(4. 6). Then it follows that h"cp+2h'cp'+hcp"+xh'cp+xhcp'+hrp=cp. Since (p'=-xcp and cp"=(x2-1) cp, we have

If h"-xh'= O, then h'=ex'12.

Putting h '= g(x)exviz we obtain

g'e2"/2+gxe2 ' 12-gxex'12 =- 1 2' Since g' _ - 2 e` 12, it follows that g(x)= e-2"12dx+cl ;

h(x)=-

2S (Se

K*(x) - 2 co( x)

x-i2dx+ c)e2'i2dx +cz;

1% e-x212

dx+Cl) e"12dx+c21.

Since K(x) is an odd function , it is seen that K(x)=-(p(x)f S (ce--,12dx)ex'/ldx+C}, l00

(4.8)

where C=-K(0) 1 S(logItl)e-1"2dt. cp(0) 1/2^c Further it follows from (2. 1) that the Fourier transform of tee-("12)1`1 is given by z

(4x2+ tc2)g(1 22_2 . (4.9) Next we shall calculate the Fourier transform of t2(log I t j )e-''' i . Putting

h(x)=2^c J_oot2(log l H(x)

2ttJ_,

t I )e-i'x-1'1 dt .

o(log I t1 )e-«x-icidt

36 Kei TAKEUCHI and Masafumi AKAHIRA

114 we obtain

(4. 10)

h(x) _ - H" W, On the other hand putting H*(x)=2r( log t)e -' tx tdt, we have H(x)

2_c50 (log ( log

2-

t)e-1tx-tdt+2j t)e itx tdt +

.(log I t I )e-"'+adt

log I t I )e':x-tdt 2zc Jo (

=H*(x)+H*(-x). Since H`(x) 2wJo -(it log t)e-'tx-tdt, applying integration by parts we obtain H*' (X) 2ic(1+ix)Jo

(log t+l)e-X12-tdt

f °° o a-,tx tdt l+ix{H*(x)+2,cJ z 1 X-1 H*(x)+ 2ic(x- i)2

(4.11)

In order to solve (4. 11) putting H*(x) =.Lxi (x

we have *I_ _ L(x) L(x) H (x) (x-i)2+ x-i =-H*(x) FLU x-t x-i

Hence it follows that Lt(x)=27c(

1 i x-i) 2ic (1+ix)'

Since L(x)=-- log (1+ix)+C

=2^[ log ( 1+x2)+i tan -'x+C where C is an integration constant , it is seen that H*(x)=2tc(1+x2){ 2

log(1 +x2)-tan-'x+C} ;

(4. 12)

1 x2){- log(l+x2) -2x tan-'x+C}. H(x) = 2^c(1+

By the value of H(0) we may be able to decide that of C. From (4. 10) and (4. 12) h(x) is also obtained.

37 August, 1976 On Gram-Charlier-Edgeworth Type Expansion of the Sums

115

For example we shall get the density function corresponding to the right-hand side of (2. 17). Since from (4. 5) and (4. 8)

i t I t I e- (c•= /2)t =-itxdt = - 1 G1'' x

I 2ir 27

Cn3 Cn /

_oo(t3logI tI )e

(c.=/Zk=- =t=dt

= C .3

Kir

c n1 n !0 +4 the density function corresponding to (2. 17) is given as follows : J^

Cn nC13Kri( C, \nl+loge {C13GlrilC-n _ Cn/^+o\lo g n/

References [1] [2] [3] [4]

[5]

Gnedenko, B. V. and Kolmogorov, A. N., " Limit Distributions for Sums of Independent Random Variables," Addison-Wesley, Cambridge, Massachussets, 1954 (Translated from Russian). Ibragimov, I. A. and Linnik, Yu. V., "Independent and Stationary Sequences of Random Variables," Wolters-Noordhoff, the Netherland (1971) (Translated from Russian). Kendall, M. G. and Stuart, A., "The Advanced Theory of Statistics Volume 1" Charles Griffin, (1969). Takeuchi, K, " Kakuritsu-Bunpu no Kinji (Approximations of Probability Distributions)," (In Japanese), Kyoiku-Shuppan, Tokyo, (1975). Takeuchi, K. and Akahira, M., "On Gram-Charlier-Edgeworth type expansion of the sums of random variables (II)," Rep. Univ. Electro-Comm. 27-1, (1976).

38 Rep. Univ . Electro-Comm . 27-1, (Sci. & Tech. Sect. ), pp. 117- 123 August, 1976

On Gram-Charlier- Edgeworth Type Expansion of the Sums of Random Variables (H)* Kei TAKEUCHI** and Masafumi AKAHIRA***

Abstract Gram-Charlier-Edgeworth type expansions of the sums of independent random variables without higher order moments are given. It is shown that the asymptotic distribution functions for some cases are stable laws with fractional characteristic exponents, and the asymptotic expansion of the density function for one case is also given.

1. Introduction In the previous paper [4], we have obtained Gram-Charlier-Edgeworth (G.-C.-E.) type expansions of the sums of independent random variables with density functions which are approximated by rational functions. Here in this paper we shall obtain G.-C.-E. expansions for the case when the density function is of the type f(x)oc(1+x)-', x>0; =0, x<0 or f (x)oc( 1 -I- I X I)' which p is not an integer.

2. Non-symmetric case Let X1, X2, • • , X,,, • • • be i. i. d. random variables having the density f p(x) as follows : P-1 x>0; .f p(x)= (1+x)'' 0, x<0,

where p>1 and p is a non-integer. We shall first obtain the characteristic function Op(t) (=E(expitX;)).

By Example 2.2

of [4] we have r

(p- 1) u ' e "` du t '-' e - 'r , t>0;

(2. 1) t<0. u p e-rudu)ItIp-'e-`r, (p-1)( l I It

If t is a small and positive number, the

p(t) may be expanded as follows.

* received on June 9, 1976 The results of this paper have first been presented at the Third USSR-Japan Symposium on Probability Theory at Tashkent, August 1975.

** University of Tokyo *** Statistical Laboratory, University of Electro-Communications

39 11 8

Kei TAKEUCHI and Masafumi AKAHIRA

Put

(2.2)

Ap+iBp=u-pe'"du. I

Since z u-p e;"-1-iu+u +•••- (iu)CP] 2 Cp]! is bounded for 0
2 Cp]!

we have for non-integer

j du

(2.3)

P(> 1),

u-pe'"du =A,+iBp+c u-pe'"du r r

=Ap+iBp L

..}.. rru-pje '"- 1- iu- ... - (:u )Jdu J l Cp]• (iu)Ci ]l +fiu-p j1+iu +•••+ du

CP]•

=ap+ijp

° Cp]! i u- p j 1+iu -F•••+ (iu)Cf J ]l du +5 Cp7•

du

=ap +i^,

j (iu)[pJ +1 -

(iu)C/J+2

pl(Cp]+1)! +(Cp7+2)! +

}du --- ldu

J 1(U p}iul v+...+ i[pJuCp]-p1 +S ( du

Cp]!

1

= a p +i8p _ 1[P1+1 4 1 i[p]+2 °^(CP]+1)! u-P+Cp]+1+u-P+CP]+2+... }du

(Cp7+2) )

[p] ik r1 +

_J

uk-p

k=O k! r

=ap+ip, i1P1 +1

t- p+Cp] +2-

(Cp] +1)!(-p+Cp7+2)

j[p] +2

([p] +2)!(- p +[p] +3

) t- p+CP] +s

[P] ik 1 [P] ik tk-p+l

= k! k-p+1- E k! -p+l = ap +ap ' +i(18

[p] +8 ')-

z k t k-p+l

kk-o k! k-p ! 1

i[P]+1

(Cp7 + F)!( - p+ CpY+2) i[P]+2

t -P+lrJ+z

(Cp7 +2)!( - p +Cp7 + 3 ) t

p +1pJ +J_...

(2.4)

40 August, 1976 On Gram-Charlier-Edgeworth Type Expansion of the Sums

119

where app and Np' denote the real part and the imaginary part of [P] ik 1

k! k-p+l respectively, and ap=Ap+Cp and ip=B,+Dp. Put ap*=ap+ap' and Np*=Np+/p1.

From (2. 1) we have for small t>O

Op(t)= e- '(p-1)tp-1Ju-pe-`ndu e,

/

z

a t3+24+...

(p_1)ap*tP-1+i(p-1)Rp*tp-1- F (p-1)ikhk k=O k!(k- p+1) (p-1)i[Pl+1 t[P]+1

([p]+1)!(-p+[p]+2) (p- 1)i[p]+z t[p]+z-... } ([p]+2)!(-p+[p]+3)

(2.5)

For example we consider the case when 2O / z u-Pe'ndu=ap+iQp-^u-p (e"-1-iu+ 2 du •' +^tu p(1+iu- 2 Idu =ap+i/3p- Z t4-P- 1 (1-t1-p) 6(4-p) p-1

p 2(1- t2- p)_2(3 1

t3- a)

p)(1-

+o(t2).

From (2. 1) we have for small t>0 t2 it3

6 -...1

ap(t)= 1-it- 2

•f l+ p-2 it+(rp+i6p)tp-1+2 3-p)t2+...} 1

-1 +

p-Zit

+( rp+i8p )

1 )(3-p) t2+

tp 1+(p

...

2 where rp is some negative constant and 8p is some real constant. Hence it follows from (2. 1) that

¢ p(t) =1 +p 12 it + rp +iBpi t l I t I P -1 Letting

an(t)= E[exp ( itZn)]

+...

+(p 2)(3- p) t2

with Zn= n- '/(P -')E ^Xi -p

12/

we obtain log

a n( t )= ( rp

i5P -j_

) I t

I

P 1+ l(P-2)(3-C)

+

2

n- (3-010-1)t2

1

(2.6)

As immediately seen from above, the asymptotic distribution of Z. is a stable law with the characteristic exponent p-1 ([2], [3]) and the order of the next term is n-c3-p>i(p-1) (> n-1) Next we shall obtain the asymptotic expansion of the density function g(x : p-1) corresponding to On(t) given by (2. 6). From (2. 6) we have for 2
41 120 Kei TAKEUCHI and Masafumi AKAHIRA

c5

(t ) = exp {(rP +i8p

[

I t I)I t I v_i}]. ll+cpn-(

3-v)/(n-1)t2+ cp2n - 2(3-v)/(v-1 ) t4 _.1....

1 -1 (p-2)(3-p) 2.

where cp= p

It follows by Theorem 2. 4. 2 of [3] ( page 55) that for 2
`=expilrp+ihpl tl)I tI P-1}dt

o f- i)^+1

1'(n(p -1)+1) r 1 1 lx n(p-1) (2.7) sinl ^tcnjp -1-(3-p) n^ L rptanZ lc(p-1) J

holds as x-.oo. Hence for 2
1 °° s+1r(n (p-1)+1) 1 8p )-7tx

o f-1)

sin 27cnp-1-(3-p) s(p 1)

n^

rptan 2 (p-1)

holds as x-->oo. Next we shall calculateexplicitely the value off1 (2. 7) for 2 < p<3. We first have 2tcJ _

( +i6 , tlll tip-IJdt ^e itXeXpj(7,

=Zx (cos tx -i sintx) expj (rp + i8p tI)It I"Idt 1 cos tx) exp {(rp+ i8p)tp-1} dt 2tc3o ( +Zn^ .(cos tx)exp {(rp-i8p)(-t)p-1}dt =2x (cos tx) pexp {( r+i8p)tp-1} dt +2x, Jo (cos tx) exp {(rp-idp)tp-1}dt (2.8) Since for non-negative integer k and for a complex number a t2ke- atp-'dt

1 a-(2k + l)/(p-1) 1' 2k+ 1 p-1 ( P-1 ) ,

it follows that 1 cos 2^ o ( cos tx ) ex 2^cJo k ((2k)! ( 1 (-1)kx2k 27c _ (2k)!

tx)2kexp {(rp+iap)tp-1)dt

Jo

t2kexp{(rp+i8p)tp-'}dt

= 1 (-1)k x2k 1 (<2k +u/(p -1)r (2k+1 rp-i8p) 2n k=o (2k)! p p-1 1 r(2k+i

2z(p -1) k=o

I'(2k+1)

(-rp-i8p )-( 2k+1) /(p - 1)x2k.

(2.9)

42 August, 1976 On Gram-Charlier-Edgeworth Type Expansion of the Sums

121

Putting a,,,,te'bp'k_(-r,- z8,)-( +1)/(P-1)

we have from ( 2. 8) and (2. 9°)°' /I tl (rP+ z8, I l it I a-i}d t -1-r e-itx exp { 1 r (2k+1 /

x(p

-1)

E (- 1)k r(2k+ 1) a,, *(cos bp, ^)xzx

(2. 10)

for 2
r/2k+i 1

1 9p(x)= x

( --1) kE (- 1)k r(2k +

1)a,.^(cosb, k)xzk. (2.11)

The result (2. 10) is essentially same as that of Feller [1], page 549.

3. Symmetric case Let X, (i = 1, 2, • • •) be i. i. d. random variables with the density function f ,(x) as follows : .fv(x)= p-1 2(1+ IxJ)a where p>1 and p is a non-integer.

oo<x

In order to calculate the characteristic function O p(t) (_ (exp i tXi)), we have (p-1)eitx Op(t)= E2(1+Ixl)pdx (p-1)ei`xdx+ (P-1)euxdx J 2(1-x) p o 2(1+x)' =r°° (p-1)e-it.=dx+f°°(p-l)eit:dx

Jo 2(1+x)' Since

TO(p-

0 2(1+x)'

1)ei=x'{2(l+x)'}dx is the conjugate complex number of

TO (p-

1)eit x'{2(1+x)'}

• dx, the characteristic function O,(t) agrees with the real part of the characteristic function (2. 1) in the case of section 2.

If p is non-integer, then it follows from (2. 5) that for every t O P( t

)={

1+(p- 1)ap * ItI'-' +A,tz +o(tz) for 1
1+Aptz+(p-1) ap*JtJ -1+o(JtJ-') for 3
where A, = 1 (2-p)(p-3). Note that A,<0 for p>3. Since ap*=

a , +a,'=A , +C, +a,',

it follows from (2. 2) and (2. 3) that a,* is the real part of (°°u pet"du +Slu''jei"-1 -iu+ uz+ ...-(zu)Cp]ldu+CL i 1

i o tt 2 [p]! J k=o k! k-p+l'

Hence it is also seen that for 1
43

122 Kei TAKEUCHI and Masatumi AKAHIRA

n

If 1
(p-1)ap* I t I P-1 _(p 2

P-3)/(P-1 )t2 _(p 21) 2 ap*a

I t 12( -1) +o(n-1)

for 1
1)2 ap * ' I t I2n 1 ) + 2 pn(P -3)/(P 1 ) t2+o(n ( P-3)/(P 1 )) for 2
<3. Hence it follows that e(P-1)aP* ItIP-1 {1 +2pn( P- 3)/( P- 1 )t2_(p

2 1)zap *=I tl2n 1 )l+o(n-1) for 1
{1-(p21)z

an(t)= J .I t I zn 1) + 2pn (P s)/( P 1)t2} +o ( n(P - 3) 1 (P - 1) ) for 23, then letting On(t)=E[exp(itZn)] with Zn=^X;IVn // log an(t)=n log gyp( n

we have

1

2 pt2+(p_ 1)ap* n(3-P)/21 t I P-1+o ( n(3-P)/2 ) for 3
for 5
2,t2-X2nY+o(n-1)

Hence it follows that elpt'

{1+(p-1)ap* n(3 -P)/21 t I P - 1} +o(n(3- P ) /2) for 3
an(t) = l elpt' 11-

2n t°)

+o(n

for 5
Considering the case when p=5/2, we have from (3. 1) 05/2(t)= 1

-rl 1 t 1 3

/2+5t2+o(t2)

where r'=- a5/z*>O. n

Letting an(t)=E[exp(itZn)] with Zn=n-213TX;, we obtain 1 log O (t)= -711 t 131 2+5n-1 /

3t2_r^n-1 I t 13+...

Hence it follows that Yn(t)=e 1+5n-1/3t2+ 2 n-2/3ta_2 -n-11 t 13+125n-lt6+...l. 2 -6 As is immediately seen fromabove, the expansion of the power of n-'/3 is obtained and the distribution function having the characteristic function e-r'111" is a stable law with characteristic exponent 3/2 ([2], [3]). It is easily seen that terms in the expansion of the density function corresponding to terms e-7'11t"'t2k is d2k t kdx2g3/2(x)r

where gs/2(x) is the density function of the stable law given by (2. 11). Finally it is remarked that when p>3, the leading term corresponds to the normal distribution but in the subsequent terms in the expansion of the characteristic function there appear

44 August, 1976 On Gram-Charlier-Edgeworth Type Expansion of the Sums 123

terms like I t I ke-t'/2 where k is not an integer, and the density function corresponding to this is tk(cos tx)e-1112dt r s"

1

tk +2! (

XJ0

1

-x2)'

I,(k+2^ +11

E 2(k+2./-1 )/ 2 Z

j=O

e- t'l2dt

(2.1)!

r(2;+1)

(-x2)7.

References [ 1 ] Feller, W., "An Introduction to Probability Theory and Its Applications, Vol. II. John Wiley and Sons, New York, (1966). [ 2 ] Gnedenko, B. V. and Kolmogorov, A. N., " Limit Distributions for Sums of Independent Random Variables," Addision-Wesley, Cambridge, Massachussets, 1954 (Translated from Russian). [ 3 ] Ibragimov, I. A. and Linnik, Yu. V., " Independent and Stationary sequences of Random Variables," Wolters-Noordhoff, the Netherlands (1971) (Translated from Russian). E4] Takeuchi, K. and Akahira, M., "On Gram-Charlier-Edgeworth type expansion of the sums of random variables (I)," Rep. Univ. Electro-Comm. 27-1, (1976).

45 ON THE SECOND ORDER ASYMPTOTIC EFFICIENCIES OF ESTIMATORS *

Kei TAKEUCHI and Masafumi AKAHIRA University of Tokyo and University of Electro- Communications

1. Introduction Second order efficiency of asymptotically efficient estimators has been discussed by R.A.Fisher [6], C.R.Rao [9], [10] and others in terms of the loss of information. Recently Chibisov ([4], [5]) has shown that a ML (maximum likelihood) estimator is second order asymptotically efficient in some sense. Pfanzagl ([7], [8]) obtained similar results. One of the authors established similar results in a book written in Japanese [11] in terms of the asymptotic distribution of the estimators. In this paper we shall present the outline of the discussion given in [11] and proceed further to the third order asymptotic efficiency. Further it is shown that the results can be extended to non-regular situations. 2. Notations and definitions Let X be an abstract sample space whose generic point is denoted by x, B a cr-field of subsets of X, and let O be a parameter space, which is assumed to be an open set in a Euclidean 1-space R1. We shall denote by (X(), 1B(')) the n-fold direct products of (X, B). For each n = 1, 2, ..., the points of ;V(n) will be denoted by xn = (Xi, ... , xn). We consider a sequence of classes of probability measures {P,,,9 : 9 E 0} (n = 1, 2, ...) each defined on (X (n), 13(m)) such that f o r each n = 1, 2, ... and each 9 E 0 the following holds: Pn,9

(B(n) ) =

Pn

+1,B (B(n) X X)

for all B(n) E ,i3(n). An estimator of 0 is defined to be a sequence {9n} of 1(n)measurable functions 9n on X(n) into 0 (n = 1, 2, ...). For simplicity we denote an estimator as 9n instead of {9 n } . For an increasing sequence of positive numbers {c,,,} (limn,,) c,,, = oo) an estimator 9n is called consistent with order {c,,} (or {c,,,}consistent for short) if for every E > 0 and every '0 E 0, there exist a *This paper is retyped with the correction of typographical errors.

46

605

sufficiently small positive number S and a sufficiently large number L satisfying the following : lim sup

n->oo

e: 1e-191 <6

Pn,e

{ cn

I On - 01 > L j < s.

The order { cn} of convergence of consistent estimators and its bound are discussed in [1] and [11 ]. In the subsequent discussions we shall deal only with the case when cn = vfn-. Let On be a {}-consistent estimator. Definition 1. On is asymptotically median unbiased (or AMU for short) if for any i9 E O there exists a positive number S such that lixn sup

lPn ,O

{,^ (en - 0)

< 0} 2 I = 0'

lim sup

I Pn,e

{v' (On - 0)

> 0} - 2I = 0.

n-ioo 9:19-+9l<6

n-oo 8:Ie-,6I <5

Definition 2. For Bn asymptotically median unbiased Fo (t) is called an asymptotic distribution of it if li^lPn,e{ ✓n(On-0)
Since On is a {Vn}-consistent estimator, it follows that Fo(-00) _ 0 and FB oo) = 1. Let 9n be a AMU estimator. Then it follows that Fo(O) = 1/2 for all 0 E R Let 00 be arbitrary and fixed in

®. Putting An(t) = {in : en(xn ) - 90 < t/i }

we have

m Pn,6o {An ( t)} = Feo(t);

lim

Pn,eo

1 +(t/.r) {An(t)} = 2.

Consider the problem of testing hypothesis H+ : 0 = 9o + (t/vrn-) (t > 0) against K : 0 = 00. We shall denote by 8n(t, an, 00) the power of the most powerful level an test (0 < an < 1). Then we obtain for each t > 0 (2.1)

Feo (t) < 'UP { {«„}: lim„^_ «n = 1 /2} 1^Qn(t, an, 0) •

Denote by foo(t) the right- hand side of (2.1). Consider next the problem of testing hypothesis H- : 0 = 00 + (t/v/n-) (t < 0) against alternative K : 0 = 0o. In a similar way as the case t > 0 we define On (t, an, 0o) and 3eo (t). Then we have for each t<0

47

606 (2.2) Fe0(t) > 1 - /390(t).

Since 00 is arbitrary, the bounds of the asymptotic distributions of AMU estimators are obtained as follows : Fo(t) < p9(t)

for all t > 0;

FB(t)> 1-QB(t)

forallt<0.

For any 0 E 0 letting ,30(0) = 1/2 we make the following definition. Definition 3. An asymptotically median unbiased estimator 0n is called asymptotically efficient if for each 0 E O FB(t) _ ,(je( t) 1-Q9(t)

for all t > 0; forallt<0.

Definition 4. 8,,, is second order asymptotically median unbiased (or second order AMU for short) if for any 79 E O, there exists a positive number b such that Vfn-

lim sup

n-+oo 0:l6-,L9l <6

lim sup n-- °° 6 : 16-191<6

Pn,g

Pn,e

0}

2

= 0;

{Vrn- (en -0) >0 -2

=0.

{v'L(e n

-0)

Definition 5. For Bn second order asymptotically median unbiased FB(t) + (1/,/n)GB(t) is called a second order asymptotic distribution of it if l

{,'

< t} - FB(t ) - (1/ f)G0 (t)I = 0. (O - 0)-

im n n-too

v " Pn,O

Let 00 be arbitrary but fixed in O. Then we have Pn,o0 {An (t)} = Fe0(t) + 1 Go,(t) + o ( I Pn,eo+(t/, ) {An(t)} = 2 + 0 (n)

Consider the problem of testing hypothesis H+ : 0 = 00 + (t/V) (t > 0) against alternative K : 0 = O. Let 3,,(t, a,,, OO) denote the power of the most powerful level an test (0 < an < 1). We assume that sup {{"„}:"n=-z+O(*) }

0. ( t, an, 00)

48 607 = fea (t) + ^'Yeo (t) + o

(

)

.

Considering the case t < 0 in a similar way as the case t > 0, we make the following definition. Definition 6. A second order asymptotically median unbiased estimator 9n is called second order asymptotically efficient if for each 9Ee FB(t) = Ge(t) =

0e (t)

for all t > 0,

1 - /3e(t)

for all t < 0,

,yo (t) -ye(t)

for all t > 0, for all t < 0,

where for each 0 E O /3o(O) = 1/2 and ye(0) = 0. In a similar and obvious way we may define the third or k-th order asymptotic efficiency of estimators. 3. Asymptotic efficiency and second order asymptotic efficiency Let X1, X2i ... , Xn, ... be independently and identically distributed random variables with a density function f (x, 9) satisfying (i), (ii) and (iii). (i) {x : f (x, 9) > 0} does not depent on 9. (ii) For almost all x[µ], f (x, 9) is twice continuously differentiable in 9.

(iii) For each 0 E O 0<1

= f {

1og f(x ,o)} 2 f(x,e)dJ1(x) 02

-

J

< 00.

{ a02 log f ( x, 0) } f (x, 0)dµ(x)

49

608 Let 00 be arbitrary but fixed in 0. Consider the problem of

testing hypothesis H+ : 0 = Bo + (t/\) (t > 0) against alternative K : 0 = 00. Then the rejection region of the most powerful test is given by n

Tn=

EZni

>C,

i=1

where Zni = log{ f (Xi, Bo)/ f (Xi, Oo +

Tn

Since

= i=1 > Z n i =

log f (Xi, 8) E=

P n a2

2ni=1 O02

log f (Xi, 0),

if 0 = 00, then Tn is asymptotically normal with mean t2leo/2 and variance t210o and if 0 = 00 + (t//), then Tn is asymptotically normal with mean - t2lo, /2 and variance t2leo. Hence it follows that Qeo(t ) _

(t Ieo)

where -:D (u) = f"o^ I e-xz/2dx . From (2.1) we have Foo(t)<(t Ieo ) forall t>0.

In a similar way as the case t > 0, we obtain from (2.2) Foo(t) > 1 - (ItI IBo) = ( t 1 /

).

Since 00 is arbitrary we have now established the following well known theorem. Theorem 3.1. Under conditions (i), (ii) and (iii), if \(0n - 0) is asymptotically normal with mean 0 and variance 1/b i then 0n is asymptotically efficient. It has been well established that under some regularity conditions the maximum likelihood estimator 0ML has the same asymptotic distributions as above, hence 0ML is asymptotically efficient.

Using Gram-Charlier expansion of the distribution of E 1 Zni we get the asymptotic series of the power of the most powerful test.

50

609

We further assume the following : (iv) f (x, 0) is three times continuously differentiable in 9. (v) There exist Je [{log = EB f(x,e)}{

log f(x,e)}]

and

^31

KB=EB [{lof(xo)

and the following hold : Ee

1ae3

log f (X, 9) = -Me - Ke.

1

We denote (a /aO) f( x, 9), (,92 /,902)f (x, 9) and by fe, fee and feee, respectively. Since f (^, e) - fee(x, 0) - 3f98(x, 83 log

B)fe(x, e) + 2 {fe (x, e)3

893 f (x, 0) {f (x, 0) }2 (

f(

,

0) - 3 {

)

{f (x,

0)1 3

2

1og f (x, o) } { 22 log f ( x, B)

- { ae log f(xo)} 3 , it follows by the last condition of (v) that f feee(x, 9)dµ(x) = 0.

Lett > 0. If 0 = 90, then

(031003)f (X, 0)

}

51

610

TnZni

Blogf (X" Bo)

i=1

1

t2 n 192 2n L 802 log

f (Xi, 00)

1

6^ ,/n-

a03 log f (Xi, 00)

Hence it follows that Eep(Tn ) 2I+6tn(3J+K), t2 t3 t Vo0(Tn)^n -I + J =t2I+ -J, n n,j Vrn3

E90 {Tn - Ee0 (Tn)}3] --K,

where I, J and K denote Ieo, Jo0 and Koo, respectively. Put 91 = 9o + (t/.,,fn-). If 0 = 91, then T. 'v

7L pe log f ( Xi, 01) ^/ /- 1 V

P n a2

+ 2n E 802 t3

log f (Xi, 01)

n a3

6n / E 803

log f(Xi,01).

Hence it follows that Eo1(Tn ) ^' - 2I'+ 6t=( 3J'+K'),

where I', J' and K' denote I01 , Jot and K917 respectively. On the other hand we have

52 611

I' NI + aele° =I +

89

J

n

=I+

l j f (x, Bo)dµ

J f ( 2 2 { BZ log f (x, Bo) } {1og f(xoo)}f(x,oo)ciit l

=I + +

f (x, Bo) { YO lob 3

f {50-

log f(x,eo)} f(x,eo)d/-z

±(2J+K).

Hence we obtain t (3J+ 2K). Ee1(Tn ) - - 2I - 6T

Since

F8 JO.;

J' J+

K'-K +

a$Ke

°,

it follows by a similar way as above that 3

Ve1(Tn) Nt2I+

(J+K); 3

Eel

[ {Tn

- Ee1(Tn)}3]

--K.

Letting an be a rejection bound, we have P, ,e1 {Tn

Putting

cn

<

n

an} =

Pri,el

I

Tn + (t2 1/2) tj.

<

an + (t2 1/2) tV

= {an + (t2I/2)}/(tv17), we obtain

Pn,o1 {Tn < an}

t2 _ (D(cn) - O( cn) {_ 613J + 2K) + (J + K)c 2fI 6IK(cn 1)^+o\1/' "n-1I

Zee-u2/2 . If Pn, a1{Tn < an} = 1/2, then it follows that cn = O(1 /vrn-) and 4 (cn) = (1/2) + cnq5 (cn). Since

where O (u) = 4)'(u) =

53 612 t2 n K cn = - 6^I (3J + 2K) + 6I + o

i7 In /

we have 2

3

an=-t2 -6I(3J+2K)+6 I+0\ 1

Then we have Pn,oo {Tn ^ an } =1 -

Pn,o0

=1 - Pn,Oo

{Tn < an}

{

-

t2

1

Tn t ("7 /2) - cn < -WI

1

+ 0(-ti) { 6 nI (3J + K) - cn + 2^I J(-t^) 6K(t2I -1)}+0(;) ,/n-11

Since 00 is arbitrary, it follows that for each t > 0 Go(t) _ 1 (t

Vie) t2 7e(t) = 6(3Jo + 2Ko)q5 (t Io) . e

In a similar way as the case t > 0 , we have for each t < 0, Go(t) _ (-tom) 'Yo (t) t2 ( 3Jo + 2Ko ) q5 (t Io) . = - 6 Io

Therefore we have now established the following which is analogous to the result of Pfanzagl [7].

Theorem 3. 2. Under conditions (i)-(v), if

54

613

(3.1)

P,,,B { v4

(O - 9) < t} = 4'(t ) + 36\/ I:12 ,(t) + ° \ 7n/

then 8n is second order asymptotically efficient. (3.1) means thet n o(9n - 9) has an asymptotic distribution with mean -{(3J8 + 2KB)/(6/iIV2)} + o(1/.) and variance 1 + 0(1/Nn-) and third moment -(3J9 + 2Ke)/(/Ie12). Let X1, X2, ... , Xn,... be independently and identically distributed random variables with an exponential distribution having the following density function f (x, 9): (3.2) f (x, 0) P x1e, x > 0, 0, x < 0.

Since log f(X, O)

B

e

- log 0,

X 1

ae log f (x, 0) = B2 0

log f (X, 9) _ - 8 + 02 ,

it follows that Ie = 4 Ee Jo KB

=

[(x -

-BSE9 [(X

0)2] -

_ 02'

9)(2X

-

9)]

- 2 2 E9 [(X - 9)2] F5 3

]

= 1 Eg [(X - e)3 = . F6 F3 2

If ^(en - 0)/9 has an asymptotic distribution with mean 1/(&A-) and variance 1 + o(1/\) and third moment 2//, then 8n is second order asymptotically efficient.

The maximum likelihood estimator of 9 is given by OML = X = Ei Xi/n. Putting OML=

(1+3n)X,

we have EB

\/ (BML-

0)

0 3v/n-'

55

614

VB Vn0 nIL =

E

-,An_(^* ML

C

1

0)

+3 n

)

+°

2

✓n

^ 3] (1 + 1 )3

2

[

n

=

n

2 +0( 1 n

n

is second order asymptotically efficient . In this case Hence 0 X is sufficient statistic and the distribution has monotone likelihood ratio . Consider the problem of testing hypothesis 0 = 91 against alternative 0 = 90, where 01 = 0o + (t//) (t > 0). Then the rejection region of the most powerful test is given by the following form: X
^(X - 9)/9 has an asymptotic distribution with mean 0 and variance 1 and third cumulant 2/J. Using Gram-Charlier expansion we have (3.3)

Pn,0

{ ^(X - 0) < c }

= 4)(c) - 3 1 O( c)(c2

Let 0 = 01(= 90 + t/^) (t > 0). If (3.3) agrees with 1/2 up to the order of 1/Ji, then the following must hold: cO(c) - 3,0 ( c)(c2 - 1) 0.

Hence it follows that c = -1/(3\). The rejection region of the level 1 / 2 test of hypothesis 0 = 01 is given by X< 1-3n)81.

56

615 This agrees asymptotically with GAIL= (1+-- X(<01)

up to the order of 1//. Assume that Vn(0n - 0) has the asymptotic normal distribution with mean 0 and variance 1/Is, and that the cumulants of the asymptotic distribution are given as follows: ,/'E9(e„-0)= 1 cl( 0) +o

=

7n

1 c2(0)+0\7n/' EB [{v'(^n_Eo(On))}] =

1 c3(0)+0(

Further the m-th (m > 4) cumulants of ,An-en is assumed to be less than order 1/,,fn-. We assume that cl (0), c2(0) and c3(0) are continuous in 0. Put on = 0n - (1/n)k(0n). Since 0n agrees with do - (1/n) k(0) up to the order of 1/n, the cumulants of the asymptotic distribution of nIe(6n - 0) are given as follows: nJBE0(6n - 0 ) = J

ne

{ci(0) - k(0)} + o ();

nIBVB(en) = 1 + Iec ) + o ( r

EB

[{

nIB

(0n - EB(0n)) }

3

✓nl 3/2 IB c3(0) + 0

1

] = ,/n- ,`

If k(0) = ci (0) - 16Iec3 (0),

then in the Gram-Charlier expansion the following holds: Pn,e

It also follows that (3.4)

{Vn- (Bn

-B) '

I=

1+0(1

57 616 _ fi(t) _0 (t) 218 )t+ 10(/2) t2 +0 (I) n vFn

Hence it is seen that ( 3.4) agrees with the bound ( 3.1) if and only if c2(9) = 0 and c3 (9) = (3JB + 2KB)/IB. If c2 (9) = 0, then c3(9) must automatically be equal to -(3JB+ 2KB)/I93. Indeed since 0(t)t2 > 0 for all t 0, it follows that (3.4) (3.1) uniformly in 9 if c3 (9) < -(3JB + 2KB)/IB. This means that the bound (3.1) fails to hold at either positive or negative t. Hence if c2 (9) = 0, we must have c3(9) = -(3JB + 2KB)/I03. This apparently seems mysterious . However it will be naturally understood from different examples that if the second moment is decided up to order of 1/\, then the third moment is done. We shall verify the above fact by the maximum likelihood estimator. The second order asymptotic efficiency of the maximum likelihood estimator will be verified using the above fact. The continuous differentiability of the likelihood function is assumed up to the necessary order . Let BML be a maximum likelihood estimator. Since a

0 1og f Xi,

FO

e

ML) 2

_ ae log f (X;, B) + { ae2 log f x B) } (

n

f (Xi, B.) + 2 { aB log

4,

2

(eML

putting Tn = n(OML - 0) we have n

n

f (X" B) + ° = 88 log n

2

882 log

f (Xi, B) 7'n

n 3

+ 2nd

E a43 log f (Xi, B*) Tom.

Put Zl =

1 n .

ae log f (X4, B);

n

2

^ {log f(xio)+Ie } ; Z2 - 2 n

3

W n ^93

log f (Xti, B*).

58 617

Then Z1 and Z2 have the asymptotic normal distributions with mean 0 and variance IB and Lo(= Ee {(a2/a02 ) log f (X, 6) + Io}2) and covariance J9, respectively and also W converges in probability to -3JB - KB. Hence it follows that 1 3JB+KB 2 Z1+ -1B+7Z2 Tn- 2/i Tn 0, Zl +

Tn -

Z

118 Zl Z2 - 3218 B

I

IB

Since covariance of Z1 and the term of the order is 0, it follows that

Hence it is seen that the maximum likelihood estimator is second order asymptotically efficient. If we indeed calculate the asymptotic cumulants, they are obtained as follows: EB(Tn) ,,, - Jo + KB 1 2/ IB +o (Tn ) ;

To EB

{Tn -

( Tn 1 ); n)13]

-

- Vrns +0 Me + 03 \-^

Hence it is shown that if the second moment is decided up to the order of 1/J, the third moment is equal to what it should be. As another example, we consider the location parameter case. Let X1, X2,..., Xn,... be a sequence of independently and identically distributed random variables with a density function f (x - 0). It is well known that under appropriate regularity conditions the best linear estimator 6n = E 1 C%nX(iln) + con is asymptotically efficient, where X(i1n) < ... < X(nin) and cin are optimal constants

as was established by Blom and others. Then the estimator 0, is second order asymptotically efficient. Indeed, if U(iin) are order statistics from the uniform distribution, then it follows that

59 618

F- 1 (X(il,,)) = B + F- 1

n+1 '(U(iln) n +

+ {F-1 (n + 1

)}

+ 2 jF-1

I

,

(n + 1

2

\n + 1) + R.

)

Let con = - Z 1 cinF- 1(i/(n + 1 )) and Putting a in =

{ F -1 (n i ) (ll \

E= 1 cn = 1.

1' ,

J

bin=SF -

1 (n+1)}"'

we have Vr_

"r

(9n

- e) =

n

2 ^' ECinain

U(i1n) -

i=1 n+1 n

+ 2 cinbin i=1

(

l2

u(fl) -

n J 1

+ V nRn.

Then the asymptotic variance is given by V( i(9n - 0)) ^` 2 2 i(n+1-i) i(n+1-j) 2n n L_ cinain ( CinCjnainajn ( n + 1)(n + 2) + n + 1)2(n + 2) i=1 i<j

+ n Cincinainbjn i<3*

i(n + 1 - j)(n + 1 - 2j)

n +13 ) (n + 2) (n -) 3

(n+1-i)(n+1-2j) +nEECinCjnainbjnj i>j (n + 1)3(n + 2)(n + 3)

Let J(u) be a function sufficiently smooth and put cin+ (1 /n)J(i/(n+ 1)). Putting h(u) = f (F-1 ( u)), we have (F-1(u))' = h(u)-1 (F-1(u))" _ -h'( u)h(u)-2.

Since V(Vn-(B,. - 0))

60

619

2 ff u(1 - v)J(u)J(v)h(u)-lh(v)-ldudv u
+ n ffu(1 - v)(1 - 2v)J(u)J(v)h(u)-lh'(v)h(v)-2 dudv u
- u)(1 - 2v) J(u)J(v) h(u)-'h'(v ) h(v)-2dudv + n Ifv(1

it follows that J(u) = -h(u ) h'(u)/I,

where I = f { f'(x)}2 If (x)dx = f {h' (u)}2 du. Then we have V(v n (en

- e)) ^'

I +o

(=)

.

Let 9* be a modified estimator which is second order AMU. Then it will be shown that O is second order asymptotically efficient. Since {-1og f(x)} { J = - dx2 2 d

1

1og f(x) } f(x)dX

_ iog.ix)} f'(x)dx

2,f {

21 =2

f

3

dx

log f (X) } f (x)dx

{h'(u)}3 du

K - 2'

the third asymptotic moment of the second order asymptotically efficient estimator must be -K/(2n2I3). Indeed, since COV

(U(i1n ) n + 1)

(U(j

i.) -

n

+ 1) '

(U(k1n) -

7l

k + 1)

2

61

620

= 2cov(U(iln), U(kIn))cov(U(iIn), U(kjf)) + o E KU(i1n) n+1) (U(iln) - n+1) (U(kIn) n+1) J -i(n+1-j)( n+1-k) i< k (n + 1)2(n + 2)(n + 3) ' - '

it follows that E

[f On -

E(Bn)

}) 3

,6

JJJ u(1 - 2v)(1 - w)J(u)J(v)J(w)h(u)-lh(v)-lh(w)-ldudvdw n2 J { f miniu v ) - uv) J(u)h(u)du} J(v)h(v)-2h'(v)dv + o 1n )

u
2

+

Putting J(u) = -h(u)h"(u)/I, we have E

[Jon - F' i(Bn)

- -n 13

1

31

J {h'(u)}3 du - n I3 f h(u) h'(u)h"(u)du

n 113 f {h'( u)}3 du + 2n 13 f{h(u)}du K

2n2I3.

Hence it is shown that the third asymptotic moment is -K/(2n 213). The second order asymptotic efficiency of ML estimators depends on the assumption of differentiability of the density function . The following example is to make this point clear. Suppose that Xi' s are independently and identically distributed according to the double ( bilateral) exponential distribution with the density function

.f (x - 9) =

2

e_1z-8I ,

where 0 is unknown location parameter. In this case it can be shown that the first order asymptotic bound is obtained with I = 1, and it can be easily shown that the ML estimator

62 621 Xi 9ML = 1med
is asymptotically efficient. The second order asymptotic bound is obtained after some algebraic manipulations (the detail is given in [111) as follows: (D(t) - 6ttO(t) sgnt.

But the asymptotic distribution of 9ML is given by a Pn,e {' (011L - 0 ) < t} ='b(t) - 2 ,^-0(t) sgnt + o (

which differs in the second term from (3.5). 4. Third order asymptotic efficiency We proceed to the problem of third order asymptotic efficiency. 6n is called third order asymptotically median unbiased if for any E 4 there exists a positive number 5 such that

I

lim sup n Pn,e { ii e: 10-01 <6

n-400

(On

- 0) < 0 } -

1 = 0; 2

lim sup nl Pn,e{v(On-0) > 0} - 1 n-foo e:1e-91<8 - 2

=0.

Let 63 be the class of all third order asymptotically median unbiased estimators. For all 9^ E 63 we shall obtain the bounds of Go and HO such that lim n Pn,9

n-aoo

{j

(On-0)
In a similar way as section 3 we consider the problem of testing hypothesis 9 = 90 + (t/Vn-) against alternative 0 = 9o and the asymptotic expansion of the power of the most powerful test of it.

The continuous differentiability of the likelihood function is assumed up to the necessary order. Putting ' = (8/09) log f (x, 0),

63 622 0" = (a2

/ae2) log f (x, e) and V)(') = (ai /aez) log f(x, e) (i = 3, 4),

we have n Tn = Zns i=1

n

t2 n

V)'(X') 2n (X) _ 6n 3(3) (Xt) 4 n

24n2

0(4) (X,) +0 ( -)

n

where Zni = log If (Xi, eo)/ If (Xi, eo + (t//))}. Define L = Ego {O(3) (X)O'(X) } M=E9o{1i"(X)2},

N = Eep {,cb " (X)'(X)2}

H =Eeo{Vi'(X)4 }.

Since under appropriate conditions the following holds: Eeo {(4)(x)} _ -4L - 3M - 6N - H,

the asymptotic cumulants are given as follows: Eeo (Tn ) - t2 + 6t^ ( 3J + K) + 24n (4L + 3M + 6N + H), Veo(Tn) t2I + t3 J + t4 (M - I2) + t4 L,

4n

3n

Eeo [{T - Eeo(T)}3] = 'Y9o (Tn) - --K - 2tn (N + 12), Eeo{T - Eeo(7'n)}4] - 3 {Veo ( Tn)}2 = beo (7'n)

t4 n

(H - 312).

If B = 91 (= 90 + (t/vfn-)), then for any measurable function g Eel {g(X)}

64

623

t fBo(X) t2

1 + fep (X) + 2n feo (X) + 6n-,/n-feo (X) + .. .

= Eep 9(X) = Eeo {g (X)

fep(X) t3fo(I (X)

{

z

1 + =V(X) + 2n ("(X) + V)I(X

)2)

t3

+ 6n3 (0(3)(X) + 3Vi"(X)i1'(X) +V/(X)3)

Hence we have 2

Eo1(Tn)

n

n VA X) + 2nnE

Eeo(Tn) - Eeo

t E

to E V), (Xi) + n t3

1

2

n 3

b"(Xa) + 6n 10 (3)A)

n

E (,i"(X;) +'^/)I (X,)2) 2n 1

n

+ 6nvrn- (0(3)(X,) +30"(XX)O'(Xj) +OI(X,)3)

+ o(1)

=-2I- 6t^(3J+2K)- 24n(4L+3M+12N+3H).

Similarly we obtain Ve1(Tn) t2I + t3 ( J + K) + t4 (4L + 3M + 18N + 6H - 312), 12n Ye,(Tn ) - K - Zn (3N+2H-3I2), 4

Se1(Tn) n (H - 3I2).

Put Vn = { Tn - Eg1 (Tn) } / Vol (V,,. )

(t/t) . If Vol (T.)

=1+Ql

+na2 +o\n)

65 624

7e, (Vn

) = t3I^ryB' (Tn)

1

1

_ 'Y1+nryz+o

C

1)

J, W

8Bl (Vn ) = 1JO1 (Tn) = nb+o (1) ,

then we have Pn,B3 {Vn < a}

_ -1,(a) - ¢(a) { 2 (vfn- +') a+ 6 vrn- + nz) (a2 - 1 ) n l \ z - 3a) + 12n (a4 + 6az +3) + (__- + 24n 8n) (a3 z l 1 + 72- (a5-10a3-15a) +0

Pn,o1

{V,
2+0\n/

Since (D (a) = 2 + aq5(a) - I a3q5'(a) + ... ,

it follows that a

+ n + 4'8171 '8171 + o n 2 n a G n __ Q \ y^ ' ryz) 1 \1) 71 _72 + 0i'Yi+0 1

6vln- n 6n n

If for 0 = 00 EB ,(Vn)-

Feo (Un)-

1

1 , a=-tvfl + ^a l

, 1 ,

VBO(Vn)= ^Nl+n^2+^

C

1) n 1

1 , + na 2

+o

(

n, 1)

66 625 1 , 1 , (1) 7i + 72 0 nJ

-yo.(V- )

n

n

(1) 1

then we have (4.1) Pn,eo =

1

{Vn > a} -Pngo

{Vn <

a}

nag = (t') + O(e ) ITn I

C2a,F

a + 2n + 2n) t'

+ ( /i + 72 +!? Q ) (tit _ 11

6^ 2n 2n J bl + ai7'i + i2 (t^3 - 3t') - (24n 6n 8n ) + 1 71

t'4 -

12n (

6t'2 +

3)

Y12 1t'S-lot'3+15t' ) }+ 72n JJ) o(),

where t' = t,/I-. Substituting in (4.1) t'2 K al = 2132 (2J + K) - 613/2' ti3 2^ a2 = 1212(4L+3M+6N+3I )+ 6 - K(J + K),

/2' -

t,J

1 - 13/21 t,2

2

QzI2(M-I2)+3I2L , K 71=-I3/2

67 626

J=72- (H-31),

we obtain Pn,B0 {Vn > a}

r _ 4, W) +

tit

ti5

OW) [g^13/2 (3J + 2K) - 72n13/2 (3J + 2K)2 ti3

- 72n13 { (3J + 2K)2 - 31(4L + 3M + 6N - H + 612) } + ti 3 {2K2+9I(2N+H-12))1 72nI .1 1

Hence it follows that for t > 0 Pn,g { n1 (9n - 0) < t} n

«(t)-0(t){

u

2n(t5 -10t3+15t)

6^V/n-+6^vfn-(t2-1)+ Q4

/32

3

+ (24n + 36n ) (

t - 3t)

Q3 2 L2 ) t'>+o(2 n \, +( 2n+72n

where 33 = -(3J + 2K)/ I3/2, Q4= j3 { 3(3J+2K) 2-I(4L+3M + 6N-H+612)), Q2 3613 { 17(3J + 2K)2 - 2K2 - 91(4L + 3M + 8N + 512)}.

Further the moments of the third order asymptotically efficient estimator must be as follows:

68 627 " 1 = 9 + 6 JI3/2 +0 nVfn-

(4.2)

Ee (9)

(4.3)

^ vo(V n dn)=

(4.4)

n9n =

I

QZ

+ nl

+o n

03

n , I3/2 + oIJ

n12

n (1)

Next we shall consider the example of the exponential distribution with the density (3.2) given in section 3. Since 8n

=f 3njX

it follows that (4.6) EB(Bn) = 0 + 3 , (4.7) Ve (/9n) = 1 1+ 2 (4.8) 'Yo (V9n) = 203 +0 n (4.9)

6 bo (v On) = n 04+0

\ 02, n

n. (1

)

Since I = 1/02, J = -2/03, K = 2/03, L = 6/04, M = 5/04, N = -5/04 and H = 9/04, it follows from: (4.2)'(4.5) that /33 = 2, ,Q'4 = 6 and l3'2 = 2/3. Then the moments of (4.2)'(4.5) are equal to those of (4.6)'(4.9), respectively. This fact should hold because X is a sufficient statistic. We next consider the maximum likelihood estimator 9ML• Putting Tn = ,,,/n(9ML - 9), we have Zl + (-1 + Z2) T. + 27n -(3J + K) + Z3^ T„ 4L+3M+6N+HT3_0 6n n '

69 628

where Z3 = (1//i) Ei {0(3)(X2) + 3J + K}. We also obtain Tn

NIZl +^I2

(z2z1

- 3J

IK)Zl

+n13 {z1z + Z12 3(3J + K)Ziz2 (3J+K)2 3 4L+3M+6N+H 3l + 212 Z1 6 Zl } .

Since E(Z1Z2 ) N 1 +I2), n E(Z13 Z2) N 31J, n E(Z14Z2) N 3(M-I2)+12J2, n

it follows that E Tn (

)

N

VTn N

-

J+K +o C11 2,12 n J J

1 7J2 + 14JK+5K2 L+4N + H+I2 +o /11 I + 2414 n13 1\nJ , 3J + 2K +0 ( n1) 1 nI

3(T„) N 12

(J+K)(2J+K) - 4L+12N+3H- 31

+0

(

1n54 n)

We define a modified maximum likelihood estimator 8ML as follows:

K9ML = BML + 6n1? L 0ML

Then k * L is third order AMU, that is, Pl,e {BnfL - 0 0} = + 0 in^

2

70 629

Further it follows that 1 0KB 2 nV (kL ) = n (1 + 6n 870 - V (OML) N V(I,n) + (3N + H)I - 2K(2J + K) 3n14

where

aeKe = a f {'fi'(x) }3 f(x)dx = 3 f z/i"(x)^/i'(x )2 f (x)dx +

J {,(x)}4 f (x)dx

=3N+H, BBIB=2J+K.

In general the maximum likelihood estimator is shown not to be third order asymptotically efficient. It is enough to consider a special case, and we take the Cauchy distribution as an example which has the following density function f (x, 0): 1

f(x 0) = 7r

(1 + (x - 0) 2 ) *

Let X1, X2,. .., X,,,, ... be independent identically distributed random variables with above density.

Since 49 log f (x, 0) = 1 + x2 80 log f (x, 0) ax 2

2

2(1

2)

log f (x, 0) = 8x2 log f (x, 0) _ (1 + x)2 2 802 03 80 3

log f (x, 0) a log f (x, 0)

41(+ xz s)

it follows that I= 1/2, J=K=O, L=-3/4, M=7/8, N= -1/8 and H = 3/8. Since the density is symmetric, the maximum likelihood estimator 0ML is third order AMU. The second and fourth cumulants are given as follows:

71 630

V (/BML)

5 :6+ n

-Y( V'nB ML) ,.,

n .

On the other hand since the values of the bound of the asymptotic distribution is given by ,33 = 1 /4,

04 = 0,

they are larger than the above moments (4.10) and (4.11). Hence BML is not third order asymptotically efficient. 5. Some further remarks

The above results can be extended to the cases (I), (II), (III), (IV). (I) When the distribution is discrete , some modification is necessary to apply the above argument. (II) Let Xi's be independent but not identically distributed random variables with density functions ff(xi, 0). Let B, be a second order AMU estim ator.When c,2(9, - 0 ) has an asymptotic distribution, we obtain the bound of the distributions of 9,,,. Let 90 be arbitrary but fixed in O. Consider the most powerful test of the problem of testing hypothesis 9 = 91 (= 9o + cola, a > 0) against alternative 9 = 00. Then the test statistic of it is given by (5.1) T,, = log

ft (Xi, B1) fb(X2,

00)

_ -{ E 0 50- log fi (Xi,Bo)} . (cn1a) l n ( 2 2 { a42 log fi (Xi, eo) j

1

-6 n

+Elkni=1

l

l

logfi(Xi,eo)

1

2 (Cn1a)

11 3 (cnla)

72

631

Put Ii = Eeo {

1o fi(xioo) }

2 I 2 l z log fi(Xi, Bo) -E90

It follows that the variance of the first term of the right-hand side of (5.1) is given by (cn-'a)2 Ei Ii. When cn = I, where in = E; Ii, it's variance converges to some finite value. If the following "Lindeberg type" condition holds: for every E > 0, [{

1n li ^ IIn Eep 1

log fj(xj,9o) }2xjfl(E)] 8B

= 0,

where 1, if log fi(Xi, 90) > 0, otherwise,

In,

E

Xin(E) _ 80

then the first term of the right-hand side of (5.1) is asymptotically normal with mean 0 and variance a2. The necessary condition for this to hold is given by limn-->oo max, 0, n

2

lim 1 E Eeo [{- 02 log fi(Xi, Bo) } X n(e)] = 0, n-+oo In

where if - 8e^ log fi(Xi, 90) 0, otherwise, 1,

Xin (E) _

> EIn,

then the second term of the right-hand side of (5.1) converges in probability to a2/2. If under alternative 0 = 0o, similar regularity conditions as above, then the test statistic Tn is asymptotically normal with mean -a2/2 and variance a2. Put cn = In. For a > 0, the supremum of the power functions is given by 4)(a). Hence if

- 0) is asymptotically normal with mean 0 and variance 1, then en is an asymptotically efficient estimator. Let BML be a maximum likelihood estimator. Then In(en

fi(Xi, B) } (BML - B) 0 = { 8 B log + { 2 n

+ Rin.

log A (Xi, B)

} (BML

-

9

12 /

73

632

(BML - 9) is asymptotically Under some regularity conditions normal with mean 0 and variance 1. Hence the maximum likelihood estimator 9ML is asymptotically efficient. In order to consider the higher order of the bound of the power functions, we must decide the orders of n

n

Kn=>K+=> Eeo

[c

131

89

log .A (X%, Bo)

and n

n

2

in = Ji = Eeo %2 log fs (Xi,Go)} {5 log A A , 00) -0i i 1.111

Then the asymptotic moments of the test statistic Tn are formally obtained as follows: 2

3JSI /Kn

Eeo (Tn) ,, 2

a3, n

Voo (Tn) .., a2 + ^J%2 a3, n

Eeo{TEe0 (Tn)}3 -j 22a3.

Similarly we have for 9 = 91 a 3Jn + 2Kn 3 Eeo(7'n)..,-26732 a, n Vei ( Tn)

-a2 + Jn

P /2 n a3,

(Kn/In/2) a3.

Eel {Tn - E81 (T)13

"/2 If in and Kn have the same order which is less than I3 and the residual terms are negligible, we can apply Gram-Charlier expansion to the asymptotic distribution of the statistic Tn. Put d,t 1 = Kn/In3/2. If lim d,,

n-,oo

it (9n

-

0)

<

O}

- 1/2

=0

uniformly in any neighborhood of 90, then for a > 0

74 633

(5.2) lim do PP ,e. { In (Bn - o) < a } - ^(a) - 3J61.3 2 Kn a 2 O(a) n00 / JJ

< 0.

-

For a < 0, a similar inequality as above holds. If the equality "=" of (5.2) holds, then 9n, may be called second order asymptotically efficient. The necessary condition for second order asymptotic efficiency is that Bn has the following asymptotic moments: E,

{

I ( 9 - o) ,,

1 N 3J+2Kn

UB{

3I^2 n l / In(Bn-o)}N1+o1 );

EB

In

}3

{

(en

- e)

+0

(

) 1 /

J+2Kn Iri^ 2

1

If On has a smooth asymptotic distribution and Yg { In (Bn - o) } - 1 + o(1/dn), then the modified estimator Bn which is second order asymptotically median unbiased is second order asymptotically efficient. Let BML =

OML + Kn (OML )

6nln(OML)2

Under regularity conditions 9ML is second order asymptotically efficient. If the following generally hold: In/n -4 I

(n -3 oo),

Jn/n --^ J

(n -^ oo),

Kn/n -* K

(n -4 oo),

then it follows that do = vfn-. Since the second term of the asymptotic distribution has order of 1/J, a similar result as the i . i.d. case holds . But the fact does not always hold. For the purpose we consider the following examples. Let X2 = Ozi + U2, i = 1, 2,. . ., where zj's are constants and Ui's are independent identically distributed ramdom variables with a density f (u). Putting

75

634

1og f(u)}2f(u)du = f { f ( )}2 du,

I=f{ J=

J {l d2 log f (u)-JJJ} {{f'(u)}31og f(u)}f(u)ciu [fhl(u)fI(u)

f (U) { f (u )}2 ] du, l

3

K{dulogf(u) } f(u)du { f/(u 3

= {f(u)} du,

we have for each i II=z3I, J,=-z3J, Ks=-z3K.

Then it follows that n

n

n

In= ^z3 I, J=- ^z3 J, K=- ^z: 1

1

1

By Lindeberg's condition we obtain n

max zs

1<4
E zs -4 0 (n -4 oo). 1

Since

< max I z` I -^ 0

IEn

(E

zi )3/2

oo),

(^1 zti )1/2

it follows that Jn /In/2 - 0

(n -4 oo),

Kn /I3/2 __+ 0

(n -+oo).

z2)312 /(E z3) . If En i z2 /n -3 m2 Then we can take do = (E (n -+ oo ) and Ei z3/n --4 m3( 0), then it follows that d, _ O(-). Then the second term of the asymptotic distribution has order of 1.

76

635

But letting zi = i, we have n(n + 1)(2n + 1)I/6 - n3I/3.

n3I/3(9n - 0) is asymptotically normal with mean 0

Hence if

and variance 1, then Bn is an asymptotically efficiency estimator. Since + 1)2

= - n2(n

Jn

J

4 (

n+

)2

n4

4J,

4

n44

it follows that do = O(ff). Hence in this case the second term of the asymptotic distribution has also order of 1/V/n-. If zi -+ 0 (i -} oo), then it follows by Lindeberg's conditions that the following must hold: n

In= z1 I-4 oo (n -goo).

But it is possible that En z2 -+ 00 (n -+ co) and En z2 < 00. do = 0(1,1/2), Since letting zz = 1/vz we obtain in = O(log n). Hence og n(0n - 0) is asymptotically normal and the second term of the asymptotic distribution has order of (log n)-1/2. Let Xi's be independently distributed random variables with the following density: f (x 0) =

xn.-le-=:/B x > 0

1

BPir(pt)

where pi's are known positive integer and 0 is an unknown parameter. Since log fi(Xi, 0) =

(Pi

a0logfi(Xi,0) ,9 2

502

log fi(Xi,

it follows that

0)

- 1) log Xi - Xi/0 - pi log 0 - log r(pi),

0i

= -

- 0i,

0 ii

+ 02 ,

77

636

/ X, pi \ 2 192 - 9 I =B2,

Ii=Ee

Ji = -Eq Ki = Ee

- P) ('X' - P)] = -,Pi

02 T _^T F2 03 lXi

92

= 2pi

pi 3

9) 93

Then we have n

In

n

n

= Epi /82, in = -2 E pi /93, 1 1

Kn

= 2 E pi /93.

1

Since do = (Ei pz)1/2, the asymptotic distribution of the second order asymptotically efficient estimator 6n is given by Pn,g

n 1 2 Pi (Bn - B) /B < a - (a) + a O(a). 3 El pi

The maximum likelihood estimator 9ML of 0 is also given by E Xi /E pi . Let + 3

9ML = (1 \

EInl pi

) 9ML.

Then 9ML is a second order asymptotically efficient estimator. Since in this case En Xi is a sufficient statistic, the second order AMU estimator based on it must be second order asymptotically efficient. Indeed it is easily shown that the asymptotic distribution of 9ML agrees with the above bound of the asymptotic distributions. (III) Let {Xi} be defined recursively by Xi=OXi-1+Ui,

i= 1,2,...,

where X0 = 0 and {Ui} is a sequence of independent identically distributed real random variables having density f with mean 0 and variance a 2. Let 00 be arbitrary but fixed in O. We consider the problem of testing hypothesis 0 = 01 (= Oo + cnla) against alternative 0 = 90. Then the test statistic of the most powerful test is given by 10 g

t- 1 Zni = ,-1

f ( Xi - 00xi-1) J xi - 91Xi-1)

78

637

By Taylor expansion we have n

n

1

f'(UU)

Zni = Cn aX _1 f(Ut) i=1 i=1

n

2

- 2 c"2a2Xs 1

aU21og f (Ui) }

i=1 n

+R.1. i =1

Since for 1 01 > 1, Xi is stochastically order of 0i , the asymptotic distribution of En 1 Zni is not always written as an easy type. If 101 < 1, then at least Xi is asymptotically stationary. If cn = V/n-, then E 1 Zni is asymptotically normal ([ 11]). From above it is shown that /i(Bn - 0) is asymptotically normal with mean 0 and variance ( 1 - 02)/Icr2, where I = f { f'(u)}2 If (u)du if 0n is an asymptotically efficient estimator.

When 101 < 1, using asymptotic moments we can formally obtain the bound of the second order asymptotic distributions of second order AMU estimators of 0 and show that a modified least squares estimator of 0 is second order asymptotically efficient (Akahira [21). (IV) When the parameter 0 is vector valued, we may decompose it to 0 = (^) where 6 is a real number and 77 is a real vector, 77

and we consider estimating 6, while q is treated as a "nuisance" parameter. Then the second order asymptotic bound for any estimator ^ of 6 is given in a similar but somewhat complicated algebraic expression, and it can be shown that a modified ML estimator attains this bound under some regularity conditions Q3]).

References [1] Akahira, M., "Asymptotic theory for estimation of location in non-regular cases , I: Order of convergence of consistent estimators ," Rep. Stat. Appl. Res., JUSE, 22, (1975). [2] Akahira, M., "A note on the second asymptotic efficiency of estimators in an autoregressive process," Rep. Univ. ElectroComm. 26-1, (1975).

79

638

[3] Akahira, M. and Takeuchi, K., "On the second order asymptotic efficiency of estimators in multiparameter cases," Rep. Univ. Electro-Comm. 26-2, (1976). [4] Chibisov, D. M., "On the normal approximation for a certain class of statistics," Proc. Sixth Berkeley Symp. on Math. Statist. and Prob. 1, (1972). [5] Chibisov, D. M., "Asymptotic expansions for Neyman's C(a) tests," Proc. Second Japan-USSR Symp. on Probability Theory. Lecture Notes in Mathematics. Springer Verlag, Berlin, (1973). [6] Fisher, R. A., "Theory of statistical estimation," Proc. Camb. Phil. Soc. 22, (1925). [7] Pfanzagl, J., "Asymptotic expansions related to minimum contrast estimators," Ann. Statist. 1, (1973). [8] Pfanzagl, J., "On asymptotically complete classes," Proc. Summer Research Inst. of Statistical Inference for Stochastic Processes, 2, (1975). [9] Rao, C. R., "Asymptotic efficiency and limiting information," Proc. Fourth Berkeley Symp. on Math. Statist. and Prob. 1, (1961). [10] Rao, C. R., "Efficient estimates and optimum inference procedures in large samples," J. Roy. Statist. Soc. 24 (B), (1962). [11] Takeuchi, K., "Tokei-teki suitei no Zenkinriron (Asymptotic Theory of Statistical Estimation)," (In Japanese) KyoikuShuppan, Tokyo, (1974). K. Takeuchi M. Akahira Faculty of Economics Department of Mathematics University of Tokyo University of Electro- Communications Hongo, Bunkyo-ku Chofugaoka, Chofu-shi

Tokyo, Japan Tokyo, Japan

80 Ann. Inst. Statist. Math. 29 (1977), Part A, 397-406

EXTENSION OF EDGEWORTH TYPE EXPANSION OF THE DISTRIBUTION OF THE SUMS OF I.I.D. RANDOM VARIABLES IN NON-REGULAR CASES KEI TAKEUCHI AND MASAFUMI AKAHIRA*

(Received Oct. 21, 1976 ; revised July 7, 1977)

Abstract The asymptotic expansions of the distributions of the sums of independent identically distributed random variables are given by Edgeworth type expansions when moments do not necessarily exist, but when the density can be approximated by rational functions. 1. Introduction Asymptotic distributions for the sums of independent and identically distributed (i.i.d.) random variables has been extensively studied for many years ([1]). Suppose that X,, X2, • • •, X„ is a sequence of i.i .d. random variables with mean p and the distribution function F(x). Then the asymptotic expansion for the distribution of Y„ ( X, -np)''^/ n up to the order n m/2+' is given by the classical Edgeworth expansion when F has the moments up to the mth order as well as the continuous density ([2], [3]). The purpose of the present paper is to extend the result to the case when the moments do not necessarily exist assuming only that the density function f (x) of F is approximated by some appropriate rational functions as I x I-> oo . Put, for some real constants M„ and V„ (>0), Z„= ^ X£- M„)I V,.

Then, as is well known , if a G(x)

is a limiting distribution of a sequence of distributions of Z's, it is necessarily a stable distribution. We shall show in the next section that if this is the case then under the assumption stated above V, must be either n or n log n or and the corresponding leading term of the asymptotic expansion is the Cauchy , or the unsymmetric stable distribution with characteristic exponent 1, or the normal distribution, respectively. The second term is shown to be of order either * Supported in part by the Sakkokai Foundation.

397

81 398 KEI TAKEUCHI AND MASAFUMI AKAHIRA

1/n, (log n)/n, 1/log n or 1/vt-n- and the density function corresponding to the second term are also given. Generally the relative magnitudes of the successive terms are complicated and is difficult to write down explicitely but may be obtained for each particular case. The details of the computation are given in [4] and may be referred to for further examples. Further the similar results for fractional characteristic exponents are given in [5] (see also Cramer [6] for this case). 2. Results We shall obtain the Edgeworth type expansions for the sums of random variables not necessarily with finite moments but with the density f (x) which satisfies the following condition : There exist two rational functions g+(x) and g-(x) such that for a positive integer m (> 2) and some 0<6<1

(a) lim x -+1+' j f(x) - g+(x) j = 0 ; (b) lim 1x1m +,+a1f(x)-g (x)f=0.

Remark 1. m depends on the order of the expansion and should not be smaller than 2 in order that we have meaningful expansions. Remark 2. It should be remarked that since the distribution has a density f (x), we have

sup fi(t) I < 1 ItI >r for any p> 0. We shall first show that the following proposition holds. PROPOSITION.

g+(x) and g-(x) may be taken of the form

g+(x) a> g-(x) _1 P> s =2 (x-1)' > =2 (x+1)'

where of and P, are real constants. PROOF. We first decompose g+(x) into the sums of partial fractions such as Baf Ear 9+ (x) = E i (x+Da)' +^ i (xE+Fax+Ga)' But each term in above expression can be approximated by a sum of the terms of the type jSfl(x+1)' up to order x-'"-1 when x is large. Therefore g+(x) may be replaced by a sum of such terms. Similar is true of g-(x).

82 EXTENSION OF EDGEWORTH TYPE EXPANSION

399

Now let us denote that 1)e`tx dx af(t)= (jo (1+x)'

then we have

5

o

(j-1)e"

(1-x)'

.. dx=#f(-t)=Of(t) •

Thus if the density f (x) satisfies the condition above, the characteristic function can be expressed as

(1) fi(t)= f

ettx f(x)dx=

Z afof(t) +E p.^of(-t) +0(t)

and since

50 I

dx Jo xml f(x)- g+(x)I
x ImI

f(x)-g ( x )I dx
the remainder term ¢(t) can differentiated m times at t=0, so that we have m

fi(t)= cptP+o(I t Im)

(2)

p=0

It is shown by Takeuchi and Akahira [4] that for small I t I > 0 and jZ2

(3)

cf(t)= (j- 1)\Jit1 u -'e du )Itlf -le tt tu

= (j -1){ Z

ik

kltik - ( j2'

a

+iP* l tIf- '-

?' Itlf

}(

M ItI'-

1-

1l og

ltl +a,

ItI' -'

it - 2 t2 +6ts+...l

-' =1+E a fkItIk+E bfkt I t 1"-'+EcJkItIk l logltI k=1

k=1

k=f

in

+Edfkt lt1k-1logItI+o(Itlm) k=j

where ak = -1/k! (k - j + 1) and a j* and JS; are real constants equal to the real part and the imaginary part of f

JI

1

u-fetudu +

u2

u-f {eiL_1 _iu+

'

i-2 Zu i-

- +. • . -

)1)! }du+Eik k=o k!(k-j+1) (j

respectively, and afk, bfk, cfk and dfk are certain complex numbers. Hence we have from (1), (2) and (3)

83 400 KEI TAKEUCHI AND MASAFUMI AKAHIRA

M

m

0(t) =1+E A kl tlk+Z Bkt

(4)

k=1

k=1

m

+1

I t Ik

-1+.Z Z

j

k=j

Cjk

ltlk

-l log ItI

m

E

+Z djk j k=j

tI

tlk -1 l og I 1+o(I I-) t

t

where Ak and Bk are certain complex numbers.

Let X1, X2, • • •, X,,, • • • be a sequence of i.i.d. random variables with the characteristic function c(t).

We define zn= (Y'. XL-Mn)/Vn whose b_1

characteristic function cSn(t) is expressed as (-i I t) $n(t)=E (e"zn )= 10(-t Vn )1n exp Since

log On(t)= n log cb (Vn) - i Vn t it follows from (4) that 1+k^

Ck

Itlk ( 5 ) log0n(t)=-iMnt+nZAk tIk Vn Vn nIt r+JBktI Vri (log ItI-log Vn)+ Dk tIk Vn

(log It! - log V)

+...}-1{ }2+ 3{ 1 2-...1. It is noted that since On(t) is a characteristic function B1 is a pure imaginary if not equal to zero, and that term can be cancelled out by an appropriate choice of Mn. Without loss of generality we put B1=0.

In order to obtain Vn and the leading term of ¢n(t) it is enough to consider the following cases : (i) (ii)

Al*0, Al=0, A2/0,

(iii) Al=A2=0, C1=D1=0, C2>0, D2=0, (iv) Al=A2=0, C1>0, D1=0. Since the asymptotic distribution of Z. is stable, it is seen from (5) that each case is as follows : Case M. Vn=n necessarily holds and then C1=0. If D1=0, then the leading term of On(t) is given by eA1111. If D1 =t-0, then Mn must be O(n log n). When D1 :t-0, we have the characteristic function an(t) with the leading term of the type exp {A1 I t I +D1t log I t I } which is the unsymmetric stable law of characteristic exponent 1. Case (ii).

,,=n necessarily holds and then B2=C1=C2=D1=D2=0.

84 EXTENSION OF EDGEWORTH TYPE EXPANSION 401

Hence the leading term of j„(t) is given by e A2t2. Case (iii). VV= n log n necessarily holds. Hence the leading term of 0„(t) is given by a-C2`2 Case (iv). V„=n log n and the leading term of c„(t) must bee °1l". If C1>0, then F belongs to the domain of attraction of the Cauchy distribution, and the density f (x) is expressed as

f(x)=^

a(l+x)-2+o(x-2)

as x->oo ,

P(1-x)-2+o(x-2)

as x -->- oo,

with a+i3>O. But this implies that F belongs to the domain of normal attraction of the stable law with characteristic exponent 1 (see [1], p. 181), which contradicts the assumption V, =n log n. Thus the case (iv) can be excluded from our consideration, except for the trivial case C,=0.

In other cases the leading term of on(t) is equal to 1 which is the degenerate case. When the leading term is obtained in each case, the remaining terms may be arranged in the order of magnitude. A few examples are in order to show the point. Example 1. Let X,'s (i=1, 2,...) be i.i.d. random variables with the density function f (x) given by

f(x)= 1 2(x2+1+-✓x2+1) Since f(x) is a symmetric function and f (x)- 1/2x2 as x--> oo , F(y) belongs to the domain of normal attraction of a stable law with characteristic exponent 1 ([1]). Note that X, can be expressed as XX _ (1/(1- UU) -1 / Ui)/2, where UU is distributed uniformly in (0, 1 ). Putting 1 2(1+x2)(1+ x2+s 1+x2) h() =

1 2(1+x2) 212 2 (1+1x)2

we have (6)

Ax)

1 - 1 + g(x) 2(1+x2 )212 2(1+x2) 1 2(1+x2)

1 -h(x)+ g(x) 2(1+ I x l)a

85 402 KEI TAKEUCHI AND MASAFUMI AKAHIRA

If we have m=2, 0<8 < 1 and g * (x)= 1 /2(1 +x2)- 1 /2(1+lxl)3, then (a) and (b) are satisfied . First we obtain Jo

log (

yzh(y)dy=

+i +x)- 2 1+x2 - 2 log (1+x) ,/

1 - \ 1 Fx -1/+ ( l+ x)z -1

= 2 log{( x2 + 1+x)/(x+1 )}

2 ✓1 Txz

( l+x)z

Hence it follows that xzh(x)dx=log 2 . Since 5

5

h(x)dx=1/2 ;

xh(x)dx=0

it is seen that (7) e"xh

(x)dx=2+ log 2 tz+o(tz) 2

Since

5:

g(x)dx = 2 -

2;

s:

xzg(x)dx =

2 -1 ,

we have

s: eilxg(x)dx=2- 2 -

(8)

( 2 - 1)t2

+o(tz)

2

It follows by Takeuchi and Akahira [4] that (9)

5

Ox

+I

X I)s

dx=l+tz log I t I -rtz+o(tz)

where Y is some constant. From (6)-(9) we obtain 0(t) = J W^ e"xf(x)dx z

_xa-l`1-1-t log Itl+ r tz-1 - og2t2+2-'E 2 2 2 2 2 2 2 1)tz

2\2-

+o(tz)

=1- 2 1 tI- 2 log It1+ 2 (1-7-log2)t2+0(t2)

86 EXTENSION OF EDGEWORTH TYPE EXPANSION 403 n

Letting ¢n(t) = E [exp itZj with Z"= Z Xi/n, we obtain 1

log O„(t)= n log ¢n )

2 I t i- 2n (log I t I + log n)+

(1- r -log 2)

n

8nt2+o(n)

0n(t)=e c"2'tir1- to 2n t2 2n t2log ItI -I 1 {4(1-r-log2)-7r2}t2]+o\n/ the asymptotic expansion of the distribution of Z„ agrees essentially with the Cauchy distribution. The second term in the expansion has the form 1 (a log n +b)fl(x)+cf2(x) , n where fl(x) and f2(x) are the functions that are Fourier transforms of

and (t2 log I t I )e< x/z)1tI

te e ( 2ltl

respectively. Hence fl(x) and f2(x) are equal to -g"(x) and -h"(x), respectively, where

7r

(c' 2+ x2)

with certain constants c' and C". Example 2 . Let X2' s (i=1, 2 , • • •) be i.i.d. random variables with the normal density with mean 0 and variance 1. Let Y1=1/XX (i=1, 2,- • • ). Then the density function f (y) of Yj is given by f (y) _

_ -e 1/zvz 112

Since f (y) is a symmetric function and f (y) 1/(V y2) as y-+ oo, F(x)

belongs to the domain of normal attraction of a stable law with the characteristic exponent 1 ([1]). Put

87

404 KEI TAKEUCHI AND MASAFUMI AKAHIRA

(10)

g(y) =.f (y) -

V-z (1+2y2)

If we have m=2, 0<6<1 and gt(x)=2/(-,f 7r - (1+2x2)), then (a) and (b) are satisfied. First we have ;r ; g(y)dy=1--Vyg(y)dy=0 .

(12) Further we obtain (13)

)dy y2g(y)dy= 1 J : (e-1/2v2 - 2y2 -✓2 r 1+2y2 2 -^ r

+ 2)}

°° 1 e 2 x2 --'/2- x2(2 x

(11 ,10

= V 2 ^^ (2u)- 3/2(e -u-

ro

{

1

&✓

dx

1 )du

1+u

l f o (1+1u)2 du} -2P(2)+2

_ 1+ 1 ^ v1"2(1-v)-u2dv V-7r 0 =-J+ B( 17Tr

--, --)

=-1+ -V-7r 2 From (10)-(13) we have fi(t) = e" f(y)dy

7r(1+2y)

exp

dy+5-0.e"vg(y)dy

('(t/V

-2- )X) dx+

r(l + x2)

S-

e2Cbg(y) dy

2 (-1+^2 t2+o(t2)

_*I 7r exp =1-/2 ltj+ 2 + o(t2) . 7b

Letting 0„(t)=E [exp (itZ„)] with Z,,= Xi/n, we obtain 1

88 EXTENSION OF EDGEWORTH TYPE EXPANSION 405

log0n(t)=

nlog¢Cn

42,

ItI

+C4

2

+2/n +o( n/

Since an(t)=( eXp(-

^2 It I

)) I'

+(

" + V2 +0(

the asymptotic expansion of the distribution of Z„ agrees essentially with 7c (a+2x2)} and the next the case when the density function is //{-,1term is (a+bx2+cx')/{/(1+x2)}1. Example 3. Xi's (i=1, 2,. • •) are distributed according to t-distribution with 2-degrees of freedom i.e. with the density 1

f (x) _ 2(1+x2)8/2 . Note that the distribution has no finite variance, still it belongs to the domain of attraction of the normal. First we have

f(x)=h(x)+ 1

2(1+IxI)8

where h(x) is given in Example 1.

The conditions (a) and (b) are satisfied with m=2, 0<3<1 and

2(1+I x I)' From Example 1 we obtain

fi(t)= ^`^ e'°2 f(x)dx=1 + n t2log It I _ loge +r t2+o(t2) where r is some constant .

Letting cn(t) = E [exp (itZ,,)] with Z„ _ X¢

l n log n, we have

log an(t) = n log 0C t ) -✓ nognnJ = 1 t2 log I t _ _ 1 + log log n + 2(log 2 + r) t2+o( 2 log n 4 1 log n

log n

1 11 =e (en/4) t2 lf1+ 1 t2 log It1 +or 2 log n `` n log n)l

where cn+ log log n + 2(log 2+r) log n log n

1) log n

89 KEI TAKEUCHI AND MASAFUMI AKAHIRA

406

The leading term of the characteristic function o(t) is equal toe `21' which corresponds to the normal distribution. And the second term g(x) is the Fourier transform of

2

(t2

log I t I)e -t2i4

that is, g(x) = - 2rG"(x) ,

where G(x)

_ 2 e-x 2iJ0 (c0 e

-s2dx) ex2dx+C}

with C 4 1

S __

(log

I t I )e-`21'dt

Acknowledgement The authors wish to thank the referee for his kind comments. UNIVERSITY OF TOKYO UNIVERSITY OF ELECTRO - COMMUNICATIONS

REFERENCES

[1]

Gnedenko , B. V. and Kolmogorov, A. N. (1954). Limit Distributions for Sums of Independent Random Variables , Addison -Wesley, Cambridge , Massachussets , (Translated from Russian). [2 ] Ibragimov , I. A. and Linnik, Yu. V. (1971). Independent and Stationary Sequences of Random Variables, Wolters-Noordhoff, the Netherlands , (Translated from Russian). [3 ] Petrov, V. V. (1975). Sums of Independent Random Variables , Springer -Verlag, (Translated from Russian). [4 ] Takeuchi, K. and Akahira, M. (1976). On Gram-Charlier- Edgeworth type expansion of the sums of random variables (I), Rep. Univ. Electro-Comm. 27-1, 95-115. [5 ] Takeuchi, K. and Akahira, M. (1976). On Gram-Charlier-Edgeworth type expansion of the sums of random variables (II), Rep. Univ. Electro- Comm . 27-1, 117-123. [6 ] Cram€r, H. (1963). On asymptotic expansions for sums of independent random variables with a limiting stable distribution , Sankhya, Ser. A, 25, 13-24.

90 Rep. Univ. Electro-Comm. 28-2 , (Sci. & Tech. Sect. ), pp. 259 - 269, February , 1978 259

On Gram-Charlier-Edgeworth Type Expansion of the Sums of Random Variables (III): Multivariate Cases* Kei TAKEUCHI* * and Masafumi AKAHIRA * * *

Abstract Gram-Charlier-Edgeworth type expansion of the sums of i.i.d. random variables without higher order moments was discussed in the previous papers. In this paper it is shown that the expansion is extended to the multidimensional cases.

1. Introduction The authors discussed in the previous papers ([ 11,[2],[31) Gram-Charlier-Edgeworth (G-C-E) type expansions for the sums of i.i . d. random variables which do not necessarily have moments up to the required order . Now it is extended to the multidimensional case. Suppose that X1, X2, ... , X,,, ... is a sequence of i.i .d. p-dimensional continuous random vectors. Now suppose that the density function f (x) is of the following form: f(x)= g(T) (1 + r)R+P+l where r = (xx) , T is vector of p -1 angles of the x-vector and R is a positive constant. Then the characteristic function of X is computed as

0(t)

= f

eit'x

= Jf

f(x)dx

(1 +rrpg(T) )R +P +1

exp { irph

(6,r)

} drdr,

where p= (t't)/ and 0 is the vector of angles of t. If we denote

0P ' 0 (u) = f (1 + r)-P+P+ 1 exp ( iur) dr, we get 0(t) =J ^p,Q(ph(0,T))dr. * Received on October 6, 1977 ** University of Tokyo *** Statistical Laboratory, University of Electro-Communications

91 Kei TAKEUCHI and Masafumi AKAFIIRA

260

n Therefore the characteristic function of Zn =aryl

{.f gyp, (arylPh (0,T))dr}

Xi - bn is given by

exp(-ibn't).

Then expanding Op,p (•) near the origin , as was done in the previous paper , we get the asymptotic expansion of the characteristic func+,on of Zn and thence its density. Explicit form of the asymptotic density for some special cases are given. Then the results are generalized to the case when f (x) = g1 (x) + ... + gp (x) + h (x) , where gi (x) are all above type and h (x) satisfies the condition r+p

lim Ix xI 2h (x) = 0 Ix'xI -' for some r > 0 . 2. Results Suppose that the density is of the form

f (x) = C1 f1 (x) + c2f2 (x) + ... + Ckfk (x) + g (x) where f f (i = 1, 2, ... , k) are functions of the form given in section 1 . If for some r Ixxl g(x)-+ 0 (x'x- °°) , then it is possible to extend the characteristic function

0 (t) =

f f (x) eit•xdx

around t - 0 up to appropriate order . Hence it is seen that the asymptotic expansion of the distribution of Zn holds. In the subsequent discussion we deal with multivariate t-distribution cases and more general cases. Let X1 i X2, ..., X,,, ... be a sequence of i.i .d. random vectors having a multivariate tdistribution with a density f(x)=c(1 +x'Ax) , v>0 where A is a positive definite matrix. ' Suppose that Y is a random vector having p-dimensional normal distribution with mean vector 0 and covariance matrix A-1, and vS2 is distributed like X2 with v degree of freedom , and vS2 and Y are mutually independent . Then it is seen that the t-distribution is equal to the distribution of Y/S. Since the conditional distribution of Y/S given S is normal with mean vector 0 and covariance matrix A - 1 IS2, it follows that the characteristic function of the tdistribution

92 February, 1978 On Gram-Charlier-Edgeworth Type Expansion of the Sums of Random Variables (m) 261

is given by

0(t) = E(expit'Y/S) = E [ exp ( - 2 t'A-1 t) /S2 } ]

t'A-1 t 00 SZ ° -1 1 v (2 )2 exp { - 2 (S2 + S2 )) dS2 2I'(2) ^o Hence it is easily seen that 0 ( t) is a function only through and t'A-1 t, Put 0 ( t) =0v (r), where r - (tA-1 t). Since the form of 0 (t) does not depend on the dimension p, it follows that in the case p = 1, > , (-\/ vI r I ) is the characteristic function of the t-distribution with v degree of freedom . When v is odd number , 1 v is easily obtained as follows : for r> 0

Y'3

(r) (1+r)e-r;

0s

(r) 1 + r + (r2 /3) } e-T

IP7

(r) 1 + T + (2r2 /5) + (r3 /15) ) e-T

When v = 1, Z„ - E X; I n = X is identically multivariate Cauchy. When v = 3, the characteristic function of Zn Xi/n - X is expanded as follows: r? 3 ) } n ^ e 2{ (1 + 3 v n

4

6

T{P3(^n 4n + 18n + o (n)) }

_ t 2 't (t'A 1t) - (tA-1 t

e { 1 + 3n 4n

)2 (t,A 1t)3 1

+

18n + o

(n) } (2.1)

Hence the asymptotic distribution of Z„ is normal with covariance matrix A-1. In order to get the below the second term of (2.1) we consider the case when A = I. This does not lose the generality since the general case is reduced to the above by appropriate linear transformation. Then it follows that 'I { `3(

r s5n

)

}

n^'

e

2

£ t?

1+

1 t • 2)1i2- (

Eti2 ) + (Ett1)3

3^ i ` 4n 18n

In the asymptotic expansion of the density corresponding to the characteristic function, the corresponding term to (t12 tj2 ...) exp (- 2 E t;2) is given by

93 262

Kei TAKEUCHI and Masafumi AKAHIRA a2

2

ax2 a

x

?) ... (V L

n )" exp( 2 i xi )

For the term of order n 2 putting G(x)

(2a f... ((Etj2)1/2{ exp (- 2 Etj - itjxi)}rl dti

z the corresponding term to (E ti2) 3i2 exp (- 1 Z t`.2) is equal to - E G 2 2i a x i (x). In order to obtain G (x) we consider the case when p = 2. Making a transformation t1 = T cos B t2 = r sin 0 , xl = r cos a, x2 = r sin a, we get G (x1, x2

27r IT f o r T2 exp { - 2 - iTr cos ( 0 - a) } d rd 0 ) = ( 27x )2 JoJo

1

(21r) 2

f0

n 2 r2 exp(_ T - irrcosO)dedr. J 2

Since cos ( 0 + 2) cos 0, it follows that

(' (XI, x2) =

fo f_ IT2

( 2W)2

T2

T2 { exp (- 2 - irrcos8)

I 2

+ exp (-

+ irrcos0) } dOdT

2

IT

(27T)2

n 2

1W r

T2 2

exp (-

2

- irr cos e) dTd6 .

Since for any real number y 1 it 2

2^

d2

- iTY

27T f e dr = (- dye)

1

y2

27T e 2

1

y

7=

2

= (1 - y2) e 2

it is seen that 1 IT G (x1, x2 ) _ (2^)3i2 f n -2

r2 cos20 (1 - r2 cos2 0) e 2 dO .

Putting g (r2) = G (x1, x2), we have a2 a2 a2r2 a2r2

dG

a x22

) dr2

(ax 12

+

ax22

)G =

(ax12 +

= 4g'(r2)+4r2g"(r2)

ar 2 ar d2G 2)

+{

(a x 1

+ (ax2)

d(r2)2

,

94 February, 1978 On Gram-Charlier-Edgeworth Type Expansion of the Sums

of Random Variables ( 81) 263

Consequently each term is expressed as a function of r2. When p 2 3, putting r2 = E xi2 and making an orthogonal transformation t1' = E xiti/r , i t2' = E c jt3, ... , tj' = E c1,, tj we obtain

G(x) =

l p f • • • f r { exp (r1 Fi ti'2 - it1'r) } H dti' . (2jr) 2 f

Further transforming r cos O p = t ' , r cos O p cos Op-1 = t2 ' , ... , we have D

G(x) _ (2x)p f ... f 7p 2n

= cp

f 0: f 0

1

2

II2 sin" B f I {exp (- 2 - irr cos 81 ) } dr II dBt 2

rp{ exp(- 2 - irrcos01 )} drdO,

where cp is some constant . For even number p, the following generally holds: n

p

G(x)=cp f (-1)2 ^p(p)( rcosO1)dol n _2 _X 2

where ^p (x) _ 2x e 2 Since for odd number p 2 Tr

f

11 -irx dx

0

7e- 2

x

=

17r

0

v (t) -

1

dt }

O (x)= G, (x) (say),

it follows that p-1 G(x)=cp ' f 2 (-1) 2 G1 (p-1)(rcos01 ) dOt = g1 (r2)• n _2

Since (^ ax{ G(x) = 2pgp' (r2) + 4r2 gp„ (r2), the asymptotic expansion of the density may be possible. For odd number v (? 5) the asymptotic expansion of the density may be similarly calculated. When v is even number , it may be impossible to express the characteristic function in a simple form . Putting v = 2m we have

95 Kei TAKEUCHI and Masafumi AKAHIRA

264

7 2 + T4 +•-•+ cmT2"+ logT+o(T2mlog T). 2(v-2) 4(v-2)(v-4)

For v = 2 we obtain T2+o(T2).

412(T)= 1 + 2 T2log T -

2

n log n is given by

Then the logarism of the characteristic function of Z,, = E X; l t T

T2

nlog 4/2 ( ) _ (log r - 1 log n - 1 log n)

nlogn 2logn 2 2 cT2

2logn

1

+ O(n).

For an appropriate number c,,, the characteristic function of c„ Z„ is expanded as T2

T2logT+o( e2(1+ 1 1 log n ) log n T2

2

The corresponding density to e - 2 is normal and that to (T2 log T) e" 2 may be calculated as follows : Putting t2

(log t) e 2 (cos tx) dt ; K1 (x) = 1 Ir J o

K2 (x) =

1 (

1J 0

t2

(t log t) e 2 (cos tx) d t ,

we have IE2

cp ( K, (D) (r cos B) dO = K, (r) for even number p ; .l

2 n 2

cp {

K2 (v) (r cos 9) dO = K2 (r) for odd number p ;

J 2

It follows from Takeuchi and Akahira [1] that Kl(x)_-^P(x)[

to

t2

{ Jo a

y2

2dt } e 2 dy + ci];

K2 (x) - Ki' (x) 42

K1 (x) =Jx e 2K1 (u) du + c2, 0

96 Edgeworth Type Expansion of the Sums of Random Variables ( 11) 265 February, 1978 On Gram- Charlier -

where cl and c2 are certain constants. Hence it is seen from the above that the asymptotic expansion of the density may be possible. We shall deal with more general densities . In the case p = 2, we consider that the density of Xi is expressed as c - g (a) v+1

f (x1, x2)

r(1+r2) 2 (a + if) - g (a). Putting where r - x 12 + x2 2, a = tan 1 x2 / x 1 and c is some constant, and g z = t12 + t22 and 0 - tan-1 t2 /t1 , we obtain the following characteristic function:

0

(t1 ,t2)

f

x2)

f J f (XI,

eit,x,

+tt2x2

dx1 dx2

1 =2 r eirtcos (e-a)g(a)drda J v+l n 2 (1+r2) 2 IT

(2 fit„ (,r cos (0 - a)) g (a) da. .)_2 n 2

For example a density f (x1, x2) is given by c

f (xi, x2) _

1+x14+x24

where c is some constant.

Since c

K

(x22 + x22)2

1 + x14 + x24 { 1 + (x12 + x22 ) } 2 x14 + x24

where K is some constant , we have K f

( x 1,

x2) =

(1 +r2 )2

1 cos4 a + sin4 a

On the other hand it follows that 'P2(0 _ (1+ItI)e-Itl (See [ 1 ] , page 99). Hence n O( t1, t

= ( 2 1+Ircos(0- a)lTeos ( 92) J IT cos4a+sin4a

2

e-17

a)Ida .l

97

Kei TAKEUCHI and Masafumi AKAHIRA

266 If v = 1, then n 2

^

e -In

t2)

01,

cos (9- a )Ig

(a)da

n

J

n 2

('

1

-7

I cos (0 - a) I g (a) da

J

n 2 IT

+ 2 J

(2.2)

cost (0 - a) g (a) da + o (T2) . 2 2

Though the last term of (2.2) is expressed as n T2 f 2 cost (6 -a) g (a) da n 2 IT 2 =

(COS2e coS2a + 2 cos

T2 f

Os 0 in cos a sin a

IT _2

+ sin20 sin2a)g(a)da = Allt12 +2A12tlt2 +A22t22 , IT

r f I cos (6 - a) I g (a) da = rh (B) (say)

is not done as a simple formula of tl and t2. Hence the characteristic function of E X;/n is the following form:

{ W tl , 1 } n~ a-Th (e) { 1 + 1 (A11 t12 + 2A 12 tl t2 + A 22 t22 + ... )+o (1) }

n

n

n

n (2.3)

The density corresponding to the first term of the right-hand side of (2.3) is given by 1 f*(xl,x2) -

() 2

f

e -rh(O ) e -iTr cos

(6-a) dtldt2

_ f

IT

27T2

2

(,(0 Te -Th ( e) cos

rr cos (0 - a) } drdO

2

_ 1

2 1 -r4 cos4 (

1( 27f2 J n

B-) d6 .

{h (0)2 + r2 cos2 (0 -a) 1 2

(2.4)

98 February, 1978 On Gram-Charlier-Edgeworth Type Expansion of the Sums of Random Variables (I) 267

In general the distribution of the density ( 2.4) is not Cauchy. Since the asymptotic distribution is a stable law, it is shown from the above that in two-dimensional case the stable law with characteristic exponent 1 is not always two -variate Cauchy . But for every a l and a2 the distribution of a1X1 + a2X2 is always Cauchy . The corresponding density to the second term of the right-hand side of (2.4) is given by 2

2

2

n (Atlaz 2 + 2A lz axax + Azz ax 2 )f* (xt, xz) t 1a z z Ifv= 3, then 4(t1, t2) = 1 - 2

(A11 t12 + 2A12t1t2 +A22t2)

3 + 3 J cos (6 -a) I3g(a)da+ O (r3).

Since {^( tt , tz ))^ a-2tAr r3 (B) of 1 ) ) , 3 n { 1+ 36n h + 1 nn it follows that the first term corresponds to a normal distribution . Making an appropriate linear r

transformation we may have A - I. Then the corresponding term of the density to r3 e 2 h3(0)

is expressed by 2 n r2Cos2'(B-a) , - (axe + aa 2 ) 2 ( 2 1 { 1-r2 cos2 (0-a))h(0)e 2 d9. t z t

Ifv is odd number, then similar expansions hold. Ifv= 2, then ¢(t1, t2 ) = 1+2 rt (log r)

f cost (0- a)g(a)da

+ 2 f { log I cos (0 - a) I) g (a) da

-

r2

2

1

cost

(0

- a) g (a) dot + o (r2 )

By a suitable linear transformation we have

(t1, t2)

1 + 2 r2 logr - 2 { c+h4 (0)) r2 + o (r2).

It is seen that the asymptotic distribution of E X,

n log n is normal . The term of order

1/log n is calculated by the same way as the multivariate t-distribution case.

99 Kei TAKEUCI II and Masafumi AKAHIRA

268

The above discussion is extended to a general dimensional case . Indeed , we consider the following density: c p-1 2 g(a) r (1+r 2)v/

f(x)

where r2 = x'x and a is a vector of (p-1) angles and c is some constant . If g (-a) = g (a), then the characteristic function of X is given by P(t) ° f 0., (7-, h (0,a))9(a)da.

Hence the same discussions as two-dimensional case may be possible in the multidimensional case. If v a 1, then the asymptotic distribution of EXi/n is a stable law with characteristic exponent 1 but it is not always multivariate Cauchy. For v - 2 and v 2, the asymptotic distributions of i

Xi / n log n and E Xi / Nfn are normal, respectively.

The asymptotic expansion of the density is essentially similar to the case p = 2. It is also seen that the symmetricity of the density is not essential . Indeed, if in case p - 2 a density is given by

f(x1,x2) =

q g(a) r(1 + r)v+f

Then the characteristic function is expressed as n 0(t1, t2)

Oq (rcos(0 -a)) g(a)da J-n itr

where

iq (t) =

qe dr. J p (1 + r)a+1

Then we may obtain the asymptotic expansion of the characteristic function of Z„ by the same way as the above symmetric case . When p 2 3, then similar discussions hold. In the discussion of this paper it is essential that the order of f (x) as x'x - °° is independent of directions . But we may avoid the point by the following way. Indeed , in the case p = 2 we consider a density

f(x1,x2) =

c 1+x12 +x24

Then the characteristic function is obtained by a 2 - cre'rr(9-a)

O(t 1,

t) 2

= 2 I f o 1 + r2 cos2a + r4sin4a

.2

dnta .

100 On Gram-Charlier-Edgeworth Type Expansion of the Sums of Random Variables ( ll<) 269

February, 1978 If

r

_

1

1+r2 cos2a+r4 sin4a (sin4a)(1+r)3

+ g (r) ,

g(r) is order of r-4. Since we may make such expansion in detail , we obtain the asymptotic expansion of 0 (tl , t2) around the origin. But in such cases it may difficult that standardized constants are common . In this case we have to consider the asymptotic distribution of (T,Xlf/n, Y-X21/n).

i

f

References (1] Takeuchi , K. and Akahira , M., "On Gram-Charlier-Edgeworth type expansion of the sums of random variables (I)," Rep . Univ. Electro-Comm. 27-1, (1976), 95-115. [2] Takeuchi , K. and Akahira, M., "On Gram-Charlier-Edgeworth type expansion of the sums of random variables ( II)," Rep. Univ. Electro-Comm . 27-1, (1976), 117-123. [3] Takeuchi, K. and Akahira, M., "Extension of Edgeworth type expansion of the distribution of the sums of i.i .d. random variables in non -regular cases," Ann. Inst . Statist . Math. 29, Part A, (1977), 397-406.

101

Rep. Univ. Electro-Conun. 28- 2 , (Sci. & Tech. Sect.), pp. 271-293 February, 1978 271

Third Order Asymptotic Efficiency of Maximum Likelihood Estimator for Multiparameter Exponential Case * Kei TAKEUCHI * * and Masafumi AKAHIRA * * *

Abstract For multiparameter exponential family distributions the asymptotic expansion of the distribution of extended regular (ER) estimators is obtained up to the order n 1. And it is shown that the (modified ) maximum likelihood estimator attains the third order asymptotic efficiency among ER best asymptotically normal (BAN) estimators.

1. Introduction Third order asymptotic efficiency of maximum likelihood (ML) estimators have been discussed by J. Pfanzagl and W. Wefelmeyer ([ 21), (who adopted the terminology) and also by J.K. Ghosh and K. Subramanyam ([ 1]), for cases when sufficient statistics exist . In this paper we shall establish more general results for the multiparameter exponential family, introducing a differential operator , and show that (modified ) ML estimator is always optimal up to the order n-1 among similar modified class of estimators. The method is not restricted to the exponential family and will be applied to more general cases in a subsequent paper ([ 3 1).

2. Formulation of the problem Suppose that X1, X2, ... , X„ are independently and identically distributed according to an exponential type distribution , i.e. with the density m

f(x,B)=h(x)c(9) exp {

s1(B)t1(x)} i=1

where 0 is a p-dimensional real vector -valued parameter which is supposed to be in an open set O C R°, and s,(9)'s are continuous real valued functions and ti(x)'s are real valued measurable functions. Define c* (st,..., sm) =

1 h (x) eXp

t-1

si t, (x)} dp (x)

* Re.eived on December 7,1977 ** University of Tokyo *** Statistical Laboratory , University of Electro -Communications

102 Kei TAKEUCHI and Masafumi AKAHIRA

272

as a real valued function of s1, ..., s,,,, and denote S*CRm be the set of (Si, ... , s,n) for which c* is finite. We assume that for every 0 E 0 , (s1( 0) , ... , sm(0)) is an inner point of S*. We have, obviously that c - c*-'. We assume that s1 (0 ), ... , sm (0) are linearly independent when 0 varies in O, and also that t 1(x), ... , t,,, (x) are linearly independent functions of x. (The domain of x does not need be specified) For the sample (X1, ... , Xn ), a sufficient statistic is given by T-(T1,..Tm); n

Ti - E ti (XX) I n . J=1 From the assumption T has the moment generating function given by m

E (exp i E ti Ti)

- c*(sl (e) + E1 , ... , 5m (0) + Em ) I c*(s, (e) , ... , sm (e))• And the joint cumulants of Tl's are given by

K o k ...

or _0_

alias,ask ...

I s1 - s1 (0), ... , sm - sm(0)

We assume that Ti's are continuously (w. r. t. the Lebesgue measure) distributed and that their joint density admits Edgeworth expansion. We also assume thatsi (0)'s are continuously differential four times. Now we shall define a class of estimators called extended regular ( ER) estimators which are expressed as 1 0 - g(TI,..., Tm) + n h(T1,..., Tm). where g is three times continuously differentiable and h is continuously differentiable, and both are independent of n. An ER estimator is consistent iff g (µl(0), ... , µm(0))

=

0

(2.1)

where µi (0) - EB(Ti) (i-1,.. .,m). and if 0 is a consistent ER estimator , (9 - 0) is asymptotically distributed according to the normal distribution with mean vector 0 and the variance-convariance matrix E - E E oiiai of f

where ai's are vectors defined by

,

ail

- ve (Ti, Ti)

103 February, 1978 Third Order Asymptotic Efficiency of Maximum Likelihood Estimator 273

ag (2.2) I Ti - Iii (0)

ai = aTi

It is well known that E - I-1 > 0 (positive semi -definite) where I is the Fisher Information matrix given by I =-E8 [

a2 logf(X,0)] a0a0I

= E0 [aa log f(X,0)' =Ee [Easi (T-ti)

ao

a0,

log f(X,0)]

asi ae

(Ti-it)]

_ as, asj i Qv ao a0, And also it is easily seen that the equality is attained iff

a,

1 asi

(2.3)

ao

We call an ER estimator which satisfies ( 2.1) and (2.3) an ER best asymptotically normal ( BAN) estimator.

3. ER BAN estimators Now we expand each component of 0 into Taylor's series as Ba 0. + E as (Ti - pi) + 2 E E b (Ti - Ili) (Tj - j. )

+ 1 E E E C,Ik (Ti - Ili) (Tj - µj) (Tk - Ilk) 6 i j k

+ 1 AC, + 1 E d^ (Ti - pi) + R.

where

a a2 a3 go a? =aTj

, bij aTiaTj ' Cak aTiaTjaTk

ah Aa=h (111, ... p,,, , di = aTi and all the derivatives are evaluated at Ti = µi (0) . By putting T; - V (Ti - p1), ( 3.1) can be rewritten as

-

104 274

Kei TAKEUCHI and Masafumi AKAHIRA

-V_n (0 - Ba ) = E a q'T, +

1 E E ba T' T ' 2N/n i j

+ 1 EEECj kTiTiTk 6n i i k

+ 1 Aa + 1 2, OT! + R , . v/ n n i It is shown that R& is stochastically of order smaller than n-1. We shall denote that Ya - 1`n (Oa -Oa) - Ra .

Then we have -,/ n

(e a - Oa )

- Y. = op (1 / n), moreover , we have Theorem 3.1 The joint distribution of NrT n(0a - Oa ) and Ya coincide up to the order

n-1, more precisely , for any bounded continuous function 0, we have n lim

n I E0 ,/n

(Y1 . .... , Yp) ] - E0 [0 (sn ( O1 - 0), ... ,

(ep

-e))] I = 0.

Proof. Observing that Pr { n I R' I > e } =o (1/n) for every e > 0, we can show that lim

n

n I E [exp E ita (Oa-Oa ) ] ^

a

-E[exp E ita ya] I = 0 a

uniformly in ( t 1 , . . . , tp) in any compact subset of Rp, which establishes the theorem. (QED) Corollary 3 .1. For any measurable set C C RP whose boundary is of Lebesgue measure 0, we have lim n-

n I Pe{YEC}-P0 {-Vn(0-0)EC} I 0.

From Theorem 3.1 we have Theorem 3.2. The joint distribution of'(Oa - Oa) (a = 1, • • •, P ) is asymptotically equal to a continuous distribution up to the order n-1 . And for the distribution of Ya's we have Theorem 3. 3. The joint distribution of Ya's admits Gram-Charlier-Edgeworth type expansion up to the order n-1. Proof. The theorem is established by algebraic evaluation of the joint characteristic function of Ya's, the detail of which is obmitted. Consequently the joint density of /-n (Oa. - 0a) (a = 1, • • •, P ) can be expanded up to the order n-1 if we have joint cumulants of Ya's up to the fourth.

4. Some preliminary results In order to evaluate the joint cumulants , we introduce the following differential operator.

105 February, 1978 Third Order Asymptotic Efficiency of Maximum Likelihood Estimator 275

D« _ I ap a8 , where I' R

denotes the element of I-1 and the random variable

U=,/REFij«1asi « « if a0p

(Ti -µi)

(D«s,) T;

f

= Dotv/n f(X,0)} The following lemma will be frequently applied in the subsequent discussions. Lemma 4. 1. Suppose that Ze is a function of T1, ... , Tm and of 0, differentiable in 0 and that a E0 (Z9) = f a { Z01If(xi,0) } lIdµ(xi) ae a0 i holds . Then we have Ee (U«Ze ) D«E0 (Zo) 7Ee (D«Z0) Proof. D« Es (Ze ) f (D«Zo) II f (xi, 0) II dli (xf ) + f ZeD« H f (xi, 0) n dp (xi)

Eo(D«D0 ) + f ZeD« (E log f (xi, 0 )) lI f (xi,0 ) II dp (xi) = Ee (D«Z0 ) + vrn Eo (Ze U« ) from which the theorem is directly derived. (QED) We shall use the following symbols. z f(X 0 ) - a0 log f(X,0)l

d«Ry = Eonaeae log -.5

R

- E E Q a2 S i

ti

a s;

'^(ae « a0a)(80 y

K«Rti = E8 [

8log f (X, 0) • a9

as.

=

EEEK „k

log f (X, 0) - a6 log f (X, 0)] a ti

as.

ask

( a0)( « a0)( R ae y

106 Kei TAKEUCFII and Masafumi AKAHIRA

276 Jap Y

- E E E Iaa Ipp IYY Ja,p,.y, a' p ry'

KapY

E E E Iaa' Ipp' P' Ky'p'y' Y' 3

Lap7•s = Ee [ aB a ae p B Y

ou

log

a3s, -

f (X, e) ' 8es

log f (X, e) l

as. -11

(aeaaepaery ) ( aes) ,

a2 a2

Map. ys - Ee [ - log f M O) aeyaes log f (X,0)1- lap Irys

a2Si a2s, o11 ( aeaaep ] ( aeyaes)

Na p Y6 - Ee [

log f (X, 0) •

a

a2 aeaaep ae y

log f (X, 0) • a log f (X, 0)]

aes

+ lap IY 6

kKiik

HapYS - Ee [TO

as, ask aes; ( aeaep ) (ay) (a es )

log f (X, e) , a8 log f (X, 0) • a log f (X, 0) a p Y

a log f (X, 0) ] - (,a# Lr6 +lay fps +1as lp7 aes

-

L' d=

Fi

asi as; ask as, (aep) (ae7 ) (aeb

k kKijki (a ea)

E E E E I as Ipp IYY Is s La'p'7'.6 ; a' p' ry' d'

MOO-0 - E E E E Iaa'lpp' J7Y, I66'Ma 'p'.Y's' a'p'ry'S' Nap'ys -

E laa 'Ipp' Ivry Iss'Na E E 'p'•ry's' a' p' Y' s,

Iaa 'Ipp'IYY' Iss'Ila.p,y,s. A°pys - E E E a' p' Y' s'

107 February, 1978 Third Order Asymptotic Efficiency of Maximum Likelihood Estimator

Using these symbols we have

E

DaIRy=

E

EE

=

D. (Qii

8BR aey

ag,

asi

(Daoi,)

E oi i

as (aR)

1 2;

asi

as,

O

as,

.

(Da

K i)k

i Ek

a'

aey )

(aO) (aey ) (aok aS

P o" X 2;

aa' , rE j

+

+

) ; if (Da ae) y (ae ) (ae ) (ai9 R y R

i +

as, a s,

aS

k

)

+ EIasEE v ij

a s,

aZSi

Qi;

aO a ,aeR) (aey a 2 S, l aSi)

aea"aey

300

= E laa Ka,,,,y + E Iaa Ja'R.y + E Iaa a, a a,

Ja-y.R

Da IRy=-E E IRR' Iyy'DaIR,y, R' y =-E E E IRR Iyy Iaa (Ka ,R y, + Ja,R'.y, + Ja'y'•R') R, 'y' a' -KaRy-JaR1Y-Jay•R .

aZSi a2sk E E (Da Qij) ( ao1 aey (aeb )

DaJRy.8=

+ oii (D a

a 2S. aeRaey

a2s. ( aey

a2s; a2s, + Ei F. O,; ( ao1aey) ( Da aea ) Iaa '

+

+

EEk

n2 S

Kiik (

aeRaey )

-.

S S

( aea )

a3 E laa EE ( s ; ) (as' ) i ; aea 'ao1aey aea a' E

or,

a 2S1 Iaa'

i I

) (

a2s,

(

ao1aey

aea'aes

N o,'

277

108 Kei TAKEUCHI and Masafumi AKAHIRA

278

(Npy.a,s + La,py.s + Mpy.a's)

E laa a'

as. aS). ask

DaKpys

k (Da Kijk ) + E E o;j

(a)

as;

as,

(ae) (aes

)

7 ask

( Da aep ) ( ae Y ) (aes )

as; as; ask

+ E ^. a;i (a ) (Dane (aes Y p +

EE

as; as, ask

Qii

aep)

(aey ) (Da aes )

Iaa (HH,pys +Na'p•ys + Na'y•$s

+ Na,6.py)

a' DaJpY" = E E E (Da Ipp) rrY I66 J$3' 's, p' Y' 6'

+ E E E I pp' (Da IYY') I ss ' Jp y s , + E E E IRp IYY (Daisy ') Jp,y, s,

+ EE EIppIYYlss DaJpy.s _ - E E E (Kapp' +Jap'p' +Jap'•p) IYY' Iss' Jp,y, s. p' Y' 6' E E (KaYY' +Ja1'Y' +Ja7"7) Ipp' Iss' Jp,y, s, p' Y' 6' E E (Kass'+Jas's,+Jas,'s)Ipp'IYY'Jp,y,.s, p' Y' 6' + E E E E Po PO,IYY Iss (Np +La,p,y,.s,+Mp,y,.a,s a' p' y' s' Introducing still other notations as aa' pp' . If of, p' J vy =

E E Iaa IYY Ja,p,.y, ay

Kai = E E Y a p'

Iaa Ipp Ka,p,y,

the above can be abreviated as

109 February, 1978

Third Order Asymptotic Efficiency of Maximum Likelihood Estimator

D«JRY .s = -

279

E (KORR ' + Ja1'R' + Ja1 'R) J7:-s R' E (K0`YY '+ j' + J ay '• y) JR.s ,y Y (Kas6

+ Jad's + Jas 's)Jbti

+ NRy•as+ L'RY•s +Mas•RY .

(4.1)

DaKRYS= E E E (Da IRR) IYY?S,KR,Y,s, + E E E IRR' (Da IYY) Iss'KR,,y,s' R' Y' b' ' Iaa' (Do j ss') + E E E p1313 K R, y, 6' #

"Y , 6

+ E E E IRR I77'I6s (DaKR y,s') - (KaRR' +J0'"' +JaR 'R) K'y 01 R - E

+JaY'Y' +Ja7'•7) K13

(Ka"'

,y

Y

E (Iass'+Jas'S'+Jas,'s)Ksy + Ha1YS +Na1'Y8 +Nay.R& +Na6 13 (4.2) We have DOUR= DaDRE s1Ti I (D(,,DRsi) T; - f n DRsi (Daµi) (DcDRsi) T; -fn E E (DRsi) oi, (Do, si) i i (DaDRsi) T'

- f n E E E E Ia a IRR

a'

R'

Qi! ( as ,) (as ) i ; aea, a0p,

(DaDRsi ) T; - f n E E Iaa IRR Ia R (DgDRsi ) T; - fn IaR. we shall denote Val = E (DDDR si) T .

110 Kei TAKEUCHI and Masafumi AKAHIRA

280

Then we have the second lemma: Lemma 4.2 If Up Ze and Ze both satisfy the condition of the lemma 4.1 we have E8(U.UpZe)_ DaEe(UpZe)+IapEe(Ze) Nrn-

E9(VapZe) - EB(UaDaZe)

IapE0 (Ze) - 1 E0 ( VapZe )+n D&DpEg(Z0)

- n DaE0 (DpZe) -n D, E9(DaZe)+n E9(DO, DpZ0) Proof. By lemma 4 .1, we have E0 (U.UpZe )= DOE8 ( UpZe ) - Ee (D.( UpZe)) N/-n-

EB { (Vap - I DO-E0 ( Up Ze ) - Nrn

ap

) Ze }

which establishes the first half of the lemma. The second half is derived by applying the lemma 4.1, to terms, E0(Up Ze) and E9 (Up Da Ze) in the above. (QED) We shall derive the moments of Ua and Vpy. Obviously it is shown that E(U,) = 0 ; E(Vap) - 0 ; E(UaUp) = Iap. We have Vap (Da Dp Si) T i = E E Da (Ipp' asi) Ti aep a2s=

- EZ(D aIpp) (as`)T+ EEEIaaIRp( aep ' ` i a' p' i p• E E (K app' +Jap'p' +Jap 'p) (asi) T' i

ae p

p'

+ EEElaa'lpp( i a' p'

azs,

aea,aep, )T'

ae a 'aep) ) T'`

111 February, 1978

Third Order Asymptotic Efficiency of Maximum Likelihood Estimator

281

Hence it follows that E (VapUy)- - E E (K"13 ' + Jap•p ' + Jap''p ) Iyy' lp,y, p' y' + E E E Iaa Ipp' Iyy Jo,,p,.y, a' p' y'

- Kapy - Japh' - Jay'p + Jap-y - - Kapy - Jay'p I E(VapVys) - E E (Kapp' + Jap•p' + Jap's') I1,y, . (Kyss' + Jys's'

+ Jy8 1- 8 )

E E (Kapp' + jag-g' + Jap'•) Iyy'Iss'j ,s, p,

- E E (Kyss'+ Jys's'+ Jys'•s)Iaa'Ipp'Jap

+ E E E E I aa IRA Iyy Iss Nla ,p, y'6' a' p' y' 6'

=E 61 (Ksp + J aR + J s9) (Kyss ' + J76•6' Jys'•s) -E Jap ,'p) Jys p, (Kapp ' + Jap'p' + p - E Kyss' + Jys's' + Jys''s) Jb13 s' + Map•y6

= s (K6o9 + Jaa ) (Kyss + J76'.6) - EJSR Jyss +Map.ys

(4.3)

"E(UaURUy) = Kapy ; .\I-n-E (Vap UyUs ) - - E E E (Kapp ' + Jap'p' + Japl'p) Iyyllss,K 1,y,s, p' y' s'

+ E E E E iaa'Ipp' Iyy'Iss ' Na,p, y,6, a' p' y' s' = _ E (Kapp' + Jap•p' + Jap''p ) Kypb + Nap-y6 P,

E (Ua Up Uy Us) =

Hapys+ lap lys+ Iay Ips+ Iab lpy n

(4.4)

112 282 Kei TAKEUCHI and Masafumi AKAHIRA

5. Asymptotic cumulants of ER BAN estimators We shall obtain asymptotic joint cumulants of Za's making use of the results of the previous section. Note that the asymptotic cumulants of Za's are equal to those of / (Ba - ea ) up to the order n-1. We can write

Ya=Ua+2a + nWa n

And we have

E(Ya) 2^E( Qa)+o(n) We shall denote 2 E(Qa) = µa , and

1 Z. = Ya - µa.

Then we have 1 DaZp=-Vn Da 6p -7Da{ .P = - f Iap - - Daµp. n First we shall obtain the variance of Za up to the order n-1. We have

E(Za - Ua)2 =

1 4n

1 V(Qa) + 0(n

and we have Cou(Z.,Zp)= E(Z.Qp) + °(n) E(UaZp) + E(UUZa) - E(Ua Up) + E(Za-Ua)2 + o(n 1 ),

Applying the lemma 4.1 of the previous section we get E(UaZp )= )=DaE(Zp) - E(DaZp) = Iag + 1 Damp n E(UpZ0)= Iap + 1 Dpµa. n

113

February, 1978 Third Order Asymptotic Efficiency of Maximum Likelihood Estimator 283

Hence we have Cou (Z«, Zp) = Cou (Y«, Y, ) = I«p + n D«llp + n Dpp« + 4n V(Q«) +o(n). As to the third cumulants , the following equation is referred to Z«ZpZy= 2 (U«ZpZ,y + up Z«Zy + UyZ«Zp)

2 U«UpUy + 2 U«(Zp-Up) (Z«-Uy)

+ 2 Up(Z«-U«) (Zy - Uy) + 2 Uy(Z« -U«) (Zp Up) + (Z«-U«) (Zp - Up) (Zy-U7)• Since the last four terms are of magnitude of order smaller than n-' , we have E (Z«ZpZy) = 2 E (U. Z# Z-t + Lb Z«Zy+UyZ«Zp )

2 E(U«UpUy ) + o(n). Furthermore we drive that E(U«ZzZy)= D«E(ZpZy) -,rn- E {(D«Zp)Zy}

E Z D Z n p { ( y)} «

_ - 1 (K«p'Y + Jap y + J«y•p) + o(1n ) Thus the third cumulant is given as E(Z«ZpZy ) _ - 1 ( 2K«pY+ J«p'Y+ J«'Yp+ J0 11") +o(1) n 1 ). =Q«py+0(n As to the fourth cumulants , we shall need the following equation,

114 284

Kei TAKEUCHI and Masafumi AKALURA

(UZ Z S Z +UZ Z« ZR Zy Zs - ?3 a y R aZ Z+UZ s y aZpZ+U d S Za Zp Zy)

- 1( ZZ Z 3 U «- UP p Z'Y y a y UYp s s ZP p y +U p U y« sS + p U& l a +UZ6 « + U Z- y + U«USZ«Zp) + 3 UaUPUyUS+ 3 U«(Zp- Up)(Z7- U7)(ZS- US)

+ Up(Z« - U«)(Zy- U.)( Zs- Us )+ 3 U7(Za- U«)( Zz- Up)

• (ZS- US) + 3 U&(Za - Ua)(Z1- UR )(Z7- Uy)

+ (Za- Ua)(Z1- 4)(Zy- U1)(Z6- US). The last four terms in the above are shown to be of magnitude of order less than n-I , hence we have E (Za Zp Zy Z6 ) - 3 E (Ua Zp Zy ZS + Up Z. Zy ZS + Uy Z. Zp Z6 + U6 Z. Zp Zy )

- 1 E(UaUpZyZs+UaUyZpZ&+U.UsZpZ7+UpU7Z.Z6

E(UaUpUyU6 )+o(3)

+U11 USZaZy+UyUSZaZp)+

3

Applying the lemma 1 in the previous section , it follows that 1 E(UZpZyZs )= ^D«E(ZpZyZs) -7 E((DoZy) ZpZ6 ) -

E((DaZR)ZyZ&)

Nrn

E((DaZ6)Z11Zy)

- 1 Da (2K fl-f6 JA-f•6 + JRs 1 + J7S'p) n

+ Iap+Daµp) E(ZyZ6)+(Ia7+ DaNy) E(ZpZ6) n n

+ (IaS+ 3 Paps) E(Z1Zy) + o (3)

- 1 Da(2KR7S+JRy•S+J116•7+Jysp) n + Iap E(ZyZs)+ Ia7 E(ZpZs)+ IaS E(ZpZy)

(5.1)

115 February, 1978 Third Order Asymptotic Efficiency of Maximum Likelihood Estimator 285

1 + n 1 ya Dalai + 1 pa Dalay + 1 py Daya ) + 0( n) . It also follows that E(° U0

Z8) = I' E(Z7Z5) -

n

E

(ap Z,Za)

+ 1 D 13 E (Z Z8) - n D. E ((D1Zy) Z6) - 1 D. E ( ZZ 4 Z6 ) n

- n 4E((DDZy)Z8) - 1 Dp n E(ZyD.Z8 + n E((DDDDZy)Z5) + n E((DaZy)(DpZB

+ n E((DB Zy) (Da Zs )) + n E' ((Da Dp Zs) Zy -IapE ( Z Z8) E Z 6 + 1 DaDplya n + (lay + n Dalay) (10+ n Dpla6) + (If+ n Dpl1y )(Ias+n Daµs ) + o(1). We shall show that E(VapZ7Z6 ) - E(VapU7U6 ) + 1 E(VapVyB) + E(VapV7)

1 +o(\I-n In order to prove this, it is first noted that E(VapZ,Z6 )a

E (apUyUb) + E(V UZ

)

-E(VapU U8 )+ o( 1 ).

And we have E (Uy

V ap

Z6 ) `

Dy E (

V

ZB ) - E (( Dy

1 - - E(VapDyZ8)

V. # ) Zs

)

116 Kei TAKEUCHI and Masafumi AKAHIRA

286

E

DyE( VapU5 ) -

E((DyVap ))Us)+o(7n=).

On the other hand , it holds that E(UyV.PUs) DyE(V.#Us) E((DyV«#)US)

Nf n-

E(VapVys ) + of- ) .

It follows that 1 E(UyV,,p Zs )= E(UyVpUs) +E(VapVys) + 0 () from which the desired relation is obtained. Thus we get E(UaUp ZyZs IapE(ZyZ6) - I E(VapUyUs n n E(Vap Vys) n - E(Vp Vsy)

+ Iay IRS + Iad Ipy + 1 D. D I'y n o(n). +n(Ia7Dpµs+ Ip'Daµs+ IasDpµy+ Ip'DaWy )+ Substituting (4.1), (4.2), ( 4.3) and (4.4) into the terms E ( Va p Uy U s ), E (Vap Vys ), E(Vap Vsy) andDaDpIys - -Da(Kpys +Jpy-s +J1 ), we obtain after some algebraic manipulations, E( U. UpZyZs) = I° E(ZyZ6) + I°y Ips + as I' +n (IsyDpµs+ IapDpuy+ IpyDaµs+ I

P6

Daµs)

- 1 (Hapys + 2Nap•'YS + N ay'pd + N as•py n + Npy'as

+ Nps'ay + Lag-f-8

+ L"psy + 2Map-ys

+ May-Ps + M as •P7 )

+ 1 E (Kat'y' + J a7.1' + J o`y'.y ) (K OS + J't OS + Jpy, ) 'y

n y.

117 February, 1978 Third Order Asymptotic Efficiency of Maximum Likelihood Estimator

+ 1 (Kass'+ Jas •s' + Jas •s ) (K 07 + n

Jap 'y

+ n

6'

s

287

07 + J;7 ) Js s

(2Kps+ 2Jy6 'yI+ Jyy's+ J7's'Y )+o(n).

Y'

Calculating similarly we have E(UaZZZ7Z6)= Ia1E(Z7Z6) + Ia7E(ZZZ6)

+ Ia8E(ZZZ7)

+ 1(Iy6 Da u + Ips Dalay + Ip' Dalap ) n --(2H ap7s + 2N ap•ys + 21V a7 •06 + 2N as •py + Np7•a6 n + N06'07 + N7s•ap + Lap7• s + L006 •7 + La76•0 + Maa •07 + Ma7 • ps + Map -'Y6 )

+ 1 1 (Kapp' + Jap'p' + Jap''p ) ( 2Kpys+ Jpya + Jy016+ n p, + 1 I n 7,

(Ka77'

+ Ja7i" + Ja7'.7 ) (2Kys+ Jys + Jys+

JS -j) 0' sp) y

+ ( Kass' + Jas's' + Jas'-6 ) (2%s7 + Js' + J sy+ Js-9) n s'

Substituting all those forms together with E(UUpU7Us)= 1 Hap-f6+ Ia0 I76+ Ia7Ips+ IQ s•py n into ( 5.1) we have the fourth cumulants as E(ZZZ7Z5 ) - E(ZaZp)E(Z7Zs )- E(Z.Z7)E(ZZs )- E(Z.Z6 ) •E(ZpZ7) (I«p- E( ZZp)) (lys- E(Z7Zs)) + (lay- E(ZaZ7)) •(116 E(ZpZ5)) + (las- E(Z0Zs))(Ip7- E(ZpZ7)) - 1 (H076 n

+ 2Nap•76 + 2Na7•p6 + 2Na6•p7 + 207•a6 + 2Nps -ay

+ 2Nys•7p + Lap7•6 + Lapses + Lot-f" + Lpys'a )

118 288 Kei TAKEUCHI and Masafumi AKAHIRA

+ 1 E (K«R + n

J«R ^) 2KP716+ J^s+ Js7)

+ 1 E(K«7'7 + n

J«7 '7) (2KRS+ Jy6+j 9)

J7.9 ) + 1 E (Ka"' + Jas-6' + J«s •a) (2K#7 s+ s s ' + JR'7 n a, + JR,Y.,Y' + J17.7 )

(2K«b+ J«•s+ J'«) 7' 7' 7'

+ 1 E (K's-s' + j135.5' + JRs `6 ) n s,

(2K6c'7+ J S7+ J78?)

+ 1 E (K766' n a'

(2Ka9+ Js 9+ J61k) + o(1). n

+ 1 E (K'37.7' n

+ J7S•s' + J7s '6 )

The first three terms in the above are of order smaller than n-1 We denote

0«P-7a - E (K«'''' R'

+ J«1'R' + J«(3'"P ) (2K76+ JY-a+ J1 ')

6 E E E E E E j « «'j'3P' j77'jbd'jtt' (K«,R,t, + + q,t,_R, t «' 13 ' 7 ' b ' t '

• (2K

,S,t,

+ JY,t, a, + Jb,t,,7,

Then the 4th cumulant is given by - 1 H«'37s _ 2 (N«R•76 + N«7''3s + N«"7 + N97•«6 + N'3s •«7 + Nays.«'3 ) n n

_n (L«'37 •s La"--f + L«7R +

+ 1 (p«'3°Y6 + p«7•06 + p«6•R7 + A#7•«6 + Op6.07 + p7s'a'3 ) n (5.2)

Summarizing the computations we have Theorem 5.1. The asymptotic cumulants of - (B« - 6«) (a = 1, • • •, P) up to the order n-1 are given by

E (5(8« - e« )) = µ«+ O (n) Cou(Jn(e«-9«),f(e1-es))= I«P+ n D«µQ+ n Dpµ«

119 February, 1978 Third Order Asymptotic Efficiency of Maximum Likelihood Estimator 289

+ 1Cov(Qa,QR)+o(1) ; 4n

K3(Vn(Ba-0. ),Yn (BR-Op ),Yn(By-By))` :^Qapy+ \/-n - (eR - eR ), \/n ( e.y - ey ) , V n (ed - 08

K4 (V n

o n);

))

where /3aRy and Qapys are given above. It is also shown that Theorem 5. 2. The joint distribution of (6a - Oa ) (a - 1, • • •, P) is equivalent up to the order n-1 to a continuous distribution , whose density is given as follows i

f(yl, ..., yp)

2 n )° [ exp - 2 { «

Iap ya Yp } ]

{ 1+ 1 E E E QapyH«py + 1 E yoHa 6 -in a p y 1/n of

+ 1 E E E 1 , Z E (3apy(3a'R"y,Hapya"R"y, 72n a p y a' p' y'

1 + 24n « Q E 2n + 1

« 0.0,Y6 Hapys

1 Q (laaµp + Daµp

+ 1 E E Cov( Qa, 4n a R

+ Dpµa)HaHR

Qp ) HaHR}+ o(n)

where Hap ... are (mlutivariate ) Hermite polynomials defined by H = E I , yR Hat= HHR - IaR

HaRy HHRH'Y - IaR H - IayHp- I pyHa 7 etc.. It should be remarked that the third and the fourth cumulants are equal for all ER BAN estimators up to the order n-1 .

120 Kei TAKEUCHI and Masafumi AKAHIRA

290

6. Asymptotic higher order efficiency of ML estimators Now we impose some higher order conditions on ER estimators . Either of the three conditions may be taken into consideration. a) higher order asymptotic unbiasedness : µ« = 0 , b) higher order asymptotic coordinate -wise median unbiasedness: limn

n --+ no

Pr {/-n

(0 « -0«)

<0}-

2

I=0

Then ^,t« should be expressed in terms of 3rd cumulants, i.e.

1

n« = 6 R« ««

c) higher order asymptotic mode unbiasedness : the asymptotic density has its mode at the origin again µ« should be expressed in terms of 3rd cumulants. Now we can consider the class S of estimators which are ER BAN estimators satisfying one of the above conditions. It should be remarked that for given g which satisfies the condition

a gI aT;

Ti=gi

= D« S;

we can determine h so that 1 µ« =

where b;j = aT

2

E E bi!ai .i + h(hi.... ,)Um)

a2 ; aTf g I Ti = µ;

satisfies the condition stated above . Therefore for any g there is an ER estimator which belongs to the class S. Now for the estimators in S, the asymptotic distribution up to the order n-1 is determined except for the term 1 4n Coy (Q«,Qp) Therefore we shall minimize the matrix A= {r«p} T«p = Cov(Q«,Qp) = nE(Z«-U«) (Zp - Up) + o(1). (Z« - U«) must satisfy the following conditions : E(UpUy(Z«- U«)) = DDE(Uy(Z«- U«)) - E ((Dp Uy)(Z«- U«))

121 February, 1978

Third Order Asymptotic Efficiency of Maximum Likelihood Estimator 291 - E( U7Dp(Za- U.))

= E( UUVpc,) + o (1) Kapy - Jpy'a + o (1) ; -v/n E(UpVys (Zo, - Us,)) = DjE(Vy6 (Z,, - Ua)) -E((DpVy6)(Z,,,- U«))-E(Vy6D#(Z.,- Ua)) = E(Vy6Vp,) + o (1) . Under these conditions nCov (Z,, - Uc,, Zp - Vp) is minimized (asymptotically ) when it is expressed as n(Z. - U.) - E E X«py Up Uy+E E Z E Xotpy6 Up Vy6+o(1) , ,Ia p y 6 p y

where J^py and Xapy6 are Lagrangean multipliers determined so that the conditions are satisfied. Now instead of determining explicitly X py and Xap-y6 , we shall show that the above equation is actually satisfied when g is equal to the ML estimator. The ML estimator is determined by the equations as• E ae^(T;-pi) = 0, a = 1,..., p.

(6.1)

Since (s, .... s,,,) and (Lzt , ... , i.i m) correspond one to one and the mapping is continuous, 0 is consistent provided that (s1 (01) ... s,,,(01)) * (s1(02)... sm(02)) if 01 002. Moreover if for all 0 and for any to,, (Eta

a8st

, . . . , Eta s0

) * (0, ... 0 ), it is shown that 2

-k Pr { 118 -0 11 > e } = o (n ) foranyk. Consequently we can expand the estimator around the true value of 0. Differentiating (6.1) with respect to Ti, we have

a0a aT;

ash

a2s

a0aa0p(T'-"`')

i a0p

aµj

as;

aq }+ a0p =o, Q=1,,P•

(6.2)

Putting T; = µ; we have

aoa as; -2; 4 aT; + aop

= 0, 0=1,...,p;

aB", = 2; 1113 as; aT; a0p which is to be expected since ML estimator is a BAN estimator . Differentiating (6.2) again with

122 Kei TAKEUCHI and Masafum i AKAHIRA

292

respect to Ti, we have

azea E

aT i aTi { k

+ a(aea) y aTi

a2S

{

- k

ao,aoy

aea

a2 Si

Substituting T,

-

aea

aey

ae

= µ i , aT Das' ' aSk

a 0R

k aoR ao

+ oa a0R y aTi

}

aZSk

(Tk -

µk )

- k

aµk

aeaaoRaoy

a2µk

ask

a2Sk aµk

+ aT i

ao.2 a00

k aTi k aoa a0Raoy

(aey)

111k

aSk

T - k ( k - µk) aoaZSk a aoR

a

}

aoy

a2 Si

Q

°

aOyaOR

a 2sk

aµk

aoa aoR

ao y

a

J- 0 -,y

a2I k

aeaaey =

Jay R + Ka137

it follows that - T, ( azoa ) IaR - E E ( Dasi ) (Dysi ) (JaR•y + Jay•R + 'J y• a + KaR'Y) a aTiaTi a y

2

+ (Dasi)

a2s E (Dsi) aoaaoR - 0 aoaaoR +

Hence we have z

01 D i ) (Das +JRb•y(JRy.s a - E E E IaR +Jys• R +K Ry s ) ( ys i) aTiaTi R y a 2

aR

+ IaR ( (Dysi) + Q I aopaoy ) P 'y

a2S

aopaoy

(Dysi)

We have x E IaR

R

azs' ( aoRaoy

) T; =

E Iyy-

Vay'

y + Jay R ) + E Iyy' ( K 0'0 y + JaR

(a00,)Ti

123 February, 1978 Third Order Asymptotic Efficiency of Maximum Likelihood Estimator 293

= s 1y8 V^b +^

s

I°« (Ka ',y +Jo'fl.y +J«'.y.p) UU .

Consequently,

EE ( i

3 z 00,

i

)

Ti T; Q. = E E E I°" (KQ'#y + Jpy•a') Up Uy aTfaT. y + 2EEIy6 VasUy 6y

which has the form required for minimizing solution of A. Thus if A* denotes the variance -covariance matrix of Q0, corresponding to the (extended) ML estimator and A be any other estimator in the set S , we have A - A* 2 0 ( non-negative definite). From this the following theorem is established Theorem 6 .1. Let 6* be the extended ML estimator in Sand 0 be any other estimator in Sand C by ane convex set in R° containing the origin . Then it holds that lim n - -

n[Pr{. (0* -0)e C } - Pr

{,/n (e -0)e C} ]

Z

0

We also have Theorem 6 .2. Suppose that L is bounded, negatively unimodal function around the origin, i.e., for any c the set { u : L (u) < c ) C R° is convex and contains the origin . Let 0* be the extended ML estimator in S and 0 be any other estimator in S. Then lim

n -^ m

n[E{L ('(0*-0))) -E{ L(/(0 -0)))] < 0

References

[1] Ghosh , J.K. and Subramanyam , K., "Second order efficiency of maximum likelihood estimators ," Sankhya Ser. A, 36, ( 1974). [21 Pfanzagl , J. and Wefelmeyer, W., "A third order optimum property of the maximum likelihood estimator ," Preprints in Statistics 26, University of Cologne ( 1977). [3] Takeuchi , K. and Akahira , M., "Third order asymptotic efficiency of maximum likelihood estimator in general case ," ( to appear).

124

Rep. Univ. Electro-Comm. 29-1, (Sci. & Tech. Sect.), pp. 37-45 August, 1978

Asymptotic Optimality of the Generalized Bayes Estimator* Kei TAKEUCHI** and Masafumi AKAHIRA***

Abstract For all symmetric loss functions, the generalized Bayes estimator is second order asymptotically efficient in the class A2 of the all second order asymptotically median unbiased (AMU) estimators and third order asymptotically efficient in the restricted class D of estimators. When the loss function is not symmetric, the generalized Bayes estimator is second order asymptotically efficient in the class A2 but not third order asymptotically efficient in the restricted wider class C than the class D.

1. Introduction The expansion of a generalized Bayes estimator with respect to a loss function of the type L(0)= I01 a(a>1 ) is obtained by Gusev ([4]). His result can be extended to all symmetric loss functions .

Strasser ([6]) also obtained asymptotic expansions of the distribution of the

generalized Bayes estimator.

It is shown in this paper that for all symmetric loss function the generalized Bayes estimator B„ is asymptotically expanded as (0)} +Z,(0)Z2 (0)-3J(0)+K(0)Z1(0)2 +op V -n (6n- 0)=Z1(0)+ 1 ao+13o {3J(0)+K I(0) V n 61( 0) 1(0)2 2I(0)3

where the symbols of the right-hand side are defined in the contents .

1 V n )'

And the asymptotic

distributions of the estimators are the same up to the order n-1 except for constant location shift.

Therefore if it is properly adjusted to be second order AMU and third order AMU,

it is second order asymptotically efficient in the class A2 of the all second order AMU estinators and also third order asymptotically efficient among the estimators belonging to the class D for c „=Vn, where D is the class whose element 6. is third order AMU and asymptotically expanded as cn(6n

-0)=

1(0) +c

n- 'Q(0)

and Q(0)=O,(1), E[Z,(0)Q (0)']=o(1) (i=1 , 2),

+op(c n -I)

where E denotes asymptotic expectation

and the distribution of cn(6n-0) admits Edgeworth expansion ([9], [10]). But when the loss function is not symmetric , the generalized Bayes estimator 0. has the asymptotic form :

V n (6n-0)=Z1(0)+ 1 [ao+Ro{3J(0)+K(0)} +Z,(0)Z2(0)-3J(0)+K(0)Z1(0)z] 1(0) V n 61(0) 1(0)2 2I(0)3 * Received on June 9, 1978 ** University of Tokyo *** Statistical Laboratory, University of Electro-Communications

125

Kei TAKEUCHI and Masafumi AKAHIRA

38

+I/n IZ2(0)-3J( BI(BK(BZ1(B)}+onl^n l and the existence of the last term excludes it from the class D, and it belongs to only the wider class C whose element satisfies all but E(Zt(8)Q( 6)') =o(1) (i=1,2) in the conditions of the class D ([9], [10]), and since the last term depends on the specific choice of the loss function , no uniform third order asymptotic efficiency property holds. Note that A2 D A3 D C D D.

2. Results Let (-', B) be a sample space. We consider a family of probability measures on B, 2= (Pe : Os9}, where the index set 9 is called a parameter space. We assume that 9 is an open set in a Euclidean 1-space R1. Consider n-fold direct products ('""), B'')) of (1, B) and corresponding product measures p6(n) of P6. An estimator of 0 is defined to be a sequence {On} of Bl")-measurable functions On on Xc") into 9 (n = 1, 2, •••).

For simplicity we

denote an estimator as Bn instead of {On}. For increasing sequence of positive numbers {cn} (Cn tending to infinity) an estimator On is called consistent with order 1c.1 (or 1C.1 -consistent for short) if for every c>0 and every 9s9 there exist a sufficiently small positive number 5 and a sufficiently large positive number L satisfying the following :

Pei "I {cn I On-0 I >L} <s lim sup e:le -al<8

([1]).

n-.oo

For each k = 1, 2, • • •, a {cn} -consistent estimator On is k-th order asymptotically median unbiased (or k-th order AMU) estimator if for any 9 E 9, there exists a positive number 6 such that lim sup C.-- 1 P'(-)[O.<01- 1 0 ; n-- 6 : 16-91<8 2 1= Jim sup Cnk-1

n-.oo 8:I6-91<8

P6<->{9.>B} - 2 2 1=0.

We have defined a k-th order AMU estimator to be k-th order asymptotically efficient in the class Ai, of the all k-th order AMU estimators if the k-th order asymptotic distribution of it attains uniformly the bound of the k-th order asymptotic distributions of the estimators in the class Ak ([2], [7]). If the class Ak is changed for the class C and the class D in the definition in the case when k=3, third order asymptotically efficient estmators in the class C and in the class D are defined, respectively ([9], [10]). It is genearally shown by Pfanzagl and Wefelmeyer ([5]) and Akahira and Takeuchi ([3], [8], [9], [10]) that there exist second order asymptotically efficient estimators in the class A2 but not third order asymptotically efficient estimators in the class A3. But it was also shown in [9] and [10] that if we restrict the class of estimators, appropriately, we have higher order asymptotically efficient estimators among the restricted class of estimators, and that the maximum likelihood estimator belongs to the class of higher order asymptotically efficient estimators.

We assume that for each 0 E 9 Pe is absolutely continuous with respect to a-finite measure n

Ec.

We denote a density dPe/d p by f (x, 0).

Then the joint density is given by f] f(x;, 0). i=1

n. Let 9=R1. Let In the subsequent discussion we shall deal with the case when c.=1 1Ln(u) be a bounded non-negative and monotone increasing function of Jul and z(0) be a

126

August, 1978 Asymptotic Optimality of the Generalized Bayes Estimator

39

non-negative function. Define a posterior density p.(8 I xn) and a posterior risk r(d8) by ill f (Xi, 0)1i(B)I Jifl

pn(B xn ) =

f (xi, 8)I ^(B)de^-1 ,

and rn(d I -t-) =L Ln(d-8)pn(O I x n )dO e respectively, where xn=(x,, X2, , xn).

Now suppose that lim Ln(uli/n)=L*(u) for all real n--

number u. We define

rn *(d I xn) =J L *(V n (d - B)) pn(B I xn)d8. L An estimator e n is called a generalized Bayes estimator with respect to a loss

function L*

and a prior density 7C if rn*(On Ixn)=infra*(d Ixn). deg

Then t n = V n (On - 0) may be also called a generalized Bayes estimator w. r. t. L* and 7C. Since lim inf n-.oo

Ln(d-B)p(B)dO-infS L*(Vn(d-8))p(8)d8 =0

de

9

deg

9

uniformly in every posterior density p(C), it follows that for a generalized Bayes estimator On lim n-.oo

inf rn(d I xn)-rn*(On I xn) deg

= 0.

Suppose that Xi, X2, • • •, X ., • • • is a sequence of i. i. d. random variables with a density f (x, 0) satisfying (i )-(iv).

(i) {x : f (x, 0)>0} does not depend on 0 ; (ii) For almost all x[,u], f (x, B) is three times continuously differentiable in 0; (iii) For each 0 E 6

2 02 0<1(0) =Ee logf (x, 0) } ]= -Ee [ ^ logf (x, 0)]
J(8)=Ee[ {a jlogf(X, 0)}{aelogf(X, B)}] and K(0) =EeI {aB log f (X, 0)}3] and the following holds : Ee[a03logf(X, 0)]=-3J(0)-K(8). By the following way we have shown in [7 ] that a maximum likelihood estimator (MLE) is second order asymptotically efficient. Let BML be an MLE. By Taylor expansion we have

0= E a logf (Xi, tML)

-1 ae

2 =

E

a log(Xi , i =tae

- 0) 0) + E { 06 a j logf (X i , 0)} (BML i =1

127

40 Kei TAKEUCHI and Masafumi AKAHIRA

R

' 82 logf(Xr,

0

*)}(8ML-0)2,

where Putting T„=V n (CML-8) we obtain

6

_l/n ;E aelogf(Xr,

0)+ n

^tE

aB21ogf(Xr, B)}Tn

1 Jn a 8 ae^ logf(X+, e * ) Tn 2 . +2n1^ n ll ^ Set Z1(8) =L

7

rE aBlogf(Xi,0);

Z2(0) =71, `E{ 2logf(Xi,0) +I(0) }; W(0) = 1

a 2 2 logf(X r, 8), n i=00'

I(0) =E9 alogf ( X,0)}2]

[

where

Hence the following

Then it follows that W(0) converges in probability to -3J(0)-K(0). theorem holds : Theorem 1 . ([4]). Under the conditions (i )-(iv) / V n (&ML

up to order n

-' 2

-8)= I(0))

+

2 V B) I(0))

3 321 n I(0))

Z1(0)2+0'

(1^

as n-,oo.

Put (/ML*=(ML+ K(BML) 6nI (8ML)2

Then 9ML* is second order AMU. From Theorem 1 we have established the following : Theorem 2. ([4]). Under the conditions (i)-(iv), OML* is second order asymptotically efficient in the class A2. It will be shown that the generalized Bayes estimator w. r. t. a loss function and a prior density is second order asymptotically efficient in the class A2. Let Bo be a true parameter of 0(E19). Further we assume the following : (v) i(0) is twice differentiable in 8. Then we have

pn(8 I X n)/ p n( Oo 1 Xn) =exp[log { pn(0 jX n) / pn (8o I Xn)} ] = exp E log { f (X i, 0)l f (X i, Bo) } + log {wc (0)l lc (Oo) } ]

[ =exp E logf(Xi, 0)-E logf(Xi, 0o) +logir(0)- 1og X (Oo)] [ j_1

0-G o )+ 1 { E a2 logf( Xr, Bo)}(0 -Oo)2 2 i1002

=expl l E logf(X i, 8o) }( l j_1 a0

128 August , 1978 Asymptotic Optimality of the Generalized Bayes Estimator

41

+ s 1_^ ae3log(x;, 8*)}(e-0o) 3+ ^(B) (0-e0-)+o(11 n )l =explv n {Z1(0o)}(0-Oo )+ 2 {V n Z2( Oo)-nI (Oo)}(0-Oo)3 + 6 W(0*)(0-00- )3+ 7r(e) (0-00-

)+0 V -n

where R=(XI, X2, • , Xn) and 10 *-Oo1510-00-1.

Since W(0) converges in probability to -3J(0)-K(0), it follows that P n(0 I k-)/p-(001 gn) r =exP {Zl (0o)}Vn (0-00)+ 1 v -I(6o)}{Vn ( 0-00)}2

61^ n Putting

{3J(0o)+K(0o)} {lam n (0-Bo)}3+V n(B(B0 ){1^n (0-00-)}+

t =11 n (0-Oo)

(1)

or(v nn lJ

we obtain

I .n) / p (0 1 9 n) 0

pn(0

=expl ZI(Bo)t+ 1 ^Z2(Bo)-I(Bo)^t2-3J(0o)+K(0o) t3+_ - (00) t+0-a L 2 n l/ ✓ n (Bo)ic V 6 Vn \ -n 1 /J =ex P[L

/ 1 11 Zl(B!2'2 Zl( 0o)2 -I(00-) j 7t' (0o) Z2(0o) 3J(Bo)+K(Bo) 3 2 Itn t + on1 ✓ I(Bo) J +2I(0) +1/n x(0o)t^ 2V n - t2+ 61^ nIJ

Z3 ( O3)3)[ pI 2 ( 0o) f t-Zi ( Oo)l2l ={exp I(0o) I 2I(0o) J LL ex

tc' (Oo) t+Z3(O0)t2-3J (0o)+K(0o ) t3+0-n 1 p 11n r(0o) 21/n 6 ✓ n (✓ n

={ex Z3(Bo)2J[ex

p 2I( 00-)

ll •1

I(8°)j t-Z10)12]

p 2

l

I(0)

Z2(00) _3J(0o)+ K(0o) I 1 1l + ✓ n tc( Oo)t+21/ n t2 61^n t3 + o'1 ✓ nJJ

=qn(t, 00- I fin) (say). Let

1 n = ✓ n (9n - Bo ).

Then the posterior risk is given by

( 2) rn* (en I xn) _ n pn( 0o I ±n) L* ( ln-t)gn (t, 0012n)dt. Further we assume the following :

(vi)

L*(u ) is a convex function ;

(vii)

L*(- u)gn(u + t, Go I zn ) du is partially differentiable with respect to t under the

integral sign. By (2) and the assumption (vi) it is shown that the generalized Bayes estimator i n w. r. t. L*(•) and 7r(•) is given by a solution u of the equation

(d/du)E L*(u-t)gn ( t, Oo I xn)dt=0. Since by (vii)

d °° du _-

L*(u-t)gn (t, Bo I xn)dt

129 Kei TAKEUCHI and Masafumi AKAHIRA

42

= d J L*(-t)gn(u+t, 0o l xn)dt du _.

_d °° dt L*(-u)gn( t+u, Bo l xn)du T I L*(-u)dq.n(t+u,BoIxn)}du, the generalized Bayes estimator (3)L*(- u)}dgn(tn +u, Since

ln

is obtained by a solution of the equation

0oIxn) }du=0.

t=1/ n (0-0o) and 1n=V n (Bn-0o), the solution On of (3) may be called to be the

generalized Bayes estimator. From (1) and (3) we have

(°° 0-

L*(-u)e

r.-Z,(eo l' -1(B°) +u r Zl(00) z r(Oo)} (-I(9o) jln +u- I Bo ll L ()

)2-3J(0o)+K(0o) •j1+ ^^(00) (1n+u)+Z2 (0o)(tn+u ll V n it(0o) 21/ n 6V n • (tn+u)3+oD

( ✓ n)}+ ✓ n(B(9o

)+ 11-n n+u)

n (B0)(tn+u )2+op(nn 1 )]du 3J(2VPutting

u=tn- j1(0o),

we obtain

{1

_L( )(u+&)' 1 ^c^(0o) Zl(Bo) (4) 0= L*(-u)e 2 I(0o)( u +u) +v n 7c (9o) (u+u + I(90) ) L l

[-

) 2 3J(Bo)+K(9o) u+u+Z1(0o) 3 +Zz(9o) (u+u+Z(9 2✓ n I(9o) 6 ✓ n ( I(00) ) +oy + tc'(0o) +Z2(0o) ( u + u+Z1(9o) 1^) } 1V n tc(0o) 11n I (Bo)) 2 • (u+u+ I(00))) +op()n)]du

3J(Bo)+K(9o) 2V n

_ L*(-u)e 2 { 1-I(Oo)u12v n

V-n [_I(e_1(oo)u-I(^(Bo

) (u+u)}u+u

+ 11(00))

1(9o)Z2( 0o) u+u u2+u2 +Z 1(00)2+2uu +2Zi(0o)u+2Zi(00)u 21, n ( ) I( 0o)2 I ( 00) I(00)

+I(9o) {3J6(V)n K(Bo)} ( u+u){u3+u3+

I!Be^33

+3u2u

1 +3u2Z1(B0) +3u2Z1(0o) +3u2u +3z1(00)2u+3Z1(0o)2u (00)2

1(90) I (90) 1

1(00)

Zi(9o)1 7c'(9o) Z2(00)Z2(00) Zl(00)Z2(00) +6uu I(00) J+ ✓ n r(9o)+ V n u+ V u+ I(0o)1,/ n 3J(0o)+K(9o)

Zio)2 221(00) 2Z1(00)

21^n u2+u2+ 1(00)2 +2uu+ I(Bo) u+ I(0) u}

130 August, 1978 Asymptotic Optimality of the Generalized Bayes Estimator 43

+ool V n )Idu. (I) Symmetric loss function. We assume the following :

L*(u) is a symmetric loss function about zero.

(viii) We define

-) U.

(5) co*= L*(u)uke 2 du.

(k=2j; j=0, 1, 2, ...).

I(Oo) uS

L*(u)u27+1e 2 du=0

Note that

(j=0, 1, 2, •••).

It follows from (4), (5) and the assumption (ix) that 0=-ISoou

2Z1(Bo) I(Bo)x' (6o) I(60) Z2(6o) + I(Bo) cpz+cpzu -V n it(Bo) (P2 2V n 2c92u

_____ 1 I(Bo){3J( ______6o)} ___2 Z1(6o2 60) +on(1) J + 1(00) 2 cpou + 6V n cP4+ 3cp2__ I(00) +6cpzu 1( Z_(6o) Z2(6o) Z1(eo)Z2(60) + Vn 1Pou+ I(6o)V n cQo + 1/ n X( 0o) Po 2Z1(00) l 1 l 3J(6o)+K(Oo) ( Z1( Bo)2 I(Bo) cpou }+I(Bo)2cpzu+ov V n 1. 21^ n i cpz+ I(6o)2 cpo+ Since I(6o){coo-I(6o)cpz}u =

3J(6o)+ K(Bo){I 7r/(00) {coo-I( (Bo)cp4-3cpz} Bo)cot }+ 6V ^C(6 o) 1^ n + Z1(00)Z2(eo)

1(6)1/

{ po-I(Bo)c02}

nn

I 36o)Z1(6o)2{cpo-I(6o)c02}+ov ( 2I (y n/ 6o )2 + n and

coo-I(Bo)cpz#0,

it follows that _ n'(6o) I(6o)co4-3coz 3J(6o)+K(6o) Z1 (0o)Z2(0o) (6) u 1(6o )ir(63)V n + coo-I( Bo)cpz 61(Oo )V n + 1(Bo)2V n 32100 V n6o)

Z1

n ). (6o)2+0'(1/'

Hence we have (7) in=Zl(60)+Zl,

I(Bo)

where u is given by (6). Since in=V n (On-Bo), we modify Bn to be second order AMU and denote it by On*. From Theorem 1, (6) and (7) it follows that the MLE BML* is asymptotically equivalent to the generalized Bayes estimator 0 . * up to order n-112. By Theorem 2 it is seen that On* is second order asymptotically efficient in the class A2. It follows from (6) and (7) that the generalized Bayes estimator On* belongs to the class D. Then the asymptotic distribution of B„* is equivalent to that of the MLE BML* up to order n-1. Since OML* is third order asymptotically efficient in the class D([9], [10]), 9n* is also so.

131

44 Kei TAKEUCHI and Masafumi AKAHIRA

Hence we have established the following : Theorem 3. Under the conditions (i )-(viii), the generalized Bayes estimator 6n* is second order asymptotically efficient in the class A2 and also third order asymptotically efficient in the class D. Remark : In the location parameter case,

L*(u)=u2, tr(9)=1,

the generalized Bayes

estimator is reduced to the Pitman estimator. It follows by Theorem 3 that the Pitman estimator is second order asymptotically efficient in the class A2 if it is properly adjusted to be second order asymptotically median unbiased.

( II) Asymmetric loss function. In the case when the loss function L*(•) is asymmetric we shall obtain a similar result as the symmetric case . Further we assume the following : (ix)

L*(u) is an asymmetric loss function ;

(x) There exists a c satisfying I(0,) ..

^ du=0. L*(c-u)ue 2 Then we define teo , u L*(c-u)u"e 2 du Put

u=

t„- ---

(k=0, 1, 2, •••).

and v=u+c.

In a similar way as the symmetric loss function we have from

(4) and the assumptions (ix)

and (x)

(8) v_

(eo) _+ I(0o)04-3q '2 .3J(Oo)+K(9o)+Z1(0o) Z2(Bo) I(9o)2V n 0o-I(0o)02 6I(6o)l/ n I(6o)_c (6o)V n _3J(6o)+K(9o)Z1(eo)2- 03 - Z2(90) 2I(00 ) 3 ✓ n 21011 (0.)021 1/ n _3J(9o)+K(Oo) Z1(eo)}+ov ( 1 nn ) I(eo)

Hence we have (9) tn=ZI(Bo)_c+v, I(6o) where v is given by (8). Since t„= ✓ n (6„-8o), we modify B„ to be second order AMU and denote it by B„*. Since there exists the fifth term of the right-hand side of (8), the generalized Bayes estimator B„* does not belong to D but to C ([9], [10]). However it follows from Theorem 1, (8) and (9) that the asymptotic distribution of B„* is equivalent to that of the MLE OML up to order n-'12 ([9], [10]). Since the fifth term of the right-hand side of (8) depends on the specific choice of the loss function, the modified generalized Bayes estimator B**(EA3) is not third order asymptotically efficient estimator in the class C ([9], [10]). Hence we have established the following : Theorem 4 . Under the conditions (i )(vii), (ix) and (x), the generalized Bayes estimator is second order asymptotically efficient in the class A2 but B„**(EA3) is not third order asymptotically efficient in the class C.

132 August, 1978 Asymptotic Optimality of the Generalized Bayes Estimator

45

References [ 1 ] Akahira, M.: "Asymptotic theory for estimation of location in non-regular cases, I : Order of convergence of consistent estimators , Rep. Stat. Appl. Res., JUSE , 22, 8-26 (1975). [ 2 ] Akahira, M. and Takeuchi, K.: "On the second order asymptotic efficiency of estimators in multiparameter cases ," Rep. Univ. Electro-Comm., 26, 261 -269 (1976).

[ 3 ] Akahira, M. and Takeuchi, K.: " Discretized likelihood methods-Asymptotic properties of discretized likelihood estimators (DLE's)," (to appear). [ 4 ] Gusev, S. I.: "Asymptotic expansions associated with some statistical estimators in the smooth case I. Expansions of random variables," Theory Prob. Applications, 20, 470-498 (1975).

[ 5 ] Pfanzagl , J. and Wefelmeyer, W.: "A third order optimum property of maximum likelihood estimator ," J. Multivariate Anal. 8, 1-29 (1978). [ 6 ] Strasser, H.: "Asymptotic expansions for Bayes procedures," In Recent Development in Statistics (J. R. Barra et al., Ed.) North-Holland 9-35, (1977). [ 7 ] Takeuchi, K. and Akahira, M.: " On the second order asymptotic efficiencies

of estimators,"

Proc. of the Third Japan-USSR Symp. on Prob. Theory, Lecture Notes in Math. 550, SpringerVerlag, 604-638 (1976). [ 8 ] Takeuchi, K. and Akahira, M.: " Third order asymptotic efficiency of maximum likelihood estimator for multiparameter exponential case," Rep. Univ. Electro-Comm. 28, 271-293 (1978). [ 9 ] Takeuchi, K. and Akahira, M.: " On the asymptotic efficiency of estimators ," (in Japanese), A report of the Symposium on Some Problems of Asymptotic Theory, Annual Meeting of the Mathematical Society of Japan, 1-24 (1978). [10] Takeuchi, K. and Akahira, M.: " Third order asymptotic efficiency of maximum likelihood estimator in general case," ( to appear).

133

Rep. Stat. Appl. Res., JUSE Vol. 26, No. 3, September, 1979

A Section

On the Second Order Asymptotic Efficiency of Unbiased Confidence Intervals MASAFUMI AKAHIRA* and KEI TAKEUCHI**

1. Introduction It has been known in [4] that the confidence interval based on maximum likelihood (or )}, On + [ua,2/-✓I(9n)}] is asympasymptotically efficient) estimator 6. given as [8n - [u12/'/ totically unbiased, and asymptotically efficient. The purpose of this paper is to discuss the problem of the second order asymptotic efficiency of confidence intervals, that is, to consider the asymptotic power of the confidence interval up to the order n- 1/2 in the neighborhood of the true value of the parameter. It will be shown that asymptotically unbiased, and second order asymptotically (most powerful) confidence interval at 0' = 9 + to 1/2 depends on the specified value of t, thus proving that there exists no uniformly second order asymptotically most powerful unbiased confidence interval. The situation is in contrast to the case of the point estimation where maximum likelihood (or usually BAN) estimators are uniformly second order asymptotically efficient ([5], [7]), and is related to the curvature of the parameter space as was discussed by Efron [3]. 2. Notations and Definitions Let X be an abstract sample space whose generic point is denoted by x, (B a o-field of subsets of 9C, and let 0 be a parameter space, which is assumed to be an open set in a Euclidean 1-space R1. We shall denote by (9C(n), (B(n)) the n-fold direct products of (9C, (B). For each n = 1, 2.... , the points of 9C (n) will be denoted by 5c. = (x1, .... xn) and the corresponding random variables by In = (XI, ... , Xn). We consider a sequence of classes of probability measures [P.,9: 0 e 0} (n = 1, 2, ...) each defined on (9C(n), G In)) such that for each n = 1, 2, . . . and each 0 E O the following holds: Pn o(B(n)) = P„+l.e(B(n) X 1t)

for all B( n) (=- (B(n). Let cn be the maximum order of convergence of consistent estimators ([1]). A subinterval of O, [Q (X3), 9n(Xn)] whose limit depends on the observed random variables Xn (properly measurable statistics) is called consistent of order _cn if for every t(# 0) lim lim Pn, o[9 + tcn I E [en(Xn). 9n(Xn)]) = 1. It I-- n- -

A consistent interval [9Xn), 9n(Xn)] is called an a-asymptotically unbiased confidence (oc-AUC) interval, 0 < a < 1, if for every E 0 and every t(# 0)

a

lim Pn, 9 [ 0

+ tCn 1 E

[9 (XZ), 9n(Xn)])

< 1 -

a,

a-

and for every 0 e 0 there exists a positive number 6 such that Received March, 1979. * Statistical Laboratory, University of Electro-Communications, Chofu, Tokyo 182, Japan ** Faculty of Economics, University of Tokyo, Hong8, Bunkyo-ku, Tokyo 113, Japan

99/29

134 M. AKAHIRA AND K. TAKEUCHI

30/100

l lm n-+oo

i nf e: 16-.'1 <6

Pn, e[ 9

E

L9n( Xn), 9 (1 )]) > 1 -

a.

Further an a-AUC interval [9n(X,), 9(Xn)] is called to be second order a-AUC if for every 0 e 0 and every t(:P& 0) 1 7 rim Cn[Pn, 6[9 + tcn 1 E Len(in), O (X,)]} - (1 - a)] ^ 0, n-,oo

and for every 0 E 0 there exists a positive number 6 such that lira inf Cn[Pn,g[9 E [en (Xn), O,,(1 ,)]) - ( 1 - a)] > 0. n-+oo

An a -AUC interval interval te,,(2,),

e: 1 a -s I <8

[Q*

(Z), Bn (Xn)] is called asymptotically efficient if for every a-AUC every 9 E O and every t(# 0)

lim [Pn, O(O + tc; n^oo

1

E Len (Xn), en (I,)]) - Pn,e[9 + tc; 1

E Len(in), en(X,)}1] < 0

A second order a-AUC interval [9,*(Xn), 0* (X1)] is called second order asymptotically efficient if for every second order a-AUC interval [On(XZ), 9n(Xn)], every 0 E O and every t (# 0) 77 lim Cn[Pn,6[0 + tc; 1 e [en ( XZ), On ( Xn)]} - Pn,e[9 + tc;1 a Len(in), 9n(Xn)])] < 0. X-Note that our definition of the asymptotically efficient confidence interval is different from that of Wald [10] (asymptotically shortest confidence interval in his terminology) in that a) he considered only exact confidence intervals, and b) he defined efficiency in terms of the supremum of the difference of powers of two intervals over all pairs of parameter values of 0, 0' which implies uniformity of convergence of the power function, which in turn implies the boundedness from below of the Fisher information; and both aspects limit the applicability of the theory. 3. Second Order Asymptotic Efficiency of Unbiased Confidence Intervals. We assume that for each 0 e 0 Pe is absolutely continuous with respect to a a-finite measure n

We denote a density dPe/d,u by f (x, 0). Then the joint density is given by n f (X,, 0). In the t= 1

subsequent discussion we shall deal with the case when cn =1/ n . Suppose that X1, X2, . . . , Xn, ... is a sequence of independently and identically distributed (i.i.d.) random variables with the density f(x, 0) satisfying (i)-'(iv).

(i) [x: f(x, 0) > 01 does not depend on 0; (ii) For almost all x[,u], f(x, 0) is three times continuously differentiable in 0; (iii) For each 0 e 0 0 < 1(0) = EaI / log.f(X, 9)}2] = -EBC2 logf(X, 0)] < co; (iv) There exist J

J(0) = Ee[Jd and

EB

0)}lag log fm 0)}]

r K (9)

and the following holds:

ff2 log f(X,

[d03

e[{a0 logf(X, 0)

=E

log f (X,

}3

]

9)] _ -3J(0) - K(0).

We put Z1(9) - nn M ae logf (X:, O);

Z2(0) = nn

l BZ logf(X1,

Rep. Stat. Appl. Res., JUSE. Vol. 26, No. 3, September, 1979

0) + 1(9);};

135 ON THE SECOND ORDER ASYMPTOTIC EFFICIENCY

W (0)

101/31

33 =

n r^'tl

log f (Xn 0).

Let 00 be arbitrary but fixed in 0. Consider the problem of testing hypothesis H: 0 = 0o against alternative K: 0 = 00 + to 1/2 (t # 0). By the fundamental lemma of Neyman and Pearson it follows that the most powerful locally unbiased test function has a rejection region of the following type :

II f( X,, 00 + to -1/2) =1 „ > A o + A ltn - 1/ 2

r

f(X, 00) r=1

a0 log f(X1, 0), (3.1)

where Ao and Al are certain real numbers , and if it is actually (grobally) unbiased it is the most powerful unbiased test function. By Taylor expansion we obtain

n f(X,, 0 + to -1/2)l H f( X:, 00)

t=

=

1

exp

(=1

{logf(Xi, 00 + to -1/2) -logf( XX, B0)}] CZ i- 1

= exp

^^ aB log Ax" 0o) + Zn t3

Yep log f(Xi, 00)

81

+ 6n.1 n s i x03 log

f(Xr, 0*)},

= exp [tzl (00) + t n [Z2(00) - ✓ n I(00)) + t n W (9*)I = exp {- 2 I(eo) + tZ1 (00) + 5.Z2(0) + Z;7 n_ W(0*)}, where

10*

-

01

< tn-1 /2. Since

W(0) converges in probability to -3J(0) - K(0), it follows that

II f( X,, 00 + tn -1 /2)/III f( Xi, r= 1

00)

r =1

(3.2)

=exP [- 2 I(eo) + tZ1 (00) + 5=Z2(Bo) - 65 n [3J(0o) + K(00)}/+ or\

✓

= e (t!/2)I(9o ). erz,(9o ) I 1 + t2 Z2 (0o) - t3 [3J(0o) + K(00)) + 0P \N ^J•

L 2V/n 6.,/n n First we consider the problem up to constant order. It follows by (3.1 ) and (3 .2) that there exists the solutions Z, and Zl (Z1 < Z1 ) of the following equation of Z1(00): e (t=/2u(00)etz,( e,) = A0 + A1tZ1(00). (3.3) Then it follows that e(t'/2) I(eo).etz 2(9o) >

Ao

+

A1tZ ,(00)

if and only if Z,(00) < Z ,, Z 1( 00) > Z 1, It can be easily seen that in order to make the oc-A UC interval we have to take Z1 = -u.12-✓I (02) and Z1 = ua/2-/I(0o) (See [4]), where ua/2 denotes upper 100x/2-percent point of the standard normal distribution. In order to determine Ao and Al we substitute Z, = -u,,,/2'✓I(Oo) and Z1 = ua/2,%/I(0o) in the equations

e-(0/2) I(eo).etzt =

Ao

+

A1tZ1.

e-a=/2),(eo).etz, = Ao + A1t21. Then we have e-W /2)I(9o).e- t„a1, ✓I(eo ) = e- (t /2)I(eo).etu-"'1 Leo ) =

AO + A 1tud /2N I(0o). Al + Altu. 12'✓I (00).

Hence we obtain A0 = e-(t=/2)ne,) cosh (tum/24/1(00)); Rep. Stat. Appl. Res., JUSE. Vol. 26, No. 3, September, 1979

136 M. AKAHIRA AND K . TAKEUCHI

32/102

1 e- (t=1 2)1(8=) Binh (tua /z✓I (6o)) •

21 =

ua/y./ I(Oo)

If there exist B„*(X„) and 6„*(X„) such that Z1(en (Xn)) = u.12,,/I(en X,,)) Z1(6n (Xn)) = -Ua/2_ ' / I (6a (X n)) ,

then the o6-AUC interval [6„*(2,), 6n (X,)] is asymptotically efficient . Hence we have established the following: Theorem 1. Under the conditions (i)-(iii), the tx-AUC interval is asymptotically efficient. We can actually construct the asymptotically efficient oc-AUC interval using the maximum likelihood estimator (MLE) 6ML. We put

en (X

n) = 6ML + n 7-

6n (XR)

= 6ML + -)^n

We have Z1(en)

=

Zl

6ML

7n

V

Y r1

1_

Z1 (6 ML)

'+'Oy(;/ n

= ,V / n Z2(6ML )

_ -

-

Y^I (6ML) + 0p

I

+

o

((

1

l

1 l \^/nl

Since Z1(en ) = u112/I(0'),

it follows that -Y/I(6ML) + 0p( 1

^n

Ua/2'`' I (6 ML) + 0, ( / n ) •

Hence we have

Y•

= -

ua/2 / - I(6ML)

By a way similar to the above we obtain U.//2 _ / I (V ML)

Hence we have established the following: Theorem 2. Under the conditions ( i) - (iii), the x -A UC interval 6ML - Ua/2 , OIL ML +

✓I(6ML)n

_ a/2 UU

✓I(6ML)ni is asymptotically efficient. Next we consider the case up to the order of n-1 /2, i.e., second order case ([6]). We put ZI(60) = ZI

+ n

in the following equation:

e- ( 9 1 / 2 ) 1 ( 0 9 ) ,erz,(e,> rl +

22^

Z2(60) -

6^

[3J(6o) +

We have [ Rep. Stat. App!. Res., JOSE. Vol. 26, No. 3, September, 1979

K(6o) )]

J

=

2 0 + 11tZ1(6o) (3.4)

137 ON THE SECOND ORDER ASYMPTOTIC EFFICIENCY

10 +Alt(2,+Ann1 = e a=iz>I,e :tz, +toi^n»

{1

+ -Z2 - 3J + K)}

n

Al {1 + Z2 -(3J+ K){. n n //

l

= e'(t

=/2) I,erz ,

(1 +

n

It follows by (3.3) and (3.4) that q A e-rz3+(t =/2)I =1+ t = + 1 + Alt

103/33

n

2

t

Z2 -

3

(3J + K).

t

1/n ,/n 21/n 67n

Hence we obtain 2

Z2 - (3J + K) .

t(1 - A l e-92, +0 712)I) 12

(3.5)

Putting Z1(90) = Z 1 + --AL in the equation (3.4), we similarly have n

2

3(

6

)

(3.6) Z2 3J + K }. t(1 - lel tz, +(t=i2>I) { 2 The rejection region like the type of (1) is given by _AL nn < Z1, Z1(00) - ^n > Z), (3.7) Z1(00) - _ where A and A' are given by (3.5) and (3.6), respectively. We have

Z1(0o) Zl(0o)

= Zl(0o) + .^ nn Z2(0o) - '

- nn - n

= Z1(00) + n Z2(00)

n - -1-

where Cl =

q

t

t2(3J + K)

2(1 - hle-tzi+(t /2)I)' c2 = 6(1 - Ale-tz .+(012)I)'

t2(3J + K)

t

Cl 2(1 - Ale tZ, +(t=/2)I)'

C2 -

6(1 - Ale-tZ^+tt=^2>I)

It follows from (3.7) that the rejection region like the type (3.1) is given by Z1(00) + n Zz(0o) < Zl + n

Z)(00) + n Z2(00) > 21 + C2 n

where cl, ci, c2 and c2 are given above. We put

Al = {: zi(eo) +.zz(oo)>2i + n A2 = f Xn : Z1(00) + Z2(00) < Zl + / .

In order to construct the second order a -A UC interval we have to determine Zl and 21 such that PP.eu(Al) + Pn.ea(A2) = a + o\ In '

(3.8)

d9Pn.eo(Al) + ^BPeo(A2) = of In

(3.9)

If XA denotes the indicator function of the set A, then (3.8) is given by En.6o(

XA)

+

En..0(XA =)

= a +

o(n

)-

(3.10)

Since /Pn. e(A) =

9)} du (n)

J

XA(n)

J

XA( n){- log tII AX I , 9) - II f (xi, 9) dju(n)

Id110

^ f(xt+

('

80 =1

i=1

Rep. Star. App!. Res., JUSE. Vol. 26, No. 3, September, 1979,

138 M. AKAHIRA AND K . TAKEUCHI

34/104

J

X A( Xn){

a9

log f(xt,

= En.B[XA(Xn)tE =1 ae

0)}tf[ f(x , 9) d ju(n)

logf(x1,

e)],

where p(n) denotes the n-fold direct product measure of 4 u, it follows that (3.9) is given by En,eo [ XA,(Xn) Z 1 (9o)l + En,eo[XA,(Xn)Zl(9o)l

(3.11)

= o(n) •

Putting Z1 (00) = Z1(00 ) +

-

=Z

2(90);

l Z*

*(90)

= Z1(00 ) + n- Z2(00),

we have from (3.10) l o( n [XA^(9,,){Z*(0o) + En,e,[XA ,(xn)

{Zl *(o o)

3n Z2(00)}]

(3.12)

5

- j n Z2(e0)}]

En,eo[XA,(. ,)ZI(e0)] - % En , eo[XA, (Xn)Z2(e0)]

+ EE,eo [XA,(Xn)Zl

*(00)]

En, Bo[XA ,( Xn)Z2(e0)l

- /

Since

En.e,[Z1(0o)Z2(eo)]

En,eo[Zi(0o)l = I(0o);

En,Bo [ Z2(00)]

= J(0o);

= M(00) - I2(00),

where

M(9) =

E0[

laez

log Ax, 0)} 2],

it follows that Vn,0o (Zl(00))

= EE.eo [ {Zt(00) +

=

Covn,eo(Zl(

En.e,[Zi(eo)]

00), Z2(00))

+

n Z2(0o)} 2] (3.13) n

n E ,0[Z1(90)Zz(0o)] + o

( .7-n

= EE,ea[Zi (eo)Z2(00)l - EE.eo[Zi (0o)]En.eo[Z2(90)l

=

En,eo [ Z1(00) +

nn

Z2(0o

(3.14)

} Zz(00) ]

n En.e,[Zi(Bo)] = En,eo[Z1(e0)Z2(00)] + c = J(90) + nn [M(00)

V12(00)1 + o(1 n

From (3.13) and (3.14) we have En,6o[Z2(00

) I Z1(00)] = Coyn,eo(Zl( 00), Z2(0o))Zi (eo) 1(e0)) VV.Bo(Z*

Hence we obtain _ En,eo[XA ,(Xn)Z2(e0)] = En,o, [XAi(Xn)En,eo(Z2(0o)) I Z1 (BO))l ` En , 6o[XA ,(Xn)Zj*(eo)] + °(^ n )

By a way similar to the above we obtain

j(9°

Rep. Stat. Appl. Res., JUSE. Vol. 26, No. 3, September, 1979

(3.15)

139 ON THE SECOND ORDER ASYMPTOTIC EFFICIENCY

Ee.eu[XA,(Xn)

E.eo[XA.(x,,.)Z1 *(90)] + o(J nn 1 •

Z2(e0)]

= I(00)E Form (3.12), (3.15) and (3.16) it follows that j1

105/35

(3.16)

/

I(eo(n E,,.e.[XA,(X»)Z*(o0)] + 1 - I(Bo), ✓ n En.e.[XA.(X.)Zi *(eo)] = o(n •

(3.17)

Itt follows by Edgeworth expansion that the probability density function f(x) of Z*(90)/Q, where a' = V,,,j.(Z*(9o))((3.13)), is given by / f(x) _ O(x) + n H3(x)O(x) + 01 n ) where O(x) = -L-e- x'/2 and Y3 is the third order cumulant of Z*/Q and H3 (x) is a Hermite polynomial , i.e. H3 (x) = x3 - 3x. Since f xH3(x)q(x) dx = f x(x3 - 3x)o(x) dx = f (x4 - 3x2)O(x) dx = f [(x4 - 6x2 + 3) + 3(x2 - 1)}O(x) dx = f [H4(x) + 3H2(x)} j(x) dx = f H4(x)((x) dx + 3 f H2(x)o(x) dx _ {-H3(x) - 3Ht(x)}O(x)

= (-x3 + 3x - 3x)o(x) = -x3/(x), where H1(x) (i = 1, 2, 3, 4) are Hermite polynomials, it follows that f xf(x) dx = f x {ct(x) + 5i.=H3(x*x)}dx + o(^ nn l

(3.18)

= f xo(x) dx - 6^3n f xH3(x)o(x) dx + of n = -O(x) -6 x30(x) + o 3(.1-n

Since by a way similar to the above , the probability density function g(x) of Z**(9o)/a'', where Q'2 = V 00(Z**(00)) = I(e0) + - nn J(Bo) + °\n)' is given by

where fl' is the third order cumulant of Z**(9o)/a ', it follows that (3.19)

f xg(x) dx = -O(x) - 6^3n x30(x) + o(- n ) •

Putting kt =Zt +^n;

ki =Zt +-/ -n

we have At = [Z* > kt}; A2 = {Z** < k;}. By (3.18) we obtain En.Ba[XA,(Z)Z*(B0)] _ [

(3.20) - 0 (a :0 /

6

P3

(Ok,

(L,

k

Rep. Stat. Appl. Res., JUSE. Vol. 26, No. 3, September, 1979

140 36/106 M. AKAHIRA AND K. TAKEUCHI

By (3.19) we also have EE,oQ[XAI (Xn)

Z **(Oo)J

= [_O(6r) c /3n (Qr )30(.L

(a, ^n Since

N3

= ri =

J

(3.21)

k'

k' )1. I

(Q)3' J

K(9o), it follows by (3.17), (3.20) and (3.21) that I117 )E^,eo(XAiZ*) + (1 - I")En .eo(XA1Zi * )

(3.22)

n){0(a)+ n 'Sal "Sa K

1.%I

(1 1../n)l^(Q)+6-✓ n \a')3^\o•' c2 =(1 -1c/n )

l +6. / n'Sa)3}"( Z l

-(1 Ian){ 1

a

a'`/n)

+61/n k,

From (3.22) we have Z1 +^n Z' +n = (1 -1.

( ^%n){1 +6.V/n ( (a')

3

(3.23) j I^ n) ll + 6 ✓n Ql)31 , 3 c'J K k, c1J - K (kl 3 1+I./n n+61/n(a'') Wn(Q)

^^

✓nWa^ 3-(a3} +n

=1 +cl./ n'J+

it follows from (3.23) that Since a2 = a'2 = I(90) + o r n// (3.24)

- 1 S (Zi + n) 2 - (Zi + n) 2} log {1 + (c1I I1

;_

+ 613/ k/ n (Z'

- Zi){ .

(c'I )J+C,61K (Z' +o ( ln) n

n

Putting Zl = ua/2

-

,/2 IV/

-✓

I(Bo) + n and Zl = -ua./2,^/I(Bo) + n, we have from (3.24) y^ uz/2 •I + ?I ' + C2 + c2

1/I -/n = (c1 n1)J + 613/K n

(I3 /2ua/2 + 13/2ua12)•

Hence we have + (c, - c)J (' u,^(/'2.^/ I 3

ua./2 - C2 - c 2 .

Since

J f( ) dx = J x

{O(x) + 6 K H3(x)O(x)} dx

_ fi(x)

-6

n

J

H3(x)o(x) dx

Rep. Stat. App/. Res., JUSE. Vol. 26, No. 3, September, 1979

(3.25)

141

ON THE SECOND ORDER ASYMPTOTIC EFFICIENCY

107/37

= (D(x) - n H2(x)O(x) = (D(x) - _ n (x2 - 1)0(x); f g(x) dx = O(x) - WK- (x2 - 1)O(x),

where fi(x) is a standard normal distribution function, it follows from (3.10) that a + 0(-n 1 = EE,Bo (XAi) + En, 90(XA,)

K + a / - W n (a12 - 110(at Hence we have Zl

Zl

.+ C2

CI

{

l r 1 -ac + °n ^( Q^n) - c` Q I n) (3.26)

+ 6,,/ n 1(QzZ - 1)o\Q' / - \a2 - 1/\ 61/} Since we put 21 = ua/2 .s/ I + 1 n

ZI = -ua/2✓ I +

1/n

we have from (3.26) ua/z / I +

(D

(

1/

Cz

n

+

-ua /2,

I

+

C2

+

^'

l

n )=1-OC+° (\n/. (3.27)

` Since

J

`

J

Q = ✓I(eo) c'I(9)n + o(^n); Q'=-✓I(9o)+ c1(8) +01^n^ it follows that

Q n

= (ua/z + \

CI

= u a/z - ua /2

n

^ j1

- I^ n + °\ %I-n)

C-J

I1/ n + c^/In + °n

-u./2'V I + C2 + 11

n ciJ c'2 + fj' 1 ^, _ ua/z + ua/2 I n + In +0 (^/

Hence we have \ua/z^,/I

-ua/2✓ I

+

+ c2 +_tJ l ( l In ^ )(-ua/z ^ c2 + ^/ \ In /

C2

+

17'

/

^( n ) = 2 + In O(ua/z) (U-1 I' J + c2 I') n + °1 ln

From (3.27) we obtain 2) - ua/2 (c1 + c'i)J + c2 -C'2 + 11-11 ' 1 0 (ua/ ,V -n ( I -'1-T / = ° \n

(3.28)

Rep. Stat. App!. Res., JUSE. Vol. 26, No. 3, September, 1979

142

M. AKAHIRA AND K. TAKEUCHI

38/108

Since by (3.28) C2 - c'2 + t7 - 17' = ua /z(c) + c'i)J ^/1 I

it follows that 17 - 11' = ua

/2(c) + ci)J - c2 + c'2. ./ I

(3.29)

From (3.25) and (3.29) we obtain ua/2(c, + c',)J - (c, - c,)J,/ I _ K,%/-I 2 6 UM/2 - c2; n i17 -T 2ua/2 tl' ua/2 (c, + c',)J - (c, - c',)J.%/ I - K,,./-I u,/2 - c2. 2^7 2ua/2 6

(3.30) (3.31)

where Cl = C

t __ t2(3J + K) 2(1 - ),l e luais ✓ ! +(6 /2)I)' C2 6(1 - ^le tua/s✓ 7 +(ts/2)!)> t t2(3J + K)

99 , 2(1 - h,elua:1i+(tsi2)I)' c2 = 6(1 - /I,,etua/s./!+(t>/z)!

with e (t'/2)'sinh (tua /2N I) + 0(1/,%/ n ).

^1 = 1 ua /2N

I(e0)

Hence we have Z, = ua/21/ I + nn ; Zl = -U./2N/ I + n where tJ and 17' are given by (3.30) and (3 . 31), respectively. We also obtain kl = ua /z ✓ I +- C2 + 17. k' = -ua / 2N I + C2 + t1'. ,,7 1/ n

We put

u(0; t) = c2 + tl; 1(0; t) = c2 + 17'. If there exist 0**(9„) and 0* *(.Xn) in the neighborhood of maximum likelihood estimator (MLE) 0ML such that

+ ^%Z2 (0 n *(X,)) = ua/z ✓I(en *(xn)) + u(0_**(nn); t) 1 (0 nn); t )n );t Z,(0 **(xn)) + 'n Z2 (0 *(xn)) = -ua / 2^✓I(#n *(xn)) + 1(0 Z1(Q* *(zn))

(3.32) (3.33)

([2]), then the second order VV a -AUC interval [B* *(2n), 0* *(. )] is second order-.asymptotically efficient . Hence we have established the following: Theorem 3. Under conditions (i) ' (iv), the second order a-AUC interval [B**(xn), 0* *(zn)] is second order asymptotically efficient. We shall actually construct the second order asymptotically efficient a-AUC interval using the MLE OML. We put e* *(xn) _ ua/2 + N I (0ML)n nr

ua/2 Bn *(xn) =BML +

+

S

N I( BML)n n

we have ZI(Bn *) + n Z2(en *)

=

Zl (

S'

p^ ua/2

ML O

+

n) .^/ n Z2(BML

" I ( ML)n

Rep. Stat. Appl. Res., JUSE. Vol. 26, No. 3, September, 1979

ud/2 N

I(0 ML)n

+8'I+vP(1)

143

ON THE SECOND ORDER ASYMPTOTIC EFFICIENCY

109/39

u^/2 + yr Zj(eML) + 1 (- ua/2 + ^r)2Z1(eML)

- \ N I( 6ML)n n 2 \ I✓I(eML)n n

+ - Z2(OML) + + r)Zy(eML) + 0,( W n ^7 ( - ^7I(OmL)n n n Since Z '1(eo) = Z2 (00) - N/ n I (eo); I'(eo) = 2J(eo) + K(eo), it follows that

Z'(eo) = Zz(e0) - 4/ n I r(eo) - -.v/ n J(90) - J n {2J(9o) + K(90)) _ -./ n {3J(eo) + K(eo)). We also obtain

Zi(e0)

n ri a 33 logf(X

" 90) + V, n Ir(6o)

0e3 logf(X, eo ) + N/ n {2J (eo) + K(e0)) n ri NI-W[-3J(00) - K(e0)) + N/ n {2J(e0) + K(00)) _ -J(eo)'1 n . Hence we have Z1(en

*) + /nz2(en *) 'V

r ua/2 +

( - `- Ice )

n

+ ua12

{-

✓2I (OML)n

(Z2(eML) - N n I (e ML)^

^/ n (3 J(9ML)

+ ua/2 (1nJ(OML)) + 01

".I(ML)fl = UM/2N I(OML) - na/2

+ K(OML))) + ' Z2(eML)

(

\✓ n

1nZ2(e ML) + / Z2(OML)

'I(e M /zJ(OML) + opN n ua/2 / (3J(OML) + K(GML)} + Clua 2I(OML)- n N146ML)

Since ✓I(B"*)= I +op\,In u(9 *, t) = u (OML; t) + 0p^

it follows by (3.32) that \1 1 us 1 22

Z2 (OML) J(OML) + _ = Z2(OML) - n.2 /2 _(3J(OML) + K(OML)J

vI( eMt)n V :V 2I(9ML)NI n Clua

+

/2J (

OML)

-ua/2(2J(OML) +

K(

OML))

2 I (#ML)n

II(9M nn

Then we have Vr = Cl

- -/ua./2 Z2 (OML)

+ Clua

/2

J(OML)

(3.34)

'. I(9ML)" J(VML) I(OML)3/2

Rep. Stat. App!. Res., JUSE. Vol. 26, No. 3, September, 1979

144 M. AKAHIRA AND K. TAKEUCHI

40/110

_ Ua/2 53J(OML)+' K(#.,,,) + 2J(OML) + K(OML) (p^ 2I(BML) I I(#+ V ML) ^I(BML)

In a similar way to the above we obtain u «/2 \

ML)

Z2Z(OML) _ J(e IUa/2 I(OML)3/2

/ l + /I(OML)) I(OML) C

f 3J(O ML) + K(OML) + 2J( OML) + K(ML) 2I(9ML) ^ I (OML) I (eML) ^ . Hence we have established the following: Theorem 4. Under conditions (i)- (iv), the second order a-AUC interval nS ,

(3 (3.35)

\\ Ua /2

CB

ML - Ua/2

+

, BML + "a/2

(3.36)

-✓I(eML)n n '✓ I(BML)n n

is second order asymptotically efficient, where 6' and 6 are given by (3.34) and (3.35), respectively. Remark. It should be noted that both of 6' and 6 depend on t and Z2(.). The second order CL-AUC interval (3.36) is most powerful but not uniformly most powerful because of the dependence on t. This testifies the fact that the asymptotically sufficient statistic up to the order n 1/2 is given by the pair (9ML, Z2(9ML)) ([9], also [8]), while in the case of point estimation 9ML is alone asymptotically efficien up to the order n l/2, i .e. second order. References [1] Akahira, M.: Asymptotic theory for estimation of location in non -regular cases, I: Order of convergence of consistent estimators, Rep. Stat. Appl. Res., JUSE, 22 (1975) 8-26. [2] Akahira, M. and Takeuchi, K.: Discretized likelihood methods-Asymptotic properties of discretized likelihood estimator (DLE's), Ann. Inst. Statist. Math., 31 (1979), Part A, 39-56. [3] Efron, B.: Defining the curvature of a statistical problem (with applications to second order efficiency), Ann. Statist., 3 (1975) 1189-1242. [4] Lehmann, E. L.: Testing Statistical Hypotheses, John Wiley (1959). [5] Pfanzagl, J.: Asymptotic expansions related to maximum contrast estimators, Ann. Statist., I (1973) 993-1026. [6] Takeuchi, K.: Asymptotic expansions for the critical limits of unbiased tests, Discussion paper No. 131, Kyoto Institute of Economics Research, Kyoto University. (1978). [7] Takeuchi, K. and Akahira, M.: On the second order asymptotic efficiencies of estimators, In Proceedings of the Third Japan-USSR Symposium on Probability Theory (G. Maruyama and J. V. Prokhorov Eds.) Lecture Notes in Mathematics 550 (1976) Springer-Verlag, Berlin, 604-638. [8] Michel, R.: Higher order asymptotic sufficiency, Sankhya, 40 (1978) 76-84. [9] Suzuki, T.: Asymptotic sufficiency up to higher orders and its applications to statistical tests and estimates, Osaka J, Math., 15 (1978) 575-588. [10] Wald, A.: Asymptotically shortest confidence intervals, Ann. Math. Statist., 13 (1942) 127-137.

Rep. Stat. Appl. Res., JUSE. Vol. 26, No. 3, September, 1979

145

Rep. Stat. App!. Res., JUSE. Vol. 26, No. 4, December, 1979

A Section

Remarks on the Asymptotic Efficiency and Inefficiency of Maximum Probability Estimators MASAFUMI AKAHIRA* AND KEi TAKEUCHI**

ABSTRACT Suppose that X1(i = 1, 2, . . . , n) are distributed according to a two-sided truncated normal distribution with unknown mean 0 . It is shown that the maximum probability estimator (MPE) of Weiss and Wolfowitz is asymptotically inadmissible and has smaller concentration of probability than the asymptotically efficient estimator BPT = (min X, + max X1)/2.

1. Introduction Weiss and Wolfowitz ([7]) defined the concept of the maximum probability estimator (MPE) which V MP is defined as the value of which maximizes B+rc, l n

II f(xi, O) dO

B-rc,. ' t=1

with appropriate choice of the sequence of constants (cn). And they proved that asymptotically PF,O(cn 1 Bn - 01
r - 01
for r'=Ar

and it is not ruled out that or may behave badly outside the fixed interval (-r, r). In regular cases, #r is asymptotically equivalent to the maximum likelihood estimator (MLE), and is asymptotically independent of r. But in non-regular cases OMP does depend on r (which is also emphasized by Weiss and Wolfowitz themselves), the asymptotic behavior of or is not well investigated. In (2], [3], [4] and [5] we defined one-sided and two-sided asymptotically efficient estimator in nonregular cases, as the one which maximizes lim P.,9[cn(Bn - 0) < a) for a > 0 n,w

or lim Pn,g[cn(On - 0) > -b)

for b > 0

and lim P e[cn 10n - 01 < c}

for c > 0

n-

Received 4 July, 1979 * Statistical Laboratory, University of Electro-Communications, Chofu, Tokyo 182, Japan ** Faculty of Economics, University of Tokyo, Hongo, Bunkyo-ku, Tokyo 113, Japan 22132

146 REMARKS ON THE ASYMPTOTIC EFFICIENCY

133/23

uniformly in 0 and a or b or c among asymptotically median unbiased (AMU) estimators which satisfy the condition lim Pn, 9[On <

0 1 = lim Pn,B[O& > 01 = 1 2

n-•.

uniformly in 0. Then the question is: if or is AMU, then is it asymptotically efficient in the sense above? The purpose of this paper is to show that the answer is not always yes, even in the case when asymptotically efficient estimator does exist, implying that the MPE may asymptotically inadmissible. The case discussed below is also covered by Weiss and Wolfowitz [8] and their Z. is basically asymptotically equivalent to 0, and at least as good as Or but they did not show explicitely the inadmissibility of the latter. 2. Results Let (S, (13) be a sample space. We consider a family of probability measures on CB, tP _ [PB: 0 E 0), where the index set 0 is called the parameter space. We assume that 0 is an open set in a Euclidean 1-space R1. Consider n-fold direct products (X(n), &n)) of (9C, (13) and the corresponding product measures Pn B of PB. An estimator of 0 is defined to be a sequence [On) of (Bcn1-measurable functions On on X(n) into 0 (n = 1, 2, ...). For simplicity we denote an estimator by Bn instead of [Vn}. For an increasing sequence of positive numbers f cn} (cn tending to infinity) an estimator On is called consistent with order f cn) (or (cn}-consistent for short) if for every e > 0 and every $ E 0 there exist a sufficiently small positive number (5 and a sufficiently large positive number L satisfying the following: lim sup n-+Po B: 18 -91
P., etc. 1

0n

-

01

> L} < E ([1]).

A [cn)-consistent estimator On is called asymptotically median unbiased (AMU) estimator if for any $ E 0 there exists a positive number 0 such that

O < 01 - I 0;

llm sup

P., Of n n-•^ B:I 8-91
lim

> 0 1 - I I = 0.

sup

I Pn,B10n X- 0:10-0 1<45 2

A distribution function FB(t) is called to be the asymptotic distribution of cn(On - 0)(or en for short) if for each real number t, Fe(t) is continuous in 0 and for any $ c 0 there exists a positive number d such that at any continuity point t of FO(t) lim sup n-o 0:I0-91
I Pn,B[Cn(

0 n - 0) S t) - FB(t) I = 0.

Let X = O = R1. Suppose that X1, X2, ... , Xn, ... is a sequence of independently and identically distributed random variables with a density f(x: 0). Suppose that 0 is a location parameter, i.e., f (x: 0) = f (x - 0). Further we assume that

Ice- (x'12) for l x i < 1; f(x) = 0 for l x i > 1, where c is some constant. In this case it is shown by Akahira [1] that there exists a consistent estimator with order [n), which is the largest possible order. Let R = (-r, r). The maximum probability estimator 61 of 0 is defined as d maximizing d+r/n n

II f (xi - 0)d0

d-r/n 1=1

(Weiss and Wolfowitz [7], [9]), and in this case O , is shown to be Rep. Stat. App!. Res., JUSE. Vol. 26, No. 4, December, 1979

147 MASAFUMI AKAHIRA AND KEI TAKEUCHI

24/184

-2rt; (min X1 + maxXX)formax X, - minXX > 2 min X, + 1- nfor X > min Xt+1- n ,max Xt -min Xt<2- nr; r MP =

maxXX-1 +nfor X < max X1 -1 +

n

, max X, - min Xt< 2 - nr ;

r

XformaxX,-1 } n<X<minX,+1-n , where ,k = - E Xt. n t., Remark. The definition of Or in the first case is not unique, but it does not affect the subsequent conclusions. First we note that the densities of the asymptotic distributions (asymptotic densities) of min Xi + 1 and max X, - 1 are given by u Ike ku for u > 0; 2.1 ) g,O = - t0 for u < 0; ( u _ lkekn for u < 0; 2.2) gz() - t0 for u > 0; ( respectively, where k = ce 112. Now we consider another estimator O. = (min Xi + max Xt)/2 which is asymptotically equivalent to the Pitman (best location invariant) estimator. Since min Xt + 1 and max Xt - 1 are asymptotically independent, the density of the asymptotic distribution of OPT = (min Xi + max X,) is given by g(u) = ke 2k " n i. (2.3) Next we shall obtain the asymptotic density of OMP. The asymptotic conditional density of (min Xt + max Xt) given max Xt - min Xi > 2 - 2r/n is obtained by

k 2ktai - ezkr) for Jul < r; hl(u) =I1 - K(r) (e Illl 0 for i ui > r; e-2kr. ) K(r) = (1 + 2kr where It can be shown that Xis asymptotically independent of min Xt and max Xt, or more precisely /X is asymptotically normally distributed independently of n(min Xt + 1) and n (max Xi - 1). Hence it follows that the asymptotic conditional density of min Xt + 1 - r/n given X > min X, + 1 - r/n and max Xt - min Xt < 2 - 2r/n is obtained by e-2kr

for I u I < r;

h4u) = k( n+r) K(r) a

for I u I > r.

Further the asymptotic conditional density of max Xi - 1 + r/n given k < max Xi - 1 + r/n and max Xt - min Xi < 2 - 2r/n is obtained by k -2kr K(r) e

h (u)= s k

K(r)

for I u I < r;

e k^' -0 for u < -r.

Since the order of the probability of max X, - 1 + r/n < X < min Xt + 1 - r /n is 1/n, the probability of X given max X, - 1 + r/n < X < min X, + 1 - r/n is negligible in this case. Hence the asymptotic density of #r is given by ke 2kl uI for Jul < r; g *(u) = I 2 e- k(lnH+r) for I uI > r. Rep. Slat. App!. Res., JUSE. Vol. 26, No. 4, December, 1979

148 REMARKS ON THE ASYMPTOTIC EFFICIENCY n

Let Pn,0 be the probability measures with the density

II f (XI r =1

-

135 /25

0).

Next we shall get the lower bound of Pn,tal n 1 9n =

-

0 1 > u}

Pn.B{ n(Un

-

0)

- u} +

^

ff

PP,o l n( Un

-

0) > u}

(2.5)

for all AMU estimators. Let u be arbitrarily positive but fixed and 6. be AMU. Let 00 be any fixed in O and put 01 = 00 - (u/n) and 9z = 90 + (u/n). Since for each real number t Pn,B{n(9n - 9) < t} converges uniformly in a neighborhood of 00, it follows that for sufficiently large n ^ u} '„ PP.90[n ( n - 9o) < u}; Pn.B,t9n :!z^ 001 = PP, ojn(Un - 91) F,,B,[Un ^

90}

=

Pn, e,1n(Un

O - 92) :< -u} - Pn,o,tn(Un - 90) :<

-u }.

In order to get the lower bound of (2.5) it is sufficient t find the maximum value of Pn,B, Un

<

90}

of

-

Pn , B,1 Un ^

00 1

(2.6)

for AMU estimators O. Let B. = 1xn • Un(xn ) S 90} and 0. be an indicator function of B. Then it follows that ...

J

^1n( n ) =II f(x, - 90) ^IJ dxr = EE , 90(On ) --^ (n -p co).

We also have 7 7 77 7n^' 1 f (Xi, 01) t i dxt 0n(xn ) 1 l jl f(xt, 02 ) t

Ir

(2.7) = En, B,(Y'n) - En.B,(^n)• In a similar way as the proof of the If the maximum value of (2.7) is obtained, then it is that of (2.6). fundamental lemma of Neyman ad Person, it is shown that 0. maximizing (2.7) is given by 11 f(Xi,

y'n ( n) = 1=1

0,

02)

II f (x ,, U2)

1=1

- 11 f(Xi,

9 1)

t=1

>

A

II f(xt, 90) ;

i=1

- II f(x r, 01) < A H f(x i, 90), 1=1

f=1

for some A. Since 91 and 02 are symmetric with respect to 90, it is clear that A = 0. Hence

1, if

II f(xi, 92) > [f f(x,, 01), that is,

i=1 i=1

f(xt, 02) > 0, II f(Xi, 01) = 0; (a) 11 i=1 1=1 or (b)

> 0, II f(xr, 0 1 ) II f(xt, 02)1-1

> 0 and x < c';

t=1

0, if II AXI, 02) < II A xi, 01), that is, r=1 r=1 (a') tH f(xr, 92) = 0, II f(Xi, 01) > 0; or 1, 02) > 0, II f(xi, 01) > 0 and x > c', (b') 1 IT f(x1=1 n

where z = - xt and c' is some constant n i=

Putting Rep. Stat. App!. Res., JUSE. Vol. 26, No. 4, December, 1979

149 26/136 MASAFUMI AKAHIRA AND KEI TAKEUCHI

A=

xnlmaxx;-1>O0- n,minxi+ 1>00

u n

B= {Xnlmax xi- 1
u u} C = {zn I max x -1 <00- ,minx;+1>Oo+ n n D={znlz

c')

we have 1, if x,, AU(CnD); )-^0, ifznEBU(CnD'). (2.8) Since X, max Xf - 1 and min X, + 1 are mutually asymptotically independent, it is seen that the conditional probability Pn,O{D I C) of D given C is given by for sufficiently large n Pn,g{D I C) = Pn,g{D}

In order that we get En,00(^n) -s 2

(n oo)

we must have PP,go{D } _,

2 (n -, co).

It follows from (2.1), (2.2) and (2.8) that for sufficiently large n En,91(^n ) -

Ea,0 (Yn )

Pn,O :(A)

+

2

Pn,o ,(C) -

PP,e,( A )

-

n,O,(C) 2 P

(2.9)

Pn,O:(A) I - e-2ku

From (2.5), (2.6), (2.7) and ( 2.9) it is shown that lim Pn, An I On

-

0 I ^!! u} ^i! e2ku

(2.10)

for all AMU estimators O. Hence the right-hand side of (10) is the desired lower bound. Therefore it is shown that the bound of the asymptotic distribution of n I On - 0I for AMU estimators 6. is given by lim P,,o(n n--

I On

-

0I

< u} < 1 - e 2k u.

(2.11).

Since both of the asymptotic densities of OPT and 6. 1p are symmetric, it is easily seen that these estimator are AMU. From (2.3) and (2.4) we have lim P,,o(n I OPT - 0 I < u} = 1 - e 2ku;

(2.12)

n-•oo

lim P,, O(n I OMP -

0I

< u} = 1 -e2ku for 0
(2.13)

Hence it is shown by (2.11), (2.12) and (2.13) that Opr is two-sided asymptotically efficient in the sense that the asymptotic distribution of OpT attains uniformly the bound given by the right-hand side of (2.11) but 0 , is not so. Remark that in this case the MLE is asymptotically (also in small sample) uniformly (in u) inadmissible which fact can be established in the following way. It is easily seen that Rep. Stat. Appl. Res., JUSE. Vol. 26, No. 4, December, 1979

150 REMARKS ON THE ASYMPTOTIC EFFICIENCY

137/27

1min X,+1 ifX>minX,+1, OML=

max Xi-1 ifX<maxXi-1,

X if max X,-1 <X<minX,+1 and asymptotically, it is equivalent to glmin X, + 1 with probability 1/2, n _ - tmax X, - 1 with probability 1/2.

Since the asymptotic density of J. is symmetric, it is easily shown that 6 ML is AMU. Thus the asymptotic distribution of n I OML - 0 1 is easily seen to be 1imPn,B{nIOML- 01 < u) = 1 -e k" n-_

which has twice as large as large scale as the asymptotic distribution of OPT, the asymptotic efficiency of 0ML compared with OPT is exactly 50 %! It should be also noted that OML is the limit of #i,, as r 0, and that O;K O. is inadmissible outside the interval -r < u < r, and that the interval converges to the null set when r goes to zero. On the order hand when r becomes large, the interval of u in which 0M , is admissible extends and the probability outside the interval also concentrates to the center, and as r -• oo, Ont, approaches to OPT which is uniformly efficient. Thus in this case the choice of larger r is always better than the choice of smaller r, which, although remarkable enough, does not hold in the other cases. It is also noted that the Pitman estimator is the generalized Bayes estimator with uniform prior and quadratic loss, but OPT is asymptotically equivalent not only to the Pitman estimator but also to any generalized Bayes estimators with symmetric loss function ([6]).

We have defined an AMU estimator 0,* to be one-sided asymptotically efficient in the sense that the asymptotic distribution of n(0* - 0) uniformly attains the bound of the asymptotic distributions of n(dn - 0) for AMU estimators 0n([3], [4], [5]). We have showned that the bound of the asymptotic distribution of n(On - 0) for AMU estimators On is given the following density: ke k l" 1 for J u I< k log 2; 1 0 for u I > - log 2, k (See the proof of Theorem 3.1 of [2]). Comparing (2.3) and (2.4), we may also conclude that OPT and 0MP are asymptotically equivalent for I u I < r but for I u I > r OPT is asymptotically better than o , in the sense that the asymptotic distribution of OPT is uniformly nearer to the bound than that of OMP. Though OM , and OPT are not one-sided asymptotically efficient in the sense that their asymptotic distributions uniformly attains the above bound ([3], [4], [5]), it is seen that OPT is asymptotically better than 9MP.

REFERENCES Akahira, M.: Asymptotic theory for estimation of location in non-regular cases, I :Order of convergence of consistent estimators. Rep. Stat. Appl. Res., JUSE 22, (1975) 8-26. Akahira, M.: Asymptotic theory for estimation of location in non-regular cases, II:Bounds of asymptotic distributions of consistent estimators. Rep. Stat. Appl. Res., JUSE 22, (1975) 99-115. Akahira, M.: On the asymptotic efficiency of estimators in an autoregressive process. Ann. Inst. Statist. Math. 28, (1976) 35-48. Takeuchi, K.: Tokei-teki Suitei no Zenkinriron (Asymptotic Theory of Statistical Estimation) (in Japanese), Kyoiku Shuppan, Tokyo. (1974) Takeuchi, K. and Akahira, M.: On the second order asymptotic efficiencies of estimators. Proc. of the 3rd Japan-USSR Symp, on Prob. Theory, Lecture Notes in Math. 550, (1976) 604-638, Springer Verlag. Rep. Stat. Appl. Res., JUSE. Vol. 26, No. 4, December, 1979

151

28/138 MASAFUMI AKAHIRA AND KEI TAKEUCHI [6] Takeuchi , K. and Akahira , M.: Asymptotic optimality of the generalized Bayes estimator . Rep. Univ. Electro-Comm ., 29, (1978) 37-45. [7] Weiss, L. and Wolfowitz , J.: Maximum probability estimators . Ann. Inst. Statist . Math ., 19, (1967) 193-206. [8] Weiss, L. and Wolfowitz , J.: Generalized maximum likelihood estimators in a particular case . Theor. Probability Appl., 13 , (1968) 622-627. [9] Weiss, L. and Wolfowitz , J.: Maximum Probability Estimators and Related Topics. Lecture Notes in Math. 424, (1974) Springer Verlag.

Rep. Stat. Appl. Res., JUSE. Vol. 26, No. 4, December, 1979

152 Ann. Inst. Statist. Math. 31 (1979), Part A, 39-56

DISCRETIZED LIKELIHOOD METHODS-ASYMPTOTIC PROPERTIES OF DISCRETIZED LIKELIHOOD ESTIMATORS (DLE'S)1' MASAFUMI AKAHIRA2) AND KEI TAKEUCHI

(Received Feb. 8, 1978; revised Feb. 13, 1979)

Abstract Suppose that X1, X2, • • •, Xn, • • • is a sequence of i.i . d. random variables with a density f (x, 0). Let cn be a maximum order of consistency. We consider a solution Bn of the discretized likelihood equation Z log f(Xi, On +rcn1)-^ log f(XX, On) = an(Bn , r) where an(0, r) is chosen so that On is asymptotically median unbiased (AMU). Then the solution On is called a discretized likelihood estimator (DLE). In this paper it is shown in comparison with DLE that a maximum likelihood estimator (MLE) is second order asymptotically efficient but not third order asymptotically efficient in the regular case . Further it is seen that the asymptotic efficiency (including higher order cases) may be systematically discussed by the discretized likelihood methods. 1. Introduction Recently second order asymptotic efficiency has been studied by Chibisov [4], [5], Pfanzagl [8], [9], Takeuchi and Akahira [3], [11], Efron [6], Ghosh and Subramanyam [7] and others. Furthermore third order asymptotic efficiency was discussed in Takeuchi and Akahira [11], [12], [13] and Pfanzagl and Wefelmeyer [10]. In this paper using the discretized likelihood method we consider the asymptotic efficiency of estimators including higher order cases. Suppose that X1, X2,. • •, Xn, • • • is a sequence of i.i.d. random variables with a density f (x, 0). Let cn be a maximum order of consistent estimator of 0. We have proposed a solution On of the discretized likeli1) The results of this paper have been presented by the first author at the meeting on "Asymptotic Methods of Statistics" at the Mathematical Institute in Oberwolfach of West Germany, November 1977. 2) Partially supported by Sakkokai Foundation. 39

153 40 MASAFUMI AKAHIRA AND KEI TAKEUCHI

hood equation n

n

logf(Xi, 0n+rcnl)-Z logf(Xi, On)=an(On, r) i=1 i =1 where an(0, r) is chosen so that Bn is asymptotically median unbiased (AMU) (the possibility of which will be shown in the context) [3]. Then the solution Bn is called a discretized likelihood estimator (DLE). If for each real number r, n

n

log f(Xi, 0+rcn1)-Y log f(Xi, 0) i=1

i=1

is locally monotone in 0, then the asymptotic distribution of the DLE On attains the bound of the asymptotic distributions (discussed below) of AMU estimators of 0 at r. It is easily seen that there is at least one estimator which attain the bound. In regular cases with cn= 4/ n the left-hand side of (1.1) is expanded as

(1.2)

e iZ log f(Xi,

en)

+ 2 ^r

a_

iZ

log f(Xi,

en)

+ ... =an(on, r) .

We derive from (1.2) the order to which the maximum likelihood estimator (MLE) is asymptotically efficient. In this paper it is shown that an MLE is second order asymptotically efficient but not third order asymptotically efficient. The motivation for the definition of the DLE is that ; when we test the hypothesis 0=00+rc; 1 against 0=00, the most powerful test is given by rejecting the hypothesis if n

n

E log f(X1, 00 +rcn1)-E log f(Xi, i=1 i=1

00)
hence if an estimator Cr is defined so that the event Br>00 is equivalent to the above inequality (at least asymptotically up to some order), then ©r is efficient (asymptotically up to some order) for specified choice of r. Therefore if ©r can be defined independently of r, then it is shown to be efficient (asymptotically up to the above mentioned order), and if not, we can establish that there does not exist any efficient (in the same sense) estimator. It is also seen that the asymptotic efficiency (including higher order cases) may be systematically discussed by the discretized likelihood method.

2. Notations and definitions Let (a', _B) be a sample space. We consider a family of probability measures on 9B, ' = { PO : 0 E 61, where the index set © is called the

154 DISCRETIZED LIKELIHOOD METHODS 41

parameter space. We assume that ® is an open set in a Euclidean 1space R'. Consider n-fold direct products (_'In), _B In,) of (X, _B) and the corresponding product measures P,,,0 of P0. An estimator of 0 is defined to be a sequence {Bn} of B`n)-measurable functions B. on _'In) into © (n=1, 2, ... ). For simplicity we denote an estimator as B„ instead of [On)

. For increasing sequence of positive numbers

infinity) an estimator

en

{ cn} (cn

is called consistent with order

tending to

{ cn}

(or

1c.1

-

consistent for short) if for every s >0 and every 9 E (9 there exist a sufficiently small positive number satisfying the following : lim sup n-> oo 0:

10 -81 < 5

a

and a sufficiently large number L

([1})

Pn,e {cnl0n-0I >L}
.

For each k=1,2,..., a {cn} -consistent estimator k is kth order asymptotically median unbiased (or kth order AMU) estimator if for any 9 E 0, there exists a positive number 8 such that lim sup n-->oo

0:

cn-'

10 -V1<5

Pn,0

=0;

{0„50} 2

lim sup cn-' Pn,l On >0} - I 0 2 n->O3 0:10-81
-lI pn

0{

cn ( 0n -0)
- G0(t,

0)- cn1(t, 0)- • • • -cn(k-1)Gk_l(t, 0 )1 =0

We note that Gti(t, 0) (i=1,-,k-1) may be generally absolute continuous functions, hence the asymptotic distributions for any fixed n may not be a distribution function. Suppose that en is AMU and has the kth order asymptotic distribution G0(t, 0)+c; 'G1(t, 0)+ • • • + c; (k-1 )Gk_1(t, 0). Letting 00 ( E 0) be arbi-

trary but fixed we consider the problem of testing hypothesis H+ : 0= (t> 0) against K: 0=00. Put p1/2 = { {l5 } : En, 0u +tcn1(cn) =1/2+ 0(c.; `k-1 )), 0 <0n(xn) < 1 for all xn E 'Cn) (n=1, 2, • • • )} . Putting Aa n , o0= 00 +tcn'

{cn(dn -00)St}, we have lim Pn, 0o+tcnl(Aen, 00) =1im Pn, 0o +tcnl {B

Oo +tcn'}

.

= 2

Hence it is seen that a sequence {XAan Bo} of the indicators (or characteristic functions) of Aen, o0 (n=1, 2,.. •) belongs to 01/2. If

155 42 MASAFUMI AKAHIRA AND KEI TAKEUCHI

sup lim c;-' {En,eo(¢n) -H+(t, B0)- c;,'H+ (t, Bo)( n) E P, /2 n-'°°

-cn (k-i>H+

i(t, Bo)} =0

then we have G0(t, Bo )< H+(t, Bo) ;

and for any positive integer j (;;5 k) if G(t, B0)=Hi+(t, 60) (i=1,• • •, j then

G,(t, e0)=H; (t, Bo) . Consider next the problem of testing hypothesis H- : 0=0o+tc;' (t
(on) E 01/ 2 n-'oo

m(k-1) Hk-i t, Bo)} =0

then we have G0(t, 0,)Z Ho (t, 0o) ;

and for any positive integer j (S k) if G(t, 90) = Hj-(t, 00) (i = 0, • • • , j then G;(t, 90)>H, (t, 00). 9n is called to be kth order asymptotically efficient if the kth order asymptotic distribution of it attains uniformly the bound of the kth order asymptotic distributions of kth order AMU estimators , that is, for each 0E H+(t, 0)

for t>0 ,

t Ht (t, 0)

for t<0 ,

G#, 0)=4 i=0, • • •, k-1 ([2], [11]). (Note that for t=0 we have G,(0, 0)=Hz+(0, 0) =Hj-(0, 0) (i=0,• • ., k-1) from the condition of kth order asymptotically median unbiasedness.) We assume that for each 0 E ® PO is absolutely continuous with respect to o-finite measure ,u. We denote a density dP5/da by f(x, 0). Let L(6; xn) be a likelihood function , that is, L(6 ;

_ f f (xt , 0), where .T~in = (x,, X2 ,** • , xn). For i=, each k=1, 2, • • • , a {cn } - consistent estimator Bn is called discretized likelixn )

hood estimator (DLE) if for each real number r, Bn satisfies the discretized likelihood equation (2.1) log L(Bn+rcn' ; xn) -log L(B,, ;

xn)=an (Bn,

r) ,

156 DISCRETIZED LIKELIHOOD METHODS 43 where an(0, r) is a function in 0 and r and it also depends on n. The function an(0, r) is not defined for the moment but will be determined in the sequel so that the solution obtained from the above equation be asymptotically median unbiased up to kth order . It should be noted that DLE Bn is required to be {cn }-consistent and we do not claim that the solution of the equation (2.1) be 1c. ) -consistent . We implicitely claim that there exists a solution of the equation in the 0(cn')-neighborhood of the true value. In the practical situation we have to obtain the DLE first by finding a {cn} -consistent estimator Bn in some way or another then find a solution of the equation in the neighborhood of On.

Suppose that for given function an(0, r),

(2.2) log L(0+ rcn' ;

xn)

- log L(O ;

xn)

- an(0, r)

is locally monotone in 0 with probability larger than 1- o(cn
cn-'IPn,B{en > 0-rcn' } - Pn,e[log L( 0, xn)

n -iC0 II: 1O-81
-log L(O -rcn',

'Cn) >an(e

-rcn', r)} 1 =0 .

Letting 00 (E 0) be arbitrary but fixed we consider the problem of testing hypothesis H : 0 = 00 - rcn,' (r> 0) against alternative K : 0=00. Putting Ae o= {cn(6n- 6)>-r} we have Pn ,o0 -rcn ' (Aen,o0)=1/2+o(cn'k-") Let QU k be the class of the all kth order AMU estimators . Set 012= ( {on }: En,eo- rc n ' (cn)= 1/2+o(cn`k-"), 0 o with the estimators Bn in `U k is contained in 0,/2. In SuffiPn,o0(Aen, o0) in CUk it is order to obtain the upper bound of lim ,

cient to find a sequence

{on }

of the tests which maximize HE E n,00 n -

(o

n)

It is shown by the Neyman-Pearson fundamental lemma that has the rejection Sn satisfying 01/2•

in

on

n log f( XX, 60-rcn')
where kn is some constant .

f(Xi, 00)

Then it follows from (2.3) that the upper

157 44 MASAFUMI AKAHIRA AND KEI TAKEUCHI

bound of lim Pn,BO(Aen,6O) in

V, n-.w

is given by lim En,6O n--

(on*).

Hence the kth

order asymptotic distribution of the DLE BDL attains the bound of the kth order asymptotic distributions of kth order AMU estimators at -r. In a similar way as the case when r>O, we also obtain for r<0 the upper bound of lim Pn,e0(Agn, e0) in V, of the same form. Hence the desired result also holds for the case r<0. In later sections it will be seen that BDL is asymptotically efficient up to second order. Note that the DLE usually depends on r in cases more than third order. In the subsequent discussion we shall deal with the case when cn =II/n.

3. Second order asymptotic efficiency Suppose that X,, X2i • • •, Xn, • • • is a sequence of i.i.d. random variables with a density f (x, 0) satisfying ( i) -(iv).

(i) {x : f (x, 0) > 0} does not depend on 0; (ii) For almost all x[p], f (x, 0) is three times continuously differentiable in 0; (iii) For each 0 E (9 0
{-j_-B - log .f (X, B)} 2]

E[

, 0)] < 0

( iv) There exist J(0)=Ee

[{ ae2-

log f(X, 0)} {

log AX, 0)} ]

and K(B) = EB L j aB

log .f (X, 0) I l

and the following holds : EB [_! log f(X, 0)] _ -3J(0)- K(B)

ao,

By the following way we have shown in [11] that an MLE is second order asymptotically efficient . Let B. be a maximum likelihood estimator. By Taylor expansion we have

O= tiZae log f(Xz , BML) =tE

0) ae log f(X^, +

{ae2 log f(Xi, B)} (BML-B)

158 DISCRETIZED LIKELIHOOD METHODS 45

E

a$

+1 2 ti=1 {ae2 log f(X£, o *)} (BML-B)2 , where 10*-015I0ML - 01. Putting T„=%/n(BML-o) we obtain U= 1 E a log Ax" 0)+ 1 {E aE log f(X" B)} T. t=1 ao n $=1 ao2 1 + 2n ^l n

a2 ae2

g f (X" , lo

o*) T„ .

Set Z1(o)= 1 E a log f(X" o) Vn =1 ao

Z2(o)=__L E { 1 n a2 log f(Xs, o) W(B)=- E n 1=1 aB Then it follows that W(o) converges in probability to -3J(6)-K(0)Hence the following theorem holds : THEOREM 1 ([11]).

Under conditions (i) ~ (iv)

) - 3J(o)+K(o) Zl(o)z+o^( 1 1 (9ML-0)= Zo(o) + f Z,(o)Z2(0 /nf I(0) N /W I(19)2 2,,/WI(O)-

(3.1)

up to order n- 1/2 as n-> oo. Put

BML

-BML+

K(&

)

6nI(0ML)2

Then 6ML is second order AMU. From Theorem 1 we have established the following : THEOREM 2 ([11]). Under conditions (i)-(iv), BML is second order asymptotically efficient.

In the sequel we obtain the same result for DLE. We further assume the following : (v) For given function a„ (o, r) log L(o+rn 1"2, z„)-log L(o,

is locally monotone in 0 with probability larger than 1-o(n 1). Remark.

In usual situation it is generally true since (1/n) (a2/ao2) .

159 MASAFUMI AKAHIRA AND KEI TAKEUCHI

46

log L(9, xn) is asymptotically equal to -I(9) (<0) and an(e, r) is smaller order than n-1, and is usually of constant order (0(1)) as is shown below. Let Bn be an DLE. Since n

n

Z log f(XX, en+rn-1/2)-j log f(Xi, en)=an ,

i=1

t=1

it follows by Taylor expansion that n

n Z ae

n

log f ( Xi , en)

2

+ 2n

Z

CF

log f(Xi, en)

a3 log f (Xi , e*) = E aeg

r2 n

+ 6n n

an

,

where

l0*-

enI <

.

r ,VW

Since (1/n) Z

{(a 2lae3) log f(Xi, 9)}

converges in probability to - 3J(0)-

i=1

K(0), it is seen that

(3.2) Z, (en)+ 2

Z2(en)} { I(an)+ ^n 6 r

{3J( Bn) +K(en)} +O ( 1 ) = an

On the other hand we have

(3.3) Z1(Bn)=Z1(0)+ 1 {Z2(0)-'%/ I(e)}Tn- 3J(0)+K(0) T7 +0P( 1 ../n 2.%/W ,V n) where T n=../ n (&-9).

Since

Z1(en)=Z1(e)+ 1 Z2(e)Tn- I(B)Tn./n

3J(6) +K(0)

2.,/W

T,^+op(

it follows that

(3.4)

Tn= I^B) {-Z1(en) +Z1(0)+

_n

Z2(0)Tn

_ 3J(6) +K(0) Tn +oP 1 2.v/-n ,/n Since I(en)=I(6)+ 1 12J(e)+K(6)}Tn-I-op( 1 4./n 4./n

1

),

160 DISCRETIZED LIKELIHOOD METHODS

47

J(&)=J(0)+ J'(B) T.+op( 1 )

K(BD) =K(8)+ 1 K'(6)T„ +oP( 1

)

Z2(e)=Z2(B)-J(B)T.+o'(1) it follows from (3.2) that

rn + 2

r {3rJ(B)+ rK(0)-3Z2(0)} +6 v/W

+ 2 / i {3J(0)+K(0)}T. +o, ) up to order n- 111.

From (3.4) we have

T.=_ an - r2(3J+K) + Z, + ZIZ2 - (3J+K)Z,2 +op r 1 ) rI 6I-1 n I I2.^/ n 21'.,/-n up to order n-112, where a„=-(rI/2)+an with an'=o(1/.,/n) so that B„ be AMU. In order to have second order asymptotic median unbiasedness of 9. we put r`(3J+K) _ rK +o\ 1 ) a`= 6-/n 61.,/W ^n and we denote by T,* the corresponding T. to this particular value of a..

Then we may also denote Then it is shown that

k is second order AMU. Hence we have established the following theorem. THEOREM 3. Under conditions (i) - (v), the DLE 6,* with a. defined above satisfies the following : T,* N/ n (en -0)

K + Z1 (0) + Z1(B)Z2(0) - 3J(0) +K(0) Z,(0)2 2I(0)a.,/ I(0)2. VW 6124,/n I(0) + r I Z2(0)- 3J(B)+K(B) Z1(0)} n +o( 1l ) n) 2I(0)^/ n I(0) ^/=

up to order n- 1/2 as n--> oo. Remark.

Since we have

EB [Z1(0) {z2(o) - 3J(0)+K(0) Zl(e)11=0 JJ I(0)

161 48 MASAFUMI AKAHIRA AND KEI TAKEUCHI

V 0 (T,*)= I(B) +o\.^l n l , and the third order cumulant is equal to that of the MLE up to the order n- '/2. Hence the asymptotic expansion of the distribution of T,* n- 1/2. Therefore the is equal to that of ^/n(9ML-B) up to the order DLE B,* is asymptotically equivalent to the MLE BML up to that order.

4. Third order asymptotic efficiency We proceed to the problem of third order asymptotic efficiency. We further assume the following : (vi) For almost all x[p] , f (x, 0) is four times continuously differentiable in 0; (vii) there exist L(O)=E, L as s log .f (X, 0)} aB log .f (X, B)}

a M(e)=EB I 1-66-F log

N(e) =E8

a 22

.f(X, }2]

log AX,

I

B)} ae log f(X, 0)}

and

f(X, 0)} 4] H(e) = EB 11ae log and the following holds : EB N log f (X, 0)] 4L(0) - 3M(O) - 6N(6) - H(0) O-14 Let 9n be an DLE. Since

E lo g f

(Xi, en+ / n ^ i log f (X4, en)=an ,

it follows that E a log f(Xi, B..)+ r E 82 log AX, Bn) 2n i=1 a82 ^/ n i=1 aB a4 as ^,3 n r2 +

an

r

E-

log f (Xi , Bn) +

E

6n./ n i=1 8B3 24n2 i=1 _FO

log f (Xi ,

6 *)

162 DISCRETIZED LIKELIHOOD METHODS 49 n

(1/n) Z (a'lao') log f(X;,, 0) converges in i=1 probability to -4L(0)-3M(0)-6N(6)-H(0), it is seen that

where

I0*-o1
Since

Z1(on)+ 2 { -I(B&)+ _ + 6n Z3(B

Z2(Bn)}

ban

{3J(9n) +K(on)}

4L(9n )+ 3M(&)+6N(Bn)+ H(Bn)}_ r , n)- 24n {

where Z3(B)= 1

' {iog f(x, 0)+ 3J(8)+K(0)}

Hence (4.1)

Z,(on)- rn + 2 I(Bn)- rn Z2(&)+ 6rn {3J(en)+K(&)} - 6n Zs(On)+- 24n {4L(Bn)+3M(Bn)+6N(Bn)+H(en)}

On the other hand we have

ZO(B )=Z1(o)+ 1 {Z2(Bn)-1/nI(0)}Tn- 3J(o)+K(o) Tn2 2 ^/n n VW 1 - 6n {4L(0)+3M(o)+6N(o)+H(0)}Tn +o^(n) where Tn=./n(B -o).

We obtain

(4.2) Tn= Z1(&)+Z1(e)+ 1 Z2(0)T.- 3J(o)+K(o) Tn I(o) ^/ n 2^/n - 6n {4L(o)+3M(o)+6N(o)+H(0)} Tn ] +oP(n) . Since

I(6n)=I(o)+ ^n {2J(0)+K(0)}TT -L) + 2n {2L(o )+ 2M(9)+5N(0)+H(0)}T„+ oP(ry1 J(en)=J(o)+ 1 {L(o)+M(o)+N(o)}Tn+op 1 ) ; ^/n K(en)=K(o)+ 1 n {3N(o)+H(o)}Tn+op( ) ; %/ 4/n

163 MASAFUMI AKAHIRA AND KEI TAKEUCHI

50

L(on)=L(O)+ 1 L'(e)T.+o,, 1 ) 4/n ^n

M(9n)= M(o)+JM'(o) TT

+o1, (^n)

N(6n)=N(o)d ^n N'(o)Tn+o^(_n) H(en)=H(e)+ 1 H'( o)TT+op(,V1n) Z2 (en)=Z#)-J(O)T. -

1

{2L(o)+M(o)+N(o)}T„+o p ( 1 ) 4/ n

and ZOn)=

Z3(B)Tn +op ( 1 )

it follows from (4.1) that Z,(9n)= an + I- r Z2+ r2(3J+K) _ r2 Z3+ r (3J+K)Tn r 2 2.,/W 6^/n 6n 24/n 3 24n (4L+3M+6N+H) r (4L+3M+6N+H)T„ +

+ 6n (3L+3M+6N+H)Tn+op(1) up to order n-1 as n--+oo. Under conditions (i)-(vii) we have from (4.2) (4.3)

Z1 +

rZ3 r3 11 rI 2 61.,/W I 21.,/W 61n 241n

T _ _ an

r r2(3J+K) +

rZ2 +

r (3J+K)Tn- r2L' Tn+ Z2 T.- 3J+K Tn 21.,/ n 21n I^/ n 21,^/ n r

2_

.LTn +o( 41n `rT n 61n p n =-an'+ K +Z,+ r (Z2_ 3J+KZ)+ Z,Z2 rI 6I .^/n I 21./n I I.,/n - (3JI3^ )Z2 + I L {Z1Z2 -(3J+K)Z12Z2- 6 41 + (3J+K)2Z1 _ (3J+K)Z1Z2 _ (3J+K)KZ1 + KZ2 212 21 61 6 _ r(3J+K) _ r2I(3J+K)Z2 + r2(3J+K)Z1 12 4 4

164

DISCRETIZED LIKELIHOOD METHODS 51

_ 3r(3J+K)Z1Z2 + r(3J+K)2Z2 + rIZ2 _ r2LE'Z1 2 41 2 2 +r(3J+K)2Z,2-r_fZI}+o(1) 21 4 _ n up to order n-1 as n--+oo, where a,=-(rI/2)-r3(3J+K)/61./n+rK/ 6I,/n +a;' with anl'=0(1/n) and 1=4L+3M-l-6N+H and £'=3L+3M +6N+H. We denote by T,* the corresponding T„ to this particular value of an. Then we may also denote T, * _ ,y/ n (9n* - 0). On the other hand under conditions (i) - (vii) we have obtained in [11] (4.4) /n(BML-0)= Il+I21

+ II { +

fz1z2-(3J2K)Z'2} ZiZ3 -

3J2jK Z

12Z2

(3J+K)2 3 _ 4L+3M+6N+H713} +o ( 1) Z, , n 61 2I

Let 0ML be a modified MLE so that it is third order AMU. Comparing (4.3) with (4.4), we see that .v/n(9ML-0) and T,* are essentially different in the order n-1. We put TML =.v/n(eML-0). Note that the difference of T L and T,* appears in the linear term of order n-1/2 and other terms of order n-1. It is shown in [12], [13] and [14] that the asymptotic cumulants are determined up to order n-1 by the terms of order up to n-1/2 if the first term is equal to Z(0)/I(0) ; and the fourth order cumulant is identical in the first term of order n-1 for all asymptotically efficient estimators. In the third order cumulants we have K3(T -E,(TL ))_ a(0) +

o(n)

K3(T,*-E, ( T.*))_ ^3 (B) + r3(B)+o(1) A/n n in) Therefore there is a difference of asymptotic distributions of "/ n (aL0) and T,* in the term of n-1 if r3(O) *0. If we denote by Theorem 3 and (4.3) T*= Z_ J+K + r L*+ 1 (Q- c)+ 6I2^/n / I ^/ n n R+op(nI) 1

L* 2I (Z2_ 3J +K K z)

165 52 MASAFUMI AKAHIRA AND KEI TAKEUCHI

Q= I2 (Z1Z2- 3J21K Z^2)

C=- J+K 212 We decompose that (4.5)

P3(B) + r3(e) +0(1) _ - 1 Eo (Z,3) + 3 Eo [Z1(T,*-Eo (T,*))2] .Vln n n 2I3 21 + 21n Eo [&(rL*+Q-c)2]+o(n )

Note that Eo (Q) = c. We have Eo (Z3) = K/4/n . For the second term of the right-hand side of (4.5) we use the following lemma (see [12], Lemma 4.1). LEMMA . Suppose that X1, X2, • • • , X,,, • • • is a sequence of i.i.d. random variables with a density f (x, B) which is differentiable in 0 for almost all x[u]. Suppose further that To is a .`"'-measurable function on 6''"' into R1 and differentiable function of 0 and that

B

EB (T6)=

J (") ao

IT, trr

f(xt, o)}

sT

du( x,)

holds. Then we have 1 a E, (To)- 1 Eo (aTo ) ^ n ao V/ n ao

Eo (Z1 To) =

By the lemma we obtain

(4.6)

Eo [Z1(T,*-Eo

(Tn*))2]

1 a Vo( Tn*)- 2 Eo [(T,*-Eo(T,*)) "/n ao ao

(T,*- Eo (Tn ))

ao (1) +n K i I z.,/+n +0 (n)

We also obtain (4.7)

Eo [Z,(rL*+Q-c)2] =2r Eo [Z^L*(Q-c)]+o(1)= r I M_ (3J+K)K l +0(1) 212 J I I

166 DISCRETIZED LIKELIHOOD METHODS 53

From (4.5), (4.6) and (4.7) we have r3(e) __ 3r [ M(0) - {3J(0) +K(0)} K(0) 21(0)2 L 1(0) 21(0)2 Since the third order asymptotic distribution of T,* attains the bound of the third order asymptotic distributions of third order AMU estimators at r, there is no third order AMU estimator which uniformly attains the bound, if 73(0) is not equal to zero. Hence we have established : THEOREM 4 . Under conditions (i) - (vii), BML is not third order asymptotically efficient if r3(0) *0.

5. Maximum log-likelihood estimator Instead of the equation (1.1) we may take a solution an of the discretized likelihood equation : n

n

(5.1)

Z 1ogfX,

i=1

Bn

+rcnl) -E lo gf( Xi, Bn- rcn1) =0

i=1

The solution an is 0 which maximizes rcnl n

Z log f(Xi, 0+t)dt=

J -rcnl 2 = 1

rcnl _rcml

log L(0+t)dt

where L(0) denotes the likelihood function. Then an is called maximum log-likelihood estimator (MLLE) ([3]). If log L(an+t) is locally linearized, an agrees with maximum probability estimator by Weiss and Wolfowitz ([15]). Now we shall consider a location parameter case when the density function f satisfies the following assumption : (viii) f(x, 0)=f(x-0) and f(x) is symmetric w.r.t. the origin. It follows by the symmetricity of f that the solution en of (5.1) is AMU. We also have

J(0) =K(0) = 0 . Let an be an MLLE. Then n

n

E logf(X,- an- (r/s/n))-Z logf(Xi- an +(r[/n)) =0 . i=1 i=1 Without loss of generality we assume that 00=0. Since n

n

Z log f(X -0n-(r/ '✓ n))-E log .f (Xi) i=1 i=1

167 54 MASAFUMI AKAHIRA AND KEI TAKEUCHI

I l =0 Z log f(Xi-on+(r/ ✓ n))-^ i=1 i=1 log f(X1), it follows by Taylor expansion around 0=0 that 2r :

[-L log f (Xt _ 0)

n

ao {

=1

J ,=0

{ 2r6,, .= n

a2

log .f (X' - o)

F L ae , Z

J 0=0

+ (_!1 t log .f (X _ 0)] e o ,9n' + 3nr3 n / L aB^ +lrren+ A lra' log AX,-0)]a_o~0 , 3 \^Pn n^/n /LaB' where 10* I S 10I . Putting T„ = .// n B,, , we obtain 2rZ,+2r (^n -I(0))Tn+{ Z3- /n(3J(0)+K(0))}

(.712 + 3 r )•

{4L(0)+3M(0 )+ 6N(0)+H(0)}

r T + r3 Tn0 ) . n

n

n

Hence it follows that Tn=

+ Z1Z2 ) +

r2 1%/ n I I2n 61n Z2 (Z1

Za+

1

21n

Zg

Z.'

I2

3 6In (4L + 3M+ 6N+ H) ( Ig

+r2 Il +op(n .

Under conditions (i) ~ (vii ) we have (5.2)

Tn=

Il + IZZ n + I n

{Z1Z2 + 2 Zi Z3- _I ( 4L+3M+6N+H)Z3}

+ rZ3 r2(4L+3M+6N + H) Z1+op 6In 612 n (in) up to order n' as n-> oo. As was stated preciously the difference in the order n-1 term between (4.4) and (5.2) does not affect the asymptotic distribution up to the order n1. Hence we have established : THEOREM 5.

Under conditions (i) ~ (iv) and (vi)-(viii), the MLLE

9n is asymptotically equivalent to the MLE 6ML up to order n1. It is shown in [12], [13] and [14] that BML maximizes the symmetric probability Pn,8 {4/nf BML-OI
168 DISCRETIZED LIKELIHOOD METHODS 55

up to the order n-1 among all regular AMU estimators. Therefore it is seen that

lira n[PP,g {,/ n ^BD1L -01 _0 n-oo

for all u, where 9DL denotes DLE. The asymptotic distribution of the MLLE OMLL is equal to that of BML up to the order n 1, but not in the n-3/2 and we can show that

lim n3'2[Pn,g { ,/n 1BMLL -01 _0 n_oo in general situations. Hence for symmetric intervals 0MLL is not fourth order asymptotically efficient.

6. Conclusion remarks on discretized likelihood methods" If we also define as asymptotic efficient estimator as

On

which max-

imize lim Pn,B {cnlOn -Oj
ing the following equation (6.1) is asymptotically efficient : (6.1) i

-1

f(Xi, Bn+rcnl) f(XX, On*-rcnl) -a f(XX, 0*) - n f(X1, Bn) t-1

where an is chosen so that B* is AMU. Then (6.1) is also expressed as (6.2) exp {log L(6,*+rcn1)-log L(B*)} -exp {log L(9*-rcn1)-log L(B*)} an . If exp (log L) is linearize , then (6.2) is reduced to the type of (5.1), where in this case the right-hand side of (5.1) is not always zero. Hence it is seen from the above that the asymptotic efficiency (including higher order cases) may be systematically discussed by the discretized likelihood methods. Further the discretized likelihood methods may be also applied to the statistical estimation theory of the fixed sample size. UNIVERSITY OF ELECTRO - COMMUNICATIONS UNIVERSITY OF TOKYO

REFERENCES [ 1 ] Akahira, M. (1975). Asymptotic theory for estimation of location in non -regular cases, I: Order of convergence of consistent estimators, Rep. Statist. Appl. Res., JUSE, 22, 8-26. [ 2 ] Akahira, M. and Takeuchi, K. (1976). On the second order asymptotic efficiency of 1) This section is based on Akahira and Takeuchi [3].

169 56 MASAFUMI AKAHIRA AND KEI TAKEUCHI estimators in multiparameter cases, Rep. Univ. Electro-Comm., 26, 261-269. [ 3 ] Akahira, M. and Takeuchi, K. (1977). Asymptotic properties of estimators obtained from discretized likelihood equations , Annual Meeting of the Mathematical Society of Japan. [ 4 ] Chibisov, D. M. (1972). On the normal approximation for a certain class of statistics, Proc. 6th Berkeley Symp. Math. Statist. Prob., 1, 153-174. [ 5 ] Chibisov, D. M. (1973). Asymptotic expansions for Neyman's C(a) tests, Proc. 2nd Japan-USSR Symp. on Prob. Theory, (Lecture Notes in Math. 330) Springer-Verlag, 16-45. [ 6 ] Efron, B. (1975). Defining the curvature of a statistical problem (with applications to second order efficiency), Ann. Statist., 3, 1189-1242. [ 7 ] Ghosh, J. K. and Subramanyam, K. (1974). Second order efficiency of maximum likelihood estimators, Sankhya, A, 36, 325-358. [ 8 ] Pfanzagl, J. (1973). Asymptotic expansions related to minimum contrast estimators, Ann. Statist., 1, 993-1026. [ 9 ] Pfanzagl, J. (1975). On asymptotically complete classes , Statistical Inference and Related Topics, Proc. of the Summer Research Institute on Statistical Inference for Stochastic Processes, 2, 1-43. [10] Pfanzagl, J. and Wefelmeyer, W. (1978). A third order optimum property of the maximum likelihood estimator, J. Multivariate Anal., 8, 1-29.

[11] Takeuchi, K. and Akahira, M. (1976). On the second order asymptotic efficiencies of estimators, Proc. of the 3rd Japan-USSR Symp. on Prob. Theory, (Lecture Notes in Math. 550) Springer-Verlag, 604-638. [12] Takeuchi, K. and Akahira, M. (1978). Third order asymptotic efficiency of maximum likelihood estimator for multiparameter exponential case, Rep. Univ. Electro-Comm., 28, 271-293. [13] Takeuchi, K. and Akahira, M. (1978). On the asymptotic efficiency of estimators, (in Japanese), A report of the Symposium on Some Problems of Asymptotic Theory, Annual Meeting of the Mathematical Society of Japan, 1-24. [14] Takeuchi, K. and Akahira, M. (1979). Third order asymptotic efficiency of maximum likelihood estimator in general case, to appear. [15] Weiss, L. and Wolfowitz, J. (1967). Maximum probability estimators, Ann. Inst. Statist. Math., 19, 193-206.

170 Ann. Inst. Statist. Math. 31 (1979), Part A, 403-415

ASYMPTOTIC OPTIMALITY OF THE GENERALIZED BAYES ESTIMATOR IN MULTIPARAMETER CASES KEI TAKEUCHI AND MASAFUMI AKAHIRA

(Received July 28, 1978; revised Oct. 24, 1979)

Abstract The higher order asymptotic efficiency of the generalized Bayes estimator is discussed in multiparameter cases. For all symmetric loss functions, the generalized Bayes estimator is second order asymptotically efficient in the class A2 of the all second order asymptotically median unbiased (AMU) estimators and third order asymptotically efficient in the restricted class D of estimators. 1. Introduction The expansion of a generalized Bayes estimator with respect to a loss function of the type L(B)=IOId (a>1) is obtained by Gusev [5]. His result can be extended to all symmetric loss functions. Strasser [8] also obtained asymptotic expansions of the distribution of the generalized Bayes estimator. In one parameter case the second order (or third order) asymptotic efficiency of the generalized Bayes estimator has been discussed by Takeuchi and Akahira [12]. It is shown in this paper that in multiparameter case for all symmetric loss function the generalized Bayes estimator d is asymptotically expanded as

^/n(e-B)=U-2^n I-IV+Vn I-IW+oP\^n where the symbols of the right -hand side are defined in the contexts. And the asymptotic distributions of the estimators are the same up to the order n-I except for constant location shift . Therefore if it is properly adjusted to be asymptotically median unbiased , it is third order asymptotically efficient among the estimators belonging to the class D ([3], [4], [11]) whose element d is third order AMU and is asymptotically expanded as 403

171

404 KEI TAKEUCHI AND MASAFUMI AKAHIRA

s/n(9-0 )=U+^n Q+op\"An l

and Q, = Op(1) (a = 1,---,p) and E [UQ' ]= o(1) (k=1 , 2) for all a, p=1, • • • , p, where E denotes asymptotic expectation and U= ( U1 , • • • , Up)' and /

Q= (Q, ,

... ' Qp)'.

2. Results Let (a', B) be a sample space . We consider a family of probability measures on B, ) = {P0 : 0 E 01, where the index set ® is called a parameter space . We assume that 0 is an open set in a Euclidean pspace R' with a norm denoted by II • II. Then an element 0 of 0 may be denoted by (01 i • • • , Bp). Consider n-fold direct products (1'`n), B`n)) of (X, B) and the corresponding product measures P' (n) of P0. An estimator of 0 is defined to be a sequence {B.} of _B (n)-measurable functions Bn on X(n) into ® (n =1, 2,. • • ). For simplicity we denote an estimator as d instead of {6n} . Then may be denoted by (61 f • • • , d,). For an increasing sequence of positive numbers {cn} (cn tending to infinity) an estimator is called consistent with order { cn} (or {cn}-consistent for short) if for every s>0 any every 9 E ® there exist a sufficiently small positive number 8 and a sufficiently large number L satisfying the following : lim sup P1n) {cnlle -0II _L} <s ([1]) . 0:10 -SI1<8

n-too

For each k=1,2,---, a {cn}-consistent estimator d is kth order asymptotically median unbiased (or kth order AMU) estimator if for each 9 E ® and each a =1, • • • , p, there exists a positive number 8 such that c,-1 Pan){B 0}-2

=0;

c,-ll Pen) {Ba>_BQ}- 1 lim sup n-» 0:110-s11<8 2

=0.

lim sup n--

0: 110-911<3

For B kth order AMU, Go(t, 0)+cnIG1(t, 0.)+ • • ' cn
lim c,-1I PBn){cn(Ba-ea)
cn 1G1( t, Ba)

... -Cn (k -1)Gk

-1(t,

0,,)I

=0

We note that G#, 0.) (i=1,..., k -1; a=1,---,p) may be generally

172

ASYMPTOTIC OPTIMALITY OF THE GENERALIZED BAYES ESTIMATOR 405

absolutely continuous functions , hence the asymptotic marginal distributions for any fixed n may not be a distribution function. Suppose that d is kth order AMU and has the kth order marginal asymptotic distribution G0(t, 0.)+cn'G1(t, 0.) + • • +cn ( k-')Gk_1 (t, O) (a=1, •, p) and the joint distribution of B admits asymptotic expansion up to kth order , i.e., the order of en ( "'. Letting Bo (E ®) be arbitrary but fixed. Denote Bo by (Bol, • • • , Bo,). Leta be arbitrary but fixed in 1, • • • , p. We consider the problem of testing hypothesis H+ : Ba=Boa+tcn' (t < > 0) against K : Oa = Boq . Put 01/2 { On} ; El0a+1,n1(0n) =1/2 + o(cnck-"), 0 _ ¢n(xn)51 for all x„ E _'`"' (n=1, 2,. • .)}. Putting Adq,00a={cn(Ba- Boa)St}, we have

lim Poa'+n 1(Aea,60.)=lim Poa+ i^n^ {BaSBoa-^ tcn'} = 2 Hence it is seen that a sequence {XA L, 0oQ } of the indicators (or characteristic functions ) of ABa, eoa (n=1, 2,. • •) belongs to 01/2 If sup lim cn -' {E(n) t, Boa) - cn 1H,+(t, Boa) - .. 80. (O n) - H+(

{^n}Eo ,a n-soo

n

(k-1)H+

i(t,

BOa)} -0

then we have

Go(t, Boa) s H+(t, Bo.) ; and for any positive integer j ( S k) if Gi(t, Boa ) = H,+(t, Boa ) (i =1, • • • , j-1) then G;(t, Boa)=Hj+(t, Boa) • Consider next the problem of testing hypothesis H- : BQ=Ba„+ tc-1 (t<0) against K : Ba = Boa . If sup Lim c'n -' {Edna (0n) -Ho (t, G0 )-cn'HC( t, Bon)- .. .

{!6n }Eo1

,2

n-->oo

-cn(k-I)Hk l(t , Boa)} =0

then we have Go(t, O..) Z Ho (t, Boa) ;

and for any positive integer j (< k) if Gilt, Boa) = H1 (t , Boa) (i =1, • • • , j-1), then G1(t, Boa)>HH (t, Boa). B is called to be kth order asymptotically efficient in the class Ak of the all kth order AMU estimators if the kth order asymptotic marginal distribution of it attains uniformly the bound of the kth order asymptotic marginal distributions of kth order AMU estimators, that

173

406 KEI TAKEUCHI AND MASAFUMI AKAHIRA

is, for each a =1, • • • , p

HI(t, O)

for t>O ,

Hz (t, O)

for t
i=0, • • •, k-1 ([2], [4], [9]). [Note that for t=0 and each a=1, • •, p we have Gi(0, 0.)=Hi+(0, O)=Hi- (0, 9Q) (i=0,• • •, k-1) from the condition of kth order asymptotically median unbiasedness.] B is called to be third order asymptotically efficient in the class D if the third order asymptotic marginal distribution of it attains uniformly the bound of the third order asymptotic marginal distributions of estimators in D. It is generally shown by Pfanzagl and Wefelmeyer [6], [7] and Akahira and Takeuchi [3], [4], [10], [11], [14] that there exist second order asymptotically efficient estimators but not third order asymptotically efficient estimators in the class A8. But it was also shown in [4], [10] and [11] that if we restrict the class of estimators appropriately, we have asymptotically efficient estimators among the restricted class of estimators and that the maximum likelihood estimator belongs to the class of higher order asymptotically efficient estimators. We assume that for each 0 E ® PB is absolutely continuous with respect to a-finite measure p. We denote a density dPB/dp by f(x, 0). n Then the joint density is given by T[ f (xi, 0). i=1

In the subsequent discussion we shall deal with the case when cn = ..l n . Let ®= RP. Let Ln(u) (u = (U1, • • • , Up) E RP) be a bounded nonnegative and quasi-convex function around the origin, i.e. for any c the set {u : L(u) S c} (c RP) is convex and contains the origin and ir(0) be a non-negative function. Define a posterior density pn(0I xn) and a posterior risk rn(d I xn) by

and

rn(d I xn) _ ^e Ln(d - 0)pn(0 I x n)d0 , respectively, where xn = (x, , ,

=L*(u) for all u E RP.

xn).

Now suppose that lim Ln(u /.,/ n )

We define

r*(d I x%) = J L *(-./ n (d- 0))pn(0 I xn)d0 . An estimator B is called a generalized Bayes estimator with respect to a loss function L* and a prior density it if

174 ASYMPTOTIC OPTIMALITY OF THE GENERALIZED BAYES ESTIMATOR 407

r:(BIxn)=inf rn(dIxn) dE8

Then i =../ n (9 - 0) may also be called a generalized Bayes estimator w.r.t. L* and a. Since Ln(d-0)5(0)d0-inf L*(%/n(d-0))5(0)d0l = 0

limlinf

dEB 8

n-- dE8 9

uniformly in every posterior density p(6), it follows that for a generalized Bayes estimator

limlinf rn(djxn)- r, (6 xn)j =0 n-•O

dE8

Suppose that X1, X2, • • • , Xn, • • • is a sequence of i.i.d. random variables with a density f (x, 0) satisfying (i)-(iv).

(i) {x: f(x, 0)>0} does not depend on 0. (ii) For almost all x[,u], f (x, 0) is three times continuously differentiable in 0. (a =1, • • • , p). (iii) For each a, R (a, j9 =1, • • • , p) 0
log f (X, 0) < 0 _ - EB I a2

aoaa6^

(iv) There exist Jad•r(6) =E8

LI

a2

a6Qaos

a log f(x, 6)} log f( x , 6)} {__

K(0)=E a L a log f(x, 6)} { a log f(x, 6)}

a6a a6^

• l a6, log

f(x, 6)} ]

and War(6

) = EB L a6sa6^a6,

log f (X, 0)]

and the following holds : Ee [ a9 log AX, 0)] _ -J ^.r(6)-J,.s(6)-J^r.a(6)-Kaar(6)

a6Qa6pa6r

It was shown by the same way in [9] that a maximum likelihood estimator (MLE) is second order asymptotically efficient. Let B be an MLE. By Taylor expansion we have

175

408 KEI TAKEUCHI AND MASAFUMI AKAHIRA

0=z E aea log.f(XX, B)

a2 log e) +E { :^ log f(X¢, o)} a aeaf(X1, a }z aoaaoo as oa)(Bo -oo + 1 E E E {E aoaao0aer log f(X=, o*)} (Ba-Ba)

(ea-

2 a a 7

(Ba-

'^

Ba)(Ba- 00)(87-B7)

where SI0*-0IISII9- 8!I. Putting T=./n(B-B) we obtain E {a2 log f(X{, e)} TTT, o= 1 {^ 8 log f(Xi, B)} Ta+1 ^/ n a a Boa n s s aoaaop + 2n ✓n

E E E {Z adaa'saor

where T=(T1f • • ., TT)'.

log f(X,, B*)} TTTfT7 ,

Set

Z-(8) =1/1n Z(O) 1

a '=1

log AX, B)

aBa

82 log f(XX, B)+Ia(B)

177 i=1 aeaae^

0) converges in probability to - {J p.r(B)+J 7.fl(B) Then it follows that W,,,( + Ja7•a(0) + Kaer(B )}. We Put Paa7(B) = J fi.7(0) + Jar-P(O) + Ja7.a( B) + K dr(0). Hence the following theorem holds : THEOREM 1.

Under conditions (i)-(iv) 2^n I-1V

+.^n I-1W+op(-V/1n) ,

where I= (I p) and P= (Pa,) are matrices and V= (Z pap, Up U), W= d r

(3^1, UZpr) and U= (U) are p- dimensional column vectors and U. = I "PZZ and Ia" denotes the (a, p)-element of the inverse matrix of the information matrix L Since the proof of the theorem is essentially same as that of one parameter case ([9]), it is omitted. Put

1 Y, where Y= (Z Up W . p,) is a column vector . d r

Then 9* is second order

176 ASYMPTOTIC OPTIMALITY OF THE GENERALIZED BAYES ESTIMATOR 409

AMU. From Theorem 1 we have established the following : THEOREM 2. Under conditions (i)-(iv) 9* is second order asymptotically efficient in the class A2.

Since the proof of the theorem is essentially same as that of one parameter case ([9]), it is omitted. It will be shown that the generalized Bayes estimator w.r.t. a loss function and a prior density is second order asymptotically efficient. Let Bo be a true parameter of B (E 9). Denote 0 and Bo by (B1, • • •, Bf,)' and (001f • • • , Bo,)' respectively. Further we assume the following : (v) For each a= 1, • • • , p, ir(B) is twice partially differentiable in Ba . Then we have pn(B I xn )/Pn(Bo I xn)

= exp [log {pn(B I xn)/pn(0 I FOX } = exp [ElOgif(X,,O)lf(X,,00 )1+10gl7r(0)17r(OO)I 1 i_1 J =exp

^

log f(xE, e) -Z 1

[, =eXP

I Y"

a i=1

Z=1

log f(x{, Bo)+log 2r(B)-log 2r(Bo)]

8 lo g f(xti, aea

Bo)}(Ba

-Boa)

2

a log f(x i, 0o)1 + 1 E E {E t=1 aoaaop

(0a-

2 a a

+6

E

{yE

aBaao^aB,

lo g

00 (00 -000

f(x" B*)}

EQ(B? • (Ba-Boa)(Ba-B(,p)(Br-B(r ) + Ea 7r(Bo) Boa)+o(1 )] -✓ n (oa-

=exp [v'i

E Z (Bo)(Ba -Boa) + 2 Ea E { ✓ nZ a(Bo) -nI a(00)}

• (Ba-BOa )(Ba-BOa )+ C / V

• (Br-Bor )

+E

E E E Waar( 0*)(0a-0Oa ) ( 0a -000

(Ba- BOa ) +oI

11

i(Bo) where 7C a(B)=air(B)/a0a (a =1,• • •, lp). It follows that pn(B I xn)/pn( 0 I xn) = eXp

E E { Zaa( Bo) _ Iaa(0 )} [Ea Z (BO) { ✓i (Ba -Boa )} + 1 2 a a ^/n {n(Ba-B0a )( Ba-Boa

)} - 6.,/n

E E Paar(Bo)

• {n^/n(Ba- Baa)(O -B°^)(B,-eo,>}

177

410 KEI TAKEUCHI AND MASAFUMI AKAHIRA 1

E ^a(B o){^/n(Ba-Boa )}+0,(,/n a

+ -✓ n7r (Bo)

Putting tQ = ,/-n (Ba - Boa ) (a =1, • • • , p) we obtain (1) pn(O I xn)/P..(eo I FO-) =exp [:p ZZ(O)t,+ 1 E E {Z-P(00) -IP( Oo)}tats 2 a P N/ n P tr r Pr 09o)tat

- 6.V/W - 5

+.Vln7r(Bo) ^a =eXp [-

2EEI

(eo)t.-FoP(1/n ) I

(O) (t.- U)(tP-UP)+

^a(Bo)ta +

+ 1

-,/n7c(Oo)

1

2 E IP(Bo)U UP

E E Z P(Bo) t atP

n P l1 E P-Pr(0O)tgtPtr+OP( 1 )] n a P T N/n 6.v/-

= (exp 2 E E I #o) U. UP) • [exp {-

E E I A00) (ta - U) (tp - UP)} ] __ 2 E 7r a(eo) ta+ 1 E E Z P(O0)t.tP • exp { 1 V/ -n ic(Oo) a n a P 6 1n E E E P.WOO)tatPtr + Op(

'^/lw ) }

_ (exp 2 E E I #o) U. UP)

• [cxp {- 2

E E IP(eo)(ta- Ua)(tP -UP)

E na(O0)t.+ 1 E E ZZ P(O0)ta tp n n(O0) a 2^/ n a P

{

E E E P-WOO )VA1 +Op( 1 )} 6,/ n a P r ^/ n =4'.(t, Bo 1x.)

where t = (t, , • • • , tp)'. given by (2) r. *(t I x.) _

(say)

Let t = ^/ n (6 - Bo).

^n

p.(Bo 1 xn)

5

Then the posterior risk is

L*(t-t)q„ (t, 60 I x„ )dt

178

ASYMPTOTIC OPTIMALITY OF THE GENERALIZED BAYES ESTIMATOR 411

Further we assume the following : (vi) L*(u) is a convex function ; (vii) For each a=1, . • • , p,

5

L*(-u)gn(u+t, Bo I xn)du is continuously

partially differentiable with respect to to under the integral sign. By (2) and the assumption (vi) it is shown that the generalized Bayes estimator t w.r.t. L*(.) and 7r(.) is given as a solution u of the equation

1

(d/dua) L*(u- t)gn(t, Bo I xn)dt = 0

(C'= 1'. - -, p) ,

Since by (vii)

d

dua

5

L*(u-t) gn(t, Bo I xn)dt d dua d

dua

5 5

L*(- t)gn(u +t, L*(- u)gn(t +u,

001

xn)dt

Bo 1 xn)du

ff +u, Bo^ x n) }du (a=1,..., p), -J L*(- u)Idtagn(t the generalized Bayes estimator t is obtained by a solution of the equation

(3) 5

L*(_v){_qn(t+u,0okn)}du__0 (a=1,..., p) dta ,

Since t=^/n(0-0o) and t=-/_n(0-00), t may be called to be the generalized Bayes estimator. From (1) and (3) we have

0

=5 L*(-u) exp [ -

[-

--

E IQa(Bo)(ta+ua -Ua)(ta+ua-UP)]

1 6(BO) ( ta +uo - Ue) 11+

1 ^/ n ^c (Bo)

LI ia(0o) (ta +

1 +2.,/n E Z-00)(ta+ua)(to+ua)

-6^n E + .^/n ^c(o)

-2^n

Pap ,(eo)( t a+ua)(to +UP)(t,+ur)}

n«(Bo ) + n Z(00) (tP up)

Z Z P-400) (to +ua ) (t r +u,)] du+ Or( `/n/

ua)

179 KEI TAKEUCHI AND MASAFUMI AKAHIRA

412

Putting i t , = to - U U (a =1, • • • , p) we obtain

0

=5

{- 2 Y Z I p(6.)( ua +ua)(up+it)}]

L*(-u)[exp

• [-LJ

+

Iap( u p +u a) {i+ , ✓ nn(Bo )

1

L

2,,/n a

Ll

p

a

7ra(60)(ua +ua+U

)

Zap(60)( ua +ua+U.)( u p +u p +Up)

+,k +U r (u , + it,+U) ua+uap + U )( up) 1a Zp papr( 60 r r )(

64./n

+ ^/n^c(6o)

7ra(6o)+ ^n

Zap(60)(up

+up +U)

-2 In E E pa460)(40 +up+U)(11r+ur+U,)]du+op( Further we assume the following. (viii) L*(u) is a symmetric loss function about the origin. We define

5

M= L*(-u) exp [-

I p(6o)u.up]du ;

2

pas L*(-u)uaup exp

E Ia0 o)u.up]du

[-i-

(a,= 1,..., A; Qapro=

J

L*(- u)uaupuru, exp [-

-i

I p(60)uaup]du (a, Q, r, 3=1,- --, p) •

Note that by (viii)

5

L*( -u)ua exp (2

Iap( 6o)u.up)du

L*(- u)uaupur exp (-

1

Iap(60 ) uaup)du=0

(a, j9, r=1,..., p) Then we have 0=J L*(-u)[exp

Ip(6o)uaup}] {1 -E E Ira(6o)urua}

• [-i] i] I p(60)up - E I p(6o)up - E 71

I p(6o)^r(6o) n xc(60)

180 ASYMPTOTIC OPTIMALITY OF THE GENERALIZED BAYES ESTIMATOR 413

( ud +u

p)(ur +ur +U')- 2 1/ i E E E II^(O0)Zra(O0) N

' (ud+ua)(urua+U1Ua+2uric,+2Ui,+2Urua)

n Ee Er Ea Ee I ft(oo )Pra e(oo)(u p +ua)(urusue +UrU'Ue

+ 1 6,/

+ 3urruaie + 3u,ur Ue + 3 Ur Uaue + 3 Ur. Uaue + 6urua Ue )

+ 1 na(Bo)+

E Z ZP(B0)ue+ 1

E Za0(9o)UU- 1

2%/ n l

.^l n 7r(Bo) P ,/ n A

• P.,'(0o)(upur+UpUr+2upiur+2Uur+2Usur)]du+oz,( 1 1 _ -ME Ip(oo)ue-

l E E Ia^(oo) r (oo) per-

I E E E IQa(eo)

P ^/ 7G P r 7r(Oo) 9 r a

• Zra(Bo)UrP 4+ 6 +

^a(oo) 7r(BO)

•

,V 1 n_

EE

E

E I p(Bo)

Prde(Bo)(Qa ra e +3UrUo Pae)

1 M+ 1 E UPZ P(oo) ME 17n a 2^/ n e

E P.P'(oo) 1

(PPr +UUU,M) + E

E I; Aoo)I 00)Pa 14N +01 ( ) B r a x/12

Using a matrix representation we obtain (IPI-MI)ic= ^n (PI-ME)7r*-{ 1n L- 2 1 (PI-ME)V + 1 (PI-ME)W+op(

1),

where E is an unit matrix , and L, V and W are column vectors with lr(oo) E E E I p(0o)Prae (Bo )Q9rae+ r 2

L= (- 16 V=

(zP r

Papr (o o)Pdr)

P, Nr(oo ) UpUr)

W=(E UAr(B0)) Since it is derived from (vi) that the matrix IPI-MI is positive definite, it follows that (4) A= 1

I-'7r* + 1 (IPI-MI )-IL- 2 1_ I-IV

181 414 KEI TAKEUCHI AND MASAFUMI AKAHIRA

Hence we have to=ua+U. (a=1,..., p) ,

(5)

where a=(u,, • • - , it,)' is given by (4). Since

Vn(B-Bo), we modify

d to be second order AMU and denote by B*. From Theorem 1, (4) and (5) it follows that the MLE 8ML is asymptotically equivalent to the generalized Bayes estimator 9* up to order n-'"2. By Theorem 2 it is seen that d* is second order asymptotically efficient in the class A2. We have defined in [4], [11] and [12] the class D as the set of the all third order AMU estimators B satisfying the following : (a) B is asymptotically expanded as

and Qa=OP(1) (a=1, , p) and E(UQk)=o(1) (k=1, 2) for all a, a=1, • • , p, where E denotes the asymptotic expectation of UUQ1 with U= 1 I°" 8 E log.f(Xi, B) ( = 1 , . . . , p ) ;

(b) The joint distribution of d admits Edgeworth expansion. It follows from (4) and (5) that the generalized Bayes estimator B* belongs to the class D. Then the asymptotic marginal distribution of B* is equivalent to that of the MLE 9ML up to order n-'.

Since BML is third order asymptotically efficient in the class D ([4], [10], [11]), 9* is also so. Hence we have established : THEOREM 3.

Under the assumptions (i )-(viii), the generalized Bayes

estimator d* is second order asymptotically efficient in the class A2 and also third order asymptotically efficient in the class D. UNIVERSITY OF TOKYO UNIVERSITY OF ELECTRO - COMMUNICATIONS

REFERENCES

[1]

Akahira, M. (1975). Asymptotic theory for estimation of location in non -regular cases, I: Order of convergence of consistent estimators , Rep. Statist. Appl. Res., JUSE, 22, 8-26. [2 ] Akahira, M. and Takeuchi, K. (1976). On the second order asymptotic efficiency of estimators in multiparameter cases, Rep. Univ. Electro-Comm ., 26, 261-269.

182

ASYMPTOTIC OPTIMALITY OF THE GENERALIZED BAYES ESTIMATOR 415 [3] Akahira , M. and Takeuchi , K. (1979). Discretized likelihood methods-Asymptotic properties of discretized likelihood estimators (DLE's ), Ann. Inst. Statist. Math., 31, A, 39-56. [4 ] Akahira , M. and Takeuchi , K. (1979). The Concept of Asymptotic Efficiency and Higher Order Asymptotic Efficiency in Statistical Estimation Theory, Lecture Note. [ 5 ] Gusev, S. I. (1975). Asymptotic expansions associated with some statistical estimators in the smooth case 1. Expansions of random variables , Theory Prob. Appl., 20, 470498. [ 6 ] Pfanzagl , J. and Wefelmeyer , W. (1978). A third-order optimum property of maximum likelihood estimator , J. Multivariate Anal., 8, 1-29. [ 7 ] Pfanzagl, J. and Wefelmeyer, W. (1979). Addendum to "A third -order optimum property of the maximum likelihood estimator ", J. Multivariate Anal., 9 , 179-182. [ 8 ] Strasser, H. (1977). Asymptotic expansions for Bayes procedures , Recent Development in Statistics (ed. J. R. Barra et al.), North -Holland, 9-35. [9 ] Takeuchi , K. and Akahira, M. (1976). On the second order asymptotic efficiencies of estimators , Proceedings of the Third Japan-USSR Symposium on Probability Theory (eds. G . Maruyama and J . V. Prokhorov), Lecture Notes in Mathematics 550, Springer-Verlag , Berlin, 604-638. [10] Takeuchi , K. and Akahira , M. (1978). Third order asymptotic efficiency of maximum likelihood estimator for multiparameter exponential case , Rep. Univ. ElectroComm ., 28, 271-293. [11] Takeuchi, K. and Akahira , M. (1978). On the asymptotic efficiency of estimators, (in Japanese), A report of the Symposium on Various Problems of Asymptotic Theory, Annual Meeting of the Mathematical Society of Japan, 1-24. [12] Takeuchi , K. and Akahira , M. (1978). Asymptotic optimality of the generalized Bayes estimator, Rep. Univ. Electro -Comm., 29, 37-45. [13] Takeuchi, K. and Akahira, M. (1979 ). Note on non-regular asymptotic estimationWhat " non-regularity " implies, Rep. Univ. Electro-Comm ., 30, 63-66. [14] Takeuchi , K. and Akahira, M. (1979). Third order asymptotic efficiency of maximum likelihood estimator in general case , (to appear).

183

Rep. Univ. Electro-Comm. 30-1, (Sci. & Tech. Sect.), pp. 63-66 August, 1979

Note on Non-Regular Asymptotic Estimation What " Non-Regularity" Implies* Kei TAKEUCHI ** and Masafumi AKAHIRA***

Abstract In the asymptotic theory of statistical estimation the authors tried to compare the regular versus non-regular situation to clarify the significance and implication of each of the regularity conditions. This paper summarizes the results far obtained.

1. Introduction Asymptotic theory of estimation is usually based on a set of regularity conditions. Regular case has been studied since Cramer [ 5 ] to Huber [ 6 ], and Pfanzagl [ 7 ] among others. Non-regular case has been studied by various authors at various occasions and Weiss and Wolfowitz [ 9 ] gave general results covering non-regular cases. The authors tried to compare the regular versus non-regular situation to clarify the significance and implication of each of the regularity conditions ([ 3 ]). In this paper we summarize the results thus far obtained.

2. Results Suppose that X1, X2, • • •, X,,, • • • are a sequence of i. i. d. random variables whose distribution Ft depends on an unknown parameter ^ESCR°.

A parameter to be estimated is

considered to be a real -valued function 9=0(^) of In the regular case it is assumed that i ) The distribution Ft is dominated by some measure p with the density f; (ii) The support , i. e. the set Ae= {x : f e(x)>0} is independent of e ; (iii)

f&) is continuous in ^ for almost all x Lit]; f t(x) is continuously differentiable (up to specified order ) in ^ for almost all x [It] ; lr ^ (v) The information matrix le = E log f t(x) }j log f t(x)} 1 exists. (iv)

Moreover it is usually assumed that ( vi) O(ff) is continuously differentiable (up to specified order) in ^. Then with some other set of regularity conditions it is shown that if the maximum likelihood estimator (m. 1. e.) is 0 =6(^) /whose ^/ is the m . 1. e. of 6, V n (6-8) is asymptotically normal with mean 0 and variance 100 )1I-' I a) and B is asymptotically efficient, and it is also shown to be second order asymptotically efficient. * Received on May 30, 1979. This paper was submitted to the Second Vilnius Conference on Probability Theory and Mathematical Statistics, 1977 to which the authors could not attend nor present it. University of Tokyo *** Statistical Laboratory, University of Electro-Communications

184 64 Kei TAKEUCHI and Masafumi AKAHIRA

The conclusion involves following statements : (a) The order of convergence of estimators is n''2 ; (b) Asymptotic bound for the asymptotically efficient estimators is given by the normal distribution. (c) The m. I. e. asymptotically attains uniformly that bound ;

(d) It also attains the bound up to the order n-112 (Pfanzagl [ 7 ], Takeuchi and Akahira [8]). If either of the above assumptions (i)-(v) fails to hold, some of the conclusions above may not necessarily hold true.

(1) In the undominated case, if the family of the probability distributions is decomposed into dominated subclasses such that E=UEa (a$9) a

and correspondingly the sample space X is also decomposed. a

and

Pr {X E 'a I B} =1

if

O E Ea ;

Pr {X E 'a I O} = 0

if

O Er Ea,

then each subclass Ea can be treated separately. For other undominated case little, if any, has been known.

(2) If the support depends on ^, the order of convergence of estimators is not necessarily to be n12. Assuming smoothness of O(ff) the order usually depends on the power a defined by

1- Pt- {At} = O(p ^ - ^' It ') as ^' approaches to ^, where PC, is the probability measure with the density f e'.

If f e(x)

is sufficiently regular, the maximum order of convergence is n'/" when 02 (Akahira [ 11. (3) Discontinuity of f E(x) in ^ plays similar role as the non-identical support (actually the later can be dealt with as the case when ft(x) = 0 for some x). (4) When the support depends on E, an (uniformly) asymptotically efficient estimator may not exist, and m. 1. e. may not be asymptotically good. In the case of convergence of order n'l"(a<2) m. 1. e. is not asymptotically efficient, nor even asymptotically admissible : Suppose that

f(x-B)-

cexpr-

2

(x-B)2}

if

Jx-BI <1 ;

0 t 1 otherwise, where c is some constant. Then the m. 1. e. is asymptotically equivalent to min X; +1 with probability 1/2 t

185

August, 1979 Note on Non- Regular Asymptotic Estimation

65

max X, -1 with probability 1/2 , i while the estimator 6* = (min Xi +max Xi )/2 is asymptotically uniformly more concentrated. (The asymptotic distributions of the two are equal except for the scale being twice as large for the m.1. e. as for 0*). In this case there exists one-sided asymptotically efficient estimator, but no estimator is asymptotically uniformly efficient. If we define two-sided efficiency by inf lim Pr { 10 -0 I
together with the asymptotically median unbiased (a. m. u.) condition of B (Takeuchi and Akahira E8]), then 0* above is uniformly (in t) asymptotically two-side efficient. Also Weiss and Wolfowitz's maximum probability estimator is uniformly inferior to 9* (Akahira and Takeuchi [3], [ 4 ]). When the order of convergence is (n log n)"2 and the bound of the asymptotic distributions is normal, then m. 1. e. is asymptotically efficient. (5) If the density is not smooth enough, while Fisher information is well defined, then in. 1. e. is asymptotically efficient but not second order (that is in the order n-"2) asymptotically efficient as in the regular case : Suppose that f(x-B)= 2 exp{- Ix-O } The asymptotic distribution of the in. I. e. 9 is given by 2

lim ✓ n Pr {V n (0-0)
where 0 (u)=Su

0(t) dt with 0(t)= Z e- t'/2, while the bound of the asymptotic distributions

of a. in . u. estimators is given by z 0(t)+ 6 n O(t)sgn(t).

which can not be uniformly attained (Takeuchi and Akahira E8]) but is attained for a specified to by an estimator B satisfying

-EIXi-8I+^ Xi-B+lo -c=0, n

where c is determined so that B be a. in. u. (Akahira and Takeuchi [ 2 ], [ 3 ]). (6) When the Fisher information is infinity, the asymptotic distribution can be obtained if the distribution of (8/aO)logf (X, 0) belongs to the attraction of some stable law, and m. 1. e. may or may not be asymptotically efficient : Suppose that Xi=(X,i, Xzi) (i =1, 2, •••) ; Xzi=6Xii+Ui (i=1, 2, ..) , where Ui's are i . i. d. with the density f (u) for which I= J {f1(.)} 2/f (u)du
186 66 Kei TAKEUCHI and Masafumi AKAHIRA

The above examples are chosen to illustrate the situation which usually happens in more general cases.

References [ 1 ] Akahira, M.: "Asymptotic theory for estimation of location in non-regular cases, I : Order of convergence of consistent estimators, Rep. Stat. Appl. Res., JUSE, 22, 8-26 (1975).

[ 2 ] Akahira, M. and Takeuchi, K.: "Discretized likelihood methods-Asymptotic properties of discretized likelihood estimators (DLE's)," Ann. Inst. Statist. Math. 31, Part A, 39-56 (1979). [ 3 ] Akahira, M. and Takeuchi, K.: "The Concept of Asymptotic Efficiency and Higher Order Asymptotic Efficiency in Statistical Estimation Theory," Lecture Note (1979).

[ 4 ] Akahira, M. and Takeuchi, K.: "Remarks on the asymptotic efficiency and inefficiency of maximum probability estimators," (to appear). 151 Cramer, H.: "Mathematical Methods of Statistics," Princeton University Press (1946).

[ 6 ] Huber, P.: "The behavior of maximum likelihood estimates under non-standard conditions," Proc. Fifth Berkeley Symp. on Math. Statist. Prob. 1, 221-233 (1967). [ 7 ] Pfanzagl, J.: "Asymptotic expansions related to minimum contrast estimators," Ann. Statist., 1, 993-1026 (1973).

[ 8 ] Takeuchi, K. and Akahira, M.: "On the second order asymptotic efficiencies of estimators," Proc. Third Japan-USSR Symp. on Probability Theory. Lecture Notes in Mathematics 550, Springer Verlag, 604-638 (1976). 191 Weiss, L. and Wolfowitz, J.: "Maximum Probability Estimators and Related Topics," Lecture Notes in Mathematics 424, Springer Verlag (1974).

187 Austral. J. Statist., 22 (3), 1980 , 332-335

A NOTE ON PREDICTION SUFFICIENCY (ADEQUACY) AND SUFFICIENCY' MASAFUMI AKAHIRA AND KEI TAKEUCHI

University of Electro- Communications and University of Tokyo Summary Let V, s$) be a measurable space and 9 a family of probability measures on s4. Let 98 and `C be sub cr-algebras of ,s4 and A0 a sub a--algebra of A. It is shown that if A is prediction sufficient (adequate) for A with respect to '6 and 90, and # is sufficient for 98'i' with respect to P then Y is sufficient for 98 V( with respect to 9a; that if 9$ is homogeneous and (980; 98, ') is Markov for 91, and 980 Ce is sufficient for 98"T with respect to 9A, then 980 is sufficient for 98 with respect to 9i'; and by example that the Markov property is necessary for the latter proposition to hold.

1. Introduction Prediction sufficiency (adequacy) was defined and discussed by, among others, Skibinsky (1967), Takeuchi & Akahira (1975), and Torgersen (1977). The purpose of this note is to show the relation with prediction sufficiency and the "usual" concept of sufficiency in the following situation. Suppose that the observations are given sequentially in time in two sets. Let (X, Y) be a set of observations where X is observed first and Y comes later. In such a situation one may be interested to know how much information in X should be reserved to be later combined with Y without any loss of information in the combined sample. Technically it amounts to giving the condition for the statistic T = t(X), that the combined statistic (T, Y) be sufficient for (X, Y). It is shown that it is sufficient that T be prediction sufficient for Y, and it is also necessary under further conditions. We may further add that in this formulation the distribution of Y can be dependent on X in a general manner so that we can include the sequential design case when the design (that is observation scheme) of Y is determined depending on X. We will, essentially, use the framework of Skibinsky (1967). The notion of adequacy in Skibinsky's paper is, however, replaced by the notion of prediction sufficiency. ' Manuscript received September 13, 1978; revised January 16, 1979.

188

A NOTE ON PREDICTION SUFFICIENCY 333

2. Results Let (T, 4) be a measurable space and 9a a family of probability measures on s4. We denote by 4; s42 the smallest o--algebra which contains each member of two subclasses sal, 42 of A Let XA denote the indicator function of a set A. 98 and T denote sub o--algebras of si, and 980 denotes a sub o--algebra of 90. Definition 1. 900 is said to be sufficient for 9o with respect to 95, symbolically

suf (9; 90),

oJ

if for each B E 90 there exists a 900-measurable function ¢B° such that for every p E9'

4s = EP"XB a.e. [p], where EP° XB denotes the conditional expectation under p of XB given moo. Definition 2. (Skibinsky, 1967; Takeuchi & Akahira, 1975). 910 is said to be prediction sufficient (adequate) for 90 with respect to ig and 9', symbolically

900 pred. suf (-60; 98,19) if Ro suf (-60; 90) and (980i 90, `e) is Markov for 9, that is, 98 and T are conditionally independent given 980 for each p E 9. Lemma. Let (d,°, A, p) be a probability space. (980; 90, W) is Markov for p if and only if for every B E 98 and every C E T

EPx'(X

B nc)

= Xc E P°( XB)

a.e.

[p].

Proof. Suppose that (980, 98, IC) is Markov for p. For every B E R, every C E ', every B0 E 900 and every C e ', ^o Ep

°nC'

XBnc dP = XBnc B„n

dp

C'

= Ep(XB„nBXc'nc) = Ep[EP°(XB„nBXc'nc)] = Ep[(EP°XB°nB)(EP°Xc,nc)1 = Ep[EP"(Xc'ncEP°XB„nB)]

Ep[Xc'nc EP"XB°nBJ = Ep[XB„nc'ncEP°XB XcEP°XB dp. B„nc'

Since {B0 fl C B0 E 980, C E '} generates the a--algebra 900vre, it follows that for every Be 90 and every C ET

EP°v'eXB = XcEp"XB

a.e. [p].

189 334 MASAFUMI AKAHIRA AND KEI TAKEUCHI

Conversely, if for every B E 98 and every C E W EP° XB = XcEP°XB a.e. [p], then for every B E 98 and every C E `' E',(XBnC) = EP°( "XBIC) = EP°(XcEP°XB) = EP°XBEP°Xc a.e. [p]. Hence (98,,;98, ') is Markov for p. Theorem 1. Let &' be a sub if-algebra of 980"`e. Suppose that 980 pred. suf (9; 98, W) and 9' suf (P; 980"`x). Then 9° suf (91; 98"<). Proof. For every B ER, every C E IC and every p E

9'

EPX B nC = EP ( Ep ° 'fXB nC) = EP(XcEP°XB)

(by Lemma)

= EP(Xc4s°) (by 98o suf (9; 98)) a.e. [p]. Since XcoOa ° is a 91 o Ig-measurable function and 9 suf (9; 980 w), it follows that there exists a 9-measurable function (Psnc such that for every pc9' 4snc= EP(Xc4s°) = EPXsnc

a.e. [p].

Since {B fl c I B E 98, C E W} generates 98"`x, it is seen that 9 suf (9 ; 98"x). For Theorem 2 we assume that 9 is homogeneous, that is, p and q are mutually absolutely continuous for every p and q in 91. Theorem 2. Suppose that (980; 98, W) is Markov for 9 and 980"'e suf (9; 98"`'). Then 980 suf (9'; 98). Proof . Since P is homogeneous and 980"` suf (-60; 98"') and (980; 98, W) is Markov for 9, it follows by the Lemma that for every B E 9V there exists a 9f 0-measurable function (AB'- such that for every pE9 EP°(XB) = EP°(EP°'XB) = EP°(4e°) = Os'

a.e. [p].

Hence 980 suf Remark. It is shown by the following counter-example that the homogeneity of 95 is necessary for Theorem 2. Suppose that 98 =If= {4, B, B`, X}, 98o = {4, '} and 9= {p, q} with p(B) = q(B`) = 1. Then (980; 98 , ') is Markov for 9' and 980"'C suf (9a; R"ig) but 980 is not sufficient for 98 with respect to 95.

190 A NOTE ON PREDICTION SUFFICIENCY 335

The following example shows that the Markov property is necessary for Theorem 2 to hold. Suppose that (X, Y, Z) are jointly normally distributed with means (0, 0, 0) where 0 is the unknown parameter and with known variances and convariances to be later specified. Let 910, y8 and' be the o -algebras induced by, respectively X, (X, Y) and Z. Let us assume that Y can be expressed as Y = X - Z + U, where U is independent of X and Z. We further assume that Cov (X, Z) = 0 and V(X) = V(Z) = V(U) = 1, where Cov and V denote covariance and variance, respectively. Then it follows that E(U) = 0, V(Y) = 3, Cov (Y, X) = 1 and Cov (Y, Z) _ -1. And given (X, Z), Y is conditionally normally distributed with mean X-Z and variance 1, hence its conditional distribution is independent of 0. Therefore (X, Z) is sufficient for (X, Y, Z) or equivalently 9"19 is sufficient for M"'. However, X is not sufficient for (X, Y), that is, 98o is not sufficient for 98, because given X, Y is conditionally normally distributed with mean X - 0 and variance 2, which is dependent on 0. Nor are Y and Z conditionally independent given X, hence (_qO, 98, ig) is not Markov. References SKIBINSKY , M. (1967). Adequate subfields and sufficiency. Ann. Math. Statist. 38, 155-161. TAKEucHI, K. & AKAHIRA , M. (1975). Characterizations of prediction sufficiency (adequacy) in terms of risk functions . Ann. Statist. 3, 1018-1024. ToRGEnsEN , E. N. (1977). Prediction sufficiency when the loss function does not depend on the unknown parameter. Ann. Statist. 5, 155-163.

191

Rep. Univ. Electro-Comm. 31-1, (Sci. & Tech. Sect.), pp. 89-96 August, 1980

Third Order Asymptotic Efficiency and Asymptotic Completeness of Estimators* Kei TAKEUCHI** and Masafumi AKAHIRA***

Abstract In the previous paper it was shown that the maximum likelihood estimator (MLE) was third order asymptotically efficient in multiparameter exponential cases. In this paper it is shown that the result is extended to more general cases. The concept of asymptotic completeness of an estimator

is introduced

and it is shown that the MLE is higher order asymptotically complete in the appropiate classes.

1. Introduction In the previous papers [ 6 ], [ 7 ] and [ 3 ] Takeuchi and Akahira have discussed the third order asymptotic efficiency of the maximum likelihood estimator (MLE) and the generalized Bayes estimator in multiparameter exponential cases.

In this parer we show that it is

possible to extend the result on the MLE to more general cases in a very straightforward manner by introducing appropriate classes of estimators. Further we introduce the concept of an estimator in the sense that it can beat asymptotically any other " regular " estimators by making appropriate transformations if necessary and show that the MLE is higher order asymptotically complete in the appropriate classes.

2. Third order asymptotic efficiency of maximum likelihood estimator in general cases In order to generalize the results in the previous paper E6], it is necessary to define appropriate classes of estimators among which asymptotic distribution is to be compared. As was shown in Akahira and Takeuchi [ 3 ] it is possible to establish the second order asymptotic efficiency of the modified MLE within the class of second order AMU estimators without any further restriction of the class of estimators to be considered. On the contrary in the previous paper [ 6 ] we restricted the class of estimators to be of a special type of functions of the sufficient statistic which we called the extended regular estimators. As will be shown later we can not usually have third order asymptotically efficient estimators without such restriction. In more general cases where sufficient statistics of finite dimension do not exist, the argument in the previous paper [ 6 ] can not be applied directly. We shall establish the third order asymptotic efficiency of the modified MLE and also of the modified generalized Bayes estimator within the classes C and D of estimators which are defined below. For the simplicity's sake * Received on June 10, 1980 ** University of Tokyo *** Statistical Laboratory, University of Electro-Communications

192 90 Kei TAKEUCHI and Masafumi AKAHIRA

we shall first consider the real parameter (one dimensional) case, and that the sample is independently and identically distributed with all necessary regularity conditions to be specified later. Let (1, B)be a sample space. We consider a family of probability measures on B, B= [Pe: 8 E 0} , where the index set 0 is called a parameter space.

We assume that 0 is an open

set in a Euclidean 1-space R'. Consider n-fold direct products (2C°' , B(n)) of (-T, B) and corresponding product measures Po,. of Pe.

For each n=1,,..., the points of _T(n) will be An estimator of 8 is defined to be a sequence {en} of B(n)-

denoted by .xn=(xi, . . . , xn).

measurable functions On on X(n) into (9 (n = 1, 2.... ).

For simplicity we denote an estimator as

On instead of {On}. For an increasing sequence of positive numbers {c.} (cn tending to infinity) an estimator On is called consistent with order {cn} (or {cn} -consistent for short) if for every E> 0 and every 9 E 0 there exists a sufficiently small positive number 6 and a sufficiently large positive number L satisfying the following :

lim sup n-+oo

8: 19-91<e

Pe. n {cn l8n-81 >L} <E

(Ell.

For each k=1, 2, . , a {cn} -consistent estimator On is called to be k-th order asymptotically median unbiased (or k-th order AMU) if for any 9 E 0 there exists a positive number S such that lim

sup

cnk-' Pe, n {6n <8} - 2

sup

cnk-' Pe,n

n-.oo e : 1e-91 <6

lim n-.oo 9 :

{B n >_8} -

1e-91<6

2

=0; =0.

For en k-th order AMU, Go(t, 8)+cn-'Gi(t, 8)+ ... +Cn-(k-')Gk-i(t, 8) is called to be the k-th order asymptotic distribution of cn(On-8)(or On for short) if Jim cnk-' I Pe,n {cn(en-8)
We denote that G1(t, 8) (i =1, ... , k-1) may be general absolutely continuous function, hence the asymptotic distribution for any fixed n may not be a distribution function. Suppose that On is k-th order AMU and has the k-th order asymptotic distribution Go(t, 8)+cn-'Gi(t, 8)+ . . . +Cn-(-')Gk-1(t, 0). Letting 8o(E(9) be arbitrary but fixed we consider the problem of testing hypothesis

H+: 8

= 8o+tcn -'( t>O) against K: 8=80. Put 01/2= { {/in} : Ee,+to„ ',n(On)=1/2+o(cn-(k-')), 055 n(.xn)<1 for all x.E_Ttn' (n=1, 2.... )}. Putting Adn,e,={cn(Bn-Bo)
lim Pe,+tC.-'> n(AB..e,)= lim Pe, +tcA ', n{On <8o +tcn-'}

n-»oo

Hence it is seen that a sequence

to

01/2.

n-+

2

{xAB. eo} of the indicators of AB., e, (n=1,2,...) belongs

If

sup lim cnk-' {Ee,,n(On)-Ho+(t, 8o)-cn-'Hl+( t, 80)- ... -cn-tk -'"Hk-1+(t, 80)} =0, (0.(E^ n-.oo

then we have Go(t, 8o)< Ho+(t, 8o) ; and for any positive integer j (
G;( t, 80)<_ H, +(t, 8 o).

193 August, 1980 Third Order Asymptotic Efficiency and Asymptotic Completeness of Estimators 91 Consider next problem of testing hypothesis H-: O=Oo+tcn-' (t<0) against K : 0=00.

If

inf lira cnk -' {Ee,.n(0n)-Ho-(t, Oo)-cn-'Hi-( t, Oo)- Oo)} =0, then we have Go(t, Oo)> Ho-(t, Oo) ; and for any poistive integer j(H[ (t, Oo). 0, is called to be k-th order asympotically efficient if the k-th order asymptotic distribution of it attains uniformly the bound of the k-th order asymptotic distributions of k-th order AMU estimators, that is, for each OEO

Gi(t, 0)- Hi+(t, 0) -{H;-(t, 0)

for t>0; for t<0,

2 ], [ 3 ], [ 4 ], [ 5 ]). [Note that for t=0 we have Gi(0, O)=Hi+(0, O)=Hi-(0, 0) (i =0, ... , k-1) from the condition of k-th order asymptotically median unbiasedness.] Thus if On* is k-th order asymptotically efficient in the above sense, then for any k-th order AMU estimator On we have

lim cnk '[Po, n {-a0.

(2. 1)

n_w

Note that the k-th order asymptotic efficiency defined in the above

for all a, b> 0 and 0 E 0.

gives a sufficient condition but not a necessary condition (2. 1). We assume that for each 0EO Pe is absolutely continuous with respect to a or-finite measure p.

We denote a density dPe/die by f (x, 0).

is given by

11 f (xi, 0). i=1

Then the joint density of (Xi, . . . , Xn)

In this section we shall deal with the case when cn=V n .

First we shall restrict our attention to the class of (first order) asymptotic efficient estimators which we shall call the. class A. It has been well established that any estimator On in the class A can be expressed as

Vn (9n-0)=I(B)1^ n a n+ op(1), n

where Ln= E log f (Xi, 0). i=1

Secondly, we call the class of estimators which are second AMU and second order asymptotically efficient as class B. Then any estimator On which is second order AMU and for which the distribution of 111 n (On-B) admits the Edgeworth expansion up to the order n-112 belongs to the class B. Now we generalize the concept into one step further. We call the class C the class of estimators On which are third order AMU and for which the distribution of ✓ n (On-0) admits the Edgeworth expansion up to the order n-' and

V n (Bn-B)I(0)V n 0 +l n-Q(0)+OP( nI where Q (0) is a quantity of stochastic order 1. We call the estimator O n belongs to the class D if in the above we have

E[IL.Q(6)^]=O'

194

92

Kei TAKEUCHI and Masafumi AKAHIRA

where E stands for the asymptotic mean. It is to be noted that Lemma 4. 1 in the previous paper [ 6 ] can be generalized straightforwardly as follows.

Lemma 2.1. Suppose that ZB is a function of X1, ... , X. and B and is differentiable in 0. Then E9(UZ9)=V n aBEe( Z9)-1 1 EB` 06 1 -W where U= V a log f (X, 0), provided that the differentiation under the integral sign of E,? I(0) 8B (Z9) is allowed. In the following discussions we assume that expectation of the relevant quantities always exist and the asymptotic value of the expectation and the asymptotic mean are the same at least up to the required order. Remark : The problem of non-existence of the relevant moments vis-a-vis asymptotic moments can be evaded in the following way. Suppose that we have a smooth and montone increasing transformation ij=0(6) so that 17 is bounded. Instead of 0 we may consider 27 as the parameter to be estimated. Since , is bounded we may assume that all the estimators are also bounded hence have moments of any order, and we can easily formulate the condition for the asymptotic moments and the moments of the asymptotic distribution being equal. If we consider the estimation of 77 instead of 0 the asymptotic theory based on moments is applicable in a straightforward way. Transforming back to the estimator of 0 by 9=0-'(fj) and considering the monotoniticy of the transformation of the distribution, the results are also applicable to the estimation of 0. Let us first consider the class C. Suppose that ✓ n (6„-B) is expanded as ✓ n (9„-B)= I(^j)+ 111 n Q(0)--On(n ),

where Q(B)=O,(1) and Zi(B)=- n

tZi - - log

f (X;, B).

Then we have the following lemma. Lemma 2.2. EB[Zi(B)Q(0)] =o(1) Proof. Define Te=V n (fin-9)- I (8))= .LQ(9)+oa(Vn). Then we have IA I,/-n Do V_n n aBE9[Q n). -0"(v 111

From which we have

(6)]+1-ynE^

+ov(n)

195 August, 1980 Third Order Asymptotic Efficiency and Asymptotic Completeness of Estimators 93 Ee[Zi(0)Q(0)]= ✓ n Ee[Zi(B)Te]+o,(1) =0,(1).

Thus we have the asymptotic cumulants of ✓ n (6n-0) up to the order n-' as Ee[V n (6n-0)]= nn iz1(0)+ /t2(B)+o(n) ;

Ve[✓n (6n- B)] = 1

+

n

v 2(B)+o (n) ;

n n)

K3[V n (6n-e)]=Tn P3(0)+ - 73(0)+O

K4[l/ n (6n-0)]= n P4(0)+0(n ). We can show in exactly similar way as in the previous papers [ 5 ] and [ 6 ] that ^3(B) = - 3J(9) +2K(6) .

I(0)3 P'(B)=-I8)•{3H*(0)+12N*(0)+4L(0)} +IB)s{2J(0)+K(0)} {J(9)+K(8)} where

L(O)= E ^{ a 303 log f (X, 0)}{aB log .f (X, 0)}] N*(O)=

E9[ f

logf(X, 0)+I(6)}{a log f(X, B)

}2]

H *(B)= Ee[{a8 log f(X, 0))4]-3I(o)2. Since the odd order cumulants affects only the asymmetric aspects of the asymptotic distribution, the asymptotic expansion of Ps,. {V n 1 6n - 01
is independent of them. Hence we have

the following theorem. Theorem 2. 1. Let OML* be the modified MLE in the class C and On be any other estimator in the class C. Then it holds that limn[Pe,n{Vn I6ML*-0j 0 n-*oo

for all a> 0 and all 6 E 6. For the class D we further have that r3(0)=0 and p1i(0)=/33(0)/6, hence all the asymptotic moments are determined up to the order n-' except V2. Since V2 is minimized for the modified MLE we have the following theorem.

Theorem 2.2. Let OML** be the modified MLE in the class D and On be any other estimator in the class D. Then it holds that lim n[Pe,n {-a
for all a>0, all b>0 and all BE 6.

3. The concept of asymptotic completeness of an estimator We consider first the case of a real-valued parameter. The concept of asymptotic efficiency and higher order asymptotic efficiency gives a very strong condition for the "goodness" of an estimator in that it uniformly maximizes asymptotically the probability of its lying within a

196

94 Kei TAKEUCHI and Masafumi AKAHIRA

neighborhood of the value irrespective of the parameter value and the choice of neighborhood. But only one somewhat artificial restriction is that the estimators considered should be (higher order) asymptotically median unbiased. It is, however, easily seen from the context that the only requirement is that the class of estimators to be compared must have the same asymptotic "location" whatever the exact meaning of the word may be, without which nothing can be derived. Therefore one may consider different types of the definition of " location ", and accordingly different type of "adjustment" of the estimator to satisfy the condition. Thus we may consider in the following way. Suppose that we want to establish asymptotic optimality of an estimator One, e. g. MLE, in some class of estimators.

What has been virtually established is that for any estimator On in the class if both On and Ono are adjusted like O n*=On+

1 g(On) ;

n

Ono*=On0+ 1 h(OnO) ; n so that On* and On0* are both (higher order) asymptotically median unbiased then 0n°* is (higher order) asymptotically uniformly "better " than On*. If we change the definition of " location " the way of "adjustment " will be changed, but the same results hold for adjusted estimators On" and On*.

This fact will lead to the following question : Instead of comparing

two adjusted estimators On* and On°*, isn't it possible to compare On and the adjusted On* ? Naturally, the way On* is adjusted depends on On to be compared, but is independent of the parameter. And we introduce the following concept. An estimator On* is called to be asymptotically complete with respect to a class M of estimators if for any On EM there is an "adjusted" estimator On **=On* +gn(On *)

so that for any a>0, b>0 and 0 we have lim [P9,- {-a0.

(3.1)

Now let L be a loss function such that L(0)=0 and L(u) is an increasing function for u>0 and a decreasing function for u<0 and it is bounded. If On* is asymptotically complete in the above sense, then for any estimator On in M there is On** which is a function of On* for which lim [ Ee, n( L( cn( On**-B)))- E9,n(L(cn( On- 0)))]<0. (3. 2) n-co

Higher order asymptoic completeness can be defined in a similar way. For each k=2, 3, ... , an estimator On* is called to be k-th order asymptotically complete with respect to a class M of estimators if for any On E M there is an "adjusted" estimator On**=On*+gn(On*) so that for any a>0, b>0 and 0 we have

lim cnA -'[Pe,n {-a0.

(3.3)

n-'oo

Further On* is called to be k-th order asymptotically symmetrically complete with respect to a class M of estimators when (3. 3) holds when a= b.

If On* is k-th order asymptotically complete

with respect to M, then for any estimator On in M there is On** which is a function of On'e`

197

August, 1980 Third Order Asymptotic Efficiency and Asymptotic Completeness of Estimators 95

for which limcnk '[ Ee, n( L( cn( Bn * *-6)))- E9,n(L(cn(Bn- 0)))]S0.

(3.4)

n-.oo

If 6n* is k-th order asymptotically symmetrically complete with respect to M, then (3. 4) holds for all symmetric loss function around the origin. Unfortunately the above definition is meaningless as such, since no estimator can asymptotically complete, which can be seen below. Let On* be an asymptotically efficient estimator (e. g. MLE) and 1/ n (0n*-0) be asymptotically distributed with mean 0 and variance 1/I, where I is the Fisher information. Let On be an AMU estimator with asymptotic variance larger than 1/I. Obviously On is not asymptotically efficient and inferior to On*. A naturally adjusted estimator of On* corresponding to On will be On**=On*+c/v n

which has the same

asymptotic mean with On. Now take (0<)b(0) large enough. Then we have lim [Pe,n{-a
which contradicts the condition of asymptotic completeness. However we may adjusted On* we can not get uniformity irrespective of a and b.

It is, of

course, possible to get the reverse inequality if we adjust On* depending on a and b, but the nice uniformity would be lost. There is nothing unnatural in this problem : If there is a bias in the estimators, the estimator with large variance may hit upon the true value by chance but one with small variance has little chance to get close to the true value. Therefore we must modify the concept of the asymptotic completeness and call On* asymptotically symmetrically complete if (3. 1) holds when a=b, then (3. 2) holds for all symmetric loss function. The class F of estimators here is more specifically defined as the class of estimators On such that cn(On-6) has the asymptotic distribution in the sense that lim Pe, n {cn(On-0)
where the convergence is locally uniform in 0 and Fo(a) is absolutely continuous in a and continuous in 6, and we further assume that cn is equal to the maximum order of consistency. Now we have the following theorem, the proof of which is omitted. Theorem 3. 1. Let On* be asymptotically efficient and the density of the asymptotic distribution of cn(On-B) is symmetric around the origin. Then On* is asymptotically symmetrically complete with respect to the class F.

In this case it is seen that On* need not be modified. Higher order asymptotic compleness has, however, more meaningful implication. Now we consider the regular case with cn=Vn and we define the classes B*, C* and D* which are defined in the same way as the class B, C and D, respectively but omitting the condition of higher order asymptotic median unbiasedness. Then the following theorems hold : Theorem 3.2. The MLE is second order asymptically complete with respect to the class B*. Theorem 3.3. The MLE is third order asymptotically symmetrically complete with respect to the class C*.

198 96 Kei TAKEUCHI and Masafumi AKAHIRA

Theorem 3.4. The MLE is third order asymptotically complete with respect to the class D*. The outline of the proof of Theorem 3. 4 is as follows :

Let 65ED* and the asymptotic bias of Bn be Ee.n(Un(0n-6))_nµi(B)+o( n and also of Bn* E9,n(V n (Bn*-0))=n Ftl*(B) 1 In ).

Let the modified estimator fi0n** be defined as 11 (7n**.e,,*+ i {1ai*(en*)- f.Li*(en*)} n

Then we can prove exactly as of Theorem 6. 1 in the previous paper [ 6 ] that

lim n[Pe, n

a < I/ n (O n **-0)0

n-+oo

for all a, b>0 and 0, because the asymptotic moments of V _n (6n-0) and V n (en**-B) are the same up to the order n-' except the second term in the variance which is smaller for On*. The proofs of Theorem 3. 2 and 3. 3 are similar and even simpler. The classes C* and D* are much natural classes than classes C and D and Theorems 3. 2, 3. 3 and 3. 4 may be nearly the final conclusion of the asymptotic derivability of the MLE because we have the problem of the choice of the way of modification which should be based on the consideration of the asymptotic bias versus asymptotic variance, which can not be decided once and for all, but what is established in the above implies that we only need to consider the estimators which are modified MLE but it is not necessary to take any other (regular) estimators into consideration.

4. Conclusion remark The conclusion of this paper can be easily generalized to the multiparameter cases and the discussion is very much similar to that in the multiparameter exponential case. So we omit the details of the derivation.

References Akahira, M.: Asymptotic theory for estimation of location in non-regular cases, I : Order of convergence of consistent estimators , Rep. Stat. Appl. Res., JUSE, 22, 8-26 (1975). [ 2 ] Akahira, M.: A note on the order asymptotic efficiency of estimators in an autoregressive process, Rep. Univ. Electro-Comm., 26, 143-149 (1975). [ 3 ] Akahira, M. and Takeuchi, K.: The Concept of Asymptotic Efficiency and Higher Order

[1]

Asymptotic Efficiency in Statistical Estimation Theory, Lecture Note (1979) ; Revised (1980). [ 4 ] Akahira, M. and Takeuchi, K.: Discretized likelihood methods : Asymptotic properties of discretized likelihood estimators (DLE's), Ann. Inst. Statist. Math., 31, Part A, 39-56 (1979). [ 5 ] Takeuchi, K. and Akahira, M.: On the second order asymptotic efficiencies of estimators, Proceedings of the Third Japan-USSR Symposium on Probability Theory, Lecture Notes in Mathematics, 550, Springer-Verlag, 604-638 (1976). [ 6 ] Takeuchi, K. and Akahira, M.: Third order asymptotic efficiency of maximum likelihood estimator for multiparameter exponential case, Rep. Univ. Electro-Comm., 28, 271-293 (1978). [ 7 ] Takeuchi, K. and Akahira, M.: Asymptotic optimality of the generalized Bayes estimator in multiparameter cases, Ann. Inst. Statist. Math., 31, Part A, 403-415 (1979).

199

Statistics & Decisions 1, 17-38 (1982) © Akademische Verlagsgesellschaft 1982

ON ASYMPTOTIC DEFICIENCY OF ESTIMATORS IN POOLED SAMPLES IN THE PRESENCE OF NUISANCE PARAMETERS Masafumi Akahira and Kei Takeuchi

Received : revised version : May 4, 1982

Abstract The concept of asymptotic deficiency is extended to the case when a common parameter 6 is estimated from m sets of independent samples each of size n , and the asymptotic deficiencies of some asymptotically efficient estimators relative to the maximum likelihood estimator based on the pooled sample are discussed in the presence of nuisance parameters.

1. Introduction. The third order asymptotic efficiency of the maximum likelihood estimator (MLE) in the case of independently and identically distributed

( i.i.d.)

samples has been established by Pfanzagl and Wefelmeyer ( 1978 ), Akahira and Takeuchi ( 1978, 1981a ) and Ghosh , Sinha and Wieand ( 1980), and it can be formulated in terms of asymptotic deficiency introduced by Hodges and Lehmann ( 1970) ( Akahira, 1981 ; Akahira and Takeuchi 1981a, 1981b). Let X1, X2, Xn , ...

be i.i.d. random variables according to

the distribution with a density function f(x,6) where 6 is a real

AMS Subject Classification ( 1980): 62F12, 62F10. Key words and phrases :

Asymptotic deficiency , asymptotically efficient

estimator , maximum likelihood estimator , pooled samples, nuisance parameter, loss of information , asymptotic unbiasedness , ancillary statistics.

200 18

AKAHIRA-TAKEUCHI

parameter . Under a suitable set of regularity conditions it can be proved that the MLE 0n of 0 can be asymptotically expanded into the form

^(6n- 0) = I Z + 1 Q* + opI 1 1 , T \Jn/ n CC where Z 1 G a0 log f(xi,0) and I = E0[{ae log f(x,0)}2] and that vrn- i=l which admits the same for any asymptotically efficient estimator

60

type of expansion x(00 0)

1 Z+ 1 Q+ op\ 1 1 n t An- 0

we have * V(Q0) > V(Q )

where V designates variance. This implies that we can construct a "modified " estimator ( relative to 0n) of the form

0** = 6* + h(0*) n n n n so that 0n* has the same asymptotic bias as

6n up to the order n-1

and that

lim n[P0 n{rI 6*n *-0 < a} - PO n n ,

{,rI 0n-0

< all > 0

for all 0 and all a > 0. Moreover, if n1 is defined so that the n (with Q**) based on the sample of size n1 has modified estimator 0 ** the same asymptotic distribution as that of the estimator

6n

(with Q0)

based on the sample of size n(>nI) up to the order n ' , we have

lim(n-n1 ) = I{V(Q0)- V(Q**)} n-o (Akahira, 1981). We call the above limit the asymptotic deficiency of 6n relative to the MLE. Note that the right-hand side of the above

201 AKAHIRA-TAKEUCHI

19

equation does not necessarily mean the difference of the variances of estimators but of the asymptotic variances , hence we do not need to bother about the remainder terms ( Akahira and Takeuchi , 1981a). In this paper we extend this concept to the case when a common parameter 0 is estimated from m independent sets of samples and discuss the deficiencies of some asymptotically efficient estimators relative to the MLE based on the pooled sample from the distribution with different nuisance parameters.

2. Asymptotic Deficiencies Of Estimators For Samples From The Distributions With Different Nuisance Parameters. Suppose that it is required to estimate an unknown real-valued parameter 0 based on m samples of size n whose values are

Xis (i = 1,...,m;

j = 1,...,n) from the distributions with different nuisance parameters Now we assume that for each i Xil, Xi2, ...' Xin

Ei (i = 1,...,m ).

are independently and identically distributed according to a density function

f(x,0,Ei)

with respect to a a- finite measure p.

We assume further regularity conditions: (A.1) For each i the set {xlf(x,0,Ei) > 0} does not depend on 0 and Ei. (A.2) For each i , for almost all x[ul, f (x,0,^i) is three times continuously partially differentiable in 0 and Ci. (A.3) For each 0 c 0 and each i

0 < I80 = E [{ae log f(Xij,0 )}2l 2 -E[a2 log f (Xij,Oi)l

< -o;

0 < Iii = E[{---L log f(X1^e ,^i)}2l i

202 AKAHIRA-TAKEUCHI

20

2 -E[a 2 log f ( X.j6,Ei)) i < CO

(A.4) The parameters are defined to be "orthogonal" in the sense that 2 E[862 log f i

(Xi^,O,Ei)^ = 0 (i

Remark that this assumption is not necessarily restrictive, since 901Y ( i = 1,...,m) so

otherwise we can redefine the parameters that they have the above orthogonality. (A.5) There exist

2 -log f (Xij,e, i)}{se log f ( Xij,0,^i)}] 000 - E[ { De 2 2 se2 log f ( Xij,6,^i)}{ai log f ( Xij,6 ,Ei)}] JeeE = E[{ 2 JUF = E[{808 1 log f

Je - E[{

(Xii ,O, i)}{a6 log f ( Xij,O,Ei)}]

2 868 1 log f(Xij,O , i)}{a i log f (Xij,e ,ci)}I

2 Je = E[ { a 2 log f (Xij,6,i)}{se log f (Xij,U,Fj)}} i

K000

= E[

{-

Ke0 = El {-56

log f(Xi3,U, i)}3 ]

log f (Xis , e , Ei)

,

}2 {al

log f ( Xis , e , i) } ]

2 2 E[{a 2 log f ( Xij,6,Ei)} 2l - 160 M0066 = 30 2 Me0 = E[{8A8ki log f(Xij,U, i)}2] , and the following hold:

203 AKAHIRA-TAKEUCHI

21

3 E[ -2

log f (Xij,A,Ei) l = -3J600-K B00 1 3 3 log ,B, i)l = -ji E[ 2 f (Xij

aA ai E[ 2 log f(Xij,B, i)J _ -Je aAa^i for i = 1, .

., M.

From the assumption ( A.4) it is noted that K88 Denote the log likelihood functions

= JBcA - JBA ' Li(8,Ci) ( i = 1,...,m) and

L(A,El,...,Em) by n c Li(8,Ci) = G log f(xij,8,^i) (i = 1,...,m) j=1 m m n L(8,^1,..., m) _ L.( 8, log f (xij,0,Ei) i=1 i=l j=l

I E

When are known, we denote by 80 the MLE of 0 based on L(0, 1,..., m).

For each i = 1, ..., m let

of 0 and based on

Ai and be the MLE's

L i,E1 (0 ) , respectively .

Let

Em

be the MLE ' s of 8 , Cl, ..., ^m based on L(0 , 1,..., m) , respectively. ( i) The MLE 80 when

are known.

Since A0 is the MLE of 0 for known parameters

^l' ..Em '

it follows that

ae L(BO,l,...,m) = 0

Then it is seen by a similar way as Akahira and Takeuchi ( 1981a, page 92) that the expansion of 0 0 is given by

'(e0 A)

+I Ze + 1 *2 z00Z0 - 31 *3 Ze2 + 0 1 I VI 2^ fn zA+ QO+0 (say) ✓n p (i)

where

204 AKAHIRA-TAKEUCHI

22

m

in

Ze, Zee

Ze

I

zee , it

=

m

i

in

i

i=1 1A6> J i=1 JeBA

m and K* _ Ke66 with i=1

Ze 1 An-

a6

Li(,Ei) 2

zee =

1 {a 2 L. ( A, i) + nIeA} n ae

Hence it is easily seen that 1 * * *2 (J*+K*) 2 (2.1) V(QO) = I *4 (M I - J ) + 21 *4 * m

where M =

I i=1

Me000*

(ii) The MLE 's 61, ..., . Since a * * aA L(8 1'..., m) = 0

0 (1 = 1,....m)

it follows by a similar way as the case (i) that the stochastic expansion of 6 is given by

r(e*-A)

* * 1 * 1 * 3J +K *2 2iT 1*3 ze 1* z0 + ,r I* z6ez0 1 m

1

i

i_

1

Z

+ YrnI* i =11 ii z6EZ I*2

i=1 T

1 6i Z ( + op l 2 \ 1 / 2i I i=1 I1 Ze + I

1

QO + 1 vrn-

Q1 r

+ o 1 p

* * Z + 1 Q + 0 1 ( say) I ,T p (^n::)

1

(say)

205 23

AKAHIRA-TAKEUCHI

where

z = 1

Li (e

a

a2

zed

Yrn-

aeaE1

Li(O, i)

(2.2) V(Q*) = V(QO) + V(Q1)

with i2 i2 1 mc 1 i Jere JO V(Q1) = I*2 1 = 1 1 i Mee - Ii - Ii

1 CmC 00 1 1

i2 1

Jere

i2 1CIl Jed

+ *2 iLl Ii I* Ii + 2I*2 i-1 Ii2 (iii) The weighted estimator by ^^a = 2/ae2)

We consider the estimator m C i

1 `fee ei

e _ i= mC

gee

iLl According to the remark by R. A. Fisher (1925) that the second order derivative of the loglikelihood function gives the "actual" information of the MLE , we may compute. By a similar way as in case ( i) we have * (2.3)

ze

+

*

In 3J1 +K1 * * 6861 He ze z e I 2 i=1

1

1I * 2 zeeze -

Iee

+

1 me 3Jieee Keee i i2 + 2v I*2 i=l l

ze ee

206 AKAHIRA-TAKEUCHI

24

+

1

,T

i f 1 Zi Zi - I m J8 Zi z*

m CC

I*

i11

I

i 1 JOSE

8 12

I*2

+ op

i=l

IU

^

r 11

=1 i I Z 21 I* _ *Z 8 I

+1Q 0 1 1 2 +p + 1 Q lr 1 Q2 0 r

z+ I

/rn- An-

1

Q + 0 1

(say)

p

vrn-

where m

Q2 =

2 21

1I 1

(3J688

*

+K660)

i

2

z 8 i Z8

(I lee

Since Cov(QO,Q2) = Cov(Q1,Q2) = 0 it follows that (2.4) V(Q) = V(QO) + V(Q1) + V(Q2) with m (3J i 888+K i888) 2 _ (3J*+K*)2 V(Q2) = 21 *2 i=l let 21 *4 (iv) The weighted estimator by Ie8-I68
m = i=1 C i i=1 188 Then the stochastic expansion of is given by

207

AKAHIRA-TAKEUCHI

1

^(e-e)

*

Ze

1

+ * ✓n I

I

m

1

i

i zeez e

i=1 186

I*2 i= 1

IN

+

i i Jeee +Ke6e

*

i2

z i2 e

Ziz

6 6

6

1 1 zi zi T I* i=1 I e

in

2^ I i=1 Iee

1 m 2J N i e+K ei66

+

1

25

i

i

+ 1 J6 e+K66E zizi v I* i=1 Ii I 00

M Ti

i 2Je^e+ KOe zZ*+o( 1 i=1 I e p

0 Z it 1 C' * L. 12 P T T 2V I i=1 I 1

Z e + 1 QO+ I

1

QI+

✓n

,r

*

ze + 1 + 0

Q3+

1

f

Q4+op 1

r

1 1 (say),

p

I

1

where in Q3 I

i i i 1 in Jeee+Keee 12 Z Z +Z e8 e IN e 21 i=1 IN

1

i=1

1 C 2Je0e+Kei i e6 i * _ 1 * * 3J*+K* *2 zeze 1 *2 zeeze + 2I*3 Ze I *2 i=1 Ii

ee

i i i in J6 6+K6e ze Ze = * Q4 I i=1 I

* 66

z

We divide Q3 into two parts Q5 and Q6 as follows: 1 m C^

Q3

I iLl

* 1 in Jeee i* zeez 6 1* i=l Ii Z6z6 ee ee

1

i

i

1

Ii z66z e

*

I*

1 me i *2 1 mC i i z e + I* icl J66eze + 2 I* ix (Jeee+Ke0e) Ii _ 1 m

ze

I* i=1 I1

ee

m

- Ze i _ J8 Z

I*

6

I

z1

Z

+ 21* ixi (Jeee + K6ee) 16 - I

es = Q5 + Q6 (say) .

z

208 26

AKAHIRA-TAKEUCHI

Since Cov(Qi,Q.) = 0 (i i j; i,j = 0, 1, 4, 5, 6) , it follows that (2.5) v(Q) = V(QO) + V(Q1) + V(Q4) + v(Q5) + v(Q6)

(v) The weighted estimator by I08_= Ie ( e,Ei) (i We consider the estimator me L 160 Ii

9* = 1=1 m

Then we have

r(ee) _

Ze +

1

QO +

1 Q1

+

1

Q4 +

1

I

Q7 + 0

P(1 An-

1 z * + 1 Q* + oP (say) I where i C^ Jeee i * J *2 __ 1 m 1 i i - 1 * 1 Cm ZeeZe I* i11 Ii Z 0Ze + I* Ze ZeeZe Q7 1* ae

1* iLl Ii

i

ee i

i

i

i2 1 m 3J66e+K6e6 i 1 m 3J66e+Ke6e X Z Z X Z +*3 21*2 i=1 i2 e I i=1 Iee e e

* *

+KZe *2 _ 3J 21*4

Since 2

m Q7 = Q5 21 *2 iLl(3J000+K000) Ze 1 ( Il e

Ze

=Q5-Q2 and Cov(Qi,Qj) = 0 (i # j; i,j = 0,1,2,4,5), it follows that (2.6) V(Q*) = V(QO) + V(Q1) + V(Q2) + V(Q4) + V(Q5) .

209

AKAHIRA-TAKEUCHI

27

(vi) The asymptotically best estimator based on 6. andi (i = 1, ,m). The estimators S and S have the property of being functions of 8i and

i (i = 1,..., m) alone. Now let us construct an estimator

6 = f61t...,6m,

1,..., m)

which is asymptotically best among the estimators which depend only on 6i and

i (i = 1,...,m).

We assume that f is twice continuously

partially differentiable in 6i and

i (i

Assuming the

condition of consistency

Denoting of

Z1 =C19 .... m Em ;

e1 b = of

k J

0 6l 61e,...,6m =6, El Ell...l Em Em 81e,...'0m 0 , El Ell..., EM =E M

a2f eQk aZlD^k for i, j, k, 2, = 1, ..., m , we have from the consistency condition that m ai = 1; bk = 0 i=1 and also

( k = 1,...,m)

210 AKAHIRA-TAKEUCHI

28

m

aai m m c = ; L c.. = 0 ; j=1 iJ ae i=1 j=l 1J C aai mm iLl dik 0 ' dik aEk ' e Qk = 0 for i, k, Q = 1, ..., in. Taylor's expansion of 0** yields that

e** = 0

in m m + ai (01 0) + 2 I cij( ei- e)(0 cci=1 i=1 j=1

j -0)

+ G X dik(el e) (Ekk) + op(n) i=l k=l **

,r(e -0 )

The asymptotic variance of

is given by

2 n 2 mCC ai n ai V(0i) L i i=1 i=1 00 m of which minimum value under the condition I ai = 1 is attained when i=1 i i ai = m.00 = 180 C i I

(i = 1 ,...,m) .

i==1 00 Then we have aai dik

aCk

2Je0+K00 s ik

Ie0(2Je^0+Kee^) *2

for i, k = 1, ..., m , where 6ik denotes Kronecker ' s delta. It follows from these conditions that m

m

m

,T(e Iie ,r(si-e) + 1 cijn(ei-e)(0.-0) I

i=1

2vrn-

i=l

j=1

m Cm n(Oi-0)(^k i=l k=l dik vrn-

k) + oplYrn)

211

AKAHIRA-TAKEUCHI

29

and substituting the stochastic expansion of 0i we get m

r(0 **-0) Z I

O+

1 *

0

Ze

Z1

Ji m Zee _ e9e + 1

i =1 7i 00 109

I

Zi * Zi

V I i

=1 I^

0

m cc i i i 1 * L Jed Z12 + 1 * C J09+K90 ZiZi n 37n I i=1 I 7 I * i== 1 I80ICc 0 i i 1 m 2JeE0+Ke0 Zi I*2

i=1

I

E

Z 0

i - d J OO+Ke0 ZiZi CC CC + 1 L L 2Vn I* i=1 j=1 Ieeie9 ij I*Iee e 9

1

Ze + 1 Q0 + 1 Ql + opl 1 / (say) , I \\ ^/ where 1

Q1

ci

J000+K990 i ^- - d . Z Z Z i 21 i=1 j=1 1601e0 ^ I*Iee 0 0

Since we have Cov(QO,Ql) = 0 we should minimize V(Ql) in order to minimize the asymptotic deficiency of 0** relative to 6* under the condition

m ta2J K I(2J*+K*) i 00000 00 + (2.7) ,l =c ij = a6 = I* - 1*2

for i = 1, ..., m. It is expressed by

V(Q*) _ * c - d J099*K000 1 2I i=1 j=1 ii i^ I 109190 and noting that cij = cji we get by applying Lagrange 's method of

212 30

AKAHIRA-TAKEUCHI

constrained minimization i

c ij

8i3

Jeea *K eee

+ 10000

^`i+Aj) (i,j = 1,...^m)

where ai (i = 1,...,m ) are Lagrangean multipliers.

From (2.7) we have

m m 0= I I ci. =-J*K +2I A1IA0 . i=1 j=l I i=1 Hence C i = - J*+K* i=1 i I 0e21 2 Again from (2.7) it follows that i

i

ci = J006*K000 + A1Ie0I* + a.Iee 160 j=1 ;1 2J6ee+Ke0e Iee(2J*+K*)

i _ J000 180(3J*+K*) ^i10e

I

21

*2

for i = 1, ..., in. * Consequently the minimum of V(Q1)

is attained when

I1 Ji +I3 Ji Ii Ii (3J*+K*) ii +Ki + a eee eee ee eee 00 000 ee 00 cij = Z* 1*2 ij 1*

and * __ 1 * m J ie00 1 3J*+K* *2 Z e Z Q1 1*2 0 i=1 Zi 0 21 *3 0 ee Hence the asymptotically best estimator a is expanded , after some rearrangements of terms into the form

/_(9**-e) Z* + 1 Q0 + 1 Q1 + 1 Q4 + 1 Q5 + 0 1 ✓n p (^n::) YST Z fn F

213

AKAHIRA-TAKEUCHI

31

and the asymptotic deficiency of 0 relative to 0 is given by I{V(Q4) + V(Q5)}. Comparing the above with the stochastic expansion of 0 , we get

0 - = - n Q6 + op (1) n m

1* (J000+K000 )( 0i-0)2 + op(n) . 21 i=1

Hence an estimator which is asymptotically equivalent to 0** up to the order n-1 is given m

0*** = 0 _ 1*

2I i=l

(3600+Ke00 )(

0i e)2

where I*, 3000 and K000 are estimators of I*, J000 and K000 respectively, obtained substituting 0 and Ci for

0 and

Ei (i = 1,

...,m). The adjusting term m 1* X (J000+K000)(0i 2I i=1

-0)2

may be considered as the estimated non-linear effect in the best combination of 0.'s. i From the above we have established the following theorem. THEOREM 2.1. Assume that (A.1) - (A.5 ) hold. The stochastic expansions of the estimators 00, 0*, 0, 0, s and 0** are given by

,T(e0-0) z0 + 1 QO + o 1 ; ,T P (,' ) Z0 + 1 Q0 + 1 Ql + op\1/ Yrnvrn✓n(6-0) = * Z0 + 1 QO + 1 Q1 + 1 Q2 + 0 1 p F ,r ,T

/

^(80) _ * Z* +1Q + 1 Q +1Q + 1 Q +1Q +o C1); 0 F 0 Fn 1 p F 5 v 6 + vrnFn 4

r(e*-0)

1

QO + 1 Ql - 1 Q2 + 1 Q4 + 1 Q5+opl 1 ) Fn F vrnFn r `vln-

Z*

I

214 32

AKAHIRA-TAKEUCHI

* Z0 + 1 QO + 1 Ql + 1 Q4 + 1 Q5 + o l rn r dn` r p rn respectively , where

Q5 and Q6 are given in the above Q0' Q1' Q2' Q4' and Cov (Qi,Q.) = 0 for i ¢ j (i,j = 0,1,2,4,5,6) except for i = 2 and j = 6. The asymptotic deficiencies of 0, e , 6* and A** relative to 0 are given in the table below.

Estimator Tn

Asymptotic deficiency of Tn relative to n I*V(Q2)

I*{V(Q4) + v(Q5) + v(Q6)} I*{V(Q2) + v(Q4) + v(Q5)}

I*{v(Q4) + v(Q5)}

6**

From Theorem 2.1 it is seen that 0 is asymptotically better than ** 0, 8, S and 0 up to order n . Using the estimator 0 we may construct an estimator 0* asymptotically equivalent to 0 as follows: in + 0* = 0 - ml ( 3J600 Ke00)(6 0)2 C i 1=1 l l `^00 2 i where J606

= J600 ( 6.) and K660 - K600 (

0i) (see Akahira and Takeuchi,

1981b). We can formulate still another estimator , i.e., the linear estimator 0 given by (m i* L 3AA 04

e* = =1m

,

i I AAA

i=1 where for each i = 1, ..., m the weight `966 is defined by 2

`^00 n a 2 Li(0'Y ae

215 AKAHIRA-TAKEUCHI

33

(0 may be replaced by and i). Then we get

Z0 + 1 QO + 1 Ql - 1 Q2 + op(1 j r Therefore the asymptotic deficiency of 0* relative to 0* is equal to that of 0. Moreover , we have from the above, i i* `^66 + `_00 c i me me

+6 1 m 2 2 iLl

iLl `^0A

iLl `'00 Thus the estimator

* 0i = 0 + op(n)

( 0* + 0)/2 is asymptotically equivalent to 0* up

to the third order, i.e., order n - l , that is , the asymptotic deficiency * * is zero , which can also be shown by the of (0 +0 / 2 relative to 0 more direct way as follows. Let 0 be the MLE .

Then we have

1ma Li(0, 0 = n 1 a0

- e )2

k.)(e*-e) + 1 1Gm a3 L i Z)(6*i = n 1 a2 L (0 (0 i' ae2 i i' i i 2n i a03

+0 (1) p n

Similarly we have 1m 0 = n 1 a6 Li(0i,Ei) 3 2 Li(0*. i 0*) +2n I a 3 Li(0*, *) (0 6*) 2+0p(1 ) = n a 2 a0 1 a0 And noting that 1 n 1

I 1

m

a3

1

m

0 3 Li(0i.^i) = T aA m

a2

*

1

n

in

a3

E

3

Li(0 * ,Z.) + op(1 )

1 a0

a2

n 1 a02 Ll(0 , i) = n ae2 Li(0 , i) + op(1) we have m 2 2 I 2 Li(0i,^i) + a 2 Li (0, n a i) (0*-0i) = 0 (n) * Z*)l

l t ae

Thus we get

ae

216 34

AKAHIRA-TAKEUCHI

m

i

i*

^

E `(Jee + use) ei i

o

ee + 00

Hence we have established the following theoremTHEOREM 2 . 2. Under the assumptions (A.1) - (A.5), the estimator a is asymptotically equivalent to the MLE 6 up to the order n -^ and its * asymptotic deficiency relative to 6* is zero , and the estimator (e +e)/2 is asymptotically equivalent to 6 up to the order n and its asymptotic deficiency relative to 6 is zero.

3. Discussion. V(Qi) (i = 0,1,...,6)

can be interpreted in the following way. V(QO)

consists of two parts, i.e., * * 2 ) and (J +K)) *4 (M*I* - J*2 I 21 the first being the "loss of information " of the MLE and the second due to the condition of (higher order asymptotic ) unbiasedness ( Amara, 1980). These two factors are unavoidable even when we know the true values of the nuisance parameter. Then for the MLE 6 we have three additional terms i2 i2 1 1 1 i _ J6c0 _ J0

rr (MeeE

I*2 i1l Ii

I1

Il

i2 1 C Je 2I*2

iY

i2 I

and i2 1 t° 1 _ 1 Je;6 1*2 i11 Ii I Ii

ee

of which the first is the "loss of information " due to the presence of

217

AKAHIRA-TAKEUCHI

35

unknown nuisance parameters , the second is the unbiasedness effect and the third represents the loss due to pooling. V(Q2) can be considered as representing the effect of the nonlinearity of the optimum combination of the MLE ' s to get the MLE based on the whole sample. Comparing 0 with 0 , it is shown that the information contained 60 (i = 1, ..., m) are lost when we use I66 (i = 1, '3 ... ,m) in its stead, and V(Q5) represents the loss with respect to 0

in the statistics

and V(Q4) with respect to Ci (i = 1,...,m). Comparison of 0 and 0 does not lead to any uniform conclusion, which may seem strange since in 0 a better estimator 0 of 0 is used instead of 0 (i = 1,...,m) , but it can be interpreted that 0 could be better since I00(0i'Ei) may be closer to J 60 than 160O'Ei). Comparing 0** with 0* , it is shown that the loss of information caused by restricting the estimators to be functions of 0i and Ci (i = 1,...,m) alone is expressed by V(Q4) + V(Q5). Of the two terms Q4 and Q5 the latter is the weighted mean of the asymptotically ancillary statistics JI

ZAe - 000 Zei

(i = 1,...,m)

lee and is easy to interpret , but the former is a weighted sample covariance between

( Z/I e0 ) - ( Z/I ) and Zi/Iii , but its meaning is harder to

grasp. In case of the symmetric location -scale problem in which the density function g(x,0,Ei )

is expressed as

1 f for each i = 1, ..., m and where f is an even function, we have for each i = 1, ..., m ,

218 AKAHIRA-TAKEUCHI

36

I00 2, 2 ; Ie = EI{ae (x, e, )}{a g(X,0, )}^ = 0 i

i

L

where A and B are constants determined as

A = with

f(x)d,; B = J x2{^' ( x)}2 f(x)dx

fi(x) = log f ( x). Further we obtain for each i = 1, ..., m

K600 = - 0 i -=0; i J600 JeO^ = 2 i3 (3A+C) Jere = 2 13 (-A+C) ; K60 _ i3 (-A-C) where C = I x{^'(x)}3 f(x)dx . And for each i = 1, ..., m , JO - 0; JO = 0 It is further calculated that for each i = 1, ..., m ,

where

D = J{4"(x)}2 f(x)dx and F = J x2{c" ( x)}2 f(x)dx . Therefore , in the estimation of the common location parameter of different populations of the same shape but with different scale parameters, we have Q2 = 0 and Q6 - 0 , thus we have 6 = 8 and 0 = 0 . Especially , when the underlying distribution is normal, we have for each i = 1, ..., m ,

J06^ 2E3 {31

x2 f(x)dx - J x4 f ( x)dx} = 0

and Je,O + Klan = 0 , which implies that Q4 0 and also Z60 = 0 hence Q1 = Q5 = 0. Then we see that 0*, 0 and 0 ** asymptotically equivalent , and the most commonly used estimator

are all

219 AKAHIRA-TAKEUCHI

37

6i/j 8=i=1 j./Z2 11 n n 0. = i = n X Xij and 1 = nll (Xij-% i2 (1 = 1,...,m) is, j=1 j=1 although not identical with the MLE , asymptotically equivalent to it up

with

to the order n 1.

Acknowledgements. This paper was written while the first author was at the Limburgs Universitair Centrum in Belgium as a guest professor .

The visit was sup-

ported by a grant of the Belgian National Scientific Foundation (N.F.W.O.). He is grateful to Professor H. Callaert of the LUC for inviting him and for his interest in this and related work.

References Akahira, M .: On asymptotic deficiency of estimators .

Austral. J . Statist.

23, 67-72 (1981). Akahira, M . and Takeuchi , K.: Asymptotic Efficiency of Statistical Estimators: Concepts and Higher Order Asymptotic Efficiency , Lecture Notes in Statistics 7, New York-Heidelberg -Berlin, Springer ( 1981a). Akahira, M . and Takeuchi , K.: On asymptotic deficiency of estimators in pooled samples .

Technical Report of the Limburgs Universitair

Centrum, Belgium ( 1981b). Amari, S. : Theory of information space : A differential -geometrical foundation of statistics. RAAG Report , 106 (1980). Fisher, R. A .: Theory of statistical estimation .

Proc. Camb . Phi.. Soc.

22, 700-725 ( 1925). Ghosh, J. K., Sinha , B. K. and Wieand , H. S.: Second order efficiency of the mle with respect to any bounded bowl-shaped loss function. Ann. Statist. 8 , 506-521 ( 1980). Hodges, J. L . and Lehmann , E. L.: Deficiency . 783-801 ( 1970).

Ann. Math. Statist. 41,

220 38

AKAHIRA-TAKEUCHI

Pfanzagl , J. and Wefelmeyer , W.: A third order optimum property of the maximum likelihood estimator. J. Multivariate Anal. 8, 1-29

( 1978).

Takeuchi , K. and Akahira , M.: Third order asymptotic efficiency of maximum likelihood estimator for multiparameter exponential case. Rep. Univ. Electro -Comm. 28, 271 -293 (1978).

Masafumi Akahira Kei Takeuchi Department of Mathematics Faculty of Economics University of Electro- Communications University of Tokyo Chofu, Tokyo 182 Hongo , Bunkyo-ku Japan Tokyo 113 Japan

221

Ann. Inst . Statist. Math. 37 (1985), Part A, 17-26

ESTIMATION OF A COMMON PARAMETER FOR POOLED SAMPLES FROM THE UNIFORM DISTRIBUTIONS MASAFUMI AKAHIRA AND KEI TAKEUCHI (Received Sept. 6, 1983; revised Mar. 12, 1984)

Summary The problem to estimate a common parameter for the pooled sample from the uniform distributions is discussed in the presence of nuisance parameters. The maximum likelihood estimator (MLE) and others are compared and it is shown that the MLE based on the pooled sample is not (asymptotically) efficient. 1. Introduction In regular cases the asymptotic deficiencies of asymptotically efficient estimators were calculated in pooled sample from the same distribution (Akahira [5]) and in the presence of nuisance parameters (Akahira and Takeuchi [7]). In non-regular cases the asymptotic optimality of estimators was discussed in Akahira [4], Akahira and Takeuchi [6], Ibragimov and Has'minskii [10], Jureckova [11] and others, and also recently a Monte Carlo study on the estimator considered in Akahira [1]-[3] has been done by Antoch [9]. In this paper we consider the problem to estimate an unknown real-valued parameter 0 based on m samples of size n from the uniform with different distributions on the interval (0-, B + ^1) nuisance parameters which is treated as a typical example in non-regular cases. In some cases the MLE and other estimators will be compared and it will be shown that the MLE based on the pooled sample is not better for both a sample of a fixed size and a large sample. Related results can be found in Akai [8].

2. Results Suppose that it is required to estimate an unknown real-valued Key words and phrases: Maximum likelihood estimator, weighted estimator, uniform distributions. 17

222 18 MASAFUMI AKAHIRA AND KEI TAKEUCHI

parameter 0 based on m samples of size n whose values are Xif (i=1, • • , m ; j =1, • • • , n) from the uniform distributions on the interval (0 0+^i ) with different nuisance parameters ^i (i=1,• • •, m). For each i let X(l)<X(2)<• .. <XX(,,) be order statistics from Xi1, XXE, • • •, Xi,,. We consider some cases. C a s e I. ^i = E (i =1, • • • , m) are unknown.

The MLE 0ML of 0 based on the pooled sample {X1} is given by d^L = 2 (min Xi( 3)+max Xi(n)) • lsism 15i$m

An estimator B„ of 0 based on the sample of size n is called to be (asymptotically ) median unbiased if Pr {&n 0} = Pr {dnB} = 2

^lim IPr {d&S0} - 1 I =lira Pr {dn 2

B} -

n--

2

=0 uniformly in some neigh-

borhood of B) (e.g. see Akahira and Takeuchi [6]). Then it is shown that dML is one-sided asymptotically efficient in the sense that for any asymptotically median unbiased estimator d„ lim [Pr {n(&IL-0)S t} - Pr {n(dn-0)St}]? 0

for all t>0 ;

{n(dn- 0) St}]S0

for all t<0 ,

or lim [Pr {n(#ML-0) St} -Pr n-.m

since in this case d,L is actually the MLE from the sample of size mn (see Akahira and Takeuchi [6]). Case II.

^i (i =1, • • • , m) are known. The MLE d ML of 0 based on the pooled sample {X f} is given by 1

dML= 1 {max(X(n)2

^i)+ min 15i;sm 1;ism

(

XX (

) +Si)}

Then it can be shown in a similar way as in [4] and [6] that 0ML is two-sided asymptotically efficient in the sense that for any asymptotically median unbiased estimator do

Lim [Pr {nIdML - 0^
for all t>0 .

n--

(For the definition see Akahira and Takeuchi [6], page 72 and Akahira [4l).

223 ESTIMATION OF A COMMON PARAMETER FOR POOLED SAMPLES 19

C a s e III.

Ei (i =1, • • • , m) are unknown. For each i the MLE's 9i and ^i of B and Ei based on the sample Xil, • • • , Xi,, is given by i=

2

(X,(1)+Xi(n )

i=

,

(Xi ( n)-Xi(l,) ,

respectively . Let &IL be the MLE of 0 based on the pooled sample {Xij} . Then it will be shown that BML is not two -sided asymptotically efficient . We define 9,*= 1 {max (9i+ Ei -E°)+ min (9i- Ei+Et)} 2 15i5m

Then it can be shown from Case II that 9n*, is asymptotically locally best estimator of 0 at Ei = EOt (i =1, • • • , m) in the sense that for any asymptotically median unbiased estimator 6. lim[ Pe{ n1,*e1
{n19n-

0I t }] >_0

n-w

for all t>0. First we shall obtain the MLE 9ML based on the pooled sample {Xij}. Let fi(x, 0) be a density function of the uniform distribution on the interval (0 - Ei, 0 +E). Since the likelihood function L(0 ; E l, • • , E m) is given by m

L(0; E 1,..., Em ) =TT

n

TT f

i(

i=1 j=1

x j, 0)

1 1 2n

n

for

xi(n

TT E i

)- E i;50SXi<1, + E i (i=1,•••,m);

i=1

0 otherwise . In order to obtain the MLE 9ML it is enough to find 0 minimizing if[ Ei i=1

under the condition Ei _ max {0 - xi(1) , xi(n,-0} for all i. Let 9* be some estimator of 0 based on the pooled sample {Xij} . For each i we put E*=max {B*- xi(l), xi,n) -B* }. Then we have for each i

E* =max {9 * - 9i+Ei, Hence the MLE

9$ L

ei

+Ei -B*}

=

Ei +I

B i -d *1 .

is given by 9* minimizing

TT (gi +I -9 *I) 9i

i=1

that is, such an estimator 9* is given by either of the estimators (i=1,•••,m). Since

224

20 MASAFUMI AKAHIRA AND KEI TAKEUCHI

TT (^i +1Bi-B *D=TT TT (1+ for sufficiently large n it is asymptotically equivalent to

TI si(1 +E

lei-e*^ 1 .

Hence it is seen that for sufficiently large n the MLE eML is asymptotically equivalent to a weighted median by the weights 1/^i (i = 1, • • • , m). Next we shall discuss the comparison among B, L, the weighted estimator and other estimators . We consider the case when m=2. Then for each i=1, 2, Xi,, • . • , Xi„ are independently , identically and uniformly distributed random variables on (0-^i , 8+ei). Then for each i=1, 2, the joint density function fn(x, y; 0, ei) of Bi and ^i is given by

n(2^n

(2.1)

fn(x,

1)

y; 8, ^i) =^

n -z

for 05y5^i and

B-^ i +y5x58+

i- 2J

;

0 otherwise .

For each i=1, 2 the density function fn(x; 0, si) of 0, is given by (

(2.2)

n n-1 (;i-Ix-BI )

for B - ^7i<x<8+^;i ;

fn(x ; 0, s^i)= 2^i 0

otherwise .

Also for each i = 1, 2 the conditional density function f.(x I y ; 0, $i) of

Bi given ti is given by (2.3)

fn(x

I y ; 0, e i ) =

1 2(e -y)

for B -^:i } y5x
0

otherwise ,

that is, the conditional distribution of Bi given ^i is uniform distribution on the interval (0-(^i-gi), 8+(e,-gi))• For two (asymptotically) median unbiased estimators B' and B2 of 0, B' is called to be (asymptotically) better than e2 if Pr {nIB'-0l tI&„ 2} _ Pr {nIB2-8I5tI^,, 2} a.e. for all t>0 (lim [Pr

{n1 B' -Oast} -Pr {nIB2- 1 _0

for all t>0), and then for

n-.Co

simplicity we denote it symbolically by B' } B2 (B' } 62), where Pr {A as.

^2} denotes the conditional probability of A given e, and ^2. From (2.3) we see that the conditional density of 6,-0 and #,-0 given ^, and 2

225

ESTIMATION OF A COMMON PARAMETER FOR POOLED SAMPLES 21

is given by

fn(

x l,

x21 ^1, 2) _

1 4T1z2

for

I x il
0 otherwise , where Ty =e2-c , (i = 1, 2). If c1 7*1 > c2T2, then the conditional density function f.(yI g1, ^2) of Bo-0 With Bo= C1B1+C2B2 given ^1 and 2 is given by 1

for y< - c 1T1+c2T2 ;

(C1z1+c2z2+y) 4clC2T1T2

1 2c1z1

(2.4) fn.(y I ^1, ^2) _

for

1

for y>c1r1-c2z2.

(c1T1+c2z2-y)

4c1c2T1z2

I

Iy
If c1z1
1

(c1Tl+c2z2 - y)

4c1C2T1z2 (2.5)

fn(yI ^ 1,

2)=

for y> - C1z1+c2z2

1 2c2z2

for

1

IyI<-C1z1+c2z2;

for y
(Clzl+C2T2 +y) 4C1C2T1T2

If c`=

^5/(^1 +&)

=c% (say)

(i j; i, j =1, 2) ,

then

(2.6)

e c C2T2 51 ( 52-^2)

('lzl 2(51-e1)

If e`=g>/( 1+^2)= C'' (say)

(i*j; i, j=1, 2) ,

then (2.7)

1 _ iG-_ _s2 i'Tl ^2 ( S1-S1) ' C2 'z2 ^1(E2-2)

Hence we have the following :

(2.8) 22 1 if and only if

1

= ^ 1 +52

Cizl 2

z

'.C 1-2

=1

2 1 ( ^l - J 2(L l - t1) C1 71

226 22 MASAFUMI AKAHIRA AND KEI TAKEUCHI

+^2 ^i '-2

(2.9)EZ1 if and only if

ee (CC tt S1 s2-S2) t1(S2-S2) C2'T2

c 2

On the other hand as is seen from the above discussion on the MLE, the MLE BML based on the pooled sample {X,5} is given by

0ML =j

& if 1<2 ,

02 if e1 > e2

The conditional density of the MLE B,^L given g, is given by (2.3). Now we consider two cases, i.e., ^1= e2 and S 1 # e2. Note that the first Case III, was also mentioned in the Case I with another optimum criterion. Case IIII .

e.

= In the case when ci,=ci=^5/(&1+^2) (i j; i, j=1, 2), S1=S 2

e1<>g2 if and only if ,(^-g 2) >S2(S - ,) ,

i.e. ,

C2r2>C,r 1.

Hence we have rl

(2.10)

e1

<> ^2 if and only if

cir1 +C27.2 < r2

We also obtain 2 CCe1e2C[(^1 t-^2) Z0 Ci'T1+C2 r2-(civ1+ C2r2) =

From (2.4) to (2.10) we have established the following theorem.

(i)

0

ri C ( r 1+ C2 r2

r

C l r 1+ C 2 r 2

Fig. 2.1. Comparison of the conditional densities of c161+c262-0 given & and S2 with (i) cz=c,1=^j/(&+t2) (i#j; i, j=1, 2) and (ii) ci=c'' =tj2/(ei+ ea) (i4j; i, j=1, 2) and of (iii) the MLE Bart given &s. They are given by (2.5) and (2.3), respectively.

227

ESTIMATION OF A COMMON PARAMETER FOR POOLED SAMPLES

THEOREM 2 .1.

23

If e1=^2 and S1<s2, then ^2B1+ ie2 2BI+b 102 >'BbiLB1 ee S1+S2 ^1+^2

if

e+1=52

and

e1 > e2,

then c2B1 +cz }, S2B1 +cie2 } ^1+ee le 22 52 1+ 2

B ML -B2

We assume that c1=c2=1/2 and ^1=^2=e. By (2.2) it follows that for each i=1, 2 the asymptotic density of n(&-B) is given by

.fi(x)= 1 e -1xW

(2.11)

Then the characteristic function qS(t) of the asymptotic density function of n[{(BI+B2)/2}-B] is given by (1+e2t2/4)z We may represent fi(t) as follows : 2 /4

^ t2

(2.12) t)-

1 + 2(1 +e 2t2/4)

Since

f - 1 exp

(- 2 IYI) exp (ity)dy = ^ exp (- Ix I) exp (i_x) dx 2 -1 1+e2t2/4 '

3 i i exp (--- iYl ) exp (ity)dy= 2

s:

IxI exp (- Ix I) exp (i

_ x)dx

Fig. 2. 2. Comparison of the asymptotic densities of n(61-8) and n[((61+ 62)/2) -0] given by (2.11) and (2.13), respectively.

228 24 MASAFUMI AKAHIRA AND KEI TAKEUCHI

1- Q t2/4 (1+S2t2/4)2

it follows from (2.12) that the asymptotic density of n[{(&+02)/2}

-e}

is given by (2.13)

Lx -1 exp (- fixi)

From (2.11) and (2.13) we have established the following theorem. THEOREM 2 .2.

If

e1 = e2=^,

then

Case III2 • $1#s2• We consider only the case when

e1

if &1«2;

02

if ccl>2

S,

<$ ,.

We assume that c1^1 * e2S 2•

By (2.2) it follows that for each i=1, 2 the asymptotic density of n(0, - 0) is given by

(2.14)

fi(x)= 2 t exp (-IxI/^{) ,

and also its characteristic function ¢,(t) is given by

01(t) = 1

1+Vt2

Hence the characteristic function ¢*(t) of the asymptotic density of n(0o-B )=n(c101+c202-0) is given by

*(t) = 1

(1 +cieit2) (1 +cz^at2)

Fig. 2 .3. Comparison of the asymptotic densities of n(9^ - B), n(9o-B')= n(c191+c292-B) and n(42-0) given by (2.14), (2.15) and (2.14), respectively, when F,«2.

229

ESTIMATION OF A COMMON PARAMETER FOR POOLED SAMPLES

25

Since (c'V C 'Ie2) 1 +c;eit2 1 + c2^2t2 ) it follows that the asymptotic density f(x) of n(Bo-0) is given by (2.15) f(x)= 2(c,e11 1 clea) {Cii exp (- c 1) -c2^2 exp (__!L ) There exists a positive number to such that

5°

50 f()d

f(x)dx ,

where f1(x) and f (x) are given by (2.14) and (2.15), respectively. Hence we have established the following theorem. THEOREM 2 .3.

Suppose that

and c1`s 1 *c2^2.

S1<S 2

Then

OML-01>- B0 as.

in some neighborhood of 0 in the sense that lim [Pr {nIBML-9ISt}-Pr {njBO-9I5t}]z 0

for all t<-t0.

noo

Further 00 >-OML =B1 IS.

in far away from 0 in the sense that lim [Pr {nI&-0jSt} -Pr {nI&L-0ISt}]>_O for all t>t0. Remark.

From (2.14) and (2.15) it is easily seen that

0O> 02

and

as.

0ML -01 > 02 . as.

The case e1>2 may be treated quite similarly. UNIVERSITY OF ELECTRO - COMMUNICATIONS UNIVERSITY OF TOKYO

REFERENCES Akahira, M. (1975). Asymptotic theory for estimation of location in non-regular cases, I: Order of convergence of consistent estimators , Rep. Statist. Appl. Res., JUSE, 22, 8-26. [ 2 ] Akahira, M. (1975). Asymptotic theory for estimation of location in non- regular cases, II: Bounds of asymptotic distributions of consistent estimators , Rep. Statist. Appl. Res., JUSE, 22, 99-115.

[1]

230 MASAFUMI AKAHIRA AND KEI TAKEUCHI [3] Akahira , M. (1976). A remark on asymptotic sufficiency of statistics in non -regular cases , Rep. Univ. Electro-Comm ., 27, 125-128. [ 4 ] Akahira , M. (1982). Asymptotic optimality of estimators in non-regular cases, Ann. Inst. Statist. Math., A, 34, 69-82. [ 5] Akahira, M. (1982). Asymptotic deficiencies of estimators for pooled samples from the same distribution , Probability Theory and Mathematical Statistics , Lecture Notes in Mathematics, 1021 , 6-14, Springer , Berlin. [6 ] Akahira , M. and Takeuchi, K. (1981 ). Asymptotic Efficiency of Statistical Estimators : Concepts and Higher Order Asymptotic Efficiency , Lecture Notes in Statistics 7, Springer, New York. [ 7 ] Akahira , M. and Takeuchi , K. (1982). On asymptotic deficiency of estimators in pooled samples in the presence of nuisance parameters , Statistics and Decisions, 1, 1738, Akademische Verlagsgesellschaft. [8 ] Akai , T. (1982). A combined estimator of a common parameter, Keio Science and Technology Reports, 35, 93-104. [ 9 ] Antoch, J. (1984). Behaviour of estimators of location in non-regular cases: A Monte Carlo study , Asymptotic Statistics 2, the Proceedings of the 3rd Prague Symposium on Asymptotic Statistics , 185-195 , North -Holland, Amsterdam. [10] Ibragimov, I. A. and Has'minskii, R. Z. (1981 ). Statistical Estimation : Asymptotic Theory, Springer, New York. [11] JureMova, J. (1981). Tail-behaviour of location estimators in non -regular cases, Commentationes Mathematicae Universitatis Carolinae , 22, 365-375.

231

Rep. Star. Appl. Res., JUSE Vol. 32, No. 3, Sept., 1985 p.p. 17-22

A-Section

A Note on the Minimum Variance Unbiased Estimation When the Fisher Information is Infinity Masafumi AKAHIRA* and Kei TAKEUCHI**

Abstract In the case when the Fisher information is infinity, it is shown that the locally minimum variance of unbiased estimators is equal to zero. Some examples are also given.

1. Introduction In non-regular cases the minimum variance unbiased estimation has been studied by Chapman and Robbins [6] , Kiefer [7], Polfeldt [8], Vincze [101, Akahira [11 , Akahira and Takeuchi [21, [31, [41, [9] , Akahira, Puri and Takeuchi [5] and others. In the previous paper by Akahira and Takeuchi [2] they introduced the concept of the one-directional distribution and discuss the locally minimum variance unbiased estimation for its family. In this paper, from another point of view we shall treat the case when the Fisher information is infinity and show that the locally minimum variance of unbiased estimators is equal to zero. Further we shall give two examples.

2. Results Let (X, Y) be a pair of random variables defined on some product space x x I of ?I and with a joint probability density function (j.p.d.f.) f0(x, y) with respect to a a-finite measure y, where 0 is a real -valued parameter. Then it follows that for almost all (x, y) [µx l1x Y] f0 (x, Y) = f0 (x I Y) f0 (Y), Received September 11, 1985 * Department of Mathematics, University of Electro-Communications. 1-5-1, Chofugaoka, Chofu-shi, Tokyo 182, Japan. (The author is visiting Queen's University in Canada from April to July 1985.) * * Faculty of Economics, University of Tokyo. 7-3-1, Hongo, Bunkyo-ku, Tokyo 113, Japan. (Key words) Locally minimum variance, Unbiased estimator, Fisher information.

232 M. Akahira and K. Takeuchi where f0 (x I y) and f0 (y) denote a conditional p.d.f. of x given y and a marginal p.d.f. of y, respectively . We assume that for almost all (x, y) [µx yJ , f0 (x,y), fe(x I y) and f0 (y) are continuously differentiable in 0. Since log f0(x, y) = log fe(x I y) + log fe(y) a.e. µx y it follows that XIY1Y 1X, (0)Y =B E[1 (0)] + (0)

where IX,Y(o), IXIY(o) and IY(0) denote the amounts of Fisher information with respect to f0(x,y), f0(xly) and f0(y), respectively, e.g.,

alo aee (x,

IX Y(0) JJ 1

y) cZ fo(x,y)dµx,y

3€xV

and E0Y(.) is the expectation w. r. t. f0(y). We denote J0(Y) = IXIY(o). Here we assume the following conditions (A.1) to (A.3). (A.1) O<J0(y) <° for almost ally [µy] and all0, and E00Y[J00(Y)]= oo and IX,Y(00) for some Oo. (A.2) There exists a sequence I Bn I of measurable sets of the domain of Y such that bn E0Y[J00(Y) XBn(Y)]
VXIy(6 (}{)) = 1 for all 0, 0 y J0 (y) where EX B ly(.) and V0 '(.) denote the conditional expectation and the conditional variance ofXgiven y. We shall show that the locally minimum variance of unbiased estimators of 0 is equal to zero. Theorem : Assume that the conditions (A.1) to (A.3) hold. Then inf V00(O(X,Y)) = 0. 6(X,Y) : unbiased Proof : Let an estimator 6* = 6 *(X,Y) be 1 o* = bn J00^')XBn(Y)

16y(X) - 0 0 ^

+ 00 .

233

Minimum Variance Unbiased Estimation By the assumptions (A.2) and (A.3) we have

EB 'Y(0 *) = bn EB [JB0(Y)XBn (Y) I EB IY (0Y(X)) - 0017+ 00 = 0

for all 0, hence 0* is an unbiased estimator of 0, where EB 'Y(.) denotes the expectation w.r.t. f0(x,y). We also obtain by (A.2) and (A.3) EX, Y (0*2) = bnz E, Y [J002 (Y)X Bn (Y) 10Y(X) - 00 } 2 ] 00 60 20 0 E0 X, Y [J00 (Y)XB (Y) {6 Y(X) - 00 11 + 002 + b n 0 n bnz E 0 [J80 (Y)xBn(Y) EX'Y (0Y(X) - 00)2 ] 00 20 + b0 EB [J00(Y)XB (Y) I E0 [Y (0Y(X)) - 00 + 002 n 0 n 0 bnz E0o [J00(Y)XBn(1')1 + 002

Then it follows that the variance of 0 at 00 is given by V X,Y 1 00 ( 6* ) = b n which tends to zero as n --^

because of (A.2). Thus we complete the proof.

Next we shall show some examples. Example 1 . Let Xi = 0Yi + Ui (i = 1, 2, ... , n), where (Yi,Ui) (i = 1, ... , n) are independently and identically distributed (i.i.d.) random vectors and for each i, Yi and Ui are mutually independent. Let f(u) be a known density function of Ui. Assume that lU= f(u)12 du< 00 J f(u) Putting Y = (YI, ... , Yn ) we have IY(0)=0 since the distribution of Y is independent of 0, and EY(J0 (Y)) = nE(Y 12)IU.

234 M. Akahira and K. Takeuchi Assume that E(Y12) _ 00 and f(u) is a density of the standard normal distribution. For any K > 0 and each i = 1, ... , n we define Yi for IYiI S K ; Yi,K =

1

0 for IYi I> K .

Since E(Y12) = 00, it is easily seen that E(Y1, 2) tends to infinity as K ^. We also define an estimator n i E YiK Xi OK= nE(Y1,K ) It is clear that 0K is an unbiased estimator of 0 . Since the variance of OK at 0 = 0 is given by n E(E YiK2) i-1 VO(0K) = E0(0K2) _ l _ n2f E(Y1 K2)I2

1 nE(Y1,K2)

it follows that lim VO(OK) = 0, K- 00 hence inf V0(0) = 0. 6 : unbiased Example 2 . Let Y be an unobservable random variable according to the chi-square distribution with one degree of freedom. Assume that X1 ... , Xn are independently, identically, normally and conditionally distributed random variables with mean 0 and variance Y given Y , where n > 5. Then we have I Y I(X1 _0 )2] ^ = nEe ( Y ) _ 00. 1X,Y(e) nE i Y I( XlY 0 ) 21 = nEe [ - EX, For any K > 0 we define an estimator

for E (Xi -X)2 > 1 ; i=1

n for E (Xi -X)2 < K i=1

235 Minimum Variance Unbiased Estimation where cK is a constant such that eK is an unbiased estimator of 6. Then we shall show that K VO(OK) = 0. n _

First we easily see that E (X1-X)2 is expressed as a product of two independent random i=1 variables Y and Z, where Z has the chi-square distribution with n-1 degrees of freedom. Let f1(y) and fn_1(z)be the density functions of Y and Z, respectively. By the unbiasedness condition of eK we obtain 1 1 f 1 y )f

1

If > yz=K We also have the variance of eK at 0 = 0 2

1 (z

(2 . 1)

)dydz cK

f l (y)f n_l (z) dydz = vK (say). nyz2

f fZ> y Putting

00 1 y fl(y) dy,

F(Kz) = f 1 Kz we have 0 F(Kz) = c' f

3 y y 2e 2 dy Kz

,

1 Kz where c and c' are constants. By (2.1) and (2.3) we obtain 00 1 = cK J F

(Kz) fn-l (z) dz = A V KCK ,

0 which implies cK = O(K - 1 /2), where A is some constant. By (2.2) and (2. 3) we also have 0 vK = c2 F^2) fn-1( z)dz = B ^ cg = O (K-1 /2 nz 0

(2.2)

236 M. Akahira and K. Takeuchi where B is some constant. Then we obtain li V0(OK) = lim^"K = 0.

Hence we have inf VO(O) = 0. 0 : unbiased

Acknowledgements The authors wish to thank the referees for their comments. The paper was written while the first author was at Queen's University in Canada as a visiting professor. The visit was supported by a grant of the Natural Sciences and Engineering Research Council of Canada. He is grateful to Professor Colin R. Blyth for inviting him. REFERENCES [1] Akahira, M. (1984): "On the Bhattacharyya inequality in non-regular case," Surikaisekikenkyusho Kbkyuroku (Prot. Symps., Res., Inst. Math. Sci, Kyoto Univ.) 538, 65-80. [2] Akahira, M. and Takeuchi, K.: "The lower bound of the variance of unbiased estimators for one-directional distributions" (Submitted for publication). [3] Akahira, M. and Takeuchi, K.: "Locally minimum variance unbiased estimator in a discontinuous density function" to appear in the Metrika. [4] Akahira, M. and Takeuchi, K. (1985): Non Regular Statistical Estimation, Monograph. [5] Akahira, M., Puri, M.L. and Takeuchi, K.: "Bhattacharyya bound of variances of unbiased estimators in non-regular cases" to appear in the Annals of the Institute of Statistical Mathematics. [6] Chapman, D.G. and Robbins, H. (1951): "Minimum variance estimation without regularity assumptions", Ann. Math. Statist., 22, 581-586. [7] Kiefer, J. (1952): "On minimum variance in non-regular estimation",Ann. Math. Statist., 23, 627-630. [8] Polfeldt, T. (1970): "The order of the minimum variance in a non-regular case", Ann. Math. Statist., 41, 667-672. [9] Takeuchi, K. and Akahira, M.: "A note on minimum variance" to appear in the Metrika. [10] Vincze, I. (1979): "On the Cramer-frechet-Rao inequality in the non-regular case", In: Contributions to Statistics. The Jaroslav Hajek Memorial Volume, Academia, Prague, 253-263.

237 Ann. Inst. Statist. Math. 38 (1986), Part A, 35-44

BHATTACHARYYA BOUND OF VARIANCES OF UNBIASED ESTIMATORS IN NONREGULAR CASES MASAFUMI AKAHIRA, MADAN L. PURI* AND KEI TAKEUCHI

(Received May 19, 1984; revised Jan. 16, 1985)

Summary Bhattacharyya bound is generalized to nonregular cases when the support of the density depends on the parameter, while it is differentiable several times with respect to the parameter within the support. Some example is discussed, where it is shown that the bound is sharp. 1. Introduction It is well known that the Cramer-Rao and the Bhattacharyya bounds are most important and very useful for the variances of unbiased estimators. They are, however, not applicable to the non-regular cases when the support of the distribution is dependent on the parameter. Same is true about more general and simpler bounds by Hammersley [6], Chapman and Robbins [2], Kiefer [8], Fraser and Guttman [5], Fend [4] and Chatterji [3], among others. (For an exposition of some of this work along with extensions in different directions, see Polfeldt [10], [11] and the recent papers of Vincze [15], Khatri [7] and Mori [9], among others). In his paper, Polfeldt [10] discussed the lower bound of the variances of the unbiased estimators when the class of probability measures is one-sided, that is, when P, is absolutely continuous with respect to P02 (symbolically, P,,02. In this note, our main interest is to obtain the Bhattacharyya bound when for any 01, 02, with 010 02, neither PBI << PB2 nor P82 << P01. 2. Results Let ' be an abstract sample space with x as its generic point, A a o-field of subsets of ', and let ® be a parameter space assumed to be an open set in the real line. Let 9= {P0 : 0 E ®} be a class of prob* Research supported by NSF Grant MCS-8301409. Key words and phrases: Cramdr-Rao bound, Bhattacharyya bound, unbiased estimator. 35

238 36 MASAFUMI AKAHIRA, MADAN L. PURI AND KEI TAKEUCHI

ability measures on (X, mil). We assume that for each 0 E 0, P0(•) is absolutely continuous with respect to a c-finite measure p. We denote dPe/d,u by f(x, 0). For each 0 E 0, we denote by A(0) the set of points in ' for which f (x, 0)>0. We shall consider the Bhattacharyya bound of variances of unbiased estimators at some specified point 0o in 0. We make the following assumptions :

(A.1) p((n ( U A(so+h)))DA(so))=0 c>0 Ihl f(x, 0), and for every 0 E (0o-s, 0o+s),

1

I Ax) I Ax, 0) dp < oo implies A(0)

U

r(x) I p(x)dp < oo •

A(B)

(A.3) For some positive integer k lim sup n-o x E U A(8o +jh) - A(80)

>-1 (_V (

j)f(x, so

+jh) <00 ,

I h I 'p(x)

J=1

i=1,•••,k. First we prove the following lemma. LEMMA.

Assume that (A.1), (A.2) and (A.3) hold. If ¢(x) is any

measurable function for which

I ¢(x) I p(x)dp < oo, then X

lim 1 h-.o h' i

( -1)' ^ ) (x)f(x0

0o + jh)dfe=0

.

U A(6o+jh)-A(go)

J=1

PROOF. By (A.2) and (A.3) it follows that for every i=1,. • • , k and every 0o E 0, there exist positive numbers s and K, such that IhH(_1)1

() f(x, 0o +jh) < KKp(x) for l ihl <e and x E A(0o)c ,

where A(0o)c denotes the complement of the set A(0o). Also, it follows from (A.1) that for every j=1,• • •, i, there exist a sequence {s1„} of positive numbers converging to zero as n-+ oo and a monotone nonincreasing sequence {S1„} of measurable sets such that l jh l < s p„ implies

239

BHATTACHARYYA BOUND OF VARIANCES OF UNBIASED ESTIMATORS 37

A(Bo+jh)-A(Bo)cSin and p ( fl Sin)= 0.

If for each j =1,• • •, i, I jhI<

n=1

e in, then

hi

i (-1)i(j)O(x)f(x, 00+jh)dp i U A(Oo+jh )- A(Oo) J=1 Ki

I qS( x ) I P( x)dp

S

{

K{I cb(x)I P(x)dp

U Sin

U A(9o + jh)-A(Oo)

i=1

1=1

Ki I cb( x ) I P( x)dp , sE J=1 J Sin

which tends to zero as n->oo . Remark.

The proof follows.

The assumption (A.2) together with the condition in the

above Lemma is satisfied with p(x) _ Z cif (x, 0) when the following i=1

holds : For each Bo and s > 0, there exist countable points 01 , 02, • • • and positive constants c1, c2,• • • such that U A(6i)DA(B) for all 0 E (Boi=1

s, 00+s ) and that

and F, cif(x, Bi)>f(x, 0) for all 0 E (Bo-s, Bo

ci
i=1

+ s) and almost all x[P].

We assume the following : (A.4) For each x E A(00), f (x, 0) is k-times continuously differentiable in 0 at 0=0g. (A.5) For each i=1,• •, k, x, 00+h) 8Bi f( lim sup <00 , h-•0 XEA (80)

where p(x) is defined in (A.2).

P(x)

We now prove our main theorem on the Bhattacharyya bound of the variances of unbiased estimators. THEOREM 2.1. Assume that (A.1) to (A.5) hold. Let g(0) be an estimable function which is k-times differentiable over ®. Let g(x) be an unbiased estimator of g(0) satisfying

5

I 9(x) I P( x )dp < oo .

A(B0)

Further, let A be a k x k non- negative definite matrix whose elements are

240

38 MASAFUMI AKAHIRA, MADAN L. PURI AND KEI TAKEUCHI

A£f= ( 1 J a£f(x, 00) of f(x, 00) d ^` J Ax, eo) 1. ae£ aef

i, j=1,•••,k.

A(80)

Assume that 2££, i =1, • • • , k are finite. If d is nonsingular at 00, then (2.1) Var (9) Z (gu)(0 o), ... , g(k)(00 ))A-i(g(i)(0 0), ... , g(x)(0o))' , 00

where g(£)( 0) is the i-th order derivative of g(0). PROOF. Denote

c(x)f(x, B)dp

go(0)= J

J f(x, B)=f(x, 0+h) -f(x, 0) .

and

A(00)

Then, by (A.5), we have (2.2)

1-

£ Wg0(0)]

'&'(00) = li m

'= 6a =lhm h£

I-0 h

g(x)dkf( x , 0o) dp

£ A(60)

= lim

9'(x ) -£ dhf(x, Bo)dp A(60)

( !j f(x, die =1m J 9(x ){ _Bo+h) 9

A(B)

9(x)

[ ae £ f( x ,

0) ] d p

A(B0)

0

where dhg(0)=dh 1(dhg(B)), i=1,• • •, k and 0<<1. Since W) = f 9(x) A(B)

• f(x, 0)d1 , we obtain for each i=1, • • •, k

(2.3)

d n(g(00) - go(00)) =E

f

=1

(-1)1 (

{g(0o+jh)-g o( Bo +jh)}

fE (-1)J^ j I l

-

=

5

J

4(x)f(x, 9o +jh)dte

A(60+fh)

9(x)f(x, eo +jh)dq}

A(B0)

£ E(-1 )f(j)

(5 - A(60+fh)

• g'(x)f(x,

A(80)f1 A (90+fh)

A(00)-A (00+fh)^

8o +j h)dp}

= J=l (-1)f (j) l^A(80+fh) J

=f^ (- 1)f (j) S

J )9(x)flx, A(B0) n A (Oo+f h)

A(60+fh)- A(80)

i(x)f(x, Bo+jh)dp

0o

+jh)dp}

241

BHATTACHARYYA BOUND OF VARIANCES OF UNBIASED ESTIMATORS 39

=fE

(-1)'(j )

t

1

gr(x)f(x, Bo+jh)dp

U A(Oo+kh)-A(Oo) k=1

f J 1)J >

l /9(x)f(x, Bo+jh)du

U A(Bo+kh)-A(eo) k=1

By (2.3) and Lemma, we have for each i=1, • •, k, [-j.g(o)]

(2.4)

=lim hi d Lg(eo) =lim h; dn(g(60)-g0(00))+li V¢ dng(eo) h-o h-0

^ =lim ht dhg(O0)= Bf go(B) e_e

0

From (2.2) and (2.4) we obtain for each i=1, • • •, k, (2.5)

L 8B' g(B)]

a-BO & A(Bp)

)[-8Bi .f (x, 0)] a_ odf,

Proceeding now as in the regular case (see e.g. Zacks [16], page 190), one can show that the Bhattacharyya bound of the variances of the unbiased estimators of g(0) is given by (2.1).

3. Example We consider the location parameter case, i.e., f(x, 0)=f(x-0), and unbiased estimators of 0.

We assume that for any pZ1, the density function f(x) is given by c(1 - x2)p-1

f(x)=

f 0

if

if

Ixk< 1 IxIz1

where c=1/B(1/2, p) with B(a, p)=sax 1(1-x)0-1dx (a>0, p>0).

Case ( i ) : Let p = 1. Then the distribution is uniform, and it is easy to check that min Vareo (0)=0 for any specific value 0o. (See Takeuchi B: unbiased

[14]). Case ( ii): Let p=2. In this case , it is easy to check that the Fisher

information

51

[f'(x)/f(x)]2f(x)dx=00.

For any s > 0 , we define an estimator B. which satisfies

242 40 MASAFUMI AKAHIRA, MADAN L. PURI AND KEI TAKEUCHI

if IxIs1 -6 ,

- c,f'(x) /f(x)

0 if 1-6<jxj51 , where c, is a constant determined from the equations

51

(3.1)

1 B,(x) f (x)dx = 0

5' 1

and

B,(x) f'(x)dx = -1 .

We shall determine B,(x) for x outside the interval [-1, 1] from the unbiasedness condition (11'1 +9

(3.2)

B,(x)f(x- 6)dx=6. J -1 +9

First consider the case 0 < 0:!9 1, and define g(6) = f 1 1+8 #,(x) f (x - 6)dx .

(3.3)

Since B,(x) and f'(x) are bounded, g(O) is differentiable and g'(6)= e,(x)f'(x-6)dx. _ -1+9

If we assume that B,(x) is bounded for 1<x52, the right hand side of (3.3) is also differentiable, and we have by (3.2) and (3.3) 1+9

(3.4) 1-g'(B)=-J

6,(x)f'(x-6)dx .

Differentiating (3.4) again, and noting that lim f'(x)= -3/2, we have x-.1-0 1+9

g"(6) 2 0,(1+0)- f 1

(3.5)

B,(x)f"(x - 6)dx

If B. satisfies (3.5), then it also satisfies (3.4) since lim g'(6) _ -1; it 9-.0

also satisfies (3.3) since lim g(6)=0 by (3.1). 9-.0

Since the integral equation (3.5) is of Volterra's second type, it follows that the solution B,(x) exists and is bounded. Repeating the same process, we have the solution e,(x) for all x>1. Similarly, we can construct 9,(x) for x<-1. On the other hand,

(3.6) Vargo (e,) = c' ,

E +&

[f'(x)lf(x)]Ef(x)dx

while from (3.1), we have (3.7) c, 1x1 1 J

f'(x)dx = 0 -1+.

Hence Vargo (9,)=c,.

and J

c, 1x1 1

-1+.

[ f'(x)/ f (x)]'f (x)dx =1 .

243

BHATTACHARYYA BOUND OF VARIANCES OF UNBIASED ESTIMATORS 41

[ f'(x)/ f (x)]2 f (x)dx = oo, we have from (3.7),

Now, since lim r 1 .moo

inf Vare, (B)=0 for any specified value Bo.

lim c,=0. Consequently , `4

9:

unbiased

Before proceeding on to the next cases, we note the following : (i=1, .. • , k) given in Theorem 2.1 are finite,

If k < p/2, then A since

5

1 -1 (1-

X 2)p-2k-l dx < oo

Also, 1 aif(x -e) aff(x-e) dx

c6+1

(3.8) 9

-1 f(x -B)

ae f

ae{

= f' (-1)t+> f(x) f"'(

x)fcf'(x)dx ; i, j = 1,..., k,

We also obtain for Ixl<1, f<1'(x)= -2c(p-1)x(1- x2)P-2 ; fc2>(x)= -2c(p-1){(1-

(3.9)

x2)P-

2 - 2(p-2)x2(1- x2)P -2]

f(2)(x) = 4c(p-1) ( p - 2) {3x(1- x2)P-2 - 2(p - 3)x2 (1- x2)P-1) ; .. .

If i+ j is an odd number, if follows by (3.8) and (3.9) that 2 = 0 since f"'(x) f 1>'(x) is an odd function. From (3.8) and (3.9), we have 211=

4c(p -1)2B(

3 , p-2)

Ail=8c(p-1)2(p-2) {2(p-3)B(2 , p-4) -3B(2 , p-3)}

222=

4c(p-1)2{B (2 , p-2)-4(p-2)B(

+4(p-2)2B(5, p

2,

p-3)

-4)}

233 = 16c(p-1)2(p-2)2 {9B (3 , p-4) -12(p-3)B(2 , p-5)

+4(p-3)2B 2 , p_6)} ;... ( Case (iii) : Let p=3,4. Then, we have, for any unbiased estimator B(X) of 0,

(3.10)

511

O(x)f(x)dx=0

244

42 MASAFUMI AKAHIRA, MADAN L. PURI AND KEI TAKEUCHI

11

(3.11)

B(x)f'(x)dx =-1

(3.12) 5 1

^(x)f(x)dx=0, k=2,..., p-1. 2

p-1

Noting that ^- [ {

ck f (k)(x) (

/f (

1 k=i

x)] dx <

implies c2= • • = c,-, = 0, we

have by Takeuchi and Akahira [13], that the infimum of Varo (B)= B2(x)f(x)dx under (3.10), (3.11) and (3.12) is given by inf Varo (B) ^11

= 1/A11 , where 21,=

5[

any 8>0 there exists

and

5'

9 : ( 3 .1 0 )-( 3 .12)

f'(x)/f(x)]2f(x)dx=(p-1 )(2p-1)/(p - 2) and for

0,(x) in (-1, 1) satisfying (3.10), (3.11) and (3.12),

B;(x) f (x)dx < (1/A,,) -{- s.

We can extend B,(x) for x outside (-1, 1+e

1) from the unbiasedness condition

0,(x) f (x - B)dx = B.

First, we

-1+B

consider the case when 0< 0<1, and define g(O) (0) =

5 1'

9,(x) f (x - B)dx. 1 +B

In a similar way to the case (ii), we have for k=2,• • •, p-1, 1+0 „

(3.13)

(- 1)k +lg(k +1)(O)= BkO,(1+0)- 0,(x)f ( k +1)(x-B)dx l Bk= lim f( k)(x)

Since the integral equation (3.13) is again of Volterra's second type, it follows that the solution B,(x) exists. Repeating the process, we can construct an unbiased estimator B,(x) for all values of x. Then, it follows that inf Var8, (B)=1/A,, for any specified value 00. 9: unbiased

Case (iv) : Let p=5, 6. Note that

J 11 [f'(x)f"(x)/f (x)]dx = 0 J11

[f"(x)/f(x)]2f(x)dx=

(p-1)(2p-1)(2p-3) (2p'-7p+8) (p- 2) (p- 3) (p-4)

= 222 (say)

and J1 imply c,= • .. =cp_1=0.

1 Y, ck f(k)(x)/f (x)I 2 f(x)dx < oo

1 k=i

Then we see that

Vargo (B)Z(1, 0)( '111 0 0 222

0 = Ali /_(1) 1

245

BHATTACHARYYA BOUND OF VARIANCES OF UNBIASED ESTIMATORS 43

for any specified 00, where 211 is defined above (3.8). Here again, as in the previous case inf Var,0 (8)=1/211 for any specific 00. 6: unbiased

Case (v) : Let p=7. In this case, we see that k=3. Using Theorem 2.1, (3.8) and (3.9), we obtain

Vargo (e)> ICI

3

1 ill = E('-_211233

)]_1

where

0

213

222

0

0

283

with 2ii=144cB(2, 5) ; 2ia=1440c {8B(2, 3 )-3B(3, 4)} 2E2 =144c {B (2 , 5) - 20B (2 , 4) +100B (2 , 3) } 233=144O0c {9B(3, 3)-48B (2, 2)+64B(2, 1)} We also obtain 2ia = {8B(5/2, 3)-3B(3/2, 4)}2 2112aa B(3/2, 5){9B(3/2, 3)-48B(5/2, 2)+64B(7/2, 1)} Here again inf Vargo (B)=[211(1-(213/211213)]-1 for any specific 0 0 , i.e. 9: unbiased

we have a sharp bound. Case (vi) : For pZ 8 we can continue in a similar manner by choosing k=[(p-1)/2], where [s] denotes the largest integer less than or equal to s. The above discussion establishes that here the bound is sharp but generally it is not attainable.

Acknowledgement The authors are indebted to Professor Michael Perlman for the very careful and critical examination of the first version of the paper.

246 44 MASAFUMI AKAHIRA, MADAN L. PURI AND KEI TAKEUCHI

His constructive comments and suggestions for improvements are gratefully acknowledged. They are also grateful to the editor and the referee for their helpful comments. UNIVERSITY OF ELECTRO - COMMUNICATIONS INDIANA UNIVERSITY UNIVERSITY OF TOKYO

REFERENCES [ 1 ] Bhattacharyya, A. (1946). On some analogues of the amount of information and their use in statistical estimation , Sankhya, 8, 1-32. [ 2 ] Chapman, D. G. and Robbins, H. (1951). Minimum variance estimation without regularity assumptions, Ann. Math. Statist., 22, 581-586. [3 ] Chatterji, S. D. (1982). A remark on the Cramer-Rao inequality, in Statistics and Probability: Essays in Honor of C. R. Rao, North Holland Publishing Company, 193196. [ 4] Fend, A. V. (1959). On the attainment of the Cramer-Rao and Bhattacharyya bounds for the variance of an estimate, Ann Math. Statist., 30, 381-388. [ 51 Fraser, D. A. S. and Guttman, I. (1951). Bhattacharyya bounds without regularity assumptions , Ann. Math. Statist., 23, 629-632. [ 61 Hammersley, J. M. (1950). On estimating restricted parameters, J. Roy. Statist. Soc., B, 12, 192-240. [ 7 ] Khatri, C. G. (1980). Unified treatment of Cramer-Rao bound for the non-regular density functions , J. Statist. Plann. Inf., 4, 75-79. [ 8 ] Kiefer, J. (1952). On minimum variance in non-regular estimation , Ann. Math. Statist., 23, 627-630. [ 9 ] M6ri, T. F. (1983). Note on the Cramer-Rao inequality in the non -regular case: The family of uniform distributions, J. Statist. Plann. Inf., 7, 353-358. [10] Polfeldt, T. (1970). Asymptotic results in non-regular estimation, Skand. Akt. Tidskr. Suppl., 1-2. [11] Polfeldt, T. (1970). The order of the minimum variance in a non -regular case, Ann. Math. Statist ., 41, 667-672. [12] Rao , C. R. (1973). Linear Statistical Inference and Its Applications, Wiley, New York. [13] Takeuchi, K. and Akahira, M. (1986). A note on minimum variance, in press in Metrika. [14] Takeuchi, K. (1961). On the fallacy of a theory of Gunner Blom, Rep. Stat. Appl. Res., JUSE, 9, 34-35. [15] Vfncze, I. (1979). On the Cram€r-Fr€chet-Rao inequality in the non- regular case, in Contributions to Statistics, The Jaroslav HAjek Memorial Volume, Academia, Prague, 253-262. [16] Zacks, S. (1971). Theory of Statistical Inference, John Wiley.

247

Metrika, Volume 33, 1986 , page 85-91

A Note on Minimum Variance By K. Takeuchi1 and M. Akahira2

Summary: Minimizing f {o(x)}2f (x)dp is discussed under the unbiasedness condition : f o(x)fr(x)di. = ci (i = 1 , ..., p) and the condition (A): fi (x) (i = 1, ..., p) are linearly independent, P f {ft(x)}2/f(x)d,
1 Introduction Let (x, B) be a sample space. We consider a family P= {P0 : 0 E 8} of probability measures on B, where the index set O is called a parameter space. In minimum variance unbiased estimation theory, the locally best unbiased estimator,y* of y = g(O) at 0 = 00 is obtained by minimizing f {'t(x)}2dFB0(x) x under the condition that f y(x)dP0(x) =g(0) x

for all 0 E

0.

(1.1)

When the parameter space 0 is a finite set {00, 01, ..., Op}, the condition (1.1) is reduced to

f y(x)dPei (x) = g(Oi) = c i (say)

x

(i = 0, 1, ..., p)

And it is easily derived that the minimizing solution y* is obtained as P

y *(x) = E

i=o

kfi (x),

1 Kei Takeuchi, Faculty of Economics , University of Tokyo, Hongo, Bunkyo-ku, Tokyo 113, Japan. 2 Masafumi Akahira, Statistical Laboratory , Department of Mathematics , University of ElectroCommunications , Chofu, Tokyo 182 , Japan. 0026-1335/86/020085-091 $2.50 © 1986 Physica-Verlag, Vienna

248 86 K. Takeuchi and M. Akahira where dei

A (x) = dPe (x), 0 provided that For each i = 1, ..., p, Pei is absolutely continuous with respect to Poo.

dPoi f x

(C.1)

2

dPeo

dPo o (x) <°° (i = 1, ...,p)•

f(x ) (i = 1 , ..., p) are linearly independent.

(C.2)

(C.3)

The purpose of this note is to investigate the situation where for some i the first and/ or the second condition does not hold. Remark that the third condition is obviously necessary. First we consider the case when we assume the first condition but not the second. To deal with the case we need a more subtle formulation of the condition in place of (C.2) which is given in the next section.

2 Results Let f(x) and fi(x) Q = 1, ...,p) be measurable functions. Then we consider to minimize

f {9(x) }2f(x)dp x under the unbiasedness conditions: f B(x)f(x)dµ = ci (i = 1, ...,p) x and the condition (A): fi(x) (i = 1, ..., p) are linearly independent, {fi (x)}2

AX) (X)

du< °°

(i= 1, ...,k;k
fx

and P { affi(x)}2 i=1

f(x) x

dp
249

A Note on Minimum Variance 87

implies ak+1=...=ap=0.

Note that in the above f and f (i = 1, ..., p) may not necessarily be density functions. In fact , in some applications we may take the derivatives of the density with respect to the parameter 0 as f and obtain some generalization of Bhattacharyya inequality. Also generally f is equal to one of fi's but it is not essential for the subsequent discussion. Let A11 be a k x k non-negative definite matrix whose elements are

__ tj X

f, (x)f.(x) dµ f(x

(i , 1= 1,...,k).

Theorem: Under the condition (A), inf f {9(x)}2 f(x)du = c(k)A11 c(k) 8: unbiased x

holds, where c(k) = (c1, ..., ck) ' is given in the above. The equality is not usually attained. Proof. We put

A n= x : 1 m a ^ x P

.f,•(x)I 1

x)
We define the class C„ of estimators such that for any B„ E Cn, en (x) = 0 for x Ef An . We also put

^^()

n An

fi(x)f (x) f(x)

dµ (i , l = 1, ..., p)

and denote the matrix (J4^")) by An . Then we have min Vare (9n) =c'An

OnE Cn: unbiased

1c,

where c = (c1, ..., cp)' is given in the condition (A). On the other hand we may make a partition of the matrix An into blocks as

A=

A(n) Ai21

n 1 A21 A22

where 1A(n) and A2Z are k x k and (p - k) x (p - k) matrixes , respectively.

250 88 K. Takeuchi and M. Akahira

Let c = (ci, c2)' with cl E Rk and c2 E Rp-k. Then the condition (A) implies that

(2.1)

lim c'An c = n-

for every vector c = (e 1, c2 )' such that c2 : 0. To prove the theorem it is necessary and sufficient to show that

as n - oo

and since the matrix An is non negative definite it is enough to show that

(A22 - A211111 ' A12 )-' - 0

as n -> °°,

that is, the minimum characteristic roots of the matrix An = A22 - AiiAii -1Aiz diverges to infinity as n tends to be large. Assume otherwise , then we have a sequence {cn } of vectors such that cn = ciw ^2n)' with 11 4n 11 = 1 and cn Ancn -C2nAne2n =pn (say)

n -> -. and p, 1' p* Then there exists a subsequence { cnr } of {.E*} } such that cnj = (c ink, c2Q' with II cinj II = 1 and as j - °°,

where c* (ci, c2 )' with Il ci II = 1. We can show that c*'Anc* -> p*

as n -) °°,

which contradicts (2.1). Assume otherwise, then there exist e > 0 and no such that

c*'Ano c* > p* + C. Since An is monotone increasing in n in the sense that c'An c is so for any vector c, it follows that for n > no cn'Anocn G cn'Ancn 5 p*.

251

A Note on Minimum Variance 89

Hence p*+e
f {6(x)}2.f(x)dµ S under the unbiasedness condition f O(x)f (x)dk = (f + f) O(x)ft (x)dp = c; (i = 1, ..., P), x S SC

where f (x) (i = 1, ..., p) are linearly independent. Assume that we have a condition similar to (A) above:

{f (x)}2 f f(x) dµ
i = 1, ..., k),

and P { Fi aif{(x)}2 i=1

S f(x)

dµ < 00

implies ak+1 = ... = aP = O,

and further we assume that fi (x) (i = 1, ..., p) are linearly independent within S. Let Ai 1 be a k x k non-negative matrix whose elements are _ .f1(x)f ( x) du (i,1= 1, ..., k). s f(x)

For any estimator 9(x) we define a vector d = (d 1, ..., d,)' by di = f 6(x)f(x)du Se

(i = 1, ...,P)•

252 90 K. Takeuchi and M . Akahira

Suppose that d is given , then the unbiasedness condition is reduced to f O(x)f(x)dµ = c, - di (i = 1, ..., p)

S

and under this condition we have from above theorem that inf

f

6: unbiased with S

{6(x ) }2f(x)dµ =(Cl-dl)'A 11 '(cl-dl),

a fixed d

where c = (C1i..., CP)' _ (C1, C2 )' and d = (d 1, d2)' W1thCl, d1 E Rk andc2, d2 E Rp-k It is obvious that the set of possible vectors dl is a linear space which is denoted by V. Then we have

inf 6: unbiased

f

{6(x) }2 f(x)dp = min (c1- dl)'Aii'(cl -dl). d lE D1

S

If the dimension of Dl is 1, we have a basis d*, ..., d7 of Dl and any d 1 E Dl can be expressed as

dl =bldi +...+bidr =D*b, where D* _ (d 1 , ..., dl ') is a k x 1 matrix and b = (b 1, ..., bl)'. Then it is straightforward to show that min (c1 -di )'A i '(c l

-dl)

dIEDi

= min (cl -D*b)'Aii'( cl -D*b) b

= ciA,* iD*'(D*'Aii iD*)-iD*Aii lci and the minimum is attained by b* = (D*'Aii'D*)-'D*Aii lcl. If, moreover, fi (i = 1, ..., p) are not linearly independent in S, the vector c -d is restricted to a linear space E and inf f {B(x)}2f(x)dji=(co-do)'Ao-'(co-do), 6: unbiased with d; S c-dEE where Ao is the maximum rank principal minor of Ai 1 and co and do are corresponding subvectors of c and d. Then do is on a hyperplane H of some dimensionality and

inf

f {6(x)}Zf(x)dp= min

9: unbiased S

d0EH

(co- do)'A0 -1(co -do).

253

A Note on Minimum Variance 91

Note that even when the parameter space © is infinite , minimizing the variance with a finite number of conditions is often applied to obtain a lower bound of the variances of unbiased estimators , as is the case of Cramer -Rao and Bhattacharyya . (See Hammersley; Chapman/Robbins ; Kiefer ; Fraser/Guttman; Fend; Sen/Ghosh ; Chatterji ; Akahira/ Puri/Takeuchi and also Isii with relation to mathematical programming)

References Akahira M , Puri ML, Takeuchi K (1986) Bhattacharyya bound of variances of unbiased estimators in non-regular cases . To appear in the Ann Inst Statist Math 38 Chapman DG, Robbins H (1951 ) Minimum variance estimation without regularity assumptions. Ann Math Statist 22:581-586 Chatterji SD (1982) A remark on the Cramer-Rao inequality. In: Kallianpur, Krishnaiah, Ghosh (eds) Statistics and probability : Essays in honor of C R Rao. North Holland , New York, pp 193-196 Fend AV (1959 ) Bounds for the variance of an estimate. Ann Math Statist 30:381-388 Fraser DAS, Guttman I (1952) Bhattacharyya bounds without regularity assumptions . Ann Math Statist 23 :629-632 Hammersley JM (1950) On estimating restricted parameters . J Roy Statist Soc (B ) 12:192-240 Isii K (1964) Inequalities of the types of Chebyshev and Cramer -Rao and mathematical programming . Ann Inst Statist Math 16:277-293 Kiefer J (1952) On minimum variance in non-regular estimation . Ann Math Statist 23:627-629 Sen PK, Ghosh BK ( 1976) Comparison of some bounds in estimation theory . Ann Statist 4: 755 -765 ; Correction. Ann Statist 5:593

Received August 11, 1983

254 Metrika, Volume 33, 1986, page 217-222

A Note on Optimum Spacing of Observations from a Continuous Time Simple Markov Process By K. Takeuchi' and M. Akahira2

Summary : Assume that X(r) is a continuous time simple Markov process with a parameter 0. The problem is to choose observation points to < rl < ... < rT which provide with the maximum possible information on 0. Suppose that the observation points are equally spaced , that is, for t = 1, ..., T, rt - rt-1 = s is constant. Then the optimum value for s is obtained.

1 Introduction In various practical situations especially in engineering applications of time series analysis, it often happens that the process itself is continuous in time, but measurements can be made only at discrete time points. Then the problem of choosing sample points arises, and its most natural formulations would be "what is the best set of sample points in continuous time points which would give the largest possible amount of information with respect to some unknown parameter given the size of the sample?" In this paper we deal with the problem in nearly the simplest possible case, that is, the underlying process is simple Markov and the measurement points are equally spaced, and calculate the optimum spacing. It is conjectured that the solution given is also optimum even when the sample points can be arbitrarily spaced, but the authors have been unable to prove it yet. The model can be also generalized in various ways, which will be discussed in subsequent papers. As far as the authors are aware, there seems to be little literature which has dealt with this problem, and we mention Taga (1966) who dealt with a similar problem in connection with the least squares prediction.

2 Optimum Spacing of Observations Suppose that X(r) is a stationary Gaussian process with a continuous time parameter r, and also suppose that E(X(r)) = 0, Var (X(-r)) = a2 and Cov (X(rl),X(r2 )) = 0 1T2 - Tl1a2, that is, X(r) is a simple Markov process with a parameter 0, where 0 < 0 < 1. We as-

1 Kei Takeuchi, Faculty of Economics , University of Tokyo, Hongo, Bunkyo-ku, Tokyo 113, Japan. 2 Masafumi Akahira, Department of Mathematics , University of Electro- Communications , Chofu, Tokyo 182, Japan. 0026 - 1335/86/03040217 - 222 $2.50 © 1986 Physica-Verlag , Vienna

255

218 K. Takeuchi and M. Akahira

sume that 0 and a2 are unknown , and it is required to estimate 0 based on T+ 1 measurements on X(T), which are denoted as X(ro ), X(Ti), ..., X(TT). The problem is to choose such observation points To < 7-1 < ... < TT that provide with the maximum possible information on 0. For simplicity 's sake we assume that for t = 1 , ..., T, Tt - Tt_ i = s is constant, that is, the observation points are equally spaced , and we want to get the optimum value for s . In order to simplify the notation we denote Xt = X(Tt) (t = 0, 1, ..., T) and also put p = 0s. Then we have the following simple Markov process Xt = pXt_1 + Ut (t = 1, 2, ...), where {Ut} is a sequence of independently, identically and normally distributed random variables with mean 0 and variance 02 ( 1 - p2), and X0 is a normal random variable with mean 0 and variance a2 and for each t, Xo is independent of Ut. The log-likelihood function of p and a2 based on the sample (Xe, Xi, ..., XT) is given by I logL(p , a2)=-

T-1 T XO+4T +(1+p2) xt -2p E xtxi_1

2 2 I 2(1p )0 t=1 t=1

T T+1 T+1 - 2log(1-P 2) - 2 loga2- 2 log21r.

The first and second order partial derivatives with respect to p and a2 are also given by T-1 T P 2 xo + XT +(I + p2) E x2 - 2p T xtxt-1 8 log L(p, a2) ap (1-p) a2 t=1 t=1 T 2 p 1 xt -

T P Z xtxt_1 + T

12)0, (1-p t1 t=1 1-p 2;

a a02 log

L(p, a2) I

T-1 JX2+xT +(1 +p2

2(1-p2) a4

T T+1 xtxt_1 E x2 -2p 2a2 t=1 t=1 T-1

a2 1 + 3p2

T

- Ti log L (P, u2) _ (I -p2) 302 xo +xT t (1 + p2) t El x2 - 2p t El xtxt_1 I 4p T-1 2 T (P -t=ixtxt t=i xt +(I-P2)2a2 + I T I 1 2 - T( (1-p2)a2 t=1 t

1 +p2).

( 1-p2)2'

(1)

256 A Note on Optimum Spacing of Observations 219

a2 8(a2 )2 log

L(P, a2 )

T-1 2 T 1 T+1 {x^x+(1+P2) i I1 At - 2p t E1 xtxt-1 2a4 (1-p2)a6

a2 apaa log

_L j L( P, a2) = Q 2

a

1 aP

log L (P, a2) +

Tp 1- 2

(2)

(3)

P j.

It follows by (1), (2) and (3) that the information amounts on p and a2 are obtained by

a2 l T(1 + p2 ) Ipp =-ELape logL(P, 0 2 ) = (1 _p2)2

1

a'

.[,,2,,2 = - E

a(a2

)2 log L(P, a2)

T+ 1 2a4

a2 2 1 TP .1p,2 E apaa2 logL(p, a ) _ (1 _p2)a2 .

Hence the inverse of the information matrix is given by

IPP

Ipa2

Y

1

T(l + p2) Tp (1 -p2)2

(1 - p2)a2

Tp T+ 1 I Ipa2 Ia2°2 )

\(1-p2)a2

2a4

where T 2p2 J(P)- 1-p2 {1+(T+1)( 1-p2)}•

We can regard J(p) as the amount of information obtainable from the sample with respect to the parameter p, and when T is large , for any best asymptotically normal estimator A (which may be the maximum likelihood estimator, the least squares estimator, etc.). T+ 1 (,3-p) is asymptotically normal with mean 0 and variance (T+ 1)J(p)-1. And for the initial parameter 0 with which we are concerned, the in-

257 220 K. Takeuchi and M. Akahira

formation is given by - I ap `z s202s - 2 ( 2T 02s I(8) - J(0s) = T + ) aB 1-Bzs T+1 1-Bzs

(4)

since p = O. Now we shall obtain such value of s which maximizes 1(0). Since (T+ 2T pz ) (logp)pz z 0 B z (log )2 I(0 T+1 1- p z ) 1 -p Putting t = ( 1 - pz)/pz we have

402(log 0 )2I(0) _ T { log (1 + )}2 + 0 (1) =It + 0( 1) (say). In order to get s maximizing 1(0) given by (4), it is enough to take the solution to of dItldt = 0, i.e., log (1 + t) = 21;/(1 +:;) which is about 3.926. It is also noted that for 1; =:;o, p _ 0.451. Hence s maximizing 1(0) is given by s =-{log (1 + to)}/(2 log 0) -0.797/log 0. Even when T is small, the amount of the information provided by the sample may have some value, and it is interesting to calculate the value of 0 which maximizes

T

z Bz(log B )zI(B)= en

{1+(T+1)(1 -e')},

(5)

where (3=2slog0. Then the optimum solution is given by the negative root of the equation 2 1 2 -- + =0 Q 1-ceQ 1-eQ which is independent of 0, where c = (T- 1)/(T + 1). Table 1 provides us with the values of Q which maximize (5) for some values of T. Note that for T = 1, i.e. the case of two observations , the optimum is given by s = 0. For comparison if T = 3 we have c = 1/2, hence 0 =-1.151, while for T - 00, i.e. c = 1, we have R = -1.594 which says that for small T, the optimum spacing is smaller than for large, but is at least 73 % of the asymptotically optimum spacing if T>3. The optimum value of s depends on the unknown parameter 0, hence we can not apply the result to the practical case but the autocorrelation of the successive measurements for the optimum spacing is independent of 0, and we may get some rough idea about the optimum spacing since in general cases we may have some prior information about the possible speed of decreasing of Cov (X(t), X(t + s)).

258 221

A Note on Optimum Spacing of Observations Table 1 T

2

3

4

5

9

-0.926

-1.151

-1.261 -1.327 -1.371 -1.402

T

9

10

20

0

-1.444

-1.459

-1.526 -1.566 -1.580 -1.594

50

6

7

8 -1.426

100

Otherwise we may resort to a two -stage procedure by first observing Tt values based on an initial guess and then observing T2 (= T - TI) values corresponding to the estimated optimum spacing based on the first sample. Now let us consider the problem of testing the hypothesis 0 = 0 o against the alternative 0 * 00 . Then asymptotically , the locally most powerful test and the locally most powerful unbiased test are equivalent to the one -sided and the two-sided tests respectively of the maximum likelihood estimator (e.g. Rao 1965). When the sample spacing is given , the hypothesis is transformed into H : p = po = 00'. Then if T is large, the tests based on the maximum likelihood estimator are given by the procedure: One-sided case; Reject H iff

T + 1 (P - po) > ua/ J(p0) (or

Two-sided case ; Reject H iff

T+1 (,a-po)
T+1 _ I^ - po I > u«/2 /

J(po),

where un denotes the a-upper quantile of the standard normal distribution. Under the contiguous alternative K : p = po + r/ T +I- the asymptotic powers of the above tests are represented as monotone increasing functions of r J(p0) , where r is a real number. Transforming back to the original parameter, the asymptotic power for the alternative 0 = Oo + t/\/7T1 is given as a function of t I dp/d0 I J = t \/i. Hence the power is maximized when I(00) is maximized. Therefore the optimum spacing is also given by s =-0.797/log 00. Remark: In the above discussion we excluded the possibility of 0 being equal to zero. It may be of some importance, however, in practice to test the hypothesis 0 = 0 against 0 > 0. Then it can be shown that the local power is larger when s is smaller and the maximum of the local power can not be attained, which can be also seen from the fact thats-0as0- 0. Acknowledgements: The authors are indebted to the referees for their helpful comments which resulted in improvements at various points of the paper . They also wish to thank Mr . H. Osawa of the University of Electro-Communications for making the numerical calculation of Table 1.

259

222

K. Takeuchi and M . Akahira: A Note on Optimum Spacing of Observations

References Rao CR (1965) Linear statistical inference and its applications. Wiley, New York Taga Y (1966) Optimum time sampling in stochastic processes (in Japanese). Proc Inst Statist Math 14:59-61

Received February 8, 1984 (Revised version July 10, 1984)

260

ON THE BOUND OF THE ASYMPTOTIC DISTRIBUTION OF ESTIMATORS WHEN THE MAXIMUM ORDER OF CONSISTENCY DEPENDS ON THE PARAMETER *

Masafumi Akahira(*) and Kei Takeuchi(**)

Abstract In this papar, the case when the order of consistency depends on the parameter is discussed, and in the simple unstable process the asymptotic means and variances of the log-likelihood ratio test statistic are obtained under the null and the alternative hypotheses. Further its asymptotic distribution is also discussed.

(*) Department of Mathematics, University of Electro-Communications, Chofu, Tokyo 182, Japan. The author is on leave and visiting Queen's University from April to July 1985. Faculty of Economics, University of Tokyo, Hongo Bunkyo-ku, Tokyo 113, Japan.

AMS Subject Classification (1980). 62F11, 62M10. Key words and phrases: Order of consistency, Asymptotically median unbiased estimator, Autoregressive process, Log-likelihood ratio, Asymptotic distribution.

This paper is retyped with the correction of typographical errors.

261

1. INTRODUCTION In the regular case it is known that the order of consistency is equal to V^n_, but in the non-regular case it is not always so, e.g.

nl/a (0 < a < 2 ),

n log n

etc., which are independent of the unknown parameter (e.g. see Akahira , 1975a; Akahira and Takeuchi, 1981; Vostrikova, 1984 ).

However , the order of consistency

may depend on it in the case of the unstable process. Here a discussion on that will be done. In the autoregressive process { Xt} with Xt = 9Xt_1+ Ut (t = 1, 2, ...), where { Ut} is a sequence of independently and identically distributed random variables and X0 = 0, it is known that the asymptotic distribution of the least squares estimator of 9 is normal for 101 < 1, Cauchy for 101 > 1 (e.g . White , 1958 ; Anderson, 1959) and some one for 101 = 1 (Rao , 1978 ). Further, in the case when 101 < 1 the asymptotic efficiency of estimators was studied by Akahira ( 1976 ) and Kabaila (1983) for the ARMA process , and their higher order asymptotic efficiency has been discussed by Akahira ( 1975b, 1979, 1982 , 1984 ) and Taniguchi ( 1983) for the ARMA process . In this paper in the first order AR process with 10 1 > 1 the asymptotic means and variances of the loglikelihood ratio test statistic are obtained under the null and the alternative hypothesis.

262

Further its asymptotic distribution is also discussed. 2. RESULTS Let (X, B) be a sample space and 0 be a parameter space, which is assumed to be an open set in an Euclidean 1-space R'. We shall denote by (X(T), 13(T)) the T-fold direct product of (X,B). For each T = 1, 2, ..., the points of X(T) will be denoted by ;VT = (x1, ... , XT). We consider a sequence of classes of probability measures {PB,T : 0 E O} (T = 1, 2, ...) each defined on (X (T), B(T)) such that for each T = 1, 2, ... and each 0 E 4 the following holds: PO,T (B(T)) = PB,T+1 (B(T) X X) for all B(T) E B(T). An estimator of 0 is defined to be a sequence {9T} of B(T)-measurable functions. For simplicity we may denote an estimator 9T instead of {9T}. For an increasing sequence {CT} (CT tending to infinity) an estimator 0T is called consistent with order {cT} (or CT-consistent for short) if for every e > 0 and every i9 E O there exists a positive number L such that

lira lim sup

L-9oo T-* oo O:18-i9! < 6

(Akahira, 1975a).

PO,T

{CTIT

-

01

_> L I = 0

263

It is known that the order of convergence of consistent estimators is equal to VTn_ is the regular case, but not always so, e.g . nl/a (0 < a < 2), n log n etc., in the non-regular case when the support of the density depends on the parameter 9. In both cases the order of consistency is independent of 9. If CT can not be decided independently of 9, we may change cT(9) instead of cT in the above definition. However, in such a definition we shall not be able to determine uniquely the value of cT (9) at Oo (Takeuchi , 1974). Then a similar phenomenon to "superefficiency" happens . Indeed , if 0T has an order cT of consistency independent of 0, then for a specified value 90 we define an estimator 9T as { d _ 1 (OT - 90 )

+"

9T

for 16T - 901 _< CT1/2 for 16T - 001 > CT

1/2

where f o r each T = 1, 2, ..., dT is a constant with dT > 1. We define 4 (0) as follows:

4(00)

= CTdT

264

and for any 0 90 CT

for

4(6) _ CT{1- 10 } for Case (i)

0

=

10 - Sol 10 -901

< cT-1 /2 >CT1

/2

90. Since IBT - 9o l < CT1 1 2 implies CTdT 1eT-

901 = CT I9T - 901, it follows that POo,T

{ 4 (oo)16


- eoI

IOT - 90 I

> L}

>Ll

+ Poo , T { I9T - 001

Case (ii) 0 90 . In the case when

001


T1 /2

OT - 00 1 >

10 - OoI

<

> CT1/2 CT1

}

/2, I9T -

implies 4 (9)IOT -9I < 4(9)I9T-9o I < 1, and CT1

/2 implies 4 (0) 16T -

91

= CT (O) IOT - 91 <

CTI9T - 91. Then we have for L > 1 PO,T

{GT(e) IOT

-91

>L}
-e1

>L

}.

-

10 - 001 > CT 1 / 2, IeT - 90 1 < CT-1 /2 implies 4 (0)IBT - e1 < 4(e)I9T - 001 < CTI9T - 91, and 001 > CT1 /2 implies I eT In the case when

- 1/2

CT(e)OT-e1 =CT

1- 10-901

IeT-e0I
265

Then we have for L > 1 PO,T {c

(9)M -91 > L }
> L}.

Hence in both cases (i) and ( ii) we obtain for L > 1

PO,T {(o)IoT -

01

> L}

L} + Peo,T {cTIOT - 0.1 > 4(2} Letting