Joint Statistical Papers of Akahira and Takeuchi

OINI^ STATISTI C AL RS AKAH IRK and

TAKEUCHI

This page is intentionally left blank

5281/tp/path (Converted)-2 7/9/08 3:38 PM Page 1

Composite

Published by World Scientific Publishing Co. Pte. Ltd. 5 Toh Tuck Link , Singapore 596224 USA office: Suite 202, 1060 Main Street, River Edge, NJ 07661 UK office: 57 Shelton Street, Covent Garden, London WC2H 9HE

British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library.

The editors and publisher would like to thank the following organizations and publishers of the various journals and books for their assistance and permission to reproduce the selected reprints found in this volume: Institute of Mathematical Statistics The University of Electro-Communications Springer-Verlag The Institute of Statistical Mathematics Union of Japanese Scientists and Engineers Blackwell Publishing Asia Oldenbourg Wissenschaftsverlag GmbH Physica-Verlag

Universit6 Pierre et Marie Curie University degli Studi di Roma VSP International Science Publishers Blackwell Science Ltd Marcel Dekker

While every effort has been made to contact the publishers of reprinted papers prior to publication , we have not been successful in some cases . Where we could not contact the publishers, we have acknowledged the source of the material. Proper credit will be accorded to these publications in future editions of this work after permission is granted.

JOINT STATISTICAL PAPERS OF AKAHIRA AND TAKEUCHI Copyright 0 2003 by World Scientific Publishing Co. Pte. Ltd. All rights reserved This book, or parts thereof may not be reproduced in any form or by any means, electronic or mechanical, including photocopying, recording or any information storage and retrieval system now known or to be invented, without written permission from the Publisher.

For photocopying of material in this volume , please pay a copying fee through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers , MA 01923 , USA. In this case permission to photocopy is not required from the publisher.

ISBN 981 -238-377-8

Printed in Singapore.

V

Foreword Collaborative work, virtually inevitable in experimental and observational work, is increasingly common in the mathematical sciences too. There can, however, be few collaborations that have lasted so long and produced such an impressive and coherent body of work as that of Professors Akahira and Takeuchi celebrated in the present volume. In the frequentist approach to statistical inference problems with an "exact" solution are important but essentially restricted to certain exponential family questions and to issues involving a transformational family, such as the scale and location model. This means that in more complex and realistic cases approximations cannot be avoided. The approach almost universally adopted is to appeal to asymptotic arguments. It is supposed that the information available is large enough to justify, for example, local linearization of nonlinear dependencies and the invoking of the classical limit laws of probability theory. In more formal terms, the problem under study is imbedded in a sequence of problems in which the sample size, or more generally a measure of information, tends to infinity. Theorems based on the laws of large numbers and, typically, the Central Limit Theorem can then be established about the limiting behaviour of estimation, testing and prediction procedures. These results justify, in some sense , many widely used techniques such as maximum likelihood estimators and associated testing procedures, such as likelihood ratio tests. There are broadly two difficulties with this approach. The passage to the limit is a fiction. Detailed results have to be applied for one or more specific sample sizes. Secondly, there are many different procedures that are equivalent in the limiting sense. While the different procedures will often give virtually the same answer in applications, this is not necessarily so and in any case some basis of choice between such alternative procedures is clearly desirable. The first point is in principle best addressed by finding bounds on the approximations involved, broadly analogous to the Berry-Esseen bound associated with the Central Limit Theorem. While there is some fascinating work of this kind, stemming, for example, from the use of Stein's method, useful results are extremely limited in scope. Instead higher-order terms in asymptotic expansions can be exploited in some generality and this is the approach taken in the present work. Of course the issue still remains of the sample sizes for which the results give good approximations. The second issue of the choice between estimators and test statistics equivalent to the first order of asymptotic theory is subtler and needs very careful formulation. Professors Akahira and Takeuchi have made important contributions to both aspects; I find their results on the second aspect particularly striking. Their broad approach stems from the Neyman-Pearson viewpoint emphasizing properties achieved in hypothetical repeated sampling. As such it is to be contrasted with studies proceeding from what might be called the Fisherian standpoint with its explicit emphasis on likelihood, sufficiency and conditioning. A detailed comparison of these approaches, and of both with Bayesian discussions , would probably be fruitful. This is not the place to discuss or single out individual papers in the volume, the authors having in any case provided a valuable summary and introduction. The careful discussion of nonregulax problems is, however, particularly to be noted.

Vi

Let me conclude by congratulating Professors Akahira and Takeuchi on their achievements; long may their joint work continue!

D. R. Cox Nuffield College Oxford

UK January 2003

vii

Introduction This volume consists of 44 joint papers on statistical inference for about a quarter of a century from 1975 to 2001. In the latter half of 1970's, the higher order asymptotics was extensively developed by many people including Prof. J. Pfanzagl's group, Prof. J. K. Ghosh's one and ourselves. In particular, the second and third order asymptotic efficiencies of estimators are discussed using the Edgeworth expansion under suitable regularity conditions in the joint papers. In cases when the regularity conditions do not hold, there are less well known "surprising" results such as the existence of zero variance unbiasdd estimators based on a sample of fixed size. The order of consistency and asymptotic efficiency of estimators are also strongly influenced from the conditions of nonregularity. Various non-regular cases are discussed in the joint papers. The papers on both of the higher order asymptotics and non-regular estimation formed two monographs of Akahira and Takeuchi (1981, 1995). Other papers are related to the problem of prediction sufficiency, the Edgeworth type expansions and miscellaneous topics. More precisely following subjects and their implications are studied in the joint papers in this volume.

1. Prediction sufficiency The concept of prediction sufficiency (adequacy) was defined and discussed by, among others, Skibinsky (1967), [1], and Torgersen (1977). It was shown by Bahadur (1955) that sufficiency as defined in terms of conditional expectations, under regularity conditions, implies "real sufficiency," i.e. sufficiency in terms of risk functions. In the paper [1], the converse is shown to hold provided the loss function depends on the unknown parameter, but no longer true if the loss function is independent of the unknown parameter. In the latter case, conditional independence still holds but ordinary sufficiency is not required. In the paper [15], we show the relation between prediction sufficiency and the sufficiency in terms of conditional expectations.

2. Edgeworth type expansions In the papers [3] and [6], we give the Edgeworth type expansions of the distribution of the sum of independent and identically distributed (i.i.d.) random variables without higher order moments. For special cases of t-type density, we obtain the exact formulas for the asymptotic expansion of the density, and the results are generalized to the case when the density is approximated by a rational function. In [4], it is also shown that the asymptotic distribution for cases without finite variance are stable laws with fractional characteristic exponents,

viii

and the asymptotic expansion of the density for some case is given . In the paper [ 7], the above expansion is extended to the multidimensional case.

3. Second order asymptotic efficiency In the papers [2] and [5], the second order asymptotic efficiency of asymptotically efficient estimator was discussed for one parameter and multiparameter cases. Under suitable regularity conditions, the maximum likelihood estimator (MLE) adjusted to be second order asymptotically median unbiased (AMU) is shown to be second order asymptotically efficient. And it is shown that the first order efficiency implies the second order efficiency , which phenomenon was also found by Pfanzagl ( 1979). The papers [9] and [13] demonstrate the second order efficiency of the generalized Bayes estimator ( GBE). In [12], we define the discretized likelihood estimator (DLE) as a solution of the discretized likelihood equation, and show that the MLE as the limit of DLE is second order asymptotically efficient. Thus the structure of the higher order asymptotics is clarified by considering the DLE. In the paper [28], from the viewpoint of the decision -theoretic approach, we get the second order asymptotic completeness of the MLE, and, in [10], discuss the problem of the second order asymptotic efficiency of the confidence intervals, that is, consider the asymptotic power of confidence interval up to the order n-1/2 in the neighborhood of the true value of the parameter, where n is a size of sample. In the papers [27] and [34], we obtain the Bhattacharyya type bound for the asymptotic variance of sequential estimation procedures under suitable regularity conditions , and show that the modified sequential maximum likelihood estimation procedure attains the bound if the stopping rule is properly determined , which can not be uniformly attained if the size of sample is fixed.

4. Third order asymptotic efficiency Already three-quater century ago, R. A. Fisher (1925) suggested that the MLE was asymptotically best in the class of all asymptotically efficient estimators. He guessed that the MLE was asymptotically best in the sense that the MLE had the asymptotically minimum loss of information in the class, and calculated the asymptotic value of the loss of information of the MLE and minimum chi-square estimator in the multinomial distribution . Later, C. R. Rao (1961 ) corrected Fisher 's calculation and defined the second order efficiency of the MLE, which corresponds to the third order asymptotic efficiency, in our terminology. However , it was not clear what the minimum of loss of information meant, and comparison of asymptotic variance or mean squared error up to the second order failed to establish uniform superiority of the MLE or any other estimators. This impasse was broken through by introducing the median-bias

ix

correction and considering the concentration probability of the estimator around the true value up to the order n-1 (see also [5], [8], Pfanzagl and Wefelmeyer (1985), Ghosh (1994)). In the paper [8], it is shown that the MLE adjusted to be third order AMU is third order asymptotically efficient for a multiparameter exponential family of distributions. Similar results were obtained by Pfanzagl and Wefelmeyer (1978) and Ghosh, Sinha and Wieand (1980). In the papers [9] and [13] we show that, for any symmetric loss function, the GBE is third order asymptotically efficient in a restricted class D of third order AMU estimators for general one-parameter and multiparameter cases. In [16] and [28] it is seen that, for any class of estimators which admit Edgeworth expansions but are not necessarily AMU, we get the third order asymptotic completeness (or sufficiency) of the MLE together with the second order derivative of the log-likelihood function evaluated at the MLE. In [17], the concept of the asymptotic deficiency of Hodges and Lehmann (1970) is extended to the case when a common parameter is estimated from m sets of independent samples of each size n, and the asymptotic deficiencies of some asymptotically efficient estimators relative to the MLE based on the pooled sample are discussed in the presence of nuisance parameters (see also Akahira (1986)). Further, under suitable regularity conditions, the third order asymptotic bound for the distribution of the sequential estimation procedures is obtained in [30]. And it is shown that the bias-adjusted maximum likelihood estimation procedure combined with appropriate stopping rule is uniformly third order asymptotically efficient in the sense that its asymptotic distribution attains the bound uniformly in stopping rules up to the third order, thus the asymptotic deficiency being zero. For the curved exponential family of distributions, the second order efficiency from the Fisher-Rao approach is discussed by Ghosh and Subramanyam (1974), and also by Efron (1975) and Amari (1985) from the viewpoint of the differential geometry. Amari's approach attracted much attention in that he could successfully connect the curvature of differential manifold with the asymptotic deficiency of the estimator.

5. Non- regular unbiased estimation For the lower bound for the variance of unbiased estimators, most famous is the so-called Cramer-Rao bound. But the Cramer-Rao bound and its Bhattacharyya extension assume a set of regularity conditions. Chapman and Robbins (1951), Kiefer (1952), and Fraser and Guttman (1951) obtained bounds with less stringent assumptions , but they still require the independence of the support of the parameter 0 or almost equivalently that the distribution with 0 Bo is absolutely continuous with respect to that with 0 = 0o when 0o is the specified parameter value at which the variance is evaluated. In the paper [20], the Bhattacharyya bound is generalized to the non-regular case when the support of the density depends on the parameter 0 while it

X

is k-times differentiable with respect to 0 within the support, and the bound is also shown to be sharp, which fails in the regular case (see also Takeuchi (1962)). In [26], we introduce the concept of one-directionality which includes both cases of location (and scale) parameter and selection parameter and other cases, and show that the bound for the variance of unbiased estimators is sharp in the sense that the actual infimum of the variance of unbiased estimators coincides with the bound for a specified 00, for this class of distributions, using the result to minimize the variance under the conditions of unbiasedness, linearly independence of functions, etc. in [21]. We also establish that for a wide class of non-regular distributions the infimum of the variance of unbiased estimators can be zero when the size of sample is not smaller than 2. In [25], we give the exact forms of locally minimum variance unbiased estimators and their variances in the case of a discontinuous density function. In the non-regular case when the amount of Fisher information is infinity, the paper [19] shows that the infimum of variance of unbiased estimators is equal to zero, and gives some examples. It is obvious to observe that for usual "regular" case, any parametric function which has unbiased estimators must be continuous or differentiable in the original parameter (e.g. see Zacks (1971) and Lehmann (1983)). However, in a sequential case or a randomized sample size case when the size of sample is not bounded, the continuity of estimable function does not necessarily follow. In the case of sequential Bernoulli trials, the paper [37] gives a sufficient condition for a parametric function to be unbiasedly estimable and shows the existence of a discontinuous unbiasedly estimable function using non-randomized sample size procedures.

6. Non- regular asymptotic estimation and test In the asymptotic theory of estimation, we try to compare the regular versus non-regular situation to clarify the significance and implication of each of the regularity conditions in [14]. Ibragimov and Has'minskii (1981) also proposed similar considerations for non-regularity. As an example, in the case of estimation of a location parameter in the symmetrically truncated normal distribution, it is shown in [11] that the maximum probability estimator of Weiss and Wolfowitz (1974) is asymptotically inadmissible and has smaller concentration probability than the midrange which is asymptotically efficient. For a family of uniform distributions, it is shown in [44] that for any small e > 0 the average mean squared error (MSE) of any estimator in the interval of 0 values of length e and centered at 00 can not be smaller than that of the midrange up to the order o(n-2) as the size n of sample tends to infinity. And the asymptotic lower bound for the average MSE is also shown to be sharp. In the paper [18], we consider the problem to estimate a common parameter for the pooled sample from the uniform distributions with scales as nuisance parameters. And we compare the MLE with others and show that the MLE based on the pooled sample is not (asymptotically) efficient.

Xi

Fisher ( 1934) calculated the loss of information of the MLE of the location parameter in the double exponential distribution , and showed that the loss is of order V/n-, unlike of constant order in the regular case, hence the MLE is not second order asymptotically efficienct. The results are caused from the fact that the density admits the first order differentiability with respect to the parameter, but not the second order . Hence the first order asymptotic theory of regular estimation can be applied, but, in the second order the situation is non-regular. The paper [32] extends these results by obtaining the (asymptotic) losses of information of order statistics and related estimators, and by comparing them via their asymptotic distributions up to the second order . In [39] , we obtain the Bhattacharyya type bound for the variance of unbiased estimators of a location parameter of the double exponential distribution and the loss of information of the MLE. In [29], we consider the estimation problem of a location parameter on a sample of size n from the two-sided Weibull type density f (x - 0) = C(a) exp(- Ix 01') for x E R', 0 E R1 and 1 < a < 3/2, where C(a) = a/{21'(1/a)}. Then we obtain the bound for the distribution of AMU estimators up to the 2a-th order, i.e. n (2 «- 1)/2, and calculate the asymptotic distribution of the MLE up to the same order. And we show that the MLE is not 2a-th order asymptotically efficient, and give the amount of the loss of asymptotic information of the MLE . In [35], we consider the estimation of a location parameter 0 of the density function with a support of a finite interval and contact of the power a - 1 at both endpoints , where 1 < a < 2. Then we obtain the bound for the asymptotic distribution of AMU estimators of 0 based on a sample from the density. It is also shown that the bias-adjusted MLE is not asymptotically efficient in the sense that its asymptotic distribution does not uniformly attain the bound. In the regular case, it is known that the order of consistency, i.e. the order of convergence of consistent estimators is equal to V/n-, but in the non-regular case with i . i.d. sample it is not always so, and could be nl/" (0 < a < 2), (n log n) 1/2 etc., but are usually independent of the unknown parameter (see Akahira ( 1975a, 1975b)). However, in the non i . i.d. sample case , the order of consistency may depend on the parameter . In [23], we consider the first order autoregressive ( AR) process with a parameter 0, and obtain the asymptotic means and variances of the log-likelihood ratio test statistic LT under the null and alternative hypotheses in the case when 101 > 1. We also discuss the asymptotic distribution of LT under both of the hypothesis . For the case of a two-sided Gamma type distribution , the paper [ 42] shows that the largest order of consistency is n2 and there exists a test with under n2 of consistency, and we obtain the asymptotic power function of the test. In [36] , we propose an amount of information between two distributions which is always well defined , symmetric and additive for independent sample and information contained in a statistic is always not greater than that in the whole sample and the equality holds if and only if the statistics is sufficient. And we discuss the relative (asymptotic) efficiency of a statistic (or an estimator) by the ratio of the amounts of information contained in the statistic and in the

Xii

sample in a systematic and unified way both in regular and non-regular cases.

7. Others For the process of continuous time observations are usually made on a finite number of discrete time points, information thus provided depends on the choice of observation points. In the paper [22], we assume that X(7-) is a continuous time simple Markov process with a parameter 0, and consider the problem choosing observation points TO < r1 < • • • < TT which provide with the maximum possible information on 0. If the observation points are equally spaced, that is, for t = 1, . . . , T, rt - Tt_1 = s is constant, we get the optimum value for s. In the asymptotic theory of estimation, the concept of asymptotic expectation is widely used, and it is usually remarked that it can be different from the asymptotic value of expectation. But, the concept itself is not sufficiently accurately defined in the literature, especially when the asymptotic distribution does not exist. The paper [24] gives a rigorous definition of the asymptotic expectation, and shows its properties, e.g. its linearity and a Markov type inequality. And we obtain the necessary and sufficient conditions for the convergences in probability and distribution. In the regular case when the dimension of the parameter is finite, asymptotic loss of information is constant order, but, for semiparametric models, it is shown in [31] that, under fairly regularity conditions, the asymptotic deficiency of the MLE or any regular best asymptotically normal estimator is infinity. In [33], we consider the sampling properties of the bootstrap process, that is, the empirical process obtained from a sample of size n (with replacement) of a fixed sample of size n of a continuous distribution. And we give the cumulants of the bootstrap process up to the order n-1, and discuss their unbiased estimation. We also demonstrate that the bootstrap process has an asymptotic minimax property for some class of distributions up to the order n-1/2, and further we suggest the bootstrap method can be improved in the next order by taking a sample of size n - 1 instead of n. Usually confidence interval is defined as an interval with preassigned confidence level 1 - a for all the values of parameters. In [38], generalizing the concept, we consider interval estimation procedures with confidence coefficient varying according to the value of the unknown parameter, and associated procedure to estimate the actual level. Such a consideration also leads to more general procedures including conditional procedures when the ancillary statistic is involved. For discrete distributions it is usually impossible to obtain a non-randomized test or confidence interval with exact given size, and an actual size is often quite different from the prescribed level. But a randomized procedure, which is quite nice in theory, is not easily acceptable to practitioners. Still, there is something to promote randomized procedures in practical applications. In the paper [41],

for a family of one-parameter discrete exponential family of distributions, we derive the higher order approximation of randomized confidence intervals from the optimum test. Indeed, it is shown that they can be asymptotically constructed by means of the Edgeworth expansion. The usefulness is seen from the numerical results in the case of Poisson and binomial distributions. The distribution of the sum of not identically but independently distributed random variables are difficult to calculate exactly, and the normal and Edgeworth type approximations can be applied when the number of independent random variables is not too small. But it is not always sufficiently accurate especially for the tail part. In such cases, large-deviation approximations were proposed in order to give better approximations especially for tails. In the paper [43], we obtain large-deviation approximations for the distribution of the sum of discrete random variables and show that they give sufficiently accurate results in various cases. It is noted that the large-deviation approximation connected with the saddlepoint approximation is widely discussed by Barndorff-Nielsen and Cox (1989) and Jensen (1995). In sampling from the finite population of size N, there is a problem how to construct a design in which the inclusing probability of k-unit in the population is equal to prescribed Irk and F_'-1 irk = n, and it is a problem whether there exists a sample design of size n which attains the condition. The problem is solved by the Minkowski-Farkas theorem.

References Akahira, M. (1975a). Asymptotic theory for estimation of location in nonregular cases, I: Order of convergence of consistent estimators. Rep. Stat. Appl. Res., JUSE 22, 8-26. Akahira, M. (1975b). Asymptotic theory for estimation of location in nonregular cases, II: Bounds of asymptotic distributions of consistent estimators. Rep. Stat. Appl. Res., JUSE 22, 99-115. Akahira, M. (1986). The Structure of Asymptotic Deficiency of Estimators. Queen's Papers in Pure and Applied Mathematics 75, Queen's University Press, Kingston, Canada. Akahira, M. and Takeuchi, K. (1981). Asymptotic Efficiency of Statistical Estimators: Concepts and Higher Order Asymptotic Efficiency. Lecture Notes in Statistics 7, Springer, New York. Akahira, M. and Takeuchi, K. (1995). Non-Regular Statistical Estimation. Lecture Notes in Statistics 107, Springer, New York. Amari, S. (1985). Differential- Geometrical Methods in Statistics. Notes in Statistics 28, Springer, Berlin.

Lecture

Xiv

Bahadur, R. R. (1955). A characterization of sufficiency. Ann. Math. Statist. 26, 286-293. Barndorff-Nielsen, O. E. and Cox, D. R. (1989). Asymptotic Techniques for Use in Statistics. Chapman and Hall, London. Chapman, D. G. and Robbins, H. (1951). Minimum variance estimation without regularity assumptions. Ann. Math. Statist. 22, 581-586. Efron, B. (1975). Defining the curvature of a statistical problem (with applications to second order efficiency). Ann. Statist. 3, 1189-1242. Fisher, R. A. (1925). Theory of statistical estimation. Proc. Camb. Phil. Soc. 22, 700-725. Fisher, R. A. (1934). Two new properties of mathematical likelihood. Proc. Roy. Soc. (London) Ser. A 144, 285-307. Fraser , D. A. S. and Guttman, I. (1951). Bhattacharyya bounds without regularity assumptions . Ann. Math. Statist. 23, 629-632. Ghosh, J. K. (1994). Higher Order Asymptotics. NSF-CBMS Regional Conference Series Probab. and Statist., 4, Inst . of Math. Statist., Hayward, California. Ghosh, J. K., Sinha, B. K. and Wieand, H. S. (1980). Second order efficiency of the mle with respect to any bounded bowl-shaped loss function. Ann. Statist. 8, 506-521. Ghosh, J. K. and Subramanyam , K. (1974). Second order efficiency of maximum likelihood estimators . Sankhya Ser. A 36, 325-358. Hodges, J. L. and Lehmann, E. L. (1970). Deficiency. Ann. Math. Statist. 41, 783-801. Ibragimov, I. A. and Has'minskii, R. Z. (1981). Statistical Estimation: Asymptotic Theory. Springer, New York. Jensen, J. L. (1995). Saddlepoint Approximations. Clarendon Press, Oxford. Kiefer, J. (1952). On minimum variance estimators. Ann. Math. Statist. 23, 627-629. Lehmann, E. L. (1983). Theory of Point Estimation. Wiley, New York. Pfanzagl, J. (1979). First order efficiency implies second order efficiency. In: Contributions to Statistics. Jaroslav Hdjek Memorial Volume. (J. Jureckova, ed.), 167-196, Academia, Prague. Pfanzagl, J. and Wefelmeyer, W. (1978). A third order optimum property of the maximum likelihood estimator. J. Multivariate Anal. 8, 1-29.

Xv

Pfanzagl, J. and Wefelmeyer, W. (1985). Asymptotic Expansions for General Statistical Models. Lecture Notes in Statistics 31, Springer, Berlin. Rao, C. R. (1961). Asymptotic efficiency and limiting information. Fourth Berkeley Symp. on Math. Statist. and Prob. 1, 531-545.

Proc.

Skibinsky, M. (1967). Adequate subfields and sufficiency. Ann. Math. Statist. 38, 155-161. Takeuchi, K. (1962). On a fallacy of Gunnar Blom's theorem. Rep. Stat. Appl. Res., JUSE 9, 34-35. Torgersen, E. N. (1977). Prediction sufficiency when the loss function does not depend on the unknown parameter. Ann. Statist. 5, 155-163. Weiss, L. and Wolfowitz, J. (1974). Maximum Probability Estimators and Related Topics. Lecture Notes in Math. 424, Springer, Berlin. Zacks, S. (1971). The Theory of Statistical Inference. Wiley, New York.

This page is intentionally left blank

Xvii

Contents

Foreword Introduction

v vii

1975 [1] Characterizations of Prediction Sufficiency (Adequacy) in terms of Risk Functions 1 Ann. Statist. 3(4) 1018-1024. 1976 [2] On the Second Order Asymptotic Efficiency of Estimators in Multiparameter Cases 8 Rep. Univ. Electro-Commun. 26(2) 261-269. [3] On Gram-Charlier-Edgeworth Type Expansion of the Sums of Random Variables (I) 17 Rep. Univ. Electro-Commun. 27(1) 95-115. [4] On Gram-Charlier-Edgeworth Type Expansion of the Sums of Random Variables (II) 38 Rep. Univ. Electro-Commun. 27(1) 117-123. [5] On the Second Order Asymptotic Efficiencies of Estimators 45 Lecture Notes in Mathematics, 550, Springer-Verlag, Berlin, 604-638.

1977 [6] Extension of Edgeworth Type Expansion of the Distribution of the Sums of I.I.D. Random Variables in Non-Regular Cases

80

Ann. Inst. Statist. Math. 29(3) 397-406. 1978 [7] On Gram-Charlier-Edgeworth Type Expansion of the Sums of Random Variables (III) Multivariate Cases Rep. Univ. Electro-Commun. 28(2) 259-269. [8] Third Order Asymptotic Efficiency of Maximum Likelihood Estimator for Multiparameter Exponential Case Rep. Univ. Electro-Commun. 28(2) 271-293. [9] Asymptotic Optimality of the Generalized Bayes Estimator Rep. Univ. Electro-Commun. 29(1) 37-45.

90

101 124

Xviii

1979 [10] On the Second Order Asymptotic Efficiency of Unbiased Confidence Intervals 133 Rep. Stat. Appl. Res. JUSE 26(3) 99-110.

[11] Remarks on the Asymptotic Efficiency and Inefficiency of Maximum Probability Estimators 145 Rep. Stat. Appl. Res. JUSE 26(4) 132-138. [12] Discretized Likelihood Methods - Asymptotic Properties of Discretized Likelihood Estimators (DLE's) 152 Ann. Inst. Statist. Math. 31(1), Part A, 39-56. [13] Asymptotic Optimality of the Generalized Bayes Estimator

in Multiparameter Cases 170 Ann. Inst. Statist. Math. 31(3), Part A, 403-415. [14] Note on Non-Regular Asymptotic Estimation - What "Non-Regularity" Implies 183 Rep. Univ. Electro-Commun. 30(1) 63-66. 1980 [15] A Note on Prediction Sufficiency (Adequacy) and Sufficiency 187 Austral. J. Statist. 22(3) 332-335. [16] Third Order Asymptotic Efficiency and Asymptotic Completeness of Estimators 191 Rep. Univ. Electro-Commun. 31(1) 89-96.

1982 [17] On Asymptotic Deficiency of Estimators in Pooled Samples in the Presence of Nuisance Parameters Statistics & Decisions 1(1) 17-38.

199

1985 [18] Estimation of a Common Parameter for Pooled Samples from the Uniform Distributions Ann. Inst. Statist. Math. 37(1), Part A , 17-26. [19] A Note on the Minimum Variance Unbiased Estimation When the

221

231

Fisher Information is Infinity

Rep. Stat . Appl. Res. JUSE 32(3) 17-22. 1986 [20] Bhattacharyya Bound of Variances of Unbiased Estimators in Non-Regular Cases (with Madan L. Puri)

Ann. Inst. Statist. Math. 38(1), Part A, 35-44.

237

XiX

[21] A Note on Minimum Variance Metrika 33(2) 85-91. [22] A Note on Optimum Spacing of Observations from a Continuous Time Simple Markov Process Metrika 33(3/4) 217-222. [23] On the Bound of the Asymptotic Distribution of Estimators when the Maximum Order of Consistency Depends on the Parameter Publ. Inst. Stat. Univ. Paris 31(1) 1-16.

247

254

260

1987 [24] On the Definition of Asymptotic Expectation Foundations of Statistical Inference, Advances in the Statistical Sciences 2, D. Reidel Publishing Company, Dordrecht-Holland, 199-208. [25] Locally Minimum Variance Unbiased Estimator in a Discontinuous Density Function Metrika 34(1) 1-15 [26] The Lower Bound for the Variance of Unbiased Estimators for One-Directional Family of Distributions Ann. Inst. Statist. Math. 39(3), Part A, 593-610.

276

286

301

1988 [27] Second Order Asymptotic Efficiency in terms of Asymptotic Variances of the Sequential Maximum Likelihood Estimation Procedures Statistical Theory and Data Analysis II (Elsevier Science Publishers B. V. North-Holland) 191-196. [28] Second and Third Order Asymptotic Completeness of the Class of Estimators (with F. Hirakawa) Probability Theory and Mathematical Statistics. Lecture Notes in Mathematics 1299 , Springer-Verlag, Berlin, 11-27.

319

325

1989 [29] Higher Order Asymptotics in Estimation for Two-sided Weibull Type Distributions Ann. Inst. Statist. Math. 41(4) 725-752. [30] Third Order Asymptotic Efficiency of the Sequential Maximum Likelihood Estimation Procedure Sequential Analysis 8(4) 333-359.

342

370

XX

1990 [31] First Order Asymptotic Efficiency in Semiparametric Models Implies Infinite Asymptotic Deficiency

397

Publ. Inst. Stat. Univ. Paris 35(1) 3-9. [32] Loss of Information Associated with the Order Statistics and Related Estimators in the Double Exponential Distribution Case

404

Austral. J. Statist. 32(3) 281-291. 1991 [33] Bootstrap Method and Empirical Process 415 Ann. Inst. Statist. Math. 43(2) 297-310.

[34] Second Order Asymptotic Efficiency in Terms of the Asymptotic Variance of Sequential Estimation Procedures in the Presence of Nuisance Parameters 429 Sequential Analysis 10(1&2 ) 27-43.

[ 35] Asymptotic Efficiency of Estimators for a Location Parameter Family of Densities with the Bounded Support 446 Rep. Stat. Appl. Res., JUSE 38(1) 1-9. [36] A Definition of Information Amount Applicable to Non-regular Cases 455 J. Comput. Information 2(1) 71-92. 1992

[37] Unbiased Estimation in Sequential Binomial Sampling (with K. Koike) 477 Rep. Stat. Appl. Res. JUSE 39(4) 1-13. [38] Interval Estimation with Varying Confidence Levels 490 Metron 50(3-4) 3-18. 1993 [39] Second Order Asymptotic Bound for the Variance of Estimators for the Double Exponential Distribution 506 Stat. Sci. 4 Data Anal., VSP Internat. Sci. Publ., Zeist (Netherlands) 375-382. [40] On the Application of the Minkowski-Farkas Theorem to Sampling Designs 514 Statistica Neerlandica 47(3) 221-223. 1997

[41] Randomized Confidence Intervals of a Parameter for a Family of Discrete Exponential Type Distributions (with K. Takahashi) 517 Commun. Statist . - Simula. 26(3), 1103-1128.

Xxi

[42] The Existence of a Test with the Largest Order of Consistency in the Case of a Two-sided Gamma Type Distribution

543

Metron 55(1-2) 93-107. 1999 [43] The Higher Order Large-Deviation Approximation for the Distribution of the Sum of Independent Discrete Random Variables (with K. Takahashi) 558 Commun. Statist. - Theory and Methods 28(3&4) 705-726. 2001 [44] Information Inequalities in a Family of Uniform Distributions 580 Ann. Inst. Statist. Math. 53(3) 427-435. Permission

589

1 The Annals of Statistics 1975, Vol. 3, No. 4, 1018-1024

CHARACTERIZATIONS OF PREDICTION SUFFICIENCY (ADEQUACY) IN TERMS OF RISK FUNCTIONS BY KEI TAKEUCHI AND MASAFUMI AKAHIRA University of Tokyo and University of Electro- Communications Prediction sufficiency (adequacy ), as it is usually defined in terms of conditional expectations, does imply "real" prediction sufficiency ; i.e. sufficiency in terms of risk functions . The converse holds provided we permit the loss to depend on the unknown parameter . This is no longer true if we insist on loss functions which do not involve the unknown parameter. Conditional independence still holds but ordinary sufficiency may fail. If, however, we require equivalence of risk functions , then ordinary sufficiency and, consequently , prediction sufficiency follows.

1. Introduction . It has been shown by Bahadur [2 ] that sufficiency as defined in terms of conditional expectations , under regularity conditions , implies "real sufficiency " i.e. sufficiency in terms of risk functions . Furthermore it follows from Theorem 11.3 in Bahadur ' s paper [ 1 ] that prediction sufficiency is equivalent to ordinary sufficiency w.r.t. a larger class of probability measures. One may therefore expect similar results to hold for prediction sufficiency (adequacy) as well . In the case of prediction problems it may be of interest to consider loss functions which depend only on the decision to be made and the quantity to be predicted . If we insist on this restriction, then prediction sufficiency in terms of risk functions no longer implies prediction sufficiency as it is defined in terms of conditional expectations . Conditional independence holds but ordinary sufficiency may fail . It will, however , be shown that equivalence of risk functions implies ordinary sufficiency and consequently prediction sufficiency. (One of the authors proved this in an earlier work [7].) We will , essentially , use the framework of Skibinsky [ 6]. The notion of adequacy in Skibinsky ' s paper is, however , replaced by the notion of prediction sufficiency. 2. Theorems . We shall assume that we are given a model consisting of a sample space (^2', Ste) and a family {P0 : 0 E 9 ) of probability measures on .S^l. A sub a-algebra . of _V summarizes what can and what can not be observed. Similarily , a sub a-algebra ' of ,S describes what we are interested in predicting . Finally, we are given a sub a-algebra M. of . and our problem is to decide if anything is lost by basing our predictions on rather than . The prediction problem is assumed to be completely described by a decision space ( T, .9 ), i.e. a measurable space and a loss function L from 9 x 2' x T to [0, oo[. It will always be assumed that L as a function on ^C'' x T for given Received January 1973; revised August 1974. AMS 1970 subject classifications. Primary 62805; Secondary 62007. Key words and phrases . Prediction sufficiency, conditional independence , equality of risk functions. 1018

2

SUFFICIENCY IN TERMS OF RISK FUNCTIONS 1019

9 e 6 is ' x 9 measurable . This implies that the loss does not depend on all of x e 1', only on the part of x which is to be predicted. A decision rule 6 will here be defined as a Markov kernel 8 (S I x) : S e .I, x C,2' which is .O' measurable when S is fixed and a probability measure on -9'- when x is fixed. If S is a decision rule then its performance characteristic Pa(. 0); 0 e 6 may be defined by defining - for each 0 - Pa(. B) as the probability measure on x .9'" defined by

Pa(C x S 9) = S c 8(S I x)Pe(dx) The risk function ra(0); 0 e 8 of a decision rule 6 is given by: 0e8. ra(B) = S [ S L0(x, t)8(dt I x)]Pe(dx) , The risk function is determined by the loss function and the performance characteristic through ra(0) = S Le dpa (. 10) , 9 e 8. A decision rule 6 will be called measurable if 8(S ^ .) is _'50. measurable for each S. DEFINITION 1. .eV and 25' are conditionally independent given

if

(i) Pe1(C .9') = P0'(C I _e,) a.e. [P,,] for all C e ' and for all 0 e 8. It is shown in Loeve [5], page 351 that (i) and (ii) are equivalent: (ii) pe."(B n C I ) = P9'(B I _,^F.)P0 "(C I A) a. e. [Pe] for all Be.,Wand all C e9andfor all 0e8. We define prediction sufficiency (adequacy ) and prediction sufficiency in the wide sense as follows: DEFINITION 2. .moo is prediction sufficient for M w.r.t. le . iffy is sufficient for M and M and ' are conditionally independent given .moo. DEFINITION 3. .-Wo is prediction sufficient in the wide sense for ' w.r.t. o and (b) there exist M. if (a) M and le' are conditionally independent given measurable sets B, and B, so that P0(B1 u B,) = 1 for all 0 e 0 and Pe (• ^ ^) is independent of 0 if x e B, and P "B; s(. I .moo) is independent of 0 if x e B,.

In the following example we shall show that ., is prediction sufficient in the wide sense for.' w.r.t. 'e but not prediction sufficient for M w.r.t. '^6'. We assume that X„ X„ • • • , X. and Y are random variables such that X„ X„ • • • , X,, are independently and identically distributed as N(6, 1) while the conditional distribution of Y given X„ X„ • • • , X. is N(0, 1) or N(0, 1) as u, Xi > a or be the a-algebras induced by, respectively, (X„ E, Xi < a. Let , A and X„ • • • , X.), min (a, E i Xi) and Y. Then o is prediction sufficient in the wide sense for W w.r.t. ;%' but not sufficient for

That sufficiency alone is insufficient in prediction problems may be seen by considering , for example , the situation where Pe does not depend on 0 and

3 1020 KEI TAKEUCHI AND MASAFUMI AKAHIRA

,V, = It is then fairly obvious that prediction of a which is not independent of ^% should not, in general , be based on .moo. It follows, as has been pointed out by Skibinsky [6], from Theorem 11.3 in Bahadur [1] that , is prediction sufficient for . w.r.t. ' if and only if is sufficient for w.r.t. all probability measures on ' of the form: B n-+ Pe(B C) ,

0 E 8 and P0(C) > 0. where C E In analogy with Theorem 10.2 in Bahadur [1] we get: THEOREM 1. Suppose A is prediction sufficient for .,W w.r.t. 1 'and there exists a regular conditional probability P°(• I ✓ ^a) of given which does not depend on B. Let 6 be any decision rule from (', _e-Wfl to (T, ,l) and put 6(S ^ x) _ S h(S I x')Pz-4 (dx' I S E -9'-, x E Then $ is .tea measurable and it has the same performance characteristic as 3. In particular & and 3 have the same risk functions. PROOF.

ue(C x S I S)

= S c a(S I x)P6(dx) = S c E' (S( S I •) I M.) dPe =ScE`''(3(S I .) I M., `e)dPe=Sc6(SI•)dPe=u0(Cx S b).

REMARK. As is immediately seen from above, we need to allow randomized decision rules. This is not always necessary for the subsequent discussions. Consequences of "risk prediction sufficiency" for various classes of loss functions. In order to show prediction sufficiency of .M:,o we must establish conditional independence and ordinary sufficiency. We will assume that we are given a certain class of loss functions and that to any loss function within that class and to any decision rule 3 corresponds a decision rule A which is M. measurable and has uniformly smaller risk than 6. The problem is to decide whether this suffices to establish conditional independence or ordinary sufficiency. It is clear that conditional independence cannot, in general, be established by only considering loss functions which do not depend on x. Similarly loss functions which do not involve 0 will, in general, be insufficient to establish ordinary sufficiency. Conditional independence may, however, be established by considering only loss functions which do not depend on 0. Similarly, and this follows from corresponding facts for sufficiency (see Bahadur [2], Blackwell [3] and Le Cam [4]), sufficiency of M. may be established by considering loss functions which do not depend on x.

Conditional independence may be established by considering the two decision problem with loss functions not depending on 0 as follows: THEOREM 2. Consider the decision space T = {0, 1} and the set of all loss functions, L, of the form L0(x, 0) = II(x) , x E , 0E8, L6(x, 1) = pIc.(x) , where p E ]0, 1[ and C E 'Y.

x E L" ; 0 E 8 ,

4 SUFFICIENCY IN TERMS OF RISK FUNCTIONS 1021

Suppose that to each decision rule 3 and to each loss function L, of the above form, there corresponds a o measurable decision rule fi so that r;(9) < ra (B)

,

0 E0.

Then and '' are conditionally independent given .moo.

Before proving the theorem a few remarks may be in order. REMARK 1. It follows from the proofs that we may restrict attention to nonrandomized decision rules. REMARK 2 . The proofs imply also that much smaller sets of loss functions will do . We may, for example , restrict C to a s-system generating 'f. REMARK 3 . The parameter space 0 does not play any role in this theorem. We may-and shall-in the proof assume that 0 consists of a single point. Conditional independence is, in this situation , equivalent to prediction sufficiency. PROOF OF THE THEOREM. We may, by Remark 3, omit the subscript 0. Furthermore a decision rule 3 may be identified with the critical function x n-^ 8(1 I x). The risk may then be written: r(d) = S L(•, 0) dP + S [L(•, 1) - L(•, 0)]S dP

= SL(•,0)dP+(p+ 1)S(p+ 1 - J)odP = S L(., 0) dP + (p + 1) S [p + 1 - P-(C I .,W)1 3 dP

S L(., 0) dP - (p + 1) S [P(cl M) P +

11

dP ,

where is obtained iff a = 0 a.e. or 3 = I a.e. as PI(C I .') < p/(p + 1) or P"(CI --51) > PAP + 1) The same argument applied to M. implies , by the assumption of the theorem, that the minimizing 6 may be chosen M. -measurable and such that 3 = 0 a.e. or 8 = 1 a . e. as P'(C I .moo) < pl (p + 1) or P""(C I A) > Pl ( P + 1). It follows that the event [P '(C I ) < p/(p + 1)] and the event [P "'(C I M^) < Pl (P + 1)] are equivalent provided P'(CI. ) * pl(p + 1) a .e. and PE'(CI.2'o) * pl(p + 1) a.e. This implies that the random variables PI (C I M) and P '(C I . .) have the same distribution . Hence, since P& (C I M.) = E° (P(C I P`'(C I M.) = ' are conditionally independent given P{'(CI ) a. e. It follows that is' and o•

REMARK . This form of the proof was suggested by one of the referees. A criterion based on least squares prediction theory is , as has been pointed out by one of the referees , even simpler to establish . Consider a sufficiently large class of square integrable and 7 =measurable random variables g. To a

1022 KEI TAKEUCHI AND MASAFUMI AKAHIRA

given g we associate the loss function L(x, t) = (g(x) - 1)2 , x E 2', t E ] - oo, co[ . Then a predictor d minimizes the risk if and only if it is a version of EI(g ^ M). If .-W, is assumed to be just as good in this situation then E' (g I ) = El'(g ^ moo) a.e. This establishes conditional independence if, for example, we admit all functions g = I, where C runs through a s-system generating '. For the Lemma and Theorem 3 we assme that {P8 : 0 E e} is dominated. Then we get the following lemma. measurable critical function cp there exists ao measurable LEMMA . If for any -,qcr.. ical function (P such that E,((P) = E,((p) for all 0 E 8 , then .,_--e. is sufficient for _-R The proof of the lemma is essentially the same as in Bahadur [2 ]. The outline is as follows: Let 0, and B, be any two points of 8. Let ¢ be a most powerful test for 0 against B,. Then for some k

fi(x) = 1 if

PP' > k 1

=0 if

dP"
and the set {x : (dPes/dP01)(x) < k} is Vie-measurable . Further for every c (including oo), the set {x : (dP92/dPB1)(x) < c} is M.-measurable . Hence .mo o is pairwise sufficient for M. Since {P0 : 0 E 8} is dominated , _,W. is sufficient for The following proposition is an immediate consequence of the lemma: Suppose that the decision space T = [ 0, 1] and the loss function L satisfies L(x, t) = t, for 0 < t < 1. If for any . 'measurable decision rule 6, there exists a .moo measurable decision rule A such that ri (0) = r8(0) for all 0 E 8, then M. is sufficient for M.

From the above , we get the following theorem. ' -measTHEOREM 3 . If for any loss function L not depending on 0 and for any cam urable decision rule 6 , there exists a A-measurable decision rule S such that ri(0) _ o is prediction sufficient for .' w.r.t. r,(B) for all 0 E 8, then Various criteria for prediction sufficiency may be obtained from these results by considering, in addition to the loss functions described above , loss functions which depend on 0. If we insist on considering only loss functions which do not depend on 0 then we will, in general, not be able to conclude prediction , ' ' is sufficiency . This follows by considering the case where ancillary and independent of .' and {Po : 0 E 81 is finite. Let, in this situation , ( T, %) be any decision space and L any loss function to (T, -). which does not depend on 0 and 6 any decision rule from Choose a to E T such that S L(x, ta)P0(dx ) < S [5 L(x, t)P0(dx)]v,, (dt) , 0 E 0,

6 SUFFICIENCY IN TERMS OF RISK FUNCTIONS 1023

where vo(S) = S 8(S I x)P0(dx). Then ro(t•)
BE®.

The necessary condition for prediction sufficiency in the wide sense is obtained as follows: THEOREM 4 . Suppose that , for each 6, there are regular conditional probabilities PO(B I geo) : B E .Gif and P,(C I Via) : C E ' '. Suppose further that :isa is prediction w.r.t. sufficient in the wide sense for Let L be a loss function not depending on 0 and assume that there is a . •-measurable function r on B, and an s > 0 so that S L(x', r)P(dx' I .%5o) < S L(x', t)P(dx' I :0) + e , t E T on B,. Then there corresponds to any decision rule 3 a ..-measurable decision rule S so that re(S) < ro(a) + e , 0 E may be defined as r on B, and as E(d(• i •) I A) on B,. REMARK. There exist , for each e > 0, a moo-measurable r on B, satisfying the desired inequality provided (i) There exists a countable subset It,, t„ • .. , t,,, • • • } of T such that for all t E T and for all x E B, =` (dx' I .c^'•) S S L(x', t)PP'*(dx' I ,^^' o) , inf. S L(x', tn)P and (ii) For every pair i, j and for any e > 0 the set M{j(s) = [x: S L(x', t,)P;" (dx' I .moo) < S L(x', tt)P.- (dx' I •) + s) is measurable. PROOF OF THE THEOREM. Put, for each 0,

5 e *(SI •) = S 3(SI x')Pe(dx'I.^) By the proof of Theorem 1:

e + re(a) = s + S [S L(x, t)be*(dtI x)]Po(dx) = S B, [ S L(x, t)S(dt I x)]P0(dx) + s + SB, {S [S L(x', t)P.(dx' I . •)]8B*(dt I x)}Po(dx) SB, [S L(x, t)S(dt I x)]Po(dx) + S B, { S L(x', b(x))P0.z(dx' I .)}Pe(dx) = S B, [ S L(x, t)5(dt I x)]P0(dx) + S B, L(x, d(x))Pe(dx)

re(b) . Acknowledgments . The authors wish to thank Mr. M. Takahashi of Osaka University for valuable suggestions and the referees of the Annals for co-operation in completing the final version.

7

1024 KEI TAKEUCHI AND MASAFUMI AKAHIRA

REFERENCES [ 1] BAHADUR , R. R. (1954). Sufficiency and statistical decision functions . Ann. Math. Statist. 25 423-462. [2] BAHADUR , R. R. (1955). A characterization of sufficiency . Ann. Math. Statist . 26 286-293. [3] BLACKWELL , D. (1953). Equivalent comparisions of experiments . Ann. Math . Statist. 24 265-272. [4] LE CAM , L. (1964). Sufficiency and approximate sufficiency . Ann. Math. Statist. 35 14191455. [5] LOEVE , M. (1963). Probability Theory, (3rd. ed .). Van Nostrand , Princeton. [6] SKIBINSKY , M. (1967). Adequate subfields and sufficiency. Ann. Math . Statist . 38 155-161. [7] TAKEUCHI , K. (1966). On some statistical prediction procedures . ( In Japanese). Keizaigaku Ronshu 32 23-31. FACULTY OF ECONOMICS DEPARTMENT OF MATHEMATICS UNIVERSITY OF TOKYO UNIVERSITY OF ELECTRO -COMMUNICATIONS HONGO, BUNKYO-KU CHOFUGAOKA , CHOFU-SHI TOKYO , JAPAN TOKYO, JAPAN

8 Rep. Univ. Electro-Comm. 26-2, (Sci. & Tech. Sect.), pp. 261-269 February, 1976 261

On the Second Order Asymptotic Efficiency of Estimators in Multiparameter Cases* Masafumi AKAHIRA** and Kei TAKEUCHI***

Abstract Suppose that Xi's(i=1, 2,..., n) are independently and identically distributed with the density f (x, 8, ^), where 8 is a real valued parameter and a is a real (vector) valued parameter. We consider a (sequence of) estimator(s) which is k-th order asymptotically median unbiased, and define k-th order asymptotic efficiency. We have a formula for the distribution of the second order asymptotically efficient estimator and show that a modified maximum likelihood estimator is second order asymptotically efficient.

1. Introduction Second order asymptotic efficiency of estimators has been discussed by Pfanzagl [2], Takeuchi and Akahira [3], Takeuchi [4] and Akahira [ 1] for one dimensional case. In this paper we extend a similar approach to multiparameter cases and obtain a straight -forward generalization of the one-parameter case. Let Xi, X2,. - -, X., ...

be a sequence of independent identically distributed random

variables with the density f (x, 0, ^), where 8 is a real valued parameter and . is a real (vector) valued parameter. We assume that ^ is a nuisance parameter . We shall define an estimator of 0 to be k -th order asymptotically efficient if the k-th order asymptotic distribution of it attains the bound of the k-th order asymptotic distributions of k-th order asymptotically median unbiased ( AMU) estimators of 0. We shall obtain the bound of the second order asymptotic distributions of second order AMU estimators and show that a modified maximum likelihood estimator is second order asymptotically efficient.

2. Notations and definitions Let X be an abstract sample space whose generic point is denoted by x, B a a-field of subsets of X and let ® and 5 be parameter spaces which are assumed to be open sets in R' and R° respectively. (We denote by R➢ a Euclidean p-space with a norm II • II.) We assume that 1 =(ci, ... , l p) (ES) is a nuisance parameter. We consider a sequence of classes of probability measures {PB, t, i : (0, 6) E®x 5} (i=1, 2, ...) each defined over (Z B).

We shall

denote by (X (" ), B"`)) the n-fold direct products of (1, B) and the corresponding product measures by Ps, e"" = PB, t, 1 x ... x Pe, e, ,,.

An estimator of 0 is defined to be a sequence {On}

* Received on December 10, 1975 ' Statistical Laboratory, University of Electro-Communications *** University of Tokyo

9

262

Masfumi AKAHIRA and Kei TAKEUCHI

of On)-

measurable functions Bn on Xin) into (9 (n=1, 2, ... ).

Definition 1.

For each k=1, 2, ... , {0n} is k-th order asymptotically median unbiased

(or k-th order AMU) estimator if for any 8o=(Oo, eo )E®XE, there exists a positive number S such that k-1

1

1

lim sup n 2 Ps(n){n2(en-8)50} - 2 =0; n-.00 9o8x8 :

Isk -sl<6

lim sup nk21lPs(n) {n2(Bn-0)>0} - 1 I =0. n-ooBeex8 : Iso-sl
Definition 2.

_ 1 k-1 8, 6)+n TG1(t, 8, e)+... +n - 2 Gk_1(t, 8, ^) is called to be the k-th order asymptotic distribution of it if

PB, E (n) { n2 (O n -0)
lim n- M

n

2l k

-n- 2Gk -1(t, 0,) =0. 1

Consider the problem of testing hypothesis H+ : 8=Bo+tn-2(t>0),

against K : 0=0o,

We shall denote by pn(t, an, Bo, ^) the power function of the most powerful level an test (00 \\ sup

k-1 limn-_ 2 {an(t, an, Bo, o)-Ho+(t, Bo, 6o)

flax) : a n = (1/2)+o(n- 'k-"/')} n-. oo

1 k-1 -n THi+( t, Bo, co )-... -n - -2 Hk- 1+(t, 0o,

o)} =0,

then we have Go(t, Bo, o)
G1(t, Bo, o)=Hi+(t, Bo, eo)

then

G1( t, Bo, eo )
(i=0, ... , j-1),

Consider next the problem of testing hypothesis H- : 0=0o+tn 2(t<0),

against K : 0

=00, ^_^. If for ^=^o and for each t<0 k-1 inf limn-2 {19n(t, an, Bo, o) -Ho-(t, Bo, ^o) Ilan} : an=(1/2)+o(n-a-1)/')} n-'oo -1 k-1 -n 2H1-(t, Bo, co)-... -n - -2 Hk-1-(t, Bo, ^o)} =0,

then we have Go(t, Bo, o) >Ho-(t, Bo, Sao) ;

Definition 3.

if

Gi(t, Bo, o)=Hi-(t, Bo, . o)

then

G1(t, Bo, eo)>Hf-(t, Bo, ^o) (j1,. .. , k).

(i=0, ... , j-1),

Suppose that {0n} is k-th order asymptically median unbiased, it is

called to be k-th order asymptotically efficient if for each OE19 and each ^EE

+(t, 8, ) for t>0, G1(t, B, ^)==-(t, e, e) for t<0, i=0,...,k-1. In the subsequent discussions we shall deal only with the second order asymptotic efficiency,

but the line of discussions can be extended to general cases.

10 February, 1976 On the Second Order Asymptotic Efficiency of Estimators in Multiparameter Cases 263

3. Second order asymptotic efficiency in multiparameter cases. We assume that 8 is an open set of R' because in the subsequent discussions there arises no substantial difference between R1 and R°(p>2). Let X1, X2,. - -, Xn,... be a sequence of independently and identically distributed random variables having a density function f (x, 0, ^). First we shall obtain the bound of the power functions. Consider the problem of testing _ 1

1

hypothesis H+: 0 =6o +tn 2(t>0), ^= ^o +un -2, against K : 0 =6o, ^ =eo, where u is an 1

1

arbitrary but fixed constant . Putting 61=6o+tn-2 and ^1=eo +un -2, we define Zin as follows: Z tn=log

f (X1, 60, e0) f (Xi, 61, 1)'

The most powerful test is given by the following rejection region n E Zin>C, i=1

where c is some constant. For each i =1, 2, ... , Zin is expanded as follows :

Zin=-

60> aBlogf(X1,

1

o)(tn Z)-F log f(X1, 6o, ^o)( un 2)

02

02

t2n-1 ) - 1 2 a62 log f (X i, 60, eo)( Bad log

1 6

2 log f (Xi, Bo, eo)( u2n-1)

f (Xi, Bo, ^o)( tun'1)

a3

a631og f (X 1, 6o, ^o)(

63 3 t3n -3 -f )- 6 a 3 log .f (X i, 60, ^o)(u3n Z )

- 2 6020 log f (X 60, ^o)(t2un

2) - 2 a6aelog

f (Xi, eo, eo)( tu2n- z )

3 +o(n 2).

Hence E Zi n is asymptotically normal. If 0=Bo and e_ ^o, then the asymptotic mean µo and i=1

n the asymptotic variance 602 of Zin are given by i=1

_1 [lo = 2 (Ioot2 + Iuu2+2loitu)+o(n Z); _1 Qo2=loot2 -}-I11u2+2Ioitu+o(n 2),

where

a

loo=Eeo. E.{86 log f(Xi, 60, eo)}2=-Eeo,eo{ a62 a 2log f(Xi, 60, eo)} Io1= Eeo. co{aB log f (Xi, 60, o)^ log f (Xi, 60, eo)} z =-Eeo,E°{aBa-c

log f(Xi,Go, o)

a

2

a2

Iu=Eeo.eo{a log f(Xi,60, o)} Eeo, Eo{a 2logf(Xi,60, o)}. If 6 = 61 ^ = ei, then if follows that

11 264

Masfumi AKAHIRA and Kei TAKEUCHI

Ee,.e,i

lo

log .f(X1, 60, o)}

) f f (Xi, 01, el = Ee.. e. {aB log f(X Bo, o

(X 1, 6o, ^o)J

=Ee..E.{aB logf(X1, 90, ^o )}Z( tn Z)

+Eo,.e.{a6 log f(X+, 6o, o)a log f(Xi,6o, Co)}(un 2)+ o(n Z) Ee,. e, {a- log f (X r, 60, o)} Ea.. e.

{aB log f (X 6o, o)

+E0,,e.

log Ax " 6o, 6)} (tn z )

.

log f(X;, Bo, eo)} (un- Z)+o(n- ).

{a

n For 6=61 and 6=E1, the asymptotic mean ft1 and the asymptotic variance a12 of E Z;n are t_1

given by 1 ,u1= - 2 (Ioot2+I11u1+2Iiotu)+ o(n z) ; 1 C18=0 + o(n Z). n

Hence the asymptotic power of the most powerful test of E Z, , is obtained as follows : i=1 O(,uo+/s1) = o( Ioot$+I11u°+2lo1tu) Oro

(3. 1)

_ n2 where fi(x) =J e a du. Since u can take arbitrary values, then the power function of the tests of the (composite) hypothesis is not larger than the infimum of (3. 1) with respect to u. A u minimizing Ioot°+ I11u°+2loltu is given by uo=-(101/111)t, and it follows that O( Loot I,luo2+2loltuo)=1,/j* t,

where I *=loo-(Io1E/I11). n

1

In order to obtain the expansion of E Z; n up to the order of n- z we put a Jooo=Ee..e.

a6'

log f(X+, 60, o)a6log f(X+, 60, o)}

z J001 =Ee.. e.{a6z log f (X 6o, Uo) , log f (X f, Bo, ^o)} z

Jo1o =Ee.. e.

U

aea log f(X 1, 6o, p log f (X

eo, 6o)}

E

Jo11=Ee.. e• aea^ log f(X 0, 6o, eo)0 log f(X,, Go, eo)} J1o =Ee.. e.

a

log f (Xi, 6o, Up log f (X 6o, U}

a J11=Ee.. e.

U 8 log f (X 1, 6o, eo) log f (X B0, o)}

;

12

February, 1976 On the Second Order Asymptotic Efficiency of Estimators in Multiparameter Cases 265

Kooo= Ee,.E,

K 001 = E0, . E, K011 = Ee,.

{a6 L {a6

3

log f(X,, 60, o)} ;

log

E'L^86 log

K 111= Ee,. E,

{bl_^

z

f

(X r, eo, o)}

{

log f (Xi, B0, 0)11

f (X r, 60, ^o)) {a log f (Xi, 60, o)} J 3

log f(Xi, 60, U}

Under suitable regularity conditions the following hold : s Es,. E, a8s log f (Xi, 60, eo) = -3Jooo -Kooo ; s

Ee,. E• a62 log f (Xi, 60, eo)a^ log f (Xi, 60, 60) _ - Joo1- 2Jolo -Koo, Ee,. E,

log

{a

f

E

(X r, 60, ^o)a$ log

f(X

+, Bo, co) = -Juo -2J011-Kou ; I

s

Ee,. E, a^s log f (Xi, 00, eo) = - 3J111-K111. n 1 If B=6o and 6=^o, then the asymptotic mean Ito of Zr„ up to the order of n _2 is given 1=1

as follows : ,1o= 2 (Ioot2+I11u2+21oltu) 1

+ Z

n Z {Jooot2+(Joo1+ 2J olo )t2u+(J11o+2Jo11)tu2+ J lllu3} 1

+

6

_

1

n 2 (Kooot3+3Kooit2u+3Kollt2u +K111u3) +o(n Z).

When u= - (Io 1 /Il i)t, Ito is written as the following form : Ito= 2 I*t2+ 6 n 2 (3J*+K*)ts+o(n 2). n

Similarly the asymptotic variance ao2 and the asymptotic third moment ro of Z;,, are given i=1 _ 1 ao2=l*t2+J*t3n

1 _

2+o(n

2);

9

1 _

1 _

ro=E9,.E,( Zi„-Ito) _K *t3n Z+ o(n z).

If 8=61 and then if follows that logf(Xi, 61, 6) f(X r, 60, eo)

=1+a8 log f (Xi, 60, eo)(tn- 2)+a,- log f (Xi, 6o, ^o)(un z ) +

2

{a 2 2

+28

log f (X r, 60 ,

as

^o)(t2n -1)+

log f (X r, 60,

eo)(u2n -1)

log f (Xi, 60, ^o)( tun 1)

+ 2 {00 log

f (X r, 60, So)} 2t2n

-1+ {a^

log f (X r, 60, eo)}Zu2n-1

13 2ss

Masfumi AKAHIRA and Kei TAKEUCHI

+2j O log f (X,•, 9o, Eo)}lad log f (X r, 9a, ^o)}(tun 1 )^+o(n-1). Hence we have /t1 = /to + ( Ioot2+Iliu2 + 21oitu) I

-n T {Jooot3+(Jooi+2Joio)t2u+(J11o+2Jo11 )tu2+Junu3} I - 2 n T (Kooot3+3Koojt2u+3Ko11tu2+K111u3). If u= -(Ioj/1oo)t, then it follows that

-1 1 1 /a1= - Z I* t2- 6 n T(3J*+2K*)t3+o(n T).

Similarly we obtain _ I a12= ao2 +n

I

z(Kooot3+3Kooit2u+3Koutu2+K111u3)+o(n 2)

1 1 _ =I *t2+n _2(J*+K*)t3+ o(n Z) ; s I I 1 r1= Ee,.e,(Ztn-,u11 =ro+o(n T) = -K *n Tt3+o(n T).

/

If we choose a c such that Pe,.e,(n) ^XZ1n>c 1 +o(n 2), -I 11 2

then c(=ci) is given by * 1 * c1=- 2* t2-3J* 62K n-Tt3+6 *n-1.

Hence we have I'e..f. EZin > c1

t

{

-z ) (3.2) tl^ 37 *+ 2K*t2n_2 +o (n

6VI

where (P (x)0(u)du with O(u)=-Zee2. The bound of the power functions of the problem of testing hypothesis is obtained by (3. 2), that is, Ho+(t, Bo, 6o )=O(t ✓I**)

(3.3)

H1+(t, 60, ^o)=O(t)(3J6 I *)t2

(3.4)

By a similar way as the case t> O it is shown that in the case t
The bound for the distributions of second order asymptotically

median unbiased estimators is given by

Go(t, Bo, Eo)=(P(tIII*) ; G1(t, 60, Eo)=o(t)(3J6 *+2K* )t2, where I*, J* and K* are defined in the context. Let 9(={&„}) and e(= {fin}) be maximum likelihood estimators of 0 and 6, respectively Since

Eaelogf(Xi,e, )=0;

14 February, 1976 On the Second Order Asymptotic Efficiency of Estimators in Multiparameter Cases 267

iE -log f(X1, 0,1)=0, expanding them in the neighborhood of B = Oo and _ ^o respectively , we have log f (Xi, Bo, o) EB +{^E aB2log f(Xi, B0, X0)(9-0o)+fE f (Xi,

1 { i=1ae E a3g log +2E

aBa-

log f(X+, Go, co)(e-eo)}

ag

log f (Xi, 9o, eo)(^ -^0)2 X0)(9 -0o)2+ E =16602

eo,

as

i=10B2a^

log f (Xi, 90, U(0-0 04-6) }

-0; no iE

log f (Xi, 9o, ^o) a3

+ E log i=1 800^

+

{i^ 2

+2E

aB2a a3

i=100 2

f(X,,

a.,

E aaglog f(X,, 90, 60)(x o) ^o)(9 -0o)+ j=1 I

log f(Xi, 90, ^o )(0-Oo)a+ E ag log (X 90, ^o)(^ -^o)2 s=1a^

log f(Xi, B0,

U( 9- 80)( - 0)}

-0. Putting 1 n a Zo=n_2 E - log f (Xi, Bo, U; i =108 1 n a

Zi=n 2 log f(Xi, Go,co); s Zoo=n 2 E { logf(Xi, Bo, o+Ioo} ; i=1 aB f(xi, 0, 0)+I1II Zit=n ZEfj1og a i=1

Z oi=n

2

^

Z, {1og f(xi, B0, ^0)+Iol},

i=1

000

we see that they are asymptotically normal with mean 0. Since O=Zo+(-Ioo+n

1 1 1 _1 Z Zoo)n2(9-9o)+(-Ioi+n _ZZoi)n2(cf-^o)

+ (-3Jooo-Kooo)n2(0 -0o)2+ 2 (-Jiro-2Jo11-Koii)n2( -eo)z +(-Joni-2Jolo-Kooi)n2(9-Oo)(e-^o)+o(n 2); 0=Zi+(-Ioi+n 2Zoi) n2(9-Bo )+(-Iii+n 2Zii)n2(E-e0) + 2 (-Jooi-2J010-Kool )n2(0-0o)2+ 2 (-3Jiii-Kni)n2(e-^0)2 +(-Jiio-2Joni-Koii )n2 (9-Oo)(E-^o)+o(n 2 ),

it follows that

15 268

Masfumi AKAHIRA and Kei TAKEUCHI

(loolii - Io12)n2(6 - 8o) =131Zo-Jo1Z1+n T(111Zoo-Io1Zo1)nT(8-Bo)+n 2(I11Zo1-IoiZii)n2 1 -1

• -Eo)- 2 n T(3Jooolii-Jootlol-2Joiolo1+Kooolll-Koollol) • {n2(B-Bo)}2- 2 n 2(Jiiolii+2Jo11111-3J1111o1+Kou111-K1111o1) 1

1

• {n2(e-^o)}2-n _T(JooiIii-Jiiolo1+2Jo1o111-2Jo111o1+Koo1111 -Ko111o1) {n a (9-Oo)(E-^o)} + o(n 2 ) Putting D=Ioo111-Io12, a=-Iol/Ill, b=-Iol/loo, we have nT(8-8) _ J (Zo+aZ,)+I2n T(Zoo+aZol)(Zo+aZi)+-j _n T(Zoi+aZ11)(Z1+bZo)

1n T(3Jooo+Jola+2Joloa+Kooo+Koola)(Zo+aZ1)2 _ 1111002 11I oo2n-

2D 2 -1112

2(

Jllo+

2 Jo11+ 3 Jiiia+ Koii+ K llla)(Zi+bZo) 2

goon-i(Joo1 +Jiioa+2Jo1o +2Jolla+Koo1 +Kooia)

D

• (Zo+aZi)(ZI+bZo)+o(n- Z ).

Since Ee..E,(Zo2)=1oo, Ee..E.(Z12)=111, Ee,.E.(ZoZJ)=1o1, Ee..E.(ZooZo)=Jooo, Ee.,E.(ZooZ1)=Jool, Ee..E,(Zo1Zo)=Jo1o, Ee,.E,(Zo1Z1)=Jo11, E9,.E,(ZIIZo)=J11o, Ee,.E,(Z11Z1)=Jill,

if follows that 1 J*+K* - -1 Ee,.E.{n2(0-0o)}=- 21*2 n 2+o(n 2), Ve,.E.{n2 (9-8o)} =I*+o(n 2), 2 EB.. E, {n 1 (0-80)} 2= -3J*I 3 K*n- i +,(n -1 -2').

If we define 9* by B*=96n1* then H* is second order AMU and second order asymptotically efficient at O = 8o and ^ _ ^o. Since K* and I* depend on Bo and 6o, we may change 9* into the following form 0**: 9**=B_ K*(8, ^) 6nI9, j) Thus we have Theorem 2.

B** is second order AMU and second order asymptotically efficient.

For any consistent estimator {8n°} if the asymptotic variance of {0n°} is given by

16 February, 1976 On the Second Order Asymptotic Efficiency of Estimators in Multiparameter Cases 269 Ve.e{n2(B„°-B)}=i +o(n z), then we can transform {0n°} into {Bn°#} such that the latter is second order AMU. In a similar way as [3], if there exists the second order asymptotic distribution, it is shown that {9n°#} must be second order asymptotic efficient. We assume that {Bn°} has a second order asymptotic distribution and _ 1 1 _ 1 EB. e {n2(Bn°-6)} =c1(0, ^)n 2 +o(n 2) ; Ve.e{n2(On°-6)} =I*(6, )+o(n z) E°.e{n2(e n°-6)}3=cs (6, s^)n 2+o(n 2).

Let 0 0(= {en °}) and

^ °(= Ono

{^n

°}) be consistent estimators of 0 and ^, respectively. Putting

*=Bno-n

_1

1 C ,( 6 0, ^°)- W11* (0 0, S°)C3(6 °, ^°)},

we see that {&n°*} is second order asymptotically efficient. References [ 1 ] Akahira, M., "A note on the second order asymptotic efficiency of estimators in an autoregessive process ," Rep. Univ. Electro-Comm. 26-1, (1975). [ 2 ] Pfanzagl , J., "On asymptotically complete classes ," Proceedings of the Summer Research Institute on Statistical Inference for Stochastic Processes Vol. 2, (1975).

[ 3 ] Takeuchi K. and Akahira, M., "On the second order asymptotic efficiencies of estimators," to appear in Proceedings of the Third USSR-Japan Symposium on Probability Theory. Lecture Notes in Mathematics. Springer Verlag. [ 4 ] Takeuchi, K., "T6kei-teki suitei no Zenkinriron (Asymptotic Theory of Statistical Estimation)," (in Japanese), Kyoiku-Shuppan, Tokyo, (1974).

17 Rep. Univ. Electro-Comm. 27-1, (Sci. & Tech. Sect.), pp. 95-115 August, 1976

On Gram-Charlier-Edgeworth Type Expansion of the Sums of Random Variables (1)* Kei TAKEUCHI** and Masafumi AKAHIRA***

Abstract Gram-Charlier-Edgeworth type expansion of the sum of i. i. d. random variables without higher order moments are given. Exact formulas for the asymptotic expansion of the density for special cases are given and it is shown that the results are generalized to the case when the density function is approximated by a rational function.

1. Introduction Gram-Charlier-Edgeworth ( G-C-E) expansion is applied to the sum of independently and identically distributed ( i. i. d.) random variables which satisfy some regularity conditions. More precisely suppose that X1, X2, • • •, X,,, • • • are i. i. d. random variables whose characteristic function 0(t) (=E(exp it X;)) satisfies the following :

(i) q5(t) is k-times continuously differentiable at t=0, which implies that moments of X; up to the k-th order exist. (ii) For any p> O

sup I O(t) I <1.

Itl>p (iii) For some m>0,

S cb (t) i mdt<

oo.

Then it can be shown in [4] that the usual Gram-Charlier-Edgeworth expansion ([3]) is ap-

plicable both to the density and the distribution functipns of TX;. 1 The above regularity conditions are not the widest possible [1] but similar necessary conditions exclude various cases which are not altogether trivial. For example the condition (ii) excludes discrete distributions, and (i) does the distributions which have infinite moments. The purpose of this paper is to show that expansions similar to the G-C-E expansion can be obtained for cases where the above regularity conditions do not hold true.

The case when Xi's are integer-valued and have moments was discussed by one of the authers in [4] where it was shown that * Received on April 27, 1976

The results of this paper have first been presented at the Third USSR-Japan Symposium on Probability Theory at Tashkent, August 1975. ** University of Tokyo *** Statistical Laboratory, University of Electro-Communications

18 96 Kei TAKEUCHI and Masafumi AKAHIRA

n

Pr{
z h5(z)-hi(z) +o 24n 2n 24vzn (--) hz(z)+hs(z)+L =O(z)-cP(z) 1 6 V3 -n where p=E(Xt), az=V(Xi), flz=E(Xi-p)3/a3, ,84= {E(Xi-ft)4/v4} -3 and P(z)=3: cp(u)du ith ip(u)=(1l

w { d' e-(z 'iz)

}.

2j)e-<" 2

and hi(z) is the Hermite 's polynomials i.e. hi (z)=(-1)' ez.'zx

Further expansions for this case was also given in [4].

In this paper we mainly deal with the case when moments do not exist. If the second order moments exist , but higher order ones do not, then we may obtain the expansion of the type

Pr{ 1

3(z)+... ng 1/ n a Cz)=O(z)+U n

but gs(z), ... are no longer equal to the derivatives of tp(z).

When even the second order

moments do not exist, but the distribution of X, belongs to the domain of attraction of the stable law of the exponent a, then we may get an expansion n

EXi-bn Pr{ I
2. Continuous cases We first note that if the characteristic function of the distribution is integrable, then it is absolutely continuous, and the density function is obtained by the Fourier transform of the characteristic function.

Example 2 .1. t-distributions with odd degrees of freedom : suppose that Xi's are i. i. d. random variables having the continuous distribution with the density f(x)=-

1

B(2 , p- 2) (1+x2)' where P =l, 2, • • • . This distribution does not have the moments of higher order than (2p-2)-th order. We shall obtain the characteristic function O,(t) of the distribution.

Put lv(•x)=2^c

Since

itl v-ie-iti-u=dt.

19 August, 1976 On Gram-Charlier-Edgeworth Type Expansion of the Sums

lP(x)=2n Jo tP-ie =ZZ,j'(P){(1

1t(l+1 p

- (i+") xdt+ 1 tP- 1e (1-i1)Xdt

+i x)P +(1-ix)P}

(2. 1)

(1-1C2x2 +1C4x4-•••)

where ,,C1 ( i = 2, 4, • • •) afire binomial coefficients , it follows that ./ P(x)=ap, il1 (x)+a,. 212 ( x)+... +ap, P IP(x),

where for each i =1, • • •, p, ap, i is some constant. Hence we have

OP(t)= ( a ,, i+aP, 2 ItI+ ... + a p. PIt I P - 1 ) e- M. Since OP(0) =1, it is seen that a,,1=1. From above we obtain

1090P(t)=- ItI+log(1 +ap,2 Itl +... +a P,PIt IP-'). (I) p>2. Since EP(x)=Sxfp(x)dx=0 (p>2), expanding log O,(t) we have log 'YP(t)=- CP.2t2+24 CP, 4t4+...

CP

+6

. 3 ItI3 .3 ItI3

1 ,5ItI5+...

2

+11 +1 CP

where for each i = 2, 3, • • •, c,, t is some constant. Since there exist the moments of Xi up to (2p-2)-th order, it follows that

,5=...= CP•3=CP

CP.2P-3=0.

Letting ^„ (t)=E[exp(itZ„ )] with Z „=E (Xt-lt)/(1/ n o), we obtain log^.(t)=n log ^P( no) 76 0 2+24 n + 720n2+••• 5

5

+6 n1/2 + 120n312 ... =e-

(012)Il+

r3

6n1t2

i'+ + 144n"2 l t

It1

3+

,4

t4+

L32

t5 +

24n 72n s

1296n5/2 I

t I9 +...

where for each i = 3, 4, • • •, 7t is some constant. For each odd number r we define gr(x) and Gr(x) as follows : g,-(x)

1 ( It I re-(t - 12)e-itmdt. 27C

Gr(x)=c gr(u)du.

Then the density function f„ (z) is given by

75 120n312

I t 15

20 Kei TAKEUCHI and Masafumi AKAHIRA

98

a

n(z)= 1 e-(:-/z)11+24nHe(z)+72nH6(z)+... 7374 +144n312

73 3 + 1296 ns/ag9(x) + ... Further the distribution function of Z. is obtained as follows : z F.(z)=O(z)-tP(z) 24nHa(z)+7 32 H5(z)+...

+ 673/-G3 (z) + 120n3/2Gs(z) + 144n3 /2G7(z)

where Gi(z)=S: g;(x)dx. Since the distribution of Z. is symmetric, it follows that Gl(0)=0. By (4. 4) of Appendix of section 4 we have Gt(x)= e-(x=12)et21 2 x o Hence it is clear that gi(x) is also calculated. Since I t se-(2 /z)-uxdt g3(x)= 1 2icJ_. 1 tz

2Z _.

t I e- (t= /z)-t txdt

z = (-1)dxagl(x) dx3G1(x), it follows from (2. 2) that g3(x) is calculated.

In general we have for each odd number r gr(x)=(-1)(r-1)

/zdxrG

l(x) ;

1

Gr(x)=(- 1)(r- 1)/zd r- G t ( x ) . Futher since

d Gi(

x)=g1(x)= 1 -xGi(x),

repeating differentials we obtain d dxaGl(x) _ -Gi(x)-xgi(x) =(x2-1)Gi(x)- ;

dt.

(2.

2)

21

August, 1976 On Gram-Charlier-Edgeworth Type Expansion of the Sums

99

3

d 3G1 (x)= 2xGi(x)+(x2-1)g=(x)-1 _ - (x'- 3x)Gl(x)+!(X2 -2); 4

d GI(x)-(x'-6x2+3)Gl(x)- IW-5x);

Hence we generally have for each odd number r T

- T 1(x)=(-1)rHr(x)Gi(x)- 4-Kr-i(x), where Kr -i is the polynomial of (r-1)-th degree. For large x, Gi(x) may be expanded as follows : e-<=*iz>ecx-0-i2 du o

Gi(x)=

= 1 e W/2)-ma du It o 2

4

_ (1+ 2 +

8

6 +... ) +48+... , du

(_L+

+...

it For example we have G3(x)-(xs-3x)Gi(x)+ (x2-2)

it (x2+x4+...t. In general Gi(x) and gi(x) have for large x the order of x-r" and that of x-r, respectively.

(II) p=2.

In this case if follows that f2(x)= 2 X(l+xz)z i(x)= 1 7t(1+x2) 1 l2(x) = It 1+x2)2 ( 1- x2). (

Since f 2(x)=11(x )+ 12(x), the

characteristic function is 02(t)=(l + I t j )e-M.

In a similar way as the case when p>2, the characteristic function ^„ (t) of Z. is given by /

an(t)=e-u'/z)(1+

8

4

6

3I1^ n 4n+18n+...

Hence the distribution function Fn(z) of Z. is expanded as follows : Fn(z)-4(z)+3 nG3(z)-co(z ){4nHs(z )- 18nHs(z) I ...j.

(III) p=1.

Since f i(x) is the Cauchy density function , it is easily seen that the distribu-

tion of Z. is also the Cauchy distribution. Example 2.2. (Non-symmetric case) Suppose that Xi's are i. i. d. random variables with the density function f v(x) as follows :

22 100 Kei TAKEUCHI and Masafumi AKAHIRA

P-1 x>0; fP(x)= (1+x)P'

0 , x<0, where p>1. We shall first obtain the characteristic function Op(t) (=E(expitX;)). Since

OP(t)= (p-1)ei, dx, (2.3) o (1+x)P we have for p>2. OP1(t)=

dtO P(t)

_ (p-1)ixe`txdx

J (1+x)P p_21oP- 1(t)-iov(t)• (2.4) On the other hand applying integration by parts to the right- hand side of (2.3) we obtain

(2.5)

0P(t)=1+pit20P-1(t) Substituting (2.5) into (2.4) we have

(2.6)

OP'(t)=(pt 11-i)OP(t)-pt In order to solve the differential equation (2.6) we put gp'(t)=(L-!_ i)(t). If t>0, then it follows that g,(t)=Cite-le-rt

(2.7)

'P(t)=O (t)g,( 0

(2.8)

where Ci is integration constant. Putting we have 'V P'(t

) _ ^P( t )g P'(t ),/+ ^P'(t)g P(t)

=( pt 1

-i)YP(t) +^P'( t )gP(t)

Then it follows that (/1 P'(t

)g P (t) =

-(p- 1)/t.

Hence it is easily seen from (2. 7) that

0P (t)=-(p-1)Ci- 1t- Pert. Considering that

0p(0) =1, we have from (2.7), (2.8)

(2.9)

and (2.9)

` OP(t)=(p-1)(^u-Per°du+C2)tP-Ie-rt r

for t>0

(2.10)

where C2 is integration constant. It is seen from (2.5) that the solution (2. 10) of (2.6) is also applicable to the case when 1
23 August, 1976 On Gram-Charlier-Edgeworth Type Expansion of the Sums

101

Since ttt1 -11 elM

0p(t )=

c°° (p-1)e'tcx+u tom o t °(l+x)° tdx

= lim r(P- 1)e`vdy 11 P

t-roo t

=0 it follows that C2=0. Hence we have for t > 0 ^(t) _ (P-1)(S°°u-Pe'"du)tp- 1e-it. r

(2. 11)

Next we consider the case when t<0. Since 0p(t) is the conjugate complex number of 0,(- t),

it follows that for every t<0 I Op(t)=(P-1)(Sltlu-Pe - iudu) t I P-le-it

(2. 12)

If t is a small and positive number, then Op(t) may be expanded as follows.

Put Ap +iBp=

S

1°°u-

Pe'udu.

Since U2 (iU)P-1 1 u ° e"`-1-iu + 2 +...JS (for integer p) (P-1)! is bounded for 0
r

UE

(iU)P-11

Cp+iD,=Sou- P e'"-1-iu + 2 -}-••.- 1 du (for integer p)

we have Sou Pe'"du =Ap+iBp+ S u-pe'udu r r 1 =Ap+iBp+S u-P er "- l-iu-...- (iu)p-1 du

r (p-1)! J

°_1 1 zu u-P(1+iu +...+ ( ) Jdu t (p-1)! p 1 =ap+ipp- Ku-Pfer"-1 - iu-...-((pu ) 1)! }du

+cr u- p l+iu +•••+ p{p!

(1 Zr)P 1

1

(P -1)!

= «n +i^p-Sou

du

(p+1)! ((p+2)! +...}du

+ u-P+iu1- °+...+ zp 1u 1 du Sr { (p- l)! }

for integer p, where ap=Ap+Cp and ,9p=Bp+Dp. If p(> 1) is integer, then

(2. 13)

24 102

Kei TAKEUCHI and Masafumi AKAHIRA

u

pe`udu = ap+ip,-

Jr

Jo

u-

p{(Z

U )p+

+1

(Z1

+ ( ZU )p +2

p! (p+ _}..2)! i)! (p

+.„}du

P-1 ik 1

+E

-S

k=Ok! r

u k -Pdu

ip t- ip+1 t2+ iv t3+ =av+i^p- -

p! 2(p +1)! 3(p+2)!

p +

- 2 zk

log t

1 ( 1-tk-P+I)- iP-1

k=0 k! k-p+ / (p-1)!

_

P-2 ik 1

iP-1

ip

+1 =(ap+ap' )+i(QP+RP')-kEk!k-p+1tk (p-1)! logt-pit

ip+l

3 2 2(p+l)!t + 3(p+2)!t +...

where ap' and 9p' denote the real part and the imaginary part of p-2 ik 1

= k! k-p+1 respectively. Putting ap*=ap+app'' and ,p*=/p+Rp' we obtain

Jr u

pe'udu=ap*+ifap*

p-2 Zk 1 E

k-ok! k-p+1

_

ip-1 iP tk-P+1_ -t log t-

(p-1)! p!

2

3

2(p+1)!t +3(p+2)!t +... From (2. 11) we have for small t>O OP(t)=(p-1)(, U-"e-'"du)tP-le-tt r /f p-2 iv-1 iP = (p- 1)f 0 Zkaktk-(p-1)I tP-11og I t I +ap*t P-1+iRv#t P- 1- *'tP

+0(1 t I P) }(1 - tt- 2 t2+ 6t9+...) =

/, 11-2 + ..h -l) k Zkaktk-( O

(C

ZP-1

p-2)!

ip

(p

tP

-(p -1) EOik+laktk+l+(p

M

- i(p- 1)a p *tP +(p- 1)Qv *tP+ 6 (p-1) E ik

+

1aktk

..//,^

n

P -l logl tI +(C- 1)ap * tP-1 + i( p- W P* -1

2

z 2)! tp

log I t

(P - 1) F, ika*tk+2

+s+...+ao(^ 21M tv

+o(I t I P)

(2.14)

For t
=a2+iP2 -Sru-21(Z1)2+ (26)2+...}du+

25 August, 1976 On Gram-Charlier-Edgeworth Type Expansion of the Sums 103

+t-1-ilogt =az-1+i/32+ t -i log t+ t + Similarly we have u- 2e"`du= a 2- I+i(32+ 1 +i

iti

I

t

logItI +ItI+....

I

2

From (2. 11) and (2. 12) we obtain

02(t)=1 - it log1 tI + (as -1)ItI +(/32-1)it

+t2logI tI +13 2t2- i(a2 -1 ) tIt I +o(t2). Since a2+i/2 = 5u 2(e'n - 1-iu)du +^u-2eiudu o i

it follows that az=S0u-2(cos u-1)du+cu-z cos udu ; ri (3z= 0u- 2(sin u -u)du+Tu-z sin udu.

J

We can calculate a2 as follows : a2=f'u-2(cos u-1)du+1 0 - sin u du+1 0 o U

= -u-' (cos u-1)

=1-2 Since the value of /32 is not easily obtained, putting /z=r+1 we have

02(t)=1- i tlogI tI -

2 I tI +irt+t2logI tI

+(r+1)t2+i 2 tI t I +o(t2) Letting 0n ( t)=E[exp(i tZn)] with Z„=(EX;-an)' bn we obtain log 0n( t)=n log 02(bn) -ibn

=-ibnt-ibnt(logl tI -logbn)-2b-I tj+ibrt z

z

+bn2 (log I t I -log bn)+(r+1)b -j+=2bn2t I t I -

n t2

2bn2(log I t I - log bn)2 +sbn2(1G2-472)

-Z2b„zt!t1 +o(bn2). If bn= n and an =nr+n log n, then

(2. 15)

26 104

Kei TAKEUCHI and Masafumi AKAHIRA

log an(t)=-it logltl - 2 ItI -lognt2+ 1 t2 log ltl + r+lt2 n n n z

+i Znt

I t I- (lo2n) t2+ on nn t2 log I t

t2(log

Tn

I t I )2+ 8n(Z2-4, 2)t2 -i2n t I t I

+o (1). n

function with the characteristic function expj - 2 It I -it log I t I }. This is the spacial case of a stable law with the characteristic exponent 1([1], [2]). By the expansion of the asymptotic distribution, it follows that the order of the next term is (log n)2/n. Next we consider the case when p=3. ` We calculate a3 and 8s as follows : a3=`ou 3jcosu-1+ 22Idu+ u scosudu

2z(cosu-1+ 2 )

- 2 cos u

I I 2 u-

1- 1

lu-2(sin u-u)du

J 020

2 sin u du

1 j9z- - 7 1 . 4 2 2 4' Qs Sou - 3(sin u -u)du+ U- 3 sin u du I 1 i

z 2

(sin u-u)

o+ 2

u-2 2 sinU ^+2

S

S

ou'2

ou 3(cos u -1)du

cosudu

=2+2=1-4. From (2. 13) we have for small t>0 21u-3ei°du=-r-

z +i(2-" -2cu 3(e"`-1-iu+ 2 du

uz +22tu-3(1+iu- 2 )du =-r- 2+2-2-3 -(1-t1-2i(1-t + log t Hence we have from (2. 11) and (2. 12)

+o(t).

I

27

August, 1976 On Gram-Charlier-Edgeworth Type Expansion of the Sums 105 a

s

03(t)= tE1 -it- 2 +it + X xlt +t:+logIt^-r-2-i2 ^tI-3t+o(ItI)] =1+it+tz log I t I -rtz-i 2 t I t I +o(ta).

(2. 16)

Letting ^„(t)=E[exp(itZ„)] with Z„=^(X;-1)/ ✓nlogn we olttain

log (t)= -4 log nt + n logos(Vn -log n) E

1 gn(logitj-log g no n)-log nt2 _i irt1tI - 1 tz+o 2logn 2logn (FO91 )

1 +loglogn+2r+1 tz logn lognl l

+ log n tz log

It - t

2log

nt t + o (

log n/'

Putting c„z=1+loglog+2r+1 log n log n

we have n(t)=(1+ logn(tzlogjtI - i 2tItI)}exp( ct2t2)

1

l 1 +o (log n

(2. 17)

It is clear that the first term of the right-hand side of (2. 17) corresponds to the normal distribution. In order to obtain the density function of Z. corresponding to the right-hand side

of (2. 17) we need the Fourier transforms of (t2 log I t I )e- /2) and it l i le- ('1/z) will be done in section 4 in detail. Further we consider the case when p=4.

Then we calculate a4 and ^4 as follows : a4=

i1-

uz -4(cos u-1+ 2 )du+S1 u-4cos udu

=-U33(cosu-1+2)l0 3`ou-3(sin u-u)du

-u3scos uIW- 3Tu -5sinudu

The discussion

28 106 Kei TAKEUCHI and Masafumi AKAHIRA

A4

I / us = f ou-4I sin u-u+ 6 ^du+^I u-4 sin u du =-u33lsin u-u+ 6310+ 3Tu 3(cosu-1+

2)du

u-3 - 3 sin u I+ U- 3 cos u du 5 as

18+ 3 7_r 36 6 From (2. 13) we have for small t>0 3^u-4e" `du= 3(a4+ii4 )- 3ru-4 ( ,i-- 1 - iu+ 22+z63 du z o \

+3f u-4(1 + iu- Z -z63'du

= 3(aa +ifi4) -z8

-11- t

)-

3 i (1 - 72

+ Z (1- t,+ 2 logt+o(t). Hence it follows from (2. 11) and (2. 12) that t3 Zt3

04(t)=I1-Zt- - - + -- - + - .. 2 6 •I 1+ 6it - 2t2+ 2tIlogIt^+...1 22t3logItI+•••

(2. 18)

Letting ^ „ ( t)=E[exp(i t Z.)] with Z„=2I E X ; - 2 )'n3we obtain

n\ log 0.(t)=_ 3ynit+n log04

t VK

/ logn t2 _4logn . s z t +0 n 3n1V n . 2 3✓

Hence it follows that an(t)=e-"-/2) 1-4 log nit3+o( log n n ( 3n 3 11-

(2. 19)

Corresponding to the right-hand side of (2. 19) the density function of Z. is expanded as follows : 2S c22i2>^1- 3V3n (z3 -3z) }+oI log J. Example 2.3. (Symmetric case) J

Let X; (i = 1, 2, • • •) be i. i. d. random variables with the density function f p(x) as follows : fp(x)=

p- 1 2(1 +1xl)p'

_oo <x
where p> 1. In order to calculate the characteristic function Op(t) (=E(expitX;)), we have

(2. 20)

29

August, 1976 On Gram-Charlier-Edgeworth Type Expansion of the Sums

107

(p_1)' it ^v(t)= J_ 0o 2( 1 +I x I)pdx 1)eitxdx (p-1)et udx+T (pJ ao 2( 1-x)° o 2(1+x)° = f°° (p-1)e-itxdx+ ( p-l)eiexdx. T . 2(1+x)' o 2(1+x)p Since

c(p- 1)e_itxl {2(1+x)p}dx

is the conjugate complex number of

{2(1+x)p}dx, the characteristic function

To (p-

1)e'txl

0p(t) agrees with the real part of the functions (2. 11)

and (2 . 12) in the case of Example 2. 2. If p is integer, then it follows from (2 . 15), (2. 16) and (2 . 14) that for every t

¢2(t)=1- Z ItI-t2logItI +(r+1)t2+0(t2); 03(t) =1+ t2 logI tI-yt2_

(2.21)

Itl3 +0(ItI3)

(2.22)

Op(t)+ - 1 - p-1 +p-1 t2+o(t2), (p>4). (2.23) { 2 2(p-3) p-2 When p=2, letting On(t)=E[exp(itZn)] with Z„=EXrln we obtain log ¢„(t)=n log cb2 n ) =- 2ItI

` 2

+n(logltl- logn)+

2 r +1 n t,

z _8n t2+o(t2).

Hence it follows that On(t)=a-c "i2 t : ji-log nt2+ n t2logl tI +8(7 8n)-x2t2 +o(t2).

(2.

24)

The distribution function with the characteristic function e-("i2)1tt is the Cauchy distribution having the following density function f (x) : 2 f (x) =tt2+4 x2 . In order to obtain the density function of Zn corresponding to the right-hand side of (2. 24), we need the Fourier transforms of tee-("12)jt l and (t2 log I t I )e-(,/2)Jt!. The discussion will be done in section 4 in detail. n

When p= 3, putting Z„= log n (

i

Xtl

l^ n log n, we have

1 1

2

t)=log n

(logItl- log n -

z rt log n

_z

loglog n

3 t

n (log n)3/2 1 2V -

I +0

n

Hence it follows that 1

1

l

an(t)=a-t'1z 1+lognt2logl t I +2(logn)Zt°(log t)2+... J (2.25)

1

30 108 Kei TAKEUCHI and Masafumi AKAHIRA

Then we note that the expansion of the power of 1/log n is obtained. In section 3 we shall get the density function corresponding to the second term of the right-hand side of (2. 25). But it is difficult to obtain the density function corresponding to the third term of the righthand side of (2. 25).

The asymptotic expansions given as above are also applicable to the other distribution cases. Example 2 .4. Let Xi's (i =1, 2, • • •) be i. i. d. random variables with the density function f (x) as follows : .f W= 2(1+ X 2)3/2

g(x)=f(x)- 1 2(1+IxI)3 we have c1129(71)d1/= 2 log

( +x)- 21/1-Fx2

- 2 log(l+x)-(l+x-1)+{(1+x)2-1{ = 2 log {( +x)/(x+ 1)} - 2 (1+x)2' Hence is follows that

J x2g(x)dx= log 2. Since (x)dx= 2 ; ^_O^xg( x)dx=0,

it is seen that log

eitzg (x)dx=

2t2+0(t2)

2+

By Example 2.3 we obtain 1 T errsf (x)dx=

err: x )sdx+ _We'rsg(x)dx

2 (1+

= 1+ 2 t2 log j t j_ log

Putting Z„=E Xr/

^+

7t2+o(t2).

n logg we see that Z. is asymptotically normal and the next order of

the expansion is given by (log n)-'. Example 2.5. Let Ur (i =1, 2, • • •) be i. i. d. random variables with the uniform distribution on the open interval (0, 1).

Put X, Since

2 (1 Ur

1(

Ur) (1=1, 2, ...).

1 _1 2 1u u

x= -

31 August, 1976

On Gram-Charlier-Edgeworth Type Expansion of the Sums 109

it follows that 1 dx__1 1 du 2 {u2 +( 1-u)2 = 2 ru2(11 u)2 -u(12 u) ^, On the other hand since X

2 (1-2u)2 _ 1 _ 1 u(1-u)'

4u2(l-u)2 4u2(1-u)2

we have

2u(1 - u) -1. Hence dx=2(x2+l +Vx2+1 ). du Therefore the density function f (x) of X; is given by AX) = 1 2(x2+ 1 +Vx2+ 1)'

g(x) __ 2(1

1 +x2)(1+x2+V1+x2)

f(x)=2(1+

2(1+x2)3/2+g(x)•

x2)

f (x)dx=1; Sg(x)dx =1-

2

+l=2-

5

2

;

5x2g(x)dx= {f(x)-g(x)}dx=

Z -1.

It follows by Example 2. 4 that the characteristic function 0(t) of X; is given as follows : fi(t)-1+ 2 e-1°1 - c log It I

tz 2 2 -1 t2 +(t2)

logjtj+(7+ Z)t 2 +o(t2).

=1- 2 jtjZ

Then the asymptotic expansion of the distribution of

X;'n agrees essentially with the case

when the density function is 1/ {2( 1+ I x 1 2)z} The above results obtained may be generalized into more general case. Suppose that Xi's are i. i. d. random variables with a density f (x) such that (i) f(x)=O

for x
(ii) there exists a bounded rational function g(x) such that for some positive integer m Tx*{f(x)-g(x)}dx=MR , exist.

k=0, 1, •••, m

32 110 Kei TAKEUCHI and Masafumi AKAHIRA

Then the characteristic function of Xi is expressed as E(eitx

i)=Jo

et1xg (x)dx+ eiDF{f(x)-g(x)}dx

= (°°ertxg(x)dx 0 +jMo+M1it+•••+ k k(it )k}+o(ItV)• On the other hand g(x) may be decomposed as g(x)=E j kEI(xc'a,)k+E s^ {r;+( xjk Qi)2}s Since g (x) is bounded , ai<0 for all j, hence

c'ketLZ dx can be expanded around t=0, o (x-ai)k

as was done in Example 2. 2., and r°° d;k eits djk eitp; eitx dik eits Jo {7i+(x-(3i)2} sdx=^ (7i+x2)s dx+Jo {ri+(x-^3i)2}sdx

(2. 26)

Then the second term of (2. 26) is expanded as in Example 2. 1 and the third one is easily This establishes the expansion of E(e112) and n we can get the asymptotic expansion of the distribution of E Xi similarly as in Examples 2. 4 1

shown to be expanded into a polynomial of t.

and 2. 5. For more general cases if the density f (x) satisfies the assumption that f +(x) and f -(x) defined by f+(x)- 0 If (x)

for x<0; for x>0;

- _ 0 for x<0; f (x)-{f(-x) for x>0; satisfy the condition (ii), then the above arguments are applicable to f + and f - separately and we get a similar expansion.

3. Lattice cases Let Xi (i=1, 2, • • •) be non- negative integer -valued random variables with P r {X i = x }= Pk ( x )=

(k-1)! x =01 ... k>2 . (x+1)•••(x+k)'

Putting. Ok(B)= E 6XPk(x)

s-o

we have E Bx(k-1)!

dk

aek j0k (0 ) 0k } =(k -1) ! x=0 1-0 Then 4'k(0)

=6 -1

( 0-0 -1 0 1 -rf

dri + x(O),

where x(0) is some polynomial of (k-1 )th degree. Hence the characteristic function of Xi is given by Ok(eit) and

33 111

August, 1976 On Gram-Charlier-Edgeworth Type Expansion of the Sums a

1

x

Pr^E Xi=x = {(Pk(ei:)}"e-ia=dt. 2n'

For example we consider the case when k=2. Then 02(0) =8 -'

r

B _ 2dr7 +a9 +b

=(1-B) 0x(1-B)+ +a9+b. 2

(3. 1)

It follows from (3. 1) that a=b=0. Hence we have 0(t)=02

(eit)=e - 2it(1 - ei`)log ( 1-ei`)+e-n.

Since log(1-eit)=log(-it)+log{(1-e1 )/(-it)} =logItI-i 2itl+log(1+2+ i Z it +o(I tI =login -i2 tl+ it follows that 0(t)=-it {logItj-i

2 ^tI-

it)+ l-it+o(ItI)

=1-itlogltl-2 ItI-it+o(ItI). Putting z=(x-nlogn + n)/n, we have Its-itlogltl-itzldt. PrfExi=x}-^1 exp(i 2ntc 2

In a similar way as the continuous case we may be able to expand up to the higher orders. Similarly we may expand the probability of E Xi for any integer k. Also as was done 1 in section 2, we can expand the characteristic function if Pr {X i = x} = p( x) satisfies the following : (i) p(x)=O

for x
(ii) there are constants a2, a3, •••, am such that

=_o

xk{p(x)_a2p2 (x)-...-a,,,pm(x)} =Mk

for k = 0, 1, •••, I exist and is finite.

4. Appendix In order to get the density functions corresponding to the characteristic functions &W given in sections 2 and 3 we need the Fourier transforms of it i t l e-i'12, tee-(-/2)111 and t2(logItl)e-11 1.

(log I t I )e-D'iz,

For the purpose we shall calculate gi(x)

Hence we have

Tt(cos tx) e-d'i2dt (4. 1)

34 112 Kei TAKEUCHI and Masafumi AKAHIRA

G(x)=^ gi(t)dt= 1 (sin tx)e °''zdt. a Z o

Applying integration by parts to (4. 1) we obtain gi(x)= 1 (-cos tx) e-t'iz o

Tx(sin tx)e-d'Izdt

-xGi(x). Putting Gl(x) = u(x) e-x''z

(4.

2)

we have gi(x)=Glt (x) =u!(x)e - x''2-xu (

x)e-x'/z

=ut (x)e-x='z-xGi(x). Then it follows that (4.3)

U ( X ) 1 cet'izdt+C Ico where C is an integration constant.

If G,(0)= 0, it is seen from (4. 2) and (4.3) that C=O. From (4. 2) and (4. 3) we have Gl(x)= e-x'12 et2t2 dt. (4.4) Next we calculate th Fourier transform of it It I e-' 1/2.

By (4.4) we have

1 `itItle-(0/2)-ttxdt

2j d 1 (I t i e ct=iz^ ttx dt

d x 2^c _ -glt (x) dz dxzGi(x) - x -(xz-1)Gi(x).

(4. 5)

t I )e-t'iz and (t2 log I t I )e-t2/2. Further we shall calculate the Fourier transforms of (log I Put K(x)

2^ (

log I t I )e - (t'iz) -its dt ;

-ttxdt. K*(x)=2^ ( log t)e-W12)

Then it follows that K(x)=K*(x)+K*(-x).

The first and second derivatives of K*(x) are

given as follows : K*t (x)

3

2x o ( 1

K*t'(x)= _

27c o Applying integration by parts we have

t log t ) e-(t =iz) -ttx dt

(t2 log t)e-(1'12)-itxdt.

35

August, 1976 On Gram-Charlier-Edgeworth Type Expansion of the Sums

113

K*"(x)= -2 (1+ log t-ixt log t)e-(`'iz)-t12dt

1 e-(x=12) -K*(x)-xK*(x). 2 ✓ 2n Hence K*(x) is the solution of the second order differential equation 1 y"+xy'+y= - 2 cp(x)' where

cp(x)=

(4.6)

e`/2. 1 12tc

In order to solve (4. 6), we put y"+xy'+y=0• (4.7) Since y"=-(xy'+y)=-(xy)', it follows that y=cie-x ' i2+c2, where cl and Cl are integration constants.

Hence y=cp(x) is the solution of (4. 7).

Putting y=h(x)cp(x), we substitute it into

(4. 6). Then it follows that h"cp+2h'cp'+hcp"+xh'cp+xhcp'+hrp=cp. Since (p'=-xcp and cp"=(x2-1) cp, we have

If h"-xh'= O, then h'=ex'12.

Putting h '= g(x)exviz we obtain

g'e2"/2+gxe2 ' 12-gxex'12 =- 1 2' Since g' _ - 2 e` 12, it follows that g(x)= e-2"12dx+cl ;

h(x)=-

2S (Se

K*(x) - 2 co( x)

x-i2dx+ c)e2'i2dx +cz;

1% e-x212

dx+Cl) e"12dx+c21.

Since K(x) is an odd function , it is seen that K(x)=-(p(x)f S (ce--,12dx)ex'/ldx+C}, l00

(4.8)

where C=-K(0) 1 S(logItl)e-1"2dt. cp(0) 1/2^c Further it follows from (2. 1) that the Fourier transform of tee-("12)1`1 is given by z

(4x2+ tc2)g(1 22_2 . (4.9) Next we shall calculate the Fourier transform of t2(log I t j )e-''' i . Putting

h(x)=2^c J_oot2(log l H(x)

2ttJ_,

t I )e-i'x-1'1 dt .

o(log I t1 )e-«x-icidt

36 Kei TAKEUCHI and Masafumi AKAHIRA

114 we obtain

(4. 10)

h(x) _ - H" W, On the other hand putting H*(x)=2r( log t)e -' tx tdt, we have H(x)

2_c50 (log ( log

2-

t)e-1tx-tdt+2j t)e itx tdt +

.(log I t I )e-"'+adt

log I t I )e':x-tdt 2zc Jo (

=H*(x)+H*(-x). Since H`(x) 2wJo -(it log t)e-'tx-tdt, applying integration by parts we obtain H*' (X) 2ic(1+ix)Jo

(log t+l)e-X12-tdt

f °° o a-,tx tdt l+ix{H*(x)+2,cJ z 1 X-1 H*(x)+ 2ic(x- i)2

(4.11)

In order to solve (4. 11) putting H*(x) =.Lxi (x

we have *I_ _ L(x) L(x) H (x) (x-i)2+ x-i =-H*(x) FLU x-t x-i

Hence it follows that Lt(x)=27c(

1 i x-i) 2ic (1+ix)'

Since L(x)=-- log (1+ix)+C

=2^[ log ( 1+x2)+i tan -'x+C where C is an integration constant , it is seen that H*(x)=2tc(1+x2){ 2

log(1 +x2)-tan-'x+C} ;

(4. 12)

1 x2){- log(l+x2) -2x tan-'x+C}. H(x) = 2^c(1+

By the value of H(0) we may be able to decide that of C. From (4. 10) and (4. 12) h(x) is also obtained.

37 August, 1976 On Gram-Charlier-Edgeworth Type Expansion of the Sums

115

For example we shall get the density function corresponding to the right-hand side of (2. 17). Since from (4. 5) and (4. 8)

i t I t I e- (c•= /2)t =-itxdt = - 1 G1'' x

I 2ir 27

Cn3 Cn /

_oo(t3logI tI )e

(c.=/Zk=- =t=dt

= C .3

Kir

c n1 n !0 +4 the density function corresponding to (2. 17) is given as follows : J^

Cn nC13Kri( C, \nl+loge {C13GlrilC-n _ Cn/^+o\lo g n/

References [1] [2] [3] [4]

[5]

Gnedenko, B. V. and Kolmogorov, A. N., " Limit Distributions for Sums of Independent Random Variables," Addison-Wesley, Cambridge, Massachussets, 1954 (Translated from Russian). Ibragimov, I. A. and Linnik, Yu. V., "Independent and Stationary Sequences of Random Variables," Wolters-Noordhoff, the Netherland (1971) (Translated from Russian). Kendall, M. G. and Stuart, A., "The Advanced Theory of Statistics Volume 1" Charles Griffin, (1969). Takeuchi, K, " Kakuritsu-Bunpu no Kinji (Approximations of Probability Distributions)," (In Japanese), Kyoiku-Shuppan, Tokyo, (1975). Takeuchi, K. and Akahira, M., "On Gram-Charlier-Edgeworth type expansion of the sums of random variables (II)," Rep. Univ. Electro-Comm. 27-1, (1976).

38 Rep. Univ . Electro-Comm . 27-1, (Sci. & Tech. Sect. ), pp. 117- 123 August, 1976

On Gram-Charlier- Edgeworth Type Expansion of the Sums of Random Variables (H)* Kei TAKEUCHI** and Masafumi AKAHIRA***

Abstract Gram-Charlier-Edgeworth type expansions of the sums of independent random variables without higher order moments are given. It is shown that the asymptotic distribution functions for some cases are stable laws with fractional characteristic exponents, and the asymptotic expansion of the density function for one case is also given.

1. Introduction In the previous paper [4], we have obtained Gram-Charlier-Edgeworth (G.-C.-E.) type expansions of the sums of independent random variables with density functions which are approximated by rational functions. Here in this paper we shall obtain G.-C.-E. expansions for the case when the density function is of the type f(x)oc(1+x)-', x>0; =0, x<0 or f (x)oc( 1 -I- I X I)' which p is not an integer.

2. Non-symmetric case Let X1, X2, • • , X,,, • • • be i. i. d. random variables having the density f p(x) as follows : P-1 x>0; .f p(x)= (1+x)'' 0, x<0,

where p>1 and p is a non-integer. We shall first obtain the characteristic function Op(t) (=E(expitX;)).

By Example 2.2

of [4] we have r

(p- 1) u ' e "` du t '-' e - 'r , t>0;

(2. 1) t<0. u p e-rudu)ItIp-'e-`r, (p-1)( l I It

If t is a small and positive number, the

p(t) may be expanded as follows.

* received on June 9, 1976 The results of this paper have first been presented at the Third USSR-Japan Symposium on Probability Theory at Tashkent, August 1975.

** University of Tokyo *** Statistical Laboratory, University of Electro-Communications

39 11 8

Kei TAKEUCHI and Masafumi AKAHIRA

Put

(2.2)

Ap+iBp=u-pe'"du. I

Since z u-p e;"-1-iu+u +•••- (iu)CP] 2 Cp]! is bounded for 0
2 Cp]!

we have for non-integer

j du

(2.3)

P(> 1),

u-pe'"du =A,+iBp+c u-pe'"du r r

=Ap+iBp L

..}.. rru-pje '"- 1- iu- ... - (:u )Jdu J l Cp]• (iu)Ci ]l +fiu-p j1+iu +•••+ du

CP]•

=ap+ijp

° Cp]! i u- p j 1+iu -F•••+ (iu)Cf J ]l du +5 Cp7•

du

=ap +i^,

j (iu)[pJ +1 -

(iu)C/J+2

pl(Cp]+1)! +(Cp7+2)! +

}du --- ldu

J 1(U p}iul v+...+ i[pJuCp]-p1 +S ( du

Cp]!

1

= a p +i8p _ 1[P1+1 4 1 i[p]+2 °^(CP]+1)! u-P+Cp]+1+u-P+CP]+2+... }du

(Cp7+2) )

[p] ik r1 +

_J

uk-p

k=O k! r

=ap+ip, i1P1 +1

t- p+Cp] +2-

(Cp] +1)!(-p+Cp7+2)

j[p] +2

([p] +2)!(- p +[p] +3

) t- p+CP] +s

[P] ik 1 [P] ik tk-p+l

= k! k-p+1- E k! -p+l = ap +ap ' +i(18

[p] +8 ')-

z k t k-p+l

kk-o k! k-p ! 1

i[P]+1

(Cp7 + F)!( - p+ CpY+2) i[P]+2

t -P+lrJ+z

(Cp7 +2)!( - p +Cp7 + 3 ) t

p +1pJ +J_...

(2.4)

40 August, 1976 On Gram-Charlier-Edgeworth Type Expansion of the Sums

119

where app and Np' denote the real part and the imaginary part of [P] ik 1

k! k-p+l respectively, and ap=Ap+Cp and ip=B,+Dp. Put ap*=ap+ap' and Np*=Np+/p1.

From (2. 1) we have for small t>O

Op(t)= e- '(p-1)tp-1Ju-pe-`ndu e,

/

z

a t3+24+...

(p_1)ap*tP-1+i(p-1)Rp*tp-1- F (p-1)ikhk k=O k!(k- p+1) (p-1)i[Pl+1 t[P]+1

([p]+1)!(-p+[p]+2) (p- 1)i[p]+z t[p]+z-... } ([p]+2)!(-p+[p]+3)

(2.5)

For example we consider the case when 2O / z u-Pe'ndu=ap+iQp-^u-p (e"-1-iu+ 2 du •' +^tu p(1+iu- 2 Idu =ap+i/3p- Z t4-P- 1 (1-t1-p) 6(4-p) p-1

p 2(1- t2- p)_2(3 1

t3- a)

p)(1-

+o(t2).

From (2. 1) we have for small t>0 t2 it3

6 -...1

ap(t)= 1-it- 2

•f l+ p-2 it+(rp+i6p)tp-1+2 3-p)t2+...} 1

-1 +

p-Zit

+( rp+i8p )

1 )(3-p) t2+

tp 1+(p

...

2 where rp is some negative constant and 8p is some real constant. Hence it follows from (2. 1) that

¢ p(t) =1 +p 12 it + rp +iBpi t l I t I P -1 Letting

an(t)= E[exp ( itZn)]

+...

+(p 2)(3- p) t2

with Zn= n- '/(P -')E ^Xi -p

12/

we obtain log

a n( t )= ( rp

i5P -j_

) I t

I

P 1+ l(P-2)(3-C)

+

2

n- (3-010-1)t2

1

(2.6)

As immediately seen from above, the asymptotic distribution of Z. is a stable law with the characteristic exponent p-1 ([2], [3]) and the order of the next term is n-c3-p>i(p-1) (> n-1) Next we shall obtain the asymptotic expansion of the density function g(x : p-1) corresponding to On(t) given by (2. 6). From (2. 6) we have for 2
41 120 Kei TAKEUCHI and Masafumi AKAHIRA

c5

(t ) = exp {(rP +i8p

[

I t I)I t I v_i}]. ll+cpn-(

3-v)/(n-1)t2+ cp2n - 2(3-v)/(v-1 ) t4 _.1....

1 -1 (p-2)(3-p) 2.

where cp= p

It follows by Theorem 2. 4. 2 of [3] ( page 55) that for 2
`=expilrp+ihpl tl)I tI P-1}dt

o f- i)^+1

1'(n(p -1)+1) r 1 1 lx n(p-1) (2.7) sinl ^tcnjp -1-(3-p) n^ L rptanZ lc(p-1) J

holds as x-.oo. Hence for 2
1 °° s+1r(n (p-1)+1) 1 8p )-7tx

o f-1)

sin 27cnp-1-(3-p) s(p 1)

n^

rptan 2 (p-1)

holds as x-->oo. Next we shall calculateexplicitely the value off1 (2. 7) for 2 < p<3. We first have 2tcJ _

( +i6 , tlll tip-IJdt ^e itXeXpj(7,

=Zx (cos tx -i sintx) expj (rp + i8p tI)It I"Idt 1 cos tx) exp {(rp+ i8p)tp-1} dt 2tc3o ( +Zn^ .(cos tx)exp {(rp-i8p)(-t)p-1}dt =2x (cos tx) pexp {( r+i8p)tp-1} dt +2x, Jo (cos tx) exp {(rp-idp)tp-1}dt (2.8) Since for non-negative integer k and for a complex number a t2ke- atp-'dt

1 a-(2k + l)/(p-1) 1' 2k+ 1 p-1 ( P-1 ) ,

it follows that 1 cos 2^ o ( cos tx ) ex 2^cJo k ((2k)! ( 1 (-1)kx2k 27c _ (2k)!

tx)2kexp {(rp+iap)tp-1)dt

Jo

t2kexp{(rp+i8p)tp-'}dt

= 1 (-1)k x2k 1 (<2k +u/(p -1)r (2k+1 rp-i8p) 2n k=o (2k)! p p-1 1 r(2k+i

2z(p -1) k=o

I'(2k+1)

(-rp-i8p )-( 2k+1) /(p - 1)x2k.

(2.9)

42 August, 1976 On Gram-Charlier-Edgeworth Type Expansion of the Sums

121

Putting a,,,,te'bp'k_(-r,- z8,)-( +1)/(P-1)

we have from ( 2. 8) and (2. 9°)°' /I tl (rP+ z8, I l it I a-i}d t -1-r e-itx exp { 1 r (2k+1 /

x(p

-1)

E (- 1)k r(2k+ 1) a,, *(cos bp, ^)xzx

(2. 10)

for 2
r/2k+i 1

1 9p(x)= x

( --1) kE (- 1)k r(2k +

1)a,.^(cosb, k)xzk. (2.11)

The result (2. 10) is essentially same as that of Feller [1], page 549.

3. Symmetric case Let X, (i = 1, 2, • • •) be i. i. d. random variables with the density function f ,(x) as follows : .fv(x)= p-1 2(1+ IxJ)a where p>1 and p is a non-integer.

oo<x

In order to calculate the characteristic function O p(t) (_ (exp i tXi)), we have (p-1)eitx Op(t)= E2(1+Ixl)pdx (p-1)ei`xdx+ (P-1)euxdx J 2(1-x) p o 2(1+x)' =r°° (p-1)e-it.=dx+f°°(p-l)eit:dx

Jo 2(1+x)' Since

TO(p-

0 2(1+x)'

1)ei=x'{2(l+x)'}dx is the conjugate complex number of

TO (p-

1)eit x'{2(1+x)'}

• dx, the characteristic function O,(t) agrees with the real part of the characteristic function (2. 1) in the case of section 2.

If p is non-integer, then it follows from (2. 5) that for every t O P( t

)={

1+(p- 1)ap * ItI'-' +A,tz +o(tz) for 1
1+Aptz+(p-1) ap*JtJ -1+o(JtJ-') for 3
where A, = 1 (2-p)(p-3). Note that A,<0 for p>3. Since ap*=

a , +a,'=A , +C, +a,',

it follows from (2. 2) and (2. 3) that a,* is the real part of (°°u pet"du +Slu''jei"-1 -iu+ uz+ ...-(zu)Cp]ldu+CL i 1

i o tt 2 [p]! J k=o k! k-p+l'

Hence it is also seen that for 1
43

122 Kei TAKEUCHI and Masatumi AKAHIRA

n

If 1
(p-1)ap* I t I P-1 _(p 2

P-3)/(P-1 )t2 _(p 21) 2 ap*a

I t 12( -1) +o(n-1)

for 1
1)2 ap * ' I t I2n 1 ) + 2 pn(P -3)/(P 1 ) t2+o(n ( P-3)/(P 1 )) for 2
<3. Hence it follows that e(P-1)aP* ItIP-1 {1 +2pn( P- 3)/( P- 1 )t2_(p

2 1)zap *=I tl2n 1 )l+o(n-1) for 1
{1-(p21)z

an(t)= J .I t I zn 1) + 2pn (P s)/( P 1)t2} +o ( n(P - 3) 1 (P - 1) ) for 23, then letting On(t)=E[exp(itZn)] with Zn=^X;IVn // log an(t)=n log gyp( n

we have

1

2 pt2+(p_ 1)ap* n(3-P)/21 t I P-1+o ( n(3-P)/2 ) for 3
for 5
2,t2-X2nY+o(n-1)

Hence it follows that elpt'

{1+(p-1)ap* n(3 -P)/21 t I P - 1} +o(n(3- P ) /2) for 3
an(t) = l elpt' 11-

2n t°)

+o(n

for 5
Considering the case when p=5/2, we have from (3. 1) 05/2(t)= 1

-rl 1 t 1 3

/2+5t2+o(t2)

where r'=- a5/z*>O. n

Letting an(t)=E[exp(itZn)] with Zn=n-213TX;, we obtain 1 log O (t)= -711 t 131 2+5n-1 /

3t2_r^n-1 I t 13+...

Hence it follows that Yn(t)=e 1+5n-1/3t2+ 2 n-2/3ta_2 -n-11 t 13+125n-lt6+...l. 2 -6 As is immediately seen fromabove, the expansion of the power of n-'/3 is obtained and the distribution function having the characteristic function e-r'111" is a stable law with characteristic exponent 3/2 ([2], [3]). It is easily seen that terms in the expansion of the density function corresponding to terms e-7'11t"'t2k is d2k t kdx2g3/2(x)r

where gs/2(x) is the density function of the stable law given by (2. 11). Finally it is remarked that when p>3, the leading term corresponds to the normal distribution but in the subsequent terms in the expansion of the characteristic function there appear

44 August, 1976 On Gram-Charlier-Edgeworth Type Expansion of the Sums 123

terms like I t I ke-t'/2 where k is not an integer, and the density function corresponding to this is tk(cos tx)e-1112dt r s"

1

tk +2! (

XJ0

1

-x2)'

I,(k+2^ +11

E 2(k+2./-1 )/ 2 Z

j=O

e- t'l2dt

(2.1)!

r(2;+1)

(-x2)7.

References [ 1 ] Feller, W., "An Introduction to Probability Theory and Its Applications, Vol. II. John Wiley and Sons, New York, (1966). [ 2 ] Gnedenko, B. V. and Kolmogorov, A. N., " Limit Distributions for Sums of Independent Random Variables," Addision-Wesley, Cambridge, Massachussets, 1954 (Translated from Russian). [ 3 ] Ibragimov, I. A. and Linnik, Yu. V., " Independent and Stationary sequences of Random Variables," Wolters-Noordhoff, the Netherlands (1971) (Translated from Russian). E4] Takeuchi, K. and Akahira, M., "On Gram-Charlier-Edgeworth type expansion of the sums of random variables (I)," Rep. Univ. Electro-Comm. 27-1, (1976).

45 ON THE SECOND ORDER ASYMPTOTIC EFFICIENCIES OF ESTIMATORS *

Kei TAKEUCHI and Masafumi AKAHIRA University of Tokyo and University of Electro- Communications

1. Introduction Second order efficiency of asymptotically efficient estimators has been discussed by R.A.Fisher [6], C.R.Rao [9], [10] and others in terms of the loss of information. Recently Chibisov ([4], [5]) has shown that a ML (maximum likelihood) estimator is second order asymptotically efficient in some sense. Pfanzagl ([7], [8]) obtained similar results. One of the authors established similar results in a book written in Japanese [11] in terms of the asymptotic distribution of the estimators. In this paper we shall present the outline of the discussion given in [11] and proceed further to the third order asymptotic efficiency. Further it is shown that the results can be extended to non-regular situations. 2. Notations and definitions Let X be an abstract sample space whose generic point is denoted by x, B a cr-field of subsets of X, and let O be a parameter space, which is assumed to be an open set in a Euclidean 1-space R1. We shall denote by (X(), 1B(')) the n-fold direct products of (X, B). For each n = 1, 2, ..., the points of ;V(n) will be denoted by xn = (Xi, ... , xn). We consider a sequence of classes of probability measures {P,,,9 : 9 E 0} (n = 1, 2, ...) each defined on (X (n), 13(m)) such that f o r each n = 1, 2, ... and each 9 E 0 the following holds: Pn,9

(B(n) ) =

Pn

+1,B (B(n) X X)

for all B(n) E ,i3(n). An estimator of 0 is defined to be a sequence {9n} of 1(n)measurable functions 9n on X(n) into 0 (n = 1, 2, ...). For simplicity we denote an estimator as 9n instead of {9 n } . For an increasing sequence of positive numbers {c,,,} (limn,,) c,,, = oo) an estimator 9n is called consistent with order {c,,} (or {c,,,}consistent for short) if for every E > 0 and every '0 E 0, there exist a *This paper is retyped with the correction of typographical errors.

46

605

sufficiently small positive number S and a sufficiently large number L satisfying the following : lim sup

n->oo

e: 1e-191 <6

Pn,e

{ cn

I On - 01 > L j < s.

The order { cn} of convergence of consistent estimators and its bound are discussed in [1] and [11 ]. In the subsequent discussions we shall deal only with the case when cn = vfn-. Let On be a {}-consistent estimator. Definition 1. On is asymptotically median unbiased (or AMU for short) if for any i9 E O there exists a positive number S such that lixn sup

lPn ,O

{,^ (en - 0)

< 0} 2 I = 0'

lim sup

I Pn,e

{v' (On - 0)

> 0} - 2I = 0.

n-ioo 9:19-+9l<6

n-oo 8:Ie-,6I <5

Definition 2. For Bn asymptotically median unbiased Fo (t) is called an asymptotic distribution of it if li^lPn,e{ ✓n(On-0)
Since On is a {Vn}-consistent estimator, it follows that Fo(-00) _ 0 and FB oo) = 1. Let 9n be a AMU estimator. Then it follows that Fo(O) = 1/2 for all 0 E R Let 00 be arbitrary and fixed in

®. Putting An(t) = {in : en(xn ) - 90 < t/i }

we have

m Pn,6o {An ( t)} = Feo(t);

lim

Pn,eo

1 +(t/.r) {An(t)} = 2.

Consider the problem of testing hypothesis H+ : 0 = 9o + (t/vrn-) (t > 0) against K : 0 = 00. We shall denote by 8n(t, an, 00) the power of the most powerful level an test (0 < an < 1). Then we obtain for each t > 0 (2.1)

Feo (t) < 'UP { {«„}: lim„^_ «n = 1 /2} 1^Qn(t, an, 0) •

Denote by foo(t) the right- hand side of (2.1). Consider next the problem of testing hypothesis H- : 0 = 00 + (t/v/n-) (t < 0) against alternative K : 0 = 0o. In a similar way as the case t > 0 we define On (t, an, 0o) and 3eo (t). Then we have for each t<0

47

606 (2.2) Fe0(t) > 1 - /390(t).

Since 00 is arbitrary, the bounds of the asymptotic distributions of AMU estimators are obtained as follows : Fo(t) < p9(t)

for all t > 0;

FB(t)> 1-QB(t)

forallt<0.

For any 0 E 0 letting ,30(0) = 1/2 we make the following definition. Definition 3. An asymptotically median unbiased estimator 0n is called asymptotically efficient if for each 0 E O FB(t) _ ,(je( t) 1-Q9(t)

for all t > 0; forallt<0.

Definition 4. 8,,, is second order asymptotically median unbiased (or second order AMU for short) if for any 79 E O, there exists a positive number b such that Vfn-

lim sup

n-+oo 0:l6-,L9l <6

lim sup n-- °° 6 : 16-191<6

Pn,g

Pn,e

0}

2

= 0;

{Vrn- (en -0) >0 -2

=0.

{v'L(e n

-0)

Definition 5. For Bn second order asymptotically median unbiased FB(t) + (1/,/n)GB(t) is called a second order asymptotic distribution of it if l

{,'

< t} - FB(t ) - (1/ f)G0 (t)I = 0. (O - 0)-

im n n-too

v " Pn,O

Let 00 be arbitrary but fixed in O. Then we have Pn,o0 {An (t)} = Fe0(t) + 1 Go,(t) + o ( I Pn,eo+(t/, ) {An(t)} = 2 + 0 (n)

Consider the problem of testing hypothesis H+ : 0 = 00 + (t/V) (t > 0) against alternative K : 0 = O. Let 3,,(t, a,,, OO) denote the power of the most powerful level an test (0 < an < 1). We assume that sup {{"„}:"n=-z+O(*) }

0. ( t, an, 00)

48 607 = fea (t) + ^'Yeo (t) + o

(

)

.

Considering the case t < 0 in a similar way as the case t > 0, we make the following definition. Definition 6. A second order asymptotically median unbiased estimator 9n is called second order asymptotically efficient if for each 9Ee FB(t) = Ge(t) =

0e (t)

for all t > 0,

1 - /3e(t)

for all t < 0,

,yo (t) -ye(t)

for all t > 0, for all t < 0,

where for each 0 E O /3o(O) = 1/2 and ye(0) = 0. In a similar and obvious way we may define the third or k-th order asymptotic efficiency of estimators. 3. Asymptotic efficiency and second order asymptotic efficiency Let X1, X2i ... , Xn, ... be independently and identically distributed random variables with a density function f (x, 9) satisfying (i), (ii) and (iii). (i) {x : f (x, 9) > 0} does not depent on 9. (ii) For almost all x[µ], f (x, 9) is twice continuously differentiable in 9.

(iii) For each 0 E O 0<1

= f {

1og f(x ,o)} 2 f(x,e)dJ1(x) 02

-

J

< 00.

{ a02 log f ( x, 0) } f (x, 0)dµ(x)

49

608 Let 00 be arbitrary but fixed in 0. Consider the problem of

testing hypothesis H+ : 0 = Bo + (t/\) (t > 0) against alternative K : 0 = 00. Then the rejection region of the most powerful test is given by n

Tn=

EZni

>C,

i=1

where Zni = log{ f (Xi, Bo)/ f (Xi, Oo +

Tn

Since

= i=1 > Z n i =

log f (Xi, 8) E=

P n a2

2ni=1 O02

log f (Xi, 0),

if 0 = 00, then Tn is asymptotically normal with mean t2leo/2 and variance t210o and if 0 = 00 + (t//), then Tn is asymptotically normal with mean - t2lo, /2 and variance t2leo. Hence it follows that Qeo(t ) _

(t Ieo)

where -:D (u) = f"o^ I e-xz/2dx . From (2.1) we have Foo(t)<(t Ieo ) forall t>0.

In a similar way as the case t > 0, we obtain from (2.2) Foo(t) > 1 - (ItI IBo) = ( t 1 /

).

Since 00 is arbitrary we have now established the following well known theorem. Theorem 3.1. Under conditions (i), (ii) and (iii), if \(0n - 0) is asymptotically normal with mean 0 and variance 1/b i then 0n is asymptotically efficient. It has been well established that under some regularity conditions the maximum likelihood estimator 0ML has the same asymptotic distributions as above, hence 0ML is asymptotically efficient.

Using Gram-Charlier expansion of the distribution of E 1 Zni we get the asymptotic series of the power of the most powerful test.

50

609

We further assume the following : (iv) f (x, 0) is three times continuously differentiable in 9. (v) There exist Je [{log = EB f(x,e)}{

log f(x,e)}]

and

^31

KB=EB [{lof(xo)

and the following hold : Ee

1ae3

log f (X, 9) = -Me - Ke.

1

We denote (a /aO) f( x, 9), (,92 /,902)f (x, 9) and by fe, fee and feee, respectively. Since f (^, e) - fee(x, 0) - 3f98(x, 83 log

B)fe(x, e) + 2 {fe (x, e)3

893 f (x, 0) {f (x, 0) }2 (

f(

,

0) - 3 {

)

{f (x,

0)1 3

2

1og f (x, o) } { 22 log f ( x, B)

- { ae log f(xo)} 3 , it follows by the last condition of (v) that f feee(x, 9)dµ(x) = 0.

Lett > 0. If 0 = 90, then

(031003)f (X, 0)

}

51

610

TnZni

Blogf (X" Bo)

i=1

1

t2 n 192 2n L 802 log

f (Xi, 00)

1

6^ ,/n-

a03 log f (Xi, 00)

Hence it follows that Eep(Tn ) 2I+6tn(3J+K), t2 t3 t Vo0(Tn)^n -I + J =t2I+ -J, n n,j Vrn3

E90 {Tn - Ee0 (Tn)}3] --K,

where I, J and K denote Ieo, Jo0 and Koo, respectively. Put 91 = 9o + (t/.,,fn-). If 0 = 91, then T. 'v

7L pe log f ( Xi, 01) ^/ /- 1 V

P n a2

+ 2n E 802 t3

log f (Xi, 01)

n a3

6n / E 803

log f(Xi,01).

Hence it follows that Eo1(Tn ) ^' - 2I'+ 6t=( 3J'+K'),

where I', J' and K' denote I01 , Jot and K917 respectively. On the other hand we have

52 611

I' NI + aele° =I +

89

J

n

=I+

l j f (x, Bo)dµ

J f ( 2 2 { BZ log f (x, Bo) } {1og f(xoo)}f(x,oo)ciit l

=I + +

f (x, Bo) { YO lob 3

f {50-

log f(x,eo)} f(x,eo)d/-z

±(2J+K).

Hence we obtain t (3J+ 2K). Ee1(Tn ) - - 2I - 6T

Since

F8 JO.;

J' J+

K'-K +

a$Ke

°,

it follows by a similar way as above that 3

Ve1(Tn) Nt2I+

(J+K); 3

Eel

[ {Tn

- Ee1(Tn)}3]

--K.

Letting an be a rejection bound, we have P, ,e1 {Tn

Putting

cn

<

n

an} =

Pri,el

I

Tn + (t2 1/2) tj.

<

an + (t2 1/2) tV

= {an + (t2I/2)}/(tv17), we obtain

Pn,o1 {Tn < an}

t2 _ (D(cn) - O( cn) {_ 613J + 2K) + (J + K)c 2fI 6IK(cn 1)^+o\1/' "n-1I

Zee-u2/2 . If Pn, a1{Tn < an} = 1/2, then it follows that cn = O(1 /vrn-) and 4 (cn) = (1/2) + cnq5 (cn). Since

where O (u) = 4)'(u) =

53 612 t2 n K cn = - 6^I (3J + 2K) + 6I + o

i7 In /

we have 2

3

an=-t2 -6I(3J+2K)+6 I+0\ 1

Then we have Pn,oo {Tn ^ an } =1 -

Pn,o0

=1 - Pn,Oo

{Tn < an}

{

-

t2

1

Tn t ("7 /2) - cn < -WI

1

+ 0(-ti) { 6 nI (3J + K) - cn + 2^I J(-t^) 6K(t2I -1)}+0(;) ,/n-11

Since 00 is arbitrary, it follows that for each t > 0 Go(t) _ 1 (t

Vie) t2 7e(t) = 6(3Jo + 2Ko)q5 (t Io) . e

In a similar way as the case t > 0 , we have for each t < 0, Go(t) _ (-tom) 'Yo (t) t2 ( 3Jo + 2Ko ) q5 (t Io) . = - 6 Io

Therefore we have now established the following which is analogous to the result of Pfanzagl [7].

Theorem 3. 2. Under conditions (i)-(v), if

54

613

(3.1)

P,,,B { v4

(O - 9) < t} = 4'(t ) + 36\/ I:12 ,(t) + ° \ 7n/

then 8n is second order asymptotically efficient. (3.1) means thet n o(9n - 9) has an asymptotic distribution with mean -{(3J8 + 2KB)/(6/iIV2)} + o(1/.) and variance 1 + 0(1/Nn-) and third moment -(3J9 + 2Ke)/(/Ie12). Let X1, X2, ... , Xn,... be independently and identically distributed random variables with an exponential distribution having the following density function f (x, 9): (3.2) f (x, 0) P x1e, x > 0, 0, x < 0.

Since log f(X, O)

B

e

- log 0,

X 1

ae log f (x, 0) = B2 0

log f (X, 9) _ - 8 + 02 ,

it follows that Ie = 4 Ee Jo KB

=

[(x -

-BSE9 [(X

0)2] -

_ 02'

9)(2X

-

9)]

- 2 2 E9 [(X - 9)2] F5 3

]

= 1 Eg [(X - e)3 = . F6 F3 2

If ^(en - 0)/9 has an asymptotic distribution with mean 1/(&A-) and variance 1 + o(1/\) and third moment 2//, then 8n is second order asymptotically efficient.

The maximum likelihood estimator of 9 is given by OML = X = Ei Xi/n. Putting OML=

(1+3n)X,

we have EB

\/ (BML-

0)

0 3v/n-'

55

614

VB Vn0 nIL =

E

-,An_(^* ML

C

1

0)

+3 n

)

+°

2

✓n

^ 3] (1 + 1 )3

2

[

n

=

n

2 +0( 1 n

n

is second order asymptotically efficient . In this case Hence 0 X is sufficient statistic and the distribution has monotone likelihood ratio . Consider the problem of testing hypothesis 0 = 91 against alternative 0 = 90, where 01 = 0o + (t//) (t > 0). Then the rejection region of the most powerful test is given by the following form: X
^(X - 9)/9 has an asymptotic distribution with mean 0 and variance 1 and third cumulant 2/J. Using Gram-Charlier expansion we have (3.3)

Pn,0

{ ^(X - 0) < c }

= 4)(c) - 3 1 O( c)(c2

Let 0 = 01(= 90 + t/^) (t > 0). If (3.3) agrees with 1/2 up to the order of 1/Ji, then the following must hold: cO(c) - 3,0 ( c)(c2 - 1) 0.

Hence it follows that c = -1/(3\). The rejection region of the level 1 / 2 test of hypothesis 0 = 01 is given by X< 1-3n)81.

56

615 This agrees asymptotically with GAIL= (1+-- X(<01)

up to the order of 1//. Assume that Vn(0n - 0) has the asymptotic normal distribution with mean 0 and variance 1/Is, and that the cumulants of the asymptotic distribution are given as follows: ,/'E9(e„-0)= 1 cl( 0) +o

=

7n

1 c2(0)+0\7n/' EB [{v'(^n_Eo(On))}] =

1 c3(0)+0(

Further the m-th (m > 4) cumulants of ,An-en is assumed to be less than order 1/,,fn-. We assume that cl (0), c2(0) and c3(0) are continuous in 0. Put on = 0n - (1/n)k(0n). Since 0n agrees with do - (1/n) k(0) up to the order of 1/n, the cumulants of the asymptotic distribution of nIe(6n - 0) are given as follows: nJBE0(6n - 0 ) = J

ne

{ci(0) - k(0)} + o ();

nIBVB(en) = 1 + Iec ) + o ( r

EB

[{

nIB

(0n - EB(0n)) }

3

✓nl 3/2 IB c3(0) + 0

1

] = ,/n- ,`

If k(0) = ci (0) - 16Iec3 (0),

then in the Gram-Charlier expansion the following holds: Pn,e

It also follows that (3.4)

{Vn- (Bn

-B) '

I=

1+0(1

57 616 _ fi(t) _0 (t) 218 )t+ 10(/2) t2 +0 (I) n vFn

Hence it is seen that ( 3.4) agrees with the bound ( 3.1) if and only if c2(9) = 0 and c3 (9) = (3JB + 2KB)/IB. If c2 (9) = 0, then c3(9) must automatically be equal to -(3JB+ 2KB)/I93. Indeed since 0(t)t2 > 0 for all t 0, it follows that (3.4) (3.1) uniformly in 9 if c3 (9) < -(3JB + 2KB)/IB. This means that the bound (3.1) fails to hold at either positive or negative t. Hence if c2 (9) = 0, we must have c3(9) = -(3JB + 2KB)/I03. This apparently seems mysterious . However it will be naturally understood from different examples that if the second moment is decided up to order of 1/\, then the third moment is done. We shall verify the above fact by the maximum likelihood estimator. The second order asymptotic efficiency of the maximum likelihood estimator will be verified using the above fact. The continuous differentiability of the likelihood function is assumed up to the necessary order . Let BML be a maximum likelihood estimator. Since a

0 1og f Xi,

FO

e

ML) 2

_ ae log f (X;, B) + { ae2 log f x B) } (

n

f (Xi, B.) + 2 { aB log

4,

2

(eML

putting Tn = n(OML - 0) we have n

n

f (X" B) + ° = 88 log n

2

882 log

f (Xi, B) 7'n

n 3

+ 2nd

E a43 log f (Xi, B*) Tom.

Put Zl =

1 n .

ae log f (X4, B);

n

2

^ {log f(xio)+Ie } ; Z2 - 2 n

3

W n ^93

log f (Xti, B*).

58 617

Then Z1 and Z2 have the asymptotic normal distributions with mean 0 and variance IB and Lo(= Ee {(a2/a02 ) log f (X, 6) + Io}2) and covariance J9, respectively and also W converges in probability to -3JB - KB. Hence it follows that 1 3JB+KB 2 Z1+ -1B+7Z2 Tn- 2/i Tn 0, Zl +

Tn -

Z

118 Zl Z2 - 3218 B

I

IB

Since covariance of Z1 and the term of the order is 0, it follows that

Hence it is seen that the maximum likelihood estimator is second order asymptotically efficient. If we indeed calculate the asymptotic cumulants, they are obtained as follows: EB(Tn) ,,, - Jo + KB 1 2/ IB +o (Tn ) ;

To EB

{Tn -

( Tn 1 ); n)13]

-

- Vrns +0 Me + 03 \-^

Hence it is shown that if the second moment is decided up to the order of 1/J, the third moment is equal to what it should be. As another example, we consider the location parameter case. Let X1, X2,..., Xn,... be a sequence of independently and identically distributed random variables with a density function f (x - 0). It is well known that under appropriate regularity conditions the best linear estimator 6n = E 1 C%nX(iln) + con is asymptotically efficient, where X(i1n) < ... < X(nin) and cin are optimal constants

as was established by Blom and others. Then the estimator 0, is second order asymptotically efficient. Indeed, if U(iin) are order statistics from the uniform distribution, then it follows that

59 618

F- 1 (X(il,,)) = B + F- 1

n+1 '(U(iln) n +

+ {F-1 (n + 1

)}

+ 2 jF-1

I

,

(n + 1

2

\n + 1) + R.

)

Let con = - Z 1 cinF- 1(i/(n + 1 )) and Putting a in =

{ F -1 (n i ) (ll \

E= 1 cn = 1.

1' ,

J

bin=SF -

1 (n+1)}"'

we have Vr_

"r

(9n

- e) =

n

2 ^' ECinain

U(i1n) -

i=1 n+1 n

+ 2 cinbin i=1

(

l2

u(fl) -

n J 1

+ V nRn.

Then the asymptotic variance is given by V( i(9n - 0)) ^` 2 2 i(n+1-i) i(n+1-j) 2n n L_ cinain ( CinCjnainajn ( n + 1)(n + 2) + n + 1)2(n + 2) i=1 i<j

+ n Cincinainbjn i<3*

i(n + 1 - j)(n + 1 - 2j)

n +13 ) (n + 2) (n -) 3

(n+1-i)(n+1-2j) +nEECinCjnainbjnj i>j (n + 1)3(n + 2)(n + 3)

Let J(u) be a function sufficiently smooth and put cin+ (1 /n)J(i/(n+ 1)). Putting h(u) = f (F-1 ( u)), we have (F-1(u))' = h(u)-1 (F-1(u))" _ -h'( u)h(u)-2.

Since V(Vn-(B,. - 0))

60

619

2 ff u(1 - v)J(u)J(v)h(u)-lh(v)-ldudv u
+ n ffu(1 - v)(1 - 2v)J(u)J(v)h(u)-lh'(v)h(v)-2 dudv u
- u)(1 - 2v) J(u)J(v) h(u)-'h'(v ) h(v)-2dudv + n Ifv(1

it follows that J(u) = -h(u ) h'(u)/I,

where I = f { f'(x)}2 If (x)dx = f {h' (u)}2 du. Then we have V(v n (en

- e)) ^'

I +o

(=)

.

Let 9* be a modified estimator which is second order AMU. Then it will be shown that O is second order asymptotically efficient. Since {-1og f(x)} { J = - dx2 2 d

1

1og f(x) } f(x)dX

_ iog.ix)} f'(x)dx

2,f {

21 =2

f

3

dx

log f (X) } f (x)dx

{h'(u)}3 du

K - 2'

the third asymptotic moment of the second order asymptotically efficient estimator must be -K/(2n2I3). Indeed, since COV

(U(i1n ) n + 1)

(U(j

i.) -

n

+ 1) '

(U(k1n) -

7l

k + 1)

2

61

620

= 2cov(U(iln), U(kIn))cov(U(iIn), U(kjf)) + o E KU(i1n) n+1) (U(iln) - n+1) (U(kIn) n+1) J -i(n+1-j)( n+1-k) i< k (n + 1)2(n + 2)(n + 3) ' - '

it follows that E

[f On -

E(Bn)

}) 3

,6

JJJ u(1 - 2v)(1 - w)J(u)J(v)J(w)h(u)-lh(v)-lh(w)-ldudvdw n2 J { f miniu v ) - uv) J(u)h(u)du} J(v)h(v)-2h'(v)dv + o 1n )

u
2

+

Putting J(u) = -h(u)h"(u)/I, we have E

[Jon - F' i(Bn)

- -n 13

1

31

J {h'(u)}3 du - n I3 f h(u) h'(u)h"(u)du

n 113 f {h'( u)}3 du + 2n 13 f{h(u)}du K

2n2I3.

Hence it is shown that the third asymptotic moment is -K/(2n 213). The second order asymptotic efficiency of ML estimators depends on the assumption of differentiability of the density function . The following example is to make this point clear. Suppose that Xi' s are independently and identically distributed according to the double ( bilateral) exponential distribution with the density function

.f (x - 9) =

2

e_1z-8I ,

where 0 is unknown location parameter. In this case it can be shown that the first order asymptotic bound is obtained with I = 1, and it can be easily shown that the ML estimator

62 621 Xi 9ML = 1med
is asymptotically efficient. The second order asymptotic bound is obtained after some algebraic manipulations (the detail is given in [111) as follows: (D(t) - 6ttO(t) sgnt.

But the asymptotic distribution of 9ML is given by a Pn,e {' (011L - 0 ) < t} ='b(t) - 2 ,^-0(t) sgnt + o (

which differs in the second term from (3.5). 4. Third order asymptotic efficiency We proceed to the problem of third order asymptotic efficiency. 6n is called third order asymptotically median unbiased if for any E 4 there exists a positive number 5 such that

I

lim sup n Pn,e { ii e: 10-01 <6

n-400

(On

- 0) < 0 } -

1 = 0; 2

lim sup nl Pn,e{v(On-0) > 0} - 1 n-foo e:1e-91<8 - 2

=0.

Let 63 be the class of all third order asymptotically median unbiased estimators. For all 9^ E 63 we shall obtain the bounds of Go and HO such that lim n Pn,9

n-aoo

{j

(On-0)
In a similar way as section 3 we consider the problem of testing hypothesis 9 = 90 + (t/Vn-) against alternative 0 = 9o and the asymptotic expansion of the power of the most powerful test of it.

The continuous differentiability of the likelihood function is assumed up to the necessary order. Putting ' = (8/09) log f (x, 0),

63 622 0" = (a2

/ae2) log f (x, e) and V)(') = (ai /aez) log f(x, e) (i = 3, 4),

we have n Tn = Zns i=1

n

t2 n

V)'(X') 2n (X) _ 6n 3(3) (Xt) 4 n

24n2

0(4) (X,) +0 ( -)

n

where Zni = log If (Xi, eo)/ If (Xi, eo + (t//))}. Define L = Ego {O(3) (X)O'(X) } M=E9o{1i"(X)2},

N = Eep {,cb " (X)'(X)2}

H =Eeo{Vi'(X)4 }.

Since under appropriate conditions the following holds: Eeo {(4)(x)} _ -4L - 3M - 6N - H,

the asymptotic cumulants are given as follows: Eeo (Tn ) - t2 + 6t^ ( 3J + K) + 24n (4L + 3M + 6N + H), Veo(Tn) t2I + t3 J + t4 (M - I2) + t4 L,

4n

3n

Eeo [{T - Eeo(T)}3] = 'Y9o (Tn) - --K - 2tn (N + 12), Eeo{T - Eeo(7'n)}4] - 3 {Veo ( Tn)}2 = beo (7'n)

t4 n

(H - 312).

If B = 91 (= 90 + (t/vfn-)), then for any measurable function g Eel {g(X)}

64

623

t fBo(X) t2

1 + fep (X) + 2n feo (X) + 6n-,/n-feo (X) + .. .

= Eep 9(X) = Eeo {g (X)

fep(X) t3fo(I (X)

{

z

1 + =V(X) + 2n ("(X) + V)I(X

)2)

t3

+ 6n3 (0(3)(X) + 3Vi"(X)i1'(X) +V/(X)3)

Hence we have 2

Eo1(Tn)

n

n VA X) + 2nnE

Eeo(Tn) - Eeo

t E

to E V), (Xi) + n t3

1

2

n 3

b"(Xa) + 6n 10 (3)A)

n

E (,i"(X;) +'^/)I (X,)2) 2n 1

n

+ 6nvrn- (0(3)(X,) +30"(XX)O'(Xj) +OI(X,)3)

+ o(1)

=-2I- 6t^(3J+2K)- 24n(4L+3M+12N+3H).

Similarly we obtain Ve1(Tn) t2I + t3 ( J + K) + t4 (4L + 3M + 18N + 6H - 312), 12n Ye,(Tn ) - K - Zn (3N+2H-3I2), 4

Se1(Tn) n (H - 3I2).

Put Vn = { Tn - Eg1 (Tn) } / Vol (V,,. )

(t/t) . If Vol (T.)

=1+Ql

+na2 +o\n)

65 624

7e, (Vn

) = t3I^ryB' (Tn)

1

1

_ 'Y1+nryz+o

C

1)

J, W

8Bl (Vn ) = 1JO1 (Tn) = nb+o (1) ,

then we have Pn,B3 {Vn < a}

_ -1,(a) - ¢(a) { 2 (vfn- +') a+ 6 vrn- + nz) (a2 - 1 ) n l \ z - 3a) + 12n (a4 + 6az +3) + (__- + 24n 8n) (a3 z l 1 + 72- (a5-10a3-15a) +0

Pn,o1

{V,
2+0\n/

Since (D (a) = 2 + aq5(a) - I a3q5'(a) + ... ,

it follows that a

+ n + 4'8171 '8171 + o n 2 n a G n __ Q \ y^ ' ryz) 1 \1) 71 _72 + 0i'Yi+0 1

6vln- n 6n n

If for 0 = 00 EB ,(Vn)-

Feo (Un)-

1

1 , a=-tvfl + ^a l

, 1 ,

VBO(Vn)= ^Nl+n^2+^

C

1) n 1

1 , + na 2

+o

(

n, 1)

66 625 1 , 1 , (1) 7i + 72 0 nJ

-yo.(V- )

n

n

(1) 1

then we have (4.1) Pn,eo =

1

{Vn > a} -Pngo

{Vn <

a}

nag = (t') + O(e ) ITn I

C2a,F

a + 2n + 2n) t'

+ ( /i + 72 +!? Q ) (tit _ 11

6^ 2n 2n J bl + ai7'i + i2 (t^3 - 3t') - (24n 6n 8n ) + 1 71

t'4 -

12n (

6t'2 +

3)

Y12 1t'S-lot'3+15t' ) }+ 72n JJ) o(),

where t' = t,/I-. Substituting in (4.1) t'2 K al = 2132 (2J + K) - 613/2' ti3 2^ a2 = 1212(4L+3M+6N+3I )+ 6 - K(J + K),

/2' -

t,J

1 - 13/21 t,2

2

QzI2(M-I2)+3I2L , K 71=-I3/2

67 626

J=72- (H-31),

we obtain Pn,B0 {Vn > a}

r _ 4, W) +

tit

ti5

OW) [g^13/2 (3J + 2K) - 72n13/2 (3J + 2K)2 ti3

- 72n13 { (3J + 2K)2 - 31(4L + 3M + 6N - H + 612) } + ti 3 {2K2+9I(2N+H-12))1 72nI .1 1

Hence it follows that for t > 0 Pn,g { n1 (9n - 0) < t} n

«(t)-0(t){

u

2n(t5 -10t3+15t)

6^V/n-+6^vfn-(t2-1)+ Q4

/32

3

+ (24n + 36n ) (

t - 3t)

Q3 2 L2 ) t'>+o(2 n \, +( 2n+72n

where 33 = -(3J + 2K)/ I3/2, Q4= j3 { 3(3J+2K) 2-I(4L+3M + 6N-H+612)), Q2 3613 { 17(3J + 2K)2 - 2K2 - 91(4L + 3M + 8N + 512)}.

Further the moments of the third order asymptotically efficient estimator must be as follows:

68 627 " 1 = 9 + 6 JI3/2 +0 nVfn-

(4.2)

Ee (9)

(4.3)

^ vo(V n dn)=

(4.4)

n9n =

I

QZ

+ nl

+o n

03

n , I3/2 + oIJ

n12

n (1)

Next we shall consider the example of the exponential distribution with the density (3.2) given in section 3. Since 8n

=f 3njX

it follows that (4.6) EB(Bn) = 0 + 3 , (4.7) Ve (/9n) = 1 1+ 2 (4.8) 'Yo (V9n) = 203 +0 n (4.9)

6 bo (v On) = n 04+0

\ 02, n

n. (1

)

Since I = 1/02, J = -2/03, K = 2/03, L = 6/04, M = 5/04, N = -5/04 and H = 9/04, it follows from: (4.2)'(4.5) that /33 = 2, ,Q'4 = 6 and l3'2 = 2/3. Then the moments of (4.2)'(4.5) are equal to those of (4.6)'(4.9), respectively. This fact should hold because X is a sufficient statistic. We next consider the maximum likelihood estimator 9ML• Putting Tn = ,,,/n(9ML - 9), we have Zl + (-1 + Z2) T. + 27n -(3J + K) + Z3^ T„ 4L+3M+6N+HT3_0 6n n '

69 628

where Z3 = (1//i) Ei {0(3)(X2) + 3J + K}. We also obtain Tn

NIZl +^I2

(z2z1

- 3J

IK)Zl

+n13 {z1z + Z12 3(3J + K)Ziz2 (3J+K)2 3 4L+3M+6N+H 3l + 212 Z1 6 Zl } .

Since E(Z1Z2 ) N 1 +I2), n E(Z13 Z2) N 31J, n E(Z14Z2) N 3(M-I2)+12J2, n

it follows that E Tn (

)

N

VTn N

-

J+K +o C11 2,12 n J J

1 7J2 + 14JK+5K2 L+4N + H+I2 +o /11 I + 2414 n13 1\nJ , 3J + 2K +0 ( n1) 1 nI

3(T„) N 12

(J+K)(2J+K) - 4L+12N+3H- 31

+0

(

1n54 n)

We define a modified maximum likelihood estimator 8ML as follows:

K9ML = BML + 6n1? L 0ML

Then k * L is third order AMU, that is, Pl,e {BnfL - 0 0} = + 0 in^

2

70 629

Further it follows that 1 0KB 2 nV (kL ) = n (1 + 6n 870 - V (OML) N V(I,n) + (3N + H)I - 2K(2J + K) 3n14

where

aeKe = a f {'fi'(x) }3 f(x)dx = 3 f z/i"(x)^/i'(x )2 f (x)dx +

J {,(x)}4 f (x)dx

=3N+H, BBIB=2J+K.

In general the maximum likelihood estimator is shown not to be third order asymptotically efficient. It is enough to consider a special case, and we take the Cauchy distribution as an example which has the following density function f (x, 0): 1

f(x 0) = 7r

(1 + (x - 0) 2 ) *

Let X1, X2,. .., X,,,, ... be independent identically distributed random variables with above density.

Since 49 log f (x, 0) = 1 + x2 80 log f (x, 0) ax 2

2

2(1

2)

log f (x, 0) = 8x2 log f (x, 0) _ (1 + x)2 2 802 03 80 3

log f (x, 0) a log f (x, 0)

41(+ xz s)

it follows that I= 1/2, J=K=O, L=-3/4, M=7/8, N= -1/8 and H = 3/8. Since the density is symmetric, the maximum likelihood estimator 0ML is third order AMU. The second and fourth cumulants are given as follows:

71 630

V (/BML)

5 :6+ n

-Y( V'nB ML) ,.,

n .

On the other hand since the values of the bound of the asymptotic distribution is given by ,33 = 1 /4,

04 = 0,

they are larger than the above moments (4.10) and (4.11). Hence BML is not third order asymptotically efficient. 5. Some further remarks

The above results can be extended to the cases (I), (II), (III), (IV). (I) When the distribution is discrete , some modification is necessary to apply the above argument. (II) Let Xi's be independent but not identically distributed random variables with density functions ff(xi, 0). Let B, be a second order AMU estim ator.When c,2(9, - 0 ) has an asymptotic distribution, we obtain the bound of the distributions of 9,,,. Let 90 be arbitrary but fixed in O. Consider the most powerful test of the problem of testing hypothesis 9 = 91 (= 9o + cola, a > 0) against alternative 9 = 00. Then the test statistic of it is given by (5.1) T,, = log

ft (Xi, B1) fb(X2,

00)

_ -{ E 0 50- log fi (Xi,Bo)} . (cn1a) l n ( 2 2 { a42 log fi (Xi, eo) j

1

-6 n

+Elkni=1

l

l

logfi(Xi,eo)

1

2 (Cn1a)

11 3 (cnla)

72

631

Put Ii = Eeo {

1o fi(xioo) }

2 I 2 l z log fi(Xi, Bo) -E90

It follows that the variance of the first term of the right-hand side of (5.1) is given by (cn-'a)2 Ei Ii. When cn = I, where in = E; Ii, it's variance converges to some finite value. If the following "Lindeberg type" condition holds: for every E > 0, [{

1n li ^ IIn Eep 1

log fj(xj,9o) }2xjfl(E)] 8B

= 0,

where 1, if log fi(Xi, 90) > 0, otherwise,

In,

E

Xin(E) _ 80

then the first term of the right-hand side of (5.1) is asymptotically normal with mean 0 and variance a2. The necessary condition for this to hold is given by limn-->oo max, 0, n

2

lim 1 E Eeo [{- 02 log fi(Xi, Bo) } X n(e)] = 0, n-+oo In

where if - 8e^ log fi(Xi, 90) 0, otherwise, 1,

Xin (E) _

> EIn,

then the second term of the right-hand side of (5.1) converges in probability to a2/2. If under alternative 0 = 0o, similar regularity conditions as above, then the test statistic Tn is asymptotically normal with mean -a2/2 and variance a2. Put cn = In. For a > 0, the supremum of the power functions is given by 4)(a). Hence if

- 0) is asymptotically normal with mean 0 and variance 1, then en is an asymptotically efficient estimator. Let BML be a maximum likelihood estimator. Then In(en

fi(Xi, B) } (BML - B) 0 = { 8 B log + { 2 n

+ Rin.

log A (Xi, B)

} (BML

-

9

12 /

73

632

(BML - 9) is asymptotically Under some regularity conditions normal with mean 0 and variance 1. Hence the maximum likelihood estimator 9ML is asymptotically efficient. In order to consider the higher order of the bound of the power functions, we must decide the orders of n

n

Kn=>K+=> Eeo

[c

131

89

log .A (X%, Bo)

and n

n

2

in = Ji = Eeo %2 log fs (Xi,Go)} {5 log A A , 00) -0i i 1.111

Then the asymptotic moments of the test statistic Tn are formally obtained as follows: 2

3JSI /Kn

Eeo (Tn) ,, 2

a3, n

Voo (Tn) .., a2 + ^J%2 a3, n

Eeo{TEe0 (Tn)}3 -j 22a3.

Similarly we have for 9 = 91 a 3Jn + 2Kn 3 Eeo(7'n)..,-26732 a, n Vei ( Tn)

-a2 + Jn

P /2 n a3,

(Kn/In/2) a3.

Eel {Tn - E81 (T)13

"/2 If in and Kn have the same order which is less than I3 and the residual terms are negligible, we can apply Gram-Charlier expansion to the asymptotic distribution of the statistic Tn. Put d,t 1 = Kn/In3/2. If lim d,,

n-,oo

it (9n

-

0)

<

O}

- 1/2

=0

uniformly in any neighborhood of 90, then for a > 0

74 633

(5.2) lim do PP ,e. { In (Bn - o) < a } - ^(a) - 3J61.3 2 Kn a 2 O(a) n00 / JJ

< 0.

-

For a < 0, a similar inequality as above holds. If the equality "=" of (5.2) holds, then 9n, may be called second order asymptotically efficient. The necessary condition for second order asymptotic efficiency is that Bn has the following asymptotic moments: E,

{

I ( 9 - o) ,,

1 N 3J+2Kn

UB{

3I^2 n l / In(Bn-o)}N1+o1 );

EB

In

}3

{

(en

- e)

+0

(

) 1 /

J+2Kn Iri^ 2

1

If On has a smooth asymptotic distribution and Yg { In (Bn - o) } - 1 + o(1/dn), then the modified estimator Bn which is second order asymptotically median unbiased is second order asymptotically efficient. Let BML =

OML + Kn (OML )

6nln(OML)2

Under regularity conditions 9ML is second order asymptotically efficient. If the following generally hold: In/n -4 I

(n -3 oo),

Jn/n --^ J

(n -^ oo),

Kn/n -* K

(n -4 oo),

then it follows that do = vfn-. Since the second term of the asymptotic distribution has order of 1/J, a similar result as the i . i.d. case holds . But the fact does not always hold. For the purpose we consider the following examples. Let X2 = Ozi + U2, i = 1, 2,. . ., where zj's are constants and Ui's are independent identically distributed ramdom variables with a density f (u). Putting

75

634

1og f(u)}2f(u)du = f { f ( )}2 du,

I=f{ J=

J {l d2 log f (u)-JJJ} {{f'(u)}31og f(u)}f(u)ciu [fhl(u)fI(u)

f (U) { f (u )}2 ] du, l

3

K{dulogf(u) } f(u)du { f/(u 3

= {f(u)} du,

we have for each i II=z3I, J,=-z3J, Ks=-z3K.

Then it follows that n

n

n

In= ^z3 I, J=- ^z3 J, K=- ^z: 1

1

1

By Lindeberg's condition we obtain n

max zs

1<4
E zs -4 0 (n -4 oo). 1

Since

< max I z` I -^ 0

IEn

(E

zi )3/2

oo),

(^1 zti )1/2

it follows that Jn /In/2 - 0

(n -4 oo),

Kn /I3/2 __+ 0

(n -+oo).

z2)312 /(E z3) . If En i z2 /n -3 m2 Then we can take do = (E (n -+ oo ) and Ei z3/n --4 m3( 0), then it follows that d, _ O(-). Then the second term of the asymptotic distribution has order of 1.

76

635

But letting zi = i, we have n(n + 1)(2n + 1)I/6 - n3I/3.

n3I/3(9n - 0) is asymptotically normal with mean 0

Hence if

and variance 1, then Bn is an asymptotically efficiency estimator. Since + 1)2

= - n2(n

Jn

J

4 (

n+

)2

n4

4J,

4

n44

it follows that do = O(ff). Hence in this case the second term of the asymptotic distribution has also order of 1/V/n-. If zi -+ 0 (i -} oo), then it follows by Lindeberg's conditions that the following must hold: n

In= z1 I-4 oo (n -goo).

But it is possible that En z2 -+ 00 (n -+ co) and En z2 < 00. do = 0(1,1/2), Since letting zz = 1/vz we obtain in = O(log n). Hence og n(0n - 0) is asymptotically normal and the second term of the asymptotic distribution has order of (log n)-1/2. Let Xi's be independently distributed random variables with the following density: f (x 0) =

xn.-le-=:/B x > 0

1

BPir(pt)

where pi's are known positive integer and 0 is an unknown parameter. Since log fi(Xi, 0) =

(Pi

a0logfi(Xi,0) ,9 2

502

log fi(Xi,

it follows that

0)

- 1) log Xi - Xi/0 - pi log 0 - log r(pi),

0i

= -

- 0i,

0 ii

+ 02 ,

77

636

/ X, pi \ 2 192 - 9 I =B2,

Ii=Ee

Ji = -Eq Ki = Ee

- P) ('X' - P)] = -,Pi

02 T _^T F2 03 lXi

92

= 2pi

pi 3

9) 93

Then we have n

In

n

n

= Epi /82, in = -2 E pi /93, 1 1

Kn

= 2 E pi /93.

1

Since do = (Ei pz)1/2, the asymptotic distribution of the second order asymptotically efficient estimator 6n is given by Pn,g

n 1 2 Pi (Bn - B) /B < a - (a) + a O(a). 3 El pi

The maximum likelihood estimator 9ML of 0 is also given by E Xi /E pi . Let + 3

9ML = (1 \

EInl pi

) 9ML.

Then 9ML is a second order asymptotically efficient estimator. Since in this case En Xi is a sufficient statistic, the second order AMU estimator based on it must be second order asymptotically efficient. Indeed it is easily shown that the asymptotic distribution of 9ML agrees with the above bound of the asymptotic distributions. (III) Let {Xi} be defined recursively by Xi=OXi-1+Ui,

i= 1,2,...,

where X0 = 0 and {Ui} is a sequence of independent identically distributed real random variables having density f with mean 0 and variance a 2. Let 00 be arbitrary but fixed in O. We consider the problem of testing hypothesis 0 = 01 (= Oo + cnla) against alternative 0 = 90. Then the test statistic of the most powerful test is given by 10 g

t- 1 Zni = ,-1

f ( Xi - 00xi-1) J xi - 91Xi-1)

78

637

By Taylor expansion we have n

n

1

f'(UU)

Zni = Cn aX _1 f(Ut) i=1 i=1

n

2

- 2 c"2a2Xs 1

aU21og f (Ui) }

i=1 n

+R.1. i =1

Since for 1 01 > 1, Xi is stochastically order of 0i , the asymptotic distribution of En 1 Zni is not always written as an easy type. If 101 < 1, then at least Xi is asymptotically stationary. If cn = V/n-, then E 1 Zni is asymptotically normal ([ 11]). From above it is shown that /i(Bn - 0) is asymptotically normal with mean 0 and variance ( 1 - 02)/Icr2, where I = f { f'(u)}2 If (u)du if 0n is an asymptotically efficient estimator.

When 101 < 1, using asymptotic moments we can formally obtain the bound of the second order asymptotic distributions of second order AMU estimators of 0 and show that a modified least squares estimator of 0 is second order asymptotically efficient (Akahira [21). (IV) When the parameter 0 is vector valued, we may decompose it to 0 = (^) where 6 is a real number and 77 is a real vector, 77

and we consider estimating 6, while q is treated as a "nuisance" parameter. Then the second order asymptotic bound for any estimator ^ of 6 is given in a similar but somewhat complicated algebraic expression, and it can be shown that a modified ML estimator attains this bound under some regularity conditions Q3]).

References [1] Akahira, M., "Asymptotic theory for estimation of location in non-regular cases , I: Order of convergence of consistent estimators ," Rep. Stat. Appl. Res., JUSE, 22, (1975). [2] Akahira, M., "A note on the second asymptotic efficiency of estimators in an autoregressive process," Rep. Univ. ElectroComm. 26-1, (1975).

79

638

[3] Akahira, M. and Takeuchi, K., "On the second order asymptotic efficiency of estimators in multiparameter cases," Rep. Univ. Electro-Comm. 26-2, (1976). [4] Chibisov, D. M., "On the normal approximation for a certain class of statistics," Proc. Sixth Berkeley Symp. on Math. Statist. and Prob. 1, (1972). [5] Chibisov, D. M., "Asymptotic expansions for Neyman's C(a) tests," Proc. Second Japan-USSR Symp. on Probability Theory. Lecture Notes in Mathematics. Springer Verlag, Berlin, (1973). [6] Fisher, R. A., "Theory of statistical estimation," Proc. Camb. Phil. Soc. 22, (1925). [7] Pfanzagl, J., "Asymptotic expansions related to minimum contrast estimators," Ann. Statist. 1, (1973). [8] Pfanzagl, J., "On asymptotically complete classes," Proc. Summer Research Inst. of Statistical Inference for Stochastic Processes, 2, (1975). [9] Rao, C. R., "Asymptotic efficiency and limiting information," Proc. Fourth Berkeley Symp. on Math. Statist. and Prob. 1, (1961). [10] Rao, C. R., "Efficient estimates and optimum inference procedures in large samples," J. Roy. Statist. Soc. 24 (B), (1962). [11] Takeuchi, K., "Tokei-teki suitei no Zenkinriron (Asymptotic Theory of Statistical Estimation)," (In Japanese) KyoikuShuppan, Tokyo, (1974). K. Takeuchi M. Akahira Faculty of Economics Department of Mathematics University of Tokyo University of Electro- Communications Hongo, Bunkyo-ku Chofugaoka, Chofu-shi

Tokyo, Japan Tokyo, Japan

80 Ann. Inst. Statist. Math. 29 (1977), Part A, 397-406

EXTENSION OF EDGEWORTH TYPE EXPANSION OF THE DISTRIBUTION OF THE SUMS OF I.I.D. RANDOM VARIABLES IN NON-REGULAR CASES KEI TAKEUCHI AND MASAFUMI AKAHIRA*

(Received Oct. 21, 1976 ; revised July 7, 1977)

Abstract The asymptotic expansions of the distributions of the sums of independent identically distributed random variables are given by Edgeworth type expansions when moments do not necessarily exist, but when the density can be approximated by rational functions. 1. Introduction Asymptotic distributions for the sums of independent and identically distributed (i.i.d.) random variables has been extensively studied for many years ([1]). Suppose that X,, X2, • • •, X„ is a sequence of i.i .d. random variables with mean p and the distribution function F(x). Then the asymptotic expansion for the distribution of Y„ ( X, -np)''^/ n up to the order n m/2+' is given by the classical Edgeworth expansion when F has the moments up to the mth order as well as the continuous density ([2], [3]). The purpose of the present paper is to extend the result to the case when the moments do not necessarily exist assuming only that the density function f (x) of F is approximated by some appropriate rational functions as I x I-> oo . Put, for some real constants M„ and V„ (>0), Z„= ^ X£- M„)I V,.

Then, as is well known , if a G(x)

is a limiting distribution of a sequence of distributions of Z's, it is necessarily a stable distribution. We shall show in the next section that if this is the case then under the assumption stated above V, must be either n or n log n or and the corresponding leading term of the asymptotic expansion is the Cauchy , or the unsymmetric stable distribution with characteristic exponent 1, or the normal distribution, respectively. The second term is shown to be of order either * Supported in part by the Sakkokai Foundation.

397

81 398 KEI TAKEUCHI AND MASAFUMI AKAHIRA

1/n, (log n)/n, 1/log n or 1/vt-n- and the density function corresponding to the second term are also given. Generally the relative magnitudes of the successive terms are complicated and is difficult to write down explicitely but may be obtained for each particular case. The details of the computation are given in [4] and may be referred to for further examples. Further the similar results for fractional characteristic exponents are given in [5] (see also Cramer [6] for this case). 2. Results We shall obtain the Edgeworth type expansions for the sums of random variables not necessarily with finite moments but with the density f (x) which satisfies the following condition : There exist two rational functions g+(x) and g-(x) such that for a positive integer m (> 2) and some 0<6<1

(a) lim x -+1+' j f(x) - g+(x) j = 0 ; (b) lim 1x1m +,+a1f(x)-g (x)f=0.

Remark 1. m depends on the order of the expansion and should not be smaller than 2 in order that we have meaningful expansions. Remark 2. It should be remarked that since the distribution has a density f (x), we have

sup fi(t) I < 1 ItI >r for any p> 0. We shall first show that the following proposition holds. PROPOSITION.

g+(x) and g-(x) may be taken of the form

g+(x) a> g-(x) _1 P> s =2 (x-1)' > =2 (x+1)'

where of and P, are real constants. PROOF. We first decompose g+(x) into the sums of partial fractions such as Baf Ear 9+ (x) = E i (x+Da)' +^ i (xE+Fax+Ga)' But each term in above expression can be approximated by a sum of the terms of the type jSfl(x+1)' up to order x-'"-1 when x is large. Therefore g+(x) may be replaced by a sum of such terms. Similar is true of g-(x).

82 EXTENSION OF EDGEWORTH TYPE EXPANSION

399

Now let us denote that 1)e`tx dx af(t)= (jo (1+x)'

then we have

5

o

(j-1)e"

(1-x)'

.. dx=#f(-t)=Of(t) •

Thus if the density f (x) satisfies the condition above, the characteristic function can be expressed as

(1) fi(t)= f

ettx f(x)dx=

Z afof(t) +E p.^of(-t) +0(t)

and since

50 I

dx Jo xml f(x)- g+(x)I
x ImI

f(x)-g ( x )I dx
the remainder term ¢(t) can differentiated m times at t=0, so that we have m

fi(t)= cptP+o(I t Im)

(2)

p=0

It is shown by Takeuchi and Akahira [4] that for small I t I > 0 and jZ2

(3)

cf(t)= (j- 1)\Jit1 u -'e du )Itlf -le tt tu

= (j -1){ Z

ik

kltik - ( j2'

a

+iP* l tIf- '-

?' Itlf

}(

M ItI'-

1-

1l og

ltl +a,

ItI' -'

it - 2 t2 +6ts+...l

-' =1+E a fkItIk+E bfkt I t 1"-'+EcJkItIk l logltI k=1

k=1

k=f

in

+Edfkt lt1k-1logItI+o(Itlm) k=j

where ak = -1/k! (k - j + 1) and a j* and JS; are real constants equal to the real part and the imaginary part of f

JI

1

u-fetudu +

u2

u-f {eiL_1 _iu+

'

i-2 Zu i-

- +. • . -

)1)! }du+Eik k=o k!(k-j+1) (j

respectively, and afk, bfk, cfk and dfk are certain complex numbers. Hence we have from (1), (2) and (3)

83 400 KEI TAKEUCHI AND MASAFUMI AKAHIRA

M

m

0(t) =1+E A kl tlk+Z Bkt

(4)

k=1

k=1

m

+1

I t Ik

-1+.Z Z

j

k=j

Cjk

ltlk

-l log ItI

m

E

+Z djk j k=j

tI

tlk -1 l og I 1+o(I I-) t

t

where Ak and Bk are certain complex numbers.

Let X1, X2, • • •, X,,, • • • be a sequence of i.i.d. random variables with the characteristic function c(t).

We define zn= (Y'. XL-Mn)/Vn whose b_1

characteristic function cSn(t) is expressed as (-i I t) $n(t)=E (e"zn )= 10(-t Vn )1n exp Since

log On(t)= n log cb (Vn) - i Vn t it follows from (4) that 1+k^

Ck

Itlk ( 5 ) log0n(t)=-iMnt+nZAk tIk Vn Vn nIt r+JBktI Vri (log ItI-log Vn)+ Dk tIk Vn

(log It! - log V)

+...}-1{ }2+ 3{ 1 2-...1. It is noted that since On(t) is a characteristic function B1 is a pure imaginary if not equal to zero, and that term can be cancelled out by an appropriate choice of Mn. Without loss of generality we put B1=0.

In order to obtain Vn and the leading term of ¢n(t) it is enough to consider the following cases : (i) (ii)

Al*0, Al=0, A2/0,

(iii) Al=A2=0, C1=D1=0, C2>0, D2=0, (iv) Al=A2=0, C1>0, D1=0. Since the asymptotic distribution of Z. is stable, it is seen from (5) that each case is as follows : Case M. Vn=n necessarily holds and then C1=0. If D1=0, then the leading term of On(t) is given by eA1111. If D1 =t-0, then Mn must be O(n log n). When D1 :t-0, we have the characteristic function an(t) with the leading term of the type exp {A1 I t I +D1t log I t I } which is the unsymmetric stable law of characteristic exponent 1. Case (ii).

,,=n necessarily holds and then B2=C1=C2=D1=D2=0.

84 EXTENSION OF EDGEWORTH TYPE EXPANSION 401

Hence the leading term of j„(t) is given by e A2t2. Case (iii). VV= n log n necessarily holds. Hence the leading term of 0„(t) is given by a-C2`2 Case (iv). V„=n log n and the leading term of c„(t) must bee °1l". If C1>0, then F belongs to the domain of attraction of the Cauchy distribution, and the density f (x) is expressed as

f(x)=^

a(l+x)-2+o(x-2)

as x->oo ,

P(1-x)-2+o(x-2)

as x -->- oo,

with a+i3>O. But this implies that F belongs to the domain of normal attraction of the stable law with characteristic exponent 1 (see [1], p. 181), which contradicts the assumption V, =n log n. Thus the case (iv) can be excluded from our consideration, except for the trivial case C,=0.

In other cases the leading term of on(t) is equal to 1 which is the degenerate case. When the leading term is obtained in each case, the remaining terms may be arranged in the order of magnitude. A few examples are in order to show the point. Example 1. Let X,'s (i=1, 2,...) be i.i.d. random variables with the density function f (x) given by

f(x)= 1 2(x2+1+-✓x2+1) Since f(x) is a symmetric function and f (x)- 1/2x2 as x--> oo , F(y) belongs to the domain of normal attraction of a stable law with characteristic exponent 1 ([1]). Note that X, can be expressed as XX _ (1/(1- UU) -1 / Ui)/2, where UU is distributed uniformly in (0, 1 ). Putting 1 2(1+x2)(1+ x2+s 1+x2) h() =

1 2(1+x2) 212 2 (1+1x)2

we have (6)

Ax)

1 - 1 + g(x) 2(1+x2 )212 2(1+x2) 1 2(1+x2)

1 -h(x)+ g(x) 2(1+ I x l)a

85 402 KEI TAKEUCHI AND MASAFUMI AKAHIRA

If we have m=2, 0<8 < 1 and g * (x)= 1 /2(1 +x2)- 1 /2(1+lxl)3, then (a) and (b) are satisfied . First we obtain Jo

log (

yzh(y)dy=

+i +x)- 2 1+x2 - 2 log (1+x) ,/

1 - \ 1 Fx -1/+ ( l+ x)z -1

= 2 log{( x2 + 1+x)/(x+1 )}

2 ✓1 Txz

( l+x)z

Hence it follows that xzh(x)dx=log 2 . Since 5

5

h(x)dx=1/2 ;

xh(x)dx=0

it is seen that (7) e"xh

(x)dx=2+ log 2 tz+o(tz) 2

Since

5:

g(x)dx = 2 -

2;

s:

xzg(x)dx =

2 -1 ,

we have

s: eilxg(x)dx=2- 2 -

(8)

( 2 - 1)t2

+o(tz)

2

It follows by Takeuchi and Akahira [4] that (9)

5

Ox

+I

X I)s

dx=l+tz log I t I -rtz+o(tz)

where Y is some constant. From (6)-(9) we obtain 0(t) = J W^ e"xf(x)dx z

_xa-l`1-1-t log Itl+ r tz-1 - og2t2+2-'E 2 2 2 2 2 2 2 1)tz

2\2-

+o(tz)

=1- 2 1 tI- 2 log It1+ 2 (1-7-log2)t2+0(t2)

86 EXTENSION OF EDGEWORTH TYPE EXPANSION 403 n

Letting ¢n(t) = E [exp itZj with Z"= Z Xi/n, we obtain 1

log O„(t)= n log ¢n )

2 I t i- 2n (log I t I + log n)+

(1- r -log 2)

n

8nt2+o(n)

0n(t)=e c"2'tir1- to 2n t2 2n t2log ItI -I 1 {4(1-r-log2)-7r2}t2]+o\n/ the asymptotic expansion of the distribution of Z„ agrees essentially with the Cauchy distribution. The second term in the expansion has the form 1 (a log n +b)fl(x)+cf2(x) , n where fl(x) and f2(x) are the functions that are Fourier transforms of

and (t2 log I t I )e< x/z)1tI

te e ( 2ltl

respectively. Hence fl(x) and f2(x) are equal to -g"(x) and -h"(x), respectively, where

7r

(c' 2+ x2)

with certain constants c' and C". Example 2 . Let X2' s (i=1, 2 , • • •) be i.i.d. random variables with the normal density with mean 0 and variance 1. Let Y1=1/XX (i=1, 2,- • • ). Then the density function f (y) of Yj is given by f (y) _

_ -e 1/zvz 112

Since f (y) is a symmetric function and f (y) 1/(V y2) as y-+ oo, F(x)

belongs to the domain of normal attraction of a stable law with the characteristic exponent 1 ([1]). Put

87

404 KEI TAKEUCHI AND MASAFUMI AKAHIRA

(10)

g(y) =.f (y) -

V-z (1+2y2)

If we have m=2, 0<6<1 and gt(x)=2/(-,f 7r - (1+2x2)), then (a) and (b) are satisfied. First we have ;r ; g(y)dy=1--Vyg(y)dy=0 .

(12) Further we obtain (13)

)dy y2g(y)dy= 1 J : (e-1/2v2 - 2y2 -✓2 r 1+2y2 2 -^ r

+ 2)}

°° 1 e 2 x2 --'/2- x2(2 x

(11 ,10

= V 2 ^^ (2u)- 3/2(e -u-

ro

{

1

&✓

dx

1 )du

1+u

l f o (1+1u)2 du} -2P(2)+2

_ 1+ 1 ^ v1"2(1-v)-u2dv V-7r 0 =-J+ B( 17Tr

--, --)

=-1+ -V-7r 2 From (10)-(13) we have fi(t) = e" f(y)dy

7r(1+2y)

exp

dy+5-0.e"vg(y)dy

('(t/V

-2- )X) dx+

r(l + x2)

S-

e2Cbg(y) dy

2 (-1+^2 t2+o(t2)

_*I 7r exp =1-/2 ltj+ 2 + o(t2) . 7b

Letting 0„(t)=E [exp (itZ„)] with Z,,= Xi/n, we obtain 1

88 EXTENSION OF EDGEWORTH TYPE EXPANSION 405

log0n(t)=

nlog¢Cn

42,

ItI

+C4

2

+2/n +o( n/

Since an(t)=( eXp(-

^2 It I

)) I'

+(

" + V2 +0(

the asymptotic expansion of the distribution of Z„ agrees essentially with 7c (a+2x2)} and the next the case when the density function is //{-,1term is (a+bx2+cx')/{/(1+x2)}1. Example 3. Xi's (i=1, 2,. • •) are distributed according to t-distribution with 2-degrees of freedom i.e. with the density 1

f (x) _ 2(1+x2)8/2 . Note that the distribution has no finite variance, still it belongs to the domain of attraction of the normal. First we have

f(x)=h(x)+ 1

2(1+IxI)8

where h(x) is given in Example 1.

The conditions (a) and (b) are satisfied with m=2, 0<3<1 and

2(1+I x I)' From Example 1 we obtain

fi(t)= ^`^ e'°2 f(x)dx=1 + n t2log It I _ loge +r t2+o(t2) where r is some constant .

Letting cn(t) = E [exp (itZ,,)] with Z„ _ X¢

l n log n, we have

log an(t) = n log 0C t ) -✓ nognnJ = 1 t2 log I t _ _ 1 + log log n + 2(log 2 + r) t2+o( 2 log n 4 1 log n

log n

1 11 =e (en/4) t2 lf1+ 1 t2 log It1 +or 2 log n `` n log n)l

where cn+ log log n + 2(log 2+r) log n log n

1) log n

89 KEI TAKEUCHI AND MASAFUMI AKAHIRA

406

The leading term of the characteristic function o(t) is equal toe `21' which corresponds to the normal distribution. And the second term g(x) is the Fourier transform of

2

(t2

log I t I)e -t2i4

that is, g(x) = - 2rG"(x) ,

where G(x)

_ 2 e-x 2iJ0 (c0 e

-s2dx) ex2dx+C}

with C 4 1

S __

(log

I t I )e-`21'dt

Acknowledgement The authors wish to thank the referee for his kind comments. UNIVERSITY OF TOKYO UNIVERSITY OF ELECTRO - COMMUNICATIONS

REFERENCES

[1]

Gnedenko , B. V. and Kolmogorov, A. N. (1954). Limit Distributions for Sums of Independent Random Variables , Addison -Wesley, Cambridge , Massachussets , (Translated from Russian). [2 ] Ibragimov , I. A. and Linnik, Yu. V. (1971). Independent and Stationary Sequences of Random Variables, Wolters-Noordhoff, the Netherlands , (Translated from Russian). [3 ] Petrov, V. V. (1975). Sums of Independent Random Variables , Springer -Verlag, (Translated from Russian). [4 ] Takeuchi, K. and Akahira, M. (1976). On Gram-Charlier- Edgeworth type expansion of the sums of random variables (I), Rep. Univ. Electro-Comm. 27-1, 95-115. [5 ] Takeuchi, K. and Akahira, M. (1976). On Gram-Charlier-Edgeworth type expansion of the sums of random variables (II), Rep. Univ. Electro- Comm . 27-1, 117-123. [6 ] Cram€r, H. (1963). On asymptotic expansions for sums of independent random variables with a limiting stable distribution , Sankhya, Ser. A, 25, 13-24.

90 Rep. Univ. Electro-Comm. 28-2 , (Sci. & Tech. Sect. ), pp. 259 - 269, February , 1978 259

On Gram-Charlier-Edgeworth Type Expansion of the Sums of Random Variables (III): Multivariate Cases* Kei TAKEUCHI* * and Masafumi AKAHIRA * * *

Abstract Gram-Charlier-Edgeworth type expansion of the sums of i.i.d. random variables without higher order moments was discussed in the previous papers. In this paper it is shown that the expansion is extended to the multidimensional cases.

1. Introduction The authors discussed in the previous papers ([ 11,[2],[31) Gram-Charlier-Edgeworth (G-C-E) type expansions for the sums of i.i . d. random variables which do not necessarily have moments up to the required order . Now it is extended to the multidimensional case. Suppose that X1, X2, ... , X,,, ... is a sequence of i.i .d. p-dimensional continuous random vectors. Now suppose that the density function f (x) is of the following form: f(x)= g(T) (1 + r)R+P+l where r = (xx) , T is vector of p -1 angles of the x-vector and R is a positive constant. Then the characteristic function of X is computed as

0(t)

= f

eit'x

= Jf

f(x)dx

(1 +rrpg(T) )R +P +1

exp { irph

(6,r)

} drdr,

where p= (t't)/ and 0 is the vector of angles of t. If we denote

0P ' 0 (u) = f (1 + r)-P+P+ 1 exp ( iur) dr, we get 0(t) =J ^p,Q(ph(0,T))dr. * Received on October 6, 1977 ** University of Tokyo *** Statistical Laboratory, University of Electro-Communications

91 Kei TAKEUCHI and Masafumi AKAFIIRA

260

n Therefore the characteristic function of Zn =aryl

{.f gyp, (arylPh (0,T))dr}

Xi - bn is given by

exp(-ibn't).

Then expanding Op,p (•) near the origin , as was done in the previous paper , we get the asymptotic expansion of the characteristic func+,on of Zn and thence its density. Explicit form of the asymptotic density for some special cases are given. Then the results are generalized to the case when f (x) = g1 (x) + ... + gp (x) + h (x) , where gi (x) are all above type and h (x) satisfies the condition r+p

lim Ix xI 2h (x) = 0 Ix'xI -' for some r > 0 . 2. Results Suppose that the density is of the form

f (x) = C1 f1 (x) + c2f2 (x) + ... + Ckfk (x) + g (x) where f f (i = 1, 2, ... , k) are functions of the form given in section 1 . If for some r Ixxl g(x)-+ 0 (x'x- °°) , then it is possible to extend the characteristic function

0 (t) =

f f (x) eit•xdx

around t - 0 up to appropriate order . Hence it is seen that the asymptotic expansion of the distribution of Zn holds. In the subsequent discussion we deal with multivariate t-distribution cases and more general cases. Let X1 i X2, ..., X,,, ... be a sequence of i.i .d. random vectors having a multivariate tdistribution with a density f(x)=c(1 +x'Ax) , v>0 where A is a positive definite matrix. ' Suppose that Y is a random vector having p-dimensional normal distribution with mean vector 0 and covariance matrix A-1, and vS2 is distributed like X2 with v degree of freedom , and vS2 and Y are mutually independent . Then it is seen that the t-distribution is equal to the distribution of Y/S. Since the conditional distribution of Y/S given S is normal with mean vector 0 and covariance matrix A - 1 IS2, it follows that the characteristic function of the tdistribution

92 February, 1978 On Gram-Charlier-Edgeworth Type Expansion of the Sums of Random Variables (m) 261

is given by

0(t) = E(expit'Y/S) = E [ exp ( - 2 t'A-1 t) /S2 } ]

t'A-1 t 00 SZ ° -1 1 v (2 )2 exp { - 2 (S2 + S2 )) dS2 2I'(2) ^o Hence it is easily seen that 0 ( t) is a function only through and t'A-1 t, Put 0 ( t) =0v (r), where r - (tA-1 t). Since the form of 0 (t) does not depend on the dimension p, it follows that in the case p = 1, > , (-\/ vI r I ) is the characteristic function of the t-distribution with v degree of freedom . When v is odd number , 1 v is easily obtained as follows : for r> 0

Y'3

(r) (1+r)e-r;

0s

(r) 1 + r + (r2 /3) } e-T

IP7

(r) 1 + T + (2r2 /5) + (r3 /15) ) e-T

When v = 1, Z„ - E X; I n = X is identically multivariate Cauchy. When v = 3, the characteristic function of Zn Xi/n - X is expanded as follows: r? 3 ) } n ^ e 2{ (1 + 3 v n

4

6

T{P3(^n 4n + 18n + o (n)) }

_ t 2 't (t'A 1t) - (tA-1 t

e { 1 + 3n 4n

)2 (t,A 1t)3 1

+

18n + o

(n) } (2.1)

Hence the asymptotic distribution of Z„ is normal with covariance matrix A-1. In order to get the below the second term of (2.1) we consider the case when A = I. This does not lose the generality since the general case is reduced to the above by appropriate linear transformation. Then it follows that 'I { `3(

r s5n

)

}

n^'

e

2

£ t?

1+

1 t • 2)1i2- (

Eti2 ) + (Ett1)3

3^ i ` 4n 18n

In the asymptotic expansion of the density corresponding to the characteristic function, the corresponding term to (t12 tj2 ...) exp (- 2 E t;2) is given by

93 262

Kei TAKEUCHI and Masafumi AKAHIRA a2

2

ax2 a

x

?) ... (V L

n )" exp( 2 i xi )

For the term of order n 2 putting G(x)

(2a f... ((Etj2)1/2{ exp (- 2 Etj - itjxi)}rl dti

z the corresponding term to (E ti2) 3i2 exp (- 1 Z t`.2) is equal to - E G 2 2i a x i (x). In order to obtain G (x) we consider the case when p = 2. Making a transformation t1 = T cos B t2 = r sin 0 , xl = r cos a, x2 = r sin a, we get G (x1, x2

27r IT f o r T2 exp { - 2 - iTr cos ( 0 - a) } d rd 0 ) = ( 27x )2 JoJo

1

(21r) 2

f0

n 2 r2 exp(_ T - irrcosO)dedr. J 2

Since cos ( 0 + 2) cos 0, it follows that

(' (XI, x2) =

fo f_ IT2

( 2W)2

T2

T2 { exp (- 2 - irrcos8)

I 2

+ exp (-

+ irrcos0) } dOdT

2

IT

(27T)2

n 2

1W r

T2 2

exp (-

2

- irr cos e) dTd6 .

Since for any real number y 1 it 2

2^

d2

- iTY

27T f e dr = (- dye)

1

y2

27T e 2

1

y

7=

2

= (1 - y2) e 2

it is seen that 1 IT G (x1, x2 ) _ (2^)3i2 f n -2

r2 cos20 (1 - r2 cos2 0) e 2 dO .

Putting g (r2) = G (x1, x2), we have a2 a2 a2r2 a2r2

dG

a x22

) dr2

(ax 12

+

ax22

)G =

(ax12 +

= 4g'(r2)+4r2g"(r2)

ar 2 ar d2G 2)

+{

(a x 1

+ (ax2)

d(r2)2

,

94 February, 1978 On Gram-Charlier-Edgeworth Type Expansion of the Sums

of Random Variables ( 81) 263

Consequently each term is expressed as a function of r2. When p 2 3, putting r2 = E xi2 and making an orthogonal transformation t1' = E xiti/r , i t2' = E c jt3, ... , tj' = E c1,, tj we obtain

G(x) =

l p f • • • f r { exp (r1 Fi ti'2 - it1'r) } H dti' . (2jr) 2 f

Further transforming r cos O p = t ' , r cos O p cos Op-1 = t2 ' , ... , we have D

G(x) _ (2x)p f ... f 7p 2n

= cp

f 0: f 0

1

2

II2 sin" B f I {exp (- 2 - irr cos 81 ) } dr II dBt 2

rp{ exp(- 2 - irrcos01 )} drdO,

where cp is some constant . For even number p, the following generally holds: n

p

G(x)=cp f (-1)2 ^p(p)( rcosO1)dol n _2 _X 2

where ^p (x) _ 2x e 2 Since for odd number p 2 Tr

f

11 -irx dx

0

7e- 2

x

=

17r

0

v (t) -

1

dt }

O (x)= G, (x) (say),

it follows that p-1 G(x)=cp ' f 2 (-1) 2 G1 (p-1)(rcos01 ) dOt = g1 (r2)• n _2

Since (^ ax{ G(x) = 2pgp' (r2) + 4r2 gp„ (r2), the asymptotic expansion of the density may be possible. For odd number v (? 5) the asymptotic expansion of the density may be similarly calculated. When v is even number , it may be impossible to express the characteristic function in a simple form . Putting v = 2m we have

95 Kei TAKEUCHI and Masafumi AKAHIRA

264

7 2 + T4 +•-•+ cmT2"+ logT+o(T2mlog T). 2(v-2) 4(v-2)(v-4)

For v = 2 we obtain T2+o(T2).

412(T)= 1 + 2 T2log T -

2

n log n is given by

Then the logarism of the characteristic function of Z,, = E X; l t T

T2

nlog 4/2 ( ) _ (log r - 1 log n - 1 log n)

nlogn 2logn 2 2 cT2

2logn

1

+ O(n).

For an appropriate number c,,, the characteristic function of c„ Z„ is expanded as T2

T2logT+o( e2(1+ 1 1 log n ) log n T2

2

The corresponding density to e - 2 is normal and that to (T2 log T) e" 2 may be calculated as follows : Putting t2

(log t) e 2 (cos tx) dt ; K1 (x) = 1 Ir J o

K2 (x) =

1 (

1J 0

t2

(t log t) e 2 (cos tx) d t ,

we have IE2

cp ( K, (D) (r cos B) dO = K, (r) for even number p ; .l

2 n 2

cp {

K2 (v) (r cos 9) dO = K2 (r) for odd number p ;

J 2

It follows from Takeuchi and Akahira [1] that Kl(x)_-^P(x)[

to

t2

{ Jo a

y2

2dt } e 2 dy + ci];

K2 (x) - Ki' (x) 42

K1 (x) =Jx e 2K1 (u) du + c2, 0

96 Edgeworth Type Expansion of the Sums of Random Variables ( 11) 265 February, 1978 On Gram- Charlier -

where cl and c2 are certain constants. Hence it is seen from the above that the asymptotic expansion of the density may be possible. We shall deal with more general densities . In the case p = 2, we consider that the density of Xi is expressed as c - g (a) v+1

f (x1, x2)

r(1+r2) 2 (a + if) - g (a). Putting where r - x 12 + x2 2, a = tan 1 x2 / x 1 and c is some constant, and g z = t12 + t22 and 0 - tan-1 t2 /t1 , we obtain the following characteristic function:

0

(t1 ,t2)

f

x2)

f J f (XI,

eit,x,

+tt2x2

dx1 dx2

1 =2 r eirtcos (e-a)g(a)drda J v+l n 2 (1+r2) 2 IT

(2 fit„ (,r cos (0 - a)) g (a) da. .)_2 n 2

For example a density f (x1, x2) is given by c

f (xi, x2) _

1+x14+x24

where c is some constant.

Since c

K

(x22 + x22)2

1 + x14 + x24 { 1 + (x12 + x22 ) } 2 x14 + x24

where K is some constant , we have K f

( x 1,

x2) =

(1 +r2 )2

1 cos4 a + sin4 a

On the other hand it follows that 'P2(0 _ (1+ItI)e-Itl (See [ 1 ] , page 99). Hence n O( t1, t

= ( 2 1+Ircos(0- a)lTeos ( 92) J IT cos4a+sin4a

2

e-17

a)Ida .l

97

Kei TAKEUCHI and Masafumi AKAHIRA

266 If v = 1, then n 2

^

e -In

t2)

01,

cos (9- a )Ig

(a)da

n

J

n 2

('

1

-7

I cos (0 - a) I g (a) da

J

n 2 IT

+ 2 J

(2.2)

cost (0 - a) g (a) da + o (T2) . 2 2

Though the last term of (2.2) is expressed as n T2 f 2 cost (6 -a) g (a) da n 2 IT 2 =

(COS2e coS2a + 2 cos

T2 f

Os 0 in cos a sin a

IT _2

+ sin20 sin2a)g(a)da = Allt12 +2A12tlt2 +A22t22 , IT

r f I cos (6 - a) I g (a) da = rh (B) (say)

is not done as a simple formula of tl and t2. Hence the characteristic function of E X;/n is the following form:

{ W tl , 1 } n~ a-Th (e) { 1 + 1 (A11 t12 + 2A 12 tl t2 + A 22 t22 + ... )+o (1) }

n

n

n

n (2.3)

The density corresponding to the first term of the right-hand side of (2.3) is given by 1 f*(xl,x2) -

() 2

f

e -rh(O ) e -iTr cos

(6-a) dtldt2

_ f

IT

27T2

2

(,(0 Te -Th ( e) cos

rr cos (0 - a) } drdO

2

_ 1

2 1 -r4 cos4 (

1( 27f2 J n

B-) d6 .

{h (0)2 + r2 cos2 (0 -a) 1 2

(2.4)

98 February, 1978 On Gram-Charlier-Edgeworth Type Expansion of the Sums of Random Variables (I) 267

In general the distribution of the density ( 2.4) is not Cauchy. Since the asymptotic distribution is a stable law, it is shown from the above that in two-dimensional case the stable law with characteristic exponent 1 is not always two -variate Cauchy . But for every a l and a2 the distribution of a1X1 + a2X2 is always Cauchy . The corresponding density to the second term of the right-hand side of (2.4) is given by 2

2

2

n (Atlaz 2 + 2A lz axax + Azz ax 2 )f* (xt, xz) t 1a z z Ifv= 3, then 4(t1, t2) = 1 - 2

(A11 t12 + 2A12t1t2 +A22t2)

3 + 3 J cos (6 -a) I3g(a)da+ O (r3).

Since {^( tt , tz ))^ a-2tAr r3 (B) of 1 ) ) , 3 n { 1+ 36n h + 1 nn it follows that the first term corresponds to a normal distribution . Making an appropriate linear r

transformation we may have A - I. Then the corresponding term of the density to r3 e 2 h3(0)

is expressed by 2 n r2Cos2'(B-a) , - (axe + aa 2 ) 2 ( 2 1 { 1-r2 cos2 (0-a))h(0)e 2 d9. t z t

Ifv is odd number, then similar expansions hold. Ifv= 2, then ¢(t1, t2 ) = 1+2 rt (log r)

f cost (0- a)g(a)da

+ 2 f { log I cos (0 - a) I) g (a) da

-

r2

2

1

cost

(0

- a) g (a) dot + o (r2 )

By a suitable linear transformation we have

(t1, t2)

1 + 2 r2 logr - 2 { c+h4 (0)) r2 + o (r2).

It is seen that the asymptotic distribution of E X,

n log n is normal . The term of order

1/log n is calculated by the same way as the multivariate t-distribution case.

99 Kei TAKEUCI II and Masafumi AKAHIRA

268

The above discussion is extended to a general dimensional case . Indeed , we consider the following density: c p-1 2 g(a) r (1+r 2)v/

f(x)

where r2 = x'x and a is a vector of (p-1) angles and c is some constant . If g (-a) = g (a), then the characteristic function of X is given by P(t) ° f 0., (7-, h (0,a))9(a)da.

Hence the same discussions as two-dimensional case may be possible in the multidimensional case. If v a 1, then the asymptotic distribution of EXi/n is a stable law with characteristic exponent 1 but it is not always multivariate Cauchy. For v - 2 and v 2, the asymptotic distributions of i

Xi / n log n and E Xi / Nfn are normal, respectively.

The asymptotic expansion of the density is essentially similar to the case p = 2. It is also seen that the symmetricity of the density is not essential . Indeed, if in case p - 2 a density is given by

f(x1,x2) =

q g(a) r(1 + r)v+f

Then the characteristic function is expressed as n 0(t1, t2)

Oq (rcos(0 -a)) g(a)da J-n itr

where

iq (t) =

qe dr. J p (1 + r)a+1

Then we may obtain the asymptotic expansion of the characteristic function of Z„ by the same way as the above symmetric case . When p 2 3, then similar discussions hold. In the discussion of this paper it is essential that the order of f (x) as x'x - °° is independent of directions . But we may avoid the point by the following way. Indeed , in the case p = 2 we consider a density

f(x1,x2) =

c 1+x12 +x24

Then the characteristic function is obtained by a 2 - cre'rr(9-a)

O(t 1,

t) 2

= 2 I f o 1 + r2 cos2a + r4sin4a

.2

dnta .

100 On Gram-Charlier-Edgeworth Type Expansion of the Sums of Random Variables ( ll<) 269

February, 1978 If

r

_

1

1+r2 cos2a+r4 sin4a (sin4a)(1+r)3

+ g (r) ,

g(r) is order of r-4. Since we may make such expansion in detail , we obtain the asymptotic expansion of 0 (tl , t2) around the origin. But in such cases it may difficult that standardized constants are common . In this case we have to consider the asymptotic distribution of (T,Xlf/n, Y-X21/n).

i

f

References (1] Takeuchi , K. and Akahira , M., "On Gram-Charlier-Edgeworth type expansion of the sums of random variables (I)," Rep . Univ. Electro-Comm. 27-1, (1976), 95-115. [2] Takeuchi , K. and Akahira, M., "On Gram-Charlier-Edgeworth type expansion of the sums of random variables ( II)," Rep. Univ. Electro-Comm . 27-1, (1976), 117-123. [3] Takeuchi, K. and Akahira, M., "Extension of Edgeworth type expansion of the distribution of the sums of i.i .d. random variables in non -regular cases," Ann. Inst . Statist . Math. 29, Part A, (1977), 397-406.

101

Rep. Univ. Electro-Conun. 28- 2 , (Sci. & Tech. Sect.), pp. 271-293 February, 1978 271

Third Order Asymptotic Efficiency of Maximum Likelihood Estimator for Multiparameter Exponential Case * Kei TAKEUCHI * * and Masafumi AKAHIRA * * *

Abstract For multiparameter exponential family distributions the asymptotic expansion of the distribution of extended regular (ER) estimators is obtained up to the order n 1. And it is shown that the (modified ) maximum likelihood estimator attains the third order asymptotic efficiency among ER best asymptotically normal (BAN) estimators.

1. Introduction Third order asymptotic efficiency of maximum likelihood (ML) estimators have been discussed by J. Pfanzagl and W. Wefelmeyer ([ 21), (who adopted the terminology) and also by J.K. Ghosh and K. Subramanyam ([ 1]), for cases when sufficient statistics exist . In this paper we shall establish more general results for the multiparameter exponential family, introducing a differential operator , and show that (modified ) ML estimator is always optimal up to the order n-1 among similar modified class of estimators. The method is not restricted to the exponential family and will be applied to more general cases in a subsequent paper ([ 3 1).

2. Formulation of the problem Suppose that X1, X2, ... , X„ are independently and identically distributed according to an exponential type distribution , i.e. with the density m

f(x,B)=h(x)c(9) exp {

s1(B)t1(x)} i=1

where 0 is a p-dimensional real vector -valued parameter which is supposed to be in an open set O C R°, and s,(9)'s are continuous real valued functions and ti(x)'s are real valued measurable functions. Define c* (st,..., sm) =

1 h (x) eXp

t-1

si t, (x)} dp (x)

* Re.eived on December 7,1977 ** University of Tokyo *** Statistical Laboratory , University of Electro -Communications

102 Kei TAKEUCHI and Masafumi AKAHIRA

272

as a real valued function of s1, ..., s,,,, and denote S*CRm be the set of (Si, ... , s,n) for which c* is finite. We assume that for every 0 E 0 , (s1( 0) , ... , sm(0)) is an inner point of S*. We have, obviously that c - c*-'. We assume that s1 (0 ), ... , sm (0) are linearly independent when 0 varies in O, and also that t 1(x), ... , t,,, (x) are linearly independent functions of x. (The domain of x does not need be specified) For the sample (X1, ... , Xn ), a sufficient statistic is given by T-(T1,..Tm); n

Ti - E ti (XX) I n . J=1 From the assumption T has the moment generating function given by m

E (exp i E ti Ti)

- c*(sl (e) + E1 , ... , 5m (0) + Em ) I c*(s, (e) , ... , sm (e))• And the joint cumulants of Tl's are given by

K o k ...

or _0_

alias,ask ...

I s1 - s1 (0), ... , sm - sm(0)

We assume that Ti's are continuously (w. r. t. the Lebesgue measure) distributed and that their joint density admits Edgeworth expansion. We also assume thatsi (0)'s are continuously differential four times. Now we shall define a class of estimators called extended regular ( ER) estimators which are expressed as 1 0 - g(TI,..., Tm) + n h(T1,..., Tm). where g is three times continuously differentiable and h is continuously differentiable, and both are independent of n. An ER estimator is consistent iff g (µl(0), ... , µm(0))

=

0

(2.1)

where µi (0) - EB(Ti) (i-1,.. .,m). and if 0 is a consistent ER estimator , (9 - 0) is asymptotically distributed according to the normal distribution with mean vector 0 and the variance-convariance matrix E - E E oiiai of f

where ai's are vectors defined by

,

ail

- ve (Ti, Ti)

103 February, 1978 Third Order Asymptotic Efficiency of Maximum Likelihood Estimator 273

ag (2.2) I Ti - Iii (0)

ai = aTi

It is well known that E - I-1 > 0 (positive semi -definite) where I is the Fisher Information matrix given by I =-E8 [

a2 logf(X,0)] a0a0I

= E0 [aa log f(X,0)' =Ee [Easi (T-ti)

ao

a0,

log f(X,0)]

asi ae

(Ti-it)]

_ as, asj i Qv ao a0, And also it is easily seen that the equality is attained iff

a,

1 asi

(2.3)

ao

We call an ER estimator which satisfies ( 2.1) and (2.3) an ER best asymptotically normal ( BAN) estimator.

3. ER BAN estimators Now we expand each component of 0 into Taylor's series as Ba 0. + E as (Ti - pi) + 2 E E b (Ti - Ili) (Tj - j. )

+ 1 E E E C,Ik (Ti - Ili) (Tj - µj) (Tk - Ilk) 6 i j k

+ 1 AC, + 1 E d^ (Ti - pi) + R.

where

a a2 a3 go a? =aTj

, bij aTiaTj ' Cak aTiaTjaTk

ah Aa=h (111, ... p,,, , di = aTi and all the derivatives are evaluated at Ti = µi (0) . By putting T; - V (Ti - p1), ( 3.1) can be rewritten as

-

104 274

Kei TAKEUCHI and Masafumi AKAHIRA

-V_n (0 - Ba ) = E a q'T, +

1 E E ba T' T ' 2N/n i j

+ 1 EEECj kTiTiTk 6n i i k

+ 1 Aa + 1 2, OT! + R , . v/ n n i It is shown that R& is stochastically of order smaller than n-1. We shall denote that Ya - 1`n (Oa -Oa) - Ra .

Then we have -,/ n

(e a - Oa )

- Y. = op (1 / n), moreover , we have Theorem 3.1 The joint distribution of NrT n(0a - Oa ) and Ya coincide up to the order

n-1, more precisely , for any bounded continuous function 0, we have n lim

n I E0 ,/n

(Y1 . .... , Yp) ] - E0 [0 (sn ( O1 - 0), ... ,

(ep

-e))] I = 0.

Proof. Observing that Pr { n I R' I > e } =o (1/n) for every e > 0, we can show that lim

n

n I E [exp E ita (Oa-Oa ) ] ^

a

-E[exp E ita ya] I = 0 a

uniformly in ( t 1 , . . . , tp) in any compact subset of Rp, which establishes the theorem. (QED) Corollary 3 .1. For any measurable set C C RP whose boundary is of Lebesgue measure 0, we have lim n-

n I Pe{YEC}-P0 {-Vn(0-0)EC} I 0.

From Theorem 3.1 we have Theorem 3.2. The joint distribution of'(Oa - Oa) (a = 1, • • •, P ) is asymptotically equal to a continuous distribution up to the order n-1 . And for the distribution of Ya's we have Theorem 3. 3. The joint distribution of Ya's admits Gram-Charlier-Edgeworth type expansion up to the order n-1. Proof. The theorem is established by algebraic evaluation of the joint characteristic function of Ya's, the detail of which is obmitted. Consequently the joint density of /-n (Oa. - 0a) (a = 1, • • •, P ) can be expanded up to the order n-1 if we have joint cumulants of Ya's up to the fourth.

4. Some preliminary results In order to evaluate the joint cumulants , we introduce the following differential operator.

105 February, 1978 Third Order Asymptotic Efficiency of Maximum Likelihood Estimator 275

D« _ I ap a8 , where I' R

denotes the element of I-1 and the random variable

U=,/REFij«1asi « « if a0p

(Ti -µi)

(D«s,) T;

f

= Dotv/n f(X,0)} The following lemma will be frequently applied in the subsequent discussions. Lemma 4. 1. Suppose that Ze is a function of T1, ... , Tm and of 0, differentiable in 0 and that a E0 (Z9) = f a { Z01If(xi,0) } lIdµ(xi) ae a0 i holds . Then we have Ee (U«Ze ) D«E0 (Zo) 7Ee (D«Z0) Proof. D« Es (Ze ) f (D«Zo) II f (xi, 0) II dli (xf ) + f ZeD« H f (xi, 0) n dp (xi)

Eo(D«D0 ) + f ZeD« (E log f (xi, 0 )) lI f (xi,0 ) II dp (xi) = Ee (D«Z0 ) + vrn Eo (Ze U« ) from which the theorem is directly derived. (QED) We shall use the following symbols. z f(X 0 ) - a0 log f(X,0)l

d«Ry = Eonaeae log -.5

R

- E E Q a2 S i

ti

a s;

'^(ae « a0a)(80 y

K«Rti = E8 [

8log f (X, 0) • a9

as.

=

EEEK „k

log f (X, 0) - a6 log f (X, 0)] a ti

as.

ask

( a0)( « a0)( R ae y

106 Kei TAKEUCFII and Masafumi AKAHIRA

276 Jap Y

- E E E Iaa Ipp IYY Ja,p,.y, a' p ry'

KapY

E E E Iaa' Ipp' P' Ky'p'y' Y' 3

Lap7•s = Ee [ aB a ae p B Y

ou

log

a3s, -

f (X, e) ' 8es

log f (X, e) l

as. -11

(aeaaepaery ) ( aes) ,

a2 a2

Map. ys - Ee [ - log f M O) aeyaes log f (X,0)1- lap Irys

a2Si a2s, o11 ( aeaaep ] ( aeyaes)

Na p Y6 - Ee [

log f (X, 0) •

a

a2 aeaaep ae y

log f (X, 0) • a log f (X, 0)]

aes

+ lap IY 6

kKiik

HapYS - Ee [TO

as, ask aes; ( aeaep ) (ay) (a es )

log f (X, e) , a8 log f (X, 0) • a log f (X, 0) a p Y

a log f (X, 0) ] - (,a# Lr6 +lay fps +1as lp7 aes

-

L' d=

Fi

asi as; ask as, (aep) (ae7 ) (aeb

k kKijki (a ea)

E E E E I as Ipp IYY Is s La'p'7'.6 ; a' p' ry' d'

MOO-0 - E E E E Iaa'lpp' J7Y, I66'Ma 'p'.Y's' a'p'ry'S' Nap'ys -

E laa 'Ipp' Ivry Iss'Na E E 'p'•ry's' a' p' Y' s,

Iaa 'Ipp'IYY' Iss'Ila.p,y,s. A°pys - E E E a' p' Y' s'

107 February, 1978 Third Order Asymptotic Efficiency of Maximum Likelihood Estimator

Using these symbols we have

E

DaIRy=

E

EE

=

D. (Qii

8BR aey

ag,

asi

(Daoi,)

E oi i

as (aR)

1 2;

asi

as,

O

as,

.

(Da

K i)k

i Ek

a'

aey )

(aO) (aey ) (aok aS

P o" X 2;

aa' , rE j

+

+

) ; if (Da ae) y (ae ) (ae ) (ai9 R y R

i +

as, a s,

aS

k

)

+ EIasEE v ij

a s,

aZSi

Qi;

aO a ,aeR) (aey a 2 S, l aSi)

aea"aey

300

= E laa Ka,,,,y + E Iaa Ja'R.y + E Iaa a, a a,

Ja-y.R

Da IRy=-E E IRR' Iyy'DaIR,y, R' y =-E E E IRR Iyy Iaa (Ka ,R y, + Ja,R'.y, + Ja'y'•R') R, 'y' a' -KaRy-JaR1Y-Jay•R .

aZSi a2sk E E (Da Qij) ( ao1 aey (aeb )

DaJRy.8=

+ oii (D a

a 2S. aeRaey

a2s. ( aey

a2s; a2s, + Ei F. O,; ( ao1aey) ( Da aea ) Iaa '

+

+

EEk

n2 S

Kiik (

aeRaey )

-.

S S

( aea )

a3 E laa EE ( s ; ) (as' ) i ; aea 'ao1aey aea a' E

or,

a 2S1 Iaa'

i I

) (

a2s,

(

ao1aey

aea'aes

N o,'

277

108 Kei TAKEUCHI and Masafumi AKAHIRA

278

(Npy.a,s + La,py.s + Mpy.a's)

E laa a'

as. aS). ask

DaKpys

k (Da Kijk ) + E E o;j

(a)

as;

as,

(ae) (aes

)

7 ask

( Da aep ) ( ae Y ) (aes )

as; as; ask

+ E ^. a;i (a ) (Dane (aes Y p +

EE

as; as, ask

Qii

aep)

(aey ) (Da aes )

Iaa (HH,pys +Na'p•ys + Na'y•$s

+ Na,6.py)

a' DaJpY" = E E E (Da Ipp) rrY I66 J$3' 's, p' Y' 6'

+ E E E I pp' (Da IYY') I ss ' Jp y s , + E E E IRp IYY (Daisy ') Jp,y, s,

+ EE EIppIYYlss DaJpy.s _ - E E E (Kapp' +Jap'p' +Jap'•p) IYY' Iss' Jp,y, s. p' Y' 6' E E (KaYY' +Ja1'Y' +Ja7"7) Ipp' Iss' Jp,y, s, p' Y' 6' E E (Kass'+Jas's,+Jas,'s)Ipp'IYY'Jp,y,.s, p' Y' 6' + E E E E Po PO,IYY Iss (Np +La,p,y,.s,+Mp,y,.a,s a' p' y' s' Introducing still other notations as aa' pp' . If of, p' J vy =

E E Iaa IYY Ja,p,.y, ay

Kai = E E Y a p'

Iaa Ipp Ka,p,y,

the above can be abreviated as

109 February, 1978

Third Order Asymptotic Efficiency of Maximum Likelihood Estimator

D«JRY .s = -

279

E (KORR ' + Ja1'R' + Ja1 'R) J7:-s R' E (K0`YY '+ j' + J ay '• y) JR.s ,y Y (Kas6

+ Jad's + Jas 's)Jbti

+ NRy•as+ L'RY•s +Mas•RY .

(4.1)

DaKRYS= E E E (Da IRR) IYY?S,KR,Y,s, + E E E IRR' (Da IYY) Iss'KR,,y,s' R' Y' b' ' Iaa' (Do j ss') + E E E p1313 K R, y, 6' #

"Y , 6

+ E E E IRR I77'I6s (DaKR y,s') - (KaRR' +J0'"' +JaR 'R) K'y 01 R - E

+JaY'Y' +Ja7'•7) K13

(Ka"'

,y

Y

E (Iass'+Jas'S'+Jas,'s)Ksy + Ha1YS +Na1'Y8 +Nay.R& +Na6 13 (4.2) We have DOUR= DaDRE s1Ti I (D(,,DRsi) T; - f n DRsi (Daµi) (DcDRsi) T; -fn E E (DRsi) oi, (Do, si) i i (DaDRsi) T'

- f n E E E E Ia a IRR

a'

R'

Qi! ( as ,) (as ) i ; aea, a0p,

(DaDRsi ) T; - f n E E Iaa IRR Ia R (DgDRsi ) T; - fn IaR. we shall denote Val = E (DDDR si) T .

110 Kei TAKEUCHI and Masafumi AKAHIRA

280

Then we have the second lemma: Lemma 4.2 If Up Ze and Ze both satisfy the condition of the lemma 4.1 we have E8(U.UpZe)_ DaEe(UpZe)+IapEe(Ze) Nrn-

E9(VapZe) - EB(UaDaZe)

IapE0 (Ze) - 1 E0 ( VapZe )+n D&DpEg(Z0)

- n DaE0 (DpZe) -n D, E9(DaZe)+n E9(DO, DpZ0) Proof. By lemma 4 .1, we have E0 (U.UpZe )= DOE8 ( UpZe ) - Ee (D.( UpZe)) N/-n-

EB { (Vap - I DO-E0 ( Up Ze ) - Nrn

ap

) Ze }

which establishes the first half of the lemma. The second half is derived by applying the lemma 4.1, to terms, E0(Up Ze) and E9 (Up Da Ze) in the above. (QED) We shall derive the moments of Ua and Vpy. Obviously it is shown that E(U,) = 0 ; E(Vap) - 0 ; E(UaUp) = Iap. We have Vap (Da Dp Si) T i = E E Da (Ipp' asi) Ti aep a2s=

- EZ(D aIpp) (as`)T+ EEEIaaIRp( aep ' ` i a' p' i p• E E (K app' +Jap'p' +Jap 'p) (asi) T' i

ae p

p'

+ EEElaa'lpp( i a' p'

azs,

aea,aep, )T'

ae a 'aep) ) T'`

111 February, 1978

Third Order Asymptotic Efficiency of Maximum Likelihood Estimator

281

Hence it follows that E (VapUy)- - E E (K"13 ' + Jap•p ' + Jap''p ) Iyy' lp,y, p' y' + E E E Iaa Ipp' Iyy Jo,,p,.y, a' p' y'

- Kapy - Japh' - Jay'p + Jap-y - - Kapy - Jay'p I E(VapVys) - E E (Kapp' + Jap•p' + Jap's') I1,y, . (Kyss' + Jys's'

+ Jy8 1- 8 )

E E (Kapp' + jag-g' + Jap'•) Iyy'Iss'j ,s, p,

- E E (Kyss'+ Jys's'+ Jys'•s)Iaa'Ipp'Jap

+ E E E E I aa IRA Iyy Iss Nla ,p, y'6' a' p' y' 6'

=E 61 (Ksp + J aR + J s9) (Kyss ' + J76•6' Jys'•s) -E Jap ,'p) Jys p, (Kapp ' + Jap'p' + p - E Kyss' + Jys's' + Jys''s) Jb13 s' + Map•y6

= s (K6o9 + Jaa ) (Kyss + J76'.6) - EJSR Jyss +Map.ys

(4.3)

"E(UaURUy) = Kapy ; .\I-n-E (Vap UyUs ) - - E E E (Kapp ' + Jap'p' + Japl'p) Iyyllss,K 1,y,s, p' y' s'

+ E E E E iaa'Ipp' Iyy'Iss ' Na,p, y,6, a' p' y' s' = _ E (Kapp' + Jap•p' + Jap''p ) Kypb + Nap-y6 P,

E (Ua Up Uy Us) =

Hapys+ lap lys+ Iay Ips+ Iab lpy n

(4.4)

112 282 Kei TAKEUCHI and Masafumi AKAHIRA

5. Asymptotic cumulants of ER BAN estimators We shall obtain asymptotic joint cumulants of Za's making use of the results of the previous section. Note that the asymptotic cumulants of Za's are equal to those of / (Ba - ea ) up to the order n-1. We can write

Ya=Ua+2a + nWa n

And we have

E(Ya) 2^E( Qa)+o(n) We shall denote 2 E(Qa) = µa , and

1 Z. = Ya - µa.

Then we have 1 DaZp=-Vn Da 6p -7Da{ .P = - f Iap - - Daµp. n First we shall obtain the variance of Za up to the order n-1. We have

E(Za - Ua)2 =

1 4n

1 V(Qa) + 0(n

and we have Cou(Z.,Zp)= E(Z.Qp) + °(n) E(UaZp) + E(UUZa) - E(Ua Up) + E(Za-Ua)2 + o(n 1 ),

Applying the lemma 4.1 of the previous section we get E(UaZp )= )=DaE(Zp) - E(DaZp) = Iag + 1 Damp n E(UpZ0)= Iap + 1 Dpµa. n

113

February, 1978 Third Order Asymptotic Efficiency of Maximum Likelihood Estimator 283

Hence we have Cou (Z«, Zp) = Cou (Y«, Y, ) = I«p + n D«llp + n Dpp« + 4n V(Q«) +o(n). As to the third cumulants , the following equation is referred to Z«ZpZy= 2 (U«ZpZ,y + up Z«Zy + UyZ«Zp)

2 U«UpUy + 2 U«(Zp-Up) (Z«-Uy)

+ 2 Up(Z«-U«) (Zy - Uy) + 2 Uy(Z« -U«) (Zp Up) + (Z«-U«) (Zp - Up) (Zy-U7)• Since the last four terms are of magnitude of order smaller than n-' , we have E (Z«ZpZy) = 2 E (U. Z# Z-t + Lb Z«Zy+UyZ«Zp )

2 E(U«UpUy ) + o(n). Furthermore we drive that E(U«ZzZy)= D«E(ZpZy) -,rn- E {(D«Zp)Zy}

E Z D Z n p { ( y)} «

_ - 1 (K«p'Y + Jap y + J«y•p) + o(1n ) Thus the third cumulant is given as E(Z«ZpZy ) _ - 1 ( 2K«pY+ J«p'Y+ J«'Yp+ J0 11") +o(1) n 1 ). =Q«py+0(n As to the fourth cumulants , we shall need the following equation,

114 284

Kei TAKEUCHI and Masafumi AKALURA

(UZ Z S Z +UZ Z« ZR Zy Zs - ?3 a y R aZ Z+UZ s y aZpZ+U d S Za Zp Zy)

- 1( ZZ Z 3 U «- UP p Z'Y y a y UYp s s ZP p y +U p U y« sS + p U& l a +UZ6 « + U Z- y + U«USZ«Zp) + 3 UaUPUyUS+ 3 U«(Zp- Up)(Z7- U7)(ZS- US)

+ Up(Z« - U«)(Zy- U.)( Zs- Us )+ 3 U7(Za- U«)( Zz- Up)

• (ZS- US) + 3 U&(Za - Ua)(Z1- UR )(Z7- Uy)

+ (Za- Ua)(Z1- 4)(Zy- U1)(Z6- US). The last four terms in the above are shown to be of magnitude of order less than n-I , hence we have E (Za Zp Zy Z6 ) - 3 E (Ua Zp Zy ZS + Up Z. Zy ZS + Uy Z. Zp Z6 + U6 Z. Zp Zy )

- 1 E(UaUpZyZs+UaUyZpZ&+U.UsZpZ7+UpU7Z.Z6

E(UaUpUyU6 )+o(3)

+U11 USZaZy+UyUSZaZp)+

3

Applying the lemma 1 in the previous section , it follows that 1 E(UZpZyZs )= ^D«E(ZpZyZs) -7 E((DoZy) ZpZ6 ) -

E((DaZR)ZyZ&)

Nrn

E((DaZ6)Z11Zy)

- 1 Da (2K fl-f6 JA-f•6 + JRs 1 + J7S'p) n

+ Iap+Daµp) E(ZyZ6)+(Ia7+ DaNy) E(ZpZ6) n n

+ (IaS+ 3 Paps) E(Z1Zy) + o (3)

- 1 Da(2KR7S+JRy•S+J116•7+Jysp) n + Iap E(ZyZs)+ Ia7 E(ZpZs)+ IaS E(ZpZy)

(5.1)

115 February, 1978 Third Order Asymptotic Efficiency of Maximum Likelihood Estimator 285

1 + n 1 ya Dalai + 1 pa Dalay + 1 py Daya ) + 0( n) . It also follows that E(° U0

Z8) = I' E(Z7Z5) -

n

E

(ap Z,Za)

+ 1 D 13 E (Z Z8) - n D. E ((D1Zy) Z6) - 1 D. E ( ZZ 4 Z6 ) n

- n 4E((DDZy)Z8) - 1 Dp n E(ZyD.Z8 + n E((DDDDZy)Z5) + n E((DaZy)(DpZB

+ n E((DB Zy) (Da Zs )) + n E' ((Da Dp Zs) Zy -IapE ( Z Z8) E Z 6 + 1 DaDplya n + (lay + n Dalay) (10+ n Dpla6) + (If+ n Dpl1y )(Ias+n Daµs ) + o(1). We shall show that E(VapZ7Z6 ) - E(VapU7U6 ) + 1 E(VapVyB) + E(VapV7)

1 +o(\I-n In order to prove this, it is first noted that E(VapZ,Z6 )a

E (apUyUb) + E(V UZ

)

-E(VapU U8 )+ o( 1 ).

And we have E (Uy

V ap

Z6 ) `

Dy E (

V

ZB ) - E (( Dy

1 - - E(VapDyZ8)

V. # ) Zs

)

116 Kei TAKEUCHI and Masafumi AKAHIRA

286

E

DyE( VapU5 ) -

E((DyVap ))Us)+o(7n=).

On the other hand , it holds that E(UyV.PUs) DyE(V.#Us) E((DyV«#)US)

Nf n-

E(VapVys ) + of- ) .

It follows that 1 E(UyV,,p Zs )= E(UyVpUs) +E(VapVys) + 0 () from which the desired relation is obtained. Thus we get E(UaUp ZyZs IapE(ZyZ6) - I E(VapUyUs n n E(Vap Vys) n - E(Vp Vsy)

+ Iay IRS + Iad Ipy + 1 D. D I'y n o(n). +n(Ia7Dpµs+ Ip'Daµs+ IasDpµy+ Ip'DaWy )+ Substituting (4.1), (4.2), ( 4.3) and (4.4) into the terms E ( Va p Uy U s ), E (Vap Vys ), E(Vap Vsy) andDaDpIys - -Da(Kpys +Jpy-s +J1 ), we obtain after some algebraic manipulations, E( U. UpZyZs) = I° E(ZyZ6) + I°y Ips + as I' +n (IsyDpµs+ IapDpuy+ IpyDaµs+ I

P6

Daµs)

- 1 (Hapys + 2Nap•'YS + N ay'pd + N as•py n + Npy'as

+ Nps'ay + Lag-f-8

+ L"psy + 2Map-ys

+ May-Ps + M as •P7 )

+ 1 E (Kat'y' + J a7.1' + J o`y'.y ) (K OS + J't OS + Jpy, ) 'y

n y.

117 February, 1978 Third Order Asymptotic Efficiency of Maximum Likelihood Estimator

+ 1 (Kass'+ Jas •s' + Jas •s ) (K 07 + n

Jap 'y

+ n

6'

s

287

07 + J;7 ) Js s

(2Kps+ 2Jy6 'yI+ Jyy's+ J7's'Y )+o(n).

Y'

Calculating similarly we have E(UaZZZ7Z6)= Ia1E(Z7Z6) + Ia7E(ZZZ6)

+ Ia8E(ZZZ7)

+ 1(Iy6 Da u + Ips Dalay + Ip' Dalap ) n --(2H ap7s + 2N ap•ys + 21V a7 •06 + 2N as •py + Np7•a6 n + N06'07 + N7s•ap + Lap7• s + L006 •7 + La76•0 + Maa •07 + Ma7 • ps + Map -'Y6 )

+ 1 1 (Kapp' + Jap'p' + Jap''p ) ( 2Kpys+ Jpya + Jy016+ n p, + 1 I n 7,

(Ka77'

+ Ja7i" + Ja7'.7 ) (2Kys+ Jys + Jys+

JS -j) 0' sp) y

+ ( Kass' + Jas's' + Jas'-6 ) (2%s7 + Js' + J sy+ Js-9) n s'

Substituting all those forms together with E(UUpU7Us)= 1 Hap-f6+ Ia0 I76+ Ia7Ips+ IQ s•py n into ( 5.1) we have the fourth cumulants as E(ZZZ7Z5 ) - E(ZaZp)E(Z7Zs )- E(Z.Z7)E(ZZs )- E(Z.Z6 ) •E(ZpZ7) (I«p- E( ZZp)) (lys- E(Z7Zs)) + (lay- E(ZaZ7)) •(116 E(ZpZ5)) + (las- E(Z0Zs))(Ip7- E(ZpZ7)) - 1 (H076 n

+ 2Nap•76 + 2Na7•p6 + 2Na6•p7 + 207•a6 + 2Nps -ay

+ 2Nys•7p + Lap7•6 + Lapses + Lot-f" + Lpys'a )

118 288 Kei TAKEUCHI and Masafumi AKAHIRA

+ 1 E (K«R + n

J«R ^) 2KP716+ J^s+ Js7)

+ 1 E(K«7'7 + n

J«7 '7) (2KRS+ Jy6+j 9)

J7.9 ) + 1 E (Ka"' + Jas-6' + J«s •a) (2K#7 s+ s s ' + JR'7 n a, + JR,Y.,Y' + J17.7 )

(2K«b+ J«•s+ J'«) 7' 7' 7'

+ 1 E (K's-s' + j135.5' + JRs `6 ) n s,

(2K6c'7+ J S7+ J78?)

+ 1 E (K766' n a'

(2Ka9+ Js 9+ J61k) + o(1). n

+ 1 E (K'37.7' n

+ J7S•s' + J7s '6 )

The first three terms in the above are of order smaller than n-1 We denote

0«P-7a - E (K«'''' R'

+ J«1'R' + J«(3'"P ) (2K76+ JY-a+ J1 ')

6 E E E E E E j « «'j'3P' j77'jbd'jtt' (K«,R,t, + + q,t,_R, t «' 13 ' 7 ' b ' t '

• (2K

,S,t,

+ JY,t, a, + Jb,t,,7,

Then the 4th cumulant is given by - 1 H«'37s _ 2 (N«R•76 + N«7''3s + N«"7 + N97•«6 + N'3s •«7 + Nays.«'3 ) n n

_n (L«'37 •s La"--f + L«7R +

+ 1 (p«'3°Y6 + p«7•06 + p«6•R7 + A#7•«6 + Op6.07 + p7s'a'3 ) n (5.2)

Summarizing the computations we have Theorem 5.1. The asymptotic cumulants of - (B« - 6«) (a = 1, • • •, P) up to the order n-1 are given by

E (5(8« - e« )) = µ«+ O (n) Cou(Jn(e«-9«),f(e1-es))= I«P+ n D«µQ+ n Dpµ«

119 February, 1978 Third Order Asymptotic Efficiency of Maximum Likelihood Estimator 289

+ 1Cov(Qa,QR)+o(1) ; 4n

K3(Vn(Ba-0. ),Yn (BR-Op ),Yn(By-By))` :^Qapy+ \/-n - (eR - eR ), \/n ( e.y - ey ) , V n (ed - 08

K4 (V n

o n);

))

where /3aRy and Qapys are given above. It is also shown that Theorem 5. 2. The joint distribution of (6a - Oa ) (a - 1, • • •, P) is equivalent up to the order n-1 to a continuous distribution , whose density is given as follows i

f(yl, ..., yp)

2 n )° [ exp - 2 { «

Iap ya Yp } ]

{ 1+ 1 E E E QapyH«py + 1 E yoHa 6 -in a p y 1/n of

+ 1 E E E 1 , Z E (3apy(3a'R"y,Hapya"R"y, 72n a p y a' p' y'

1 + 24n « Q E 2n + 1

« 0.0,Y6 Hapys

1 Q (laaµp + Daµp

+ 1 E E Cov( Qa, 4n a R

+ Dpµa)HaHR

Qp ) HaHR}+ o(n)

where Hap ... are (mlutivariate ) Hermite polynomials defined by H = E I , yR Hat= HHR - IaR

HaRy HHRH'Y - IaR H - IayHp- I pyHa 7 etc.. It should be remarked that the third and the fourth cumulants are equal for all ER BAN estimators up to the order n-1 .

120 Kei TAKEUCHI and Masafumi AKAHIRA

290

6. Asymptotic higher order efficiency of ML estimators Now we impose some higher order conditions on ER estimators . Either of the three conditions may be taken into consideration. a) higher order asymptotic unbiasedness : µ« = 0 , b) higher order asymptotic coordinate -wise median unbiasedness: limn

n --+ no

Pr {/-n

(0 « -0«)

<0}-

2

I=0

Then ^,t« should be expressed in terms of 3rd cumulants, i.e.

1

n« = 6 R« ««

c) higher order asymptotic mode unbiasedness : the asymptotic density has its mode at the origin again µ« should be expressed in terms of 3rd cumulants. Now we can consider the class S of estimators which are ER BAN estimators satisfying one of the above conditions. It should be remarked that for given g which satisfies the condition

a gI aT;

Ti=gi

= D« S;

we can determine h so that 1 µ« =

where b;j = aT

2

E E bi!ai .i + h(hi.... ,)Um)

a2 ; aTf g I Ti = µ;

satisfies the condition stated above . Therefore for any g there is an ER estimator which belongs to the class S. Now for the estimators in S, the asymptotic distribution up to the order n-1 is determined except for the term 1 4n Coy (Q«,Qp) Therefore we shall minimize the matrix A= {r«p} T«p = Cov(Q«,Qp) = nE(Z«-U«) (Zp - Up) + o(1). (Z« - U«) must satisfy the following conditions : E(UpUy(Z«- U«)) = DDE(Uy(Z«- U«)) - E ((Dp Uy)(Z«- U«))

121 February, 1978

Third Order Asymptotic Efficiency of Maximum Likelihood Estimator 291 - E( U7Dp(Za- U.))

= E( UUVpc,) + o (1) Kapy - Jpy'a + o (1) ; -v/n E(UpVys (Zo, - Us,)) = DjE(Vy6 (Z,, - Ua)) -E((DpVy6)(Z,,,- U«))-E(Vy6D#(Z.,- Ua)) = E(Vy6Vp,) + o (1) . Under these conditions nCov (Z,, - Uc,, Zp - Vp) is minimized (asymptotically ) when it is expressed as n(Z. - U.) - E E X«py Up Uy+E E Z E Xotpy6 Up Vy6+o(1) , ,Ia p y 6 p y

where J^py and Xapy6 are Lagrangean multipliers determined so that the conditions are satisfied. Now instead of determining explicitly X py and Xap-y6 , we shall show that the above equation is actually satisfied when g is equal to the ML estimator. The ML estimator is determined by the equations as• E ae^(T;-pi) = 0, a = 1,..., p.

(6.1)

Since (s, .... s,,,) and (Lzt , ... , i.i m) correspond one to one and the mapping is continuous, 0 is consistent provided that (s1 (01) ... s,,,(01)) * (s1(02)... sm(02)) if 01 002. Moreover if for all 0 and for any to,, (Eta

a8st

, . . . , Eta s0

) * (0, ... 0 ), it is shown that 2

-k Pr { 118 -0 11 > e } = o (n ) foranyk. Consequently we can expand the estimator around the true value of 0. Differentiating (6.1) with respect to Ti, we have

a0a aT;

ash

a2s

a0aa0p(T'-"`')

i a0p

aµj

as;

aq }+ a0p =o, Q=1,,P•

(6.2)

Putting T; = µ; we have

aoa as; -2; 4 aT; + aop

= 0, 0=1,...,p;

aB", = 2; 1113 as; aT; a0p which is to be expected since ML estimator is a BAN estimator . Differentiating (6.2) again with

122 Kei TAKEUCHI and Masafum i AKAHIRA

292

respect to Ti, we have

azea E

aT i aTi { k

+ a(aea) y aTi

a2S

{

- k

ao,aoy

aea

a2 Si

Substituting T,

-

aea

aey

ae

= µ i , aT Das' ' aSk

a 0R

k aoR ao

+ oa a0R y aTi

}

aZSk

(Tk -

µk )

- k

aµk

aeaaoRaoy

a2µk

ask

a2Sk aµk

+ aT i

ao.2 a00

k aTi k aoa a0Raoy

(aey)

111k

aSk

T - k ( k - µk) aoaZSk a aoR

a

}

aoy

a2 Si

Q

°

aOyaOR

a 2sk

aµk

aoa aoR

ao y

a

J- 0 -,y

a2I k

aeaaey =

Jay R + Ka137

it follows that - T, ( azoa ) IaR - E E ( Dasi ) (Dysi ) (JaR•y + Jay•R + 'J y• a + KaR'Y) a aTiaTi a y

2

+ (Dasi)

a2s E (Dsi) aoaaoR - 0 aoaaoR +

Hence we have z

01 D i ) (Das +JRb•y(JRy.s a - E E E IaR +Jys• R +K Ry s ) ( ys i) aTiaTi R y a 2

aR

+ IaR ( (Dysi) + Q I aopaoy ) P 'y

a2S

aopaoy

(Dysi)

We have x E IaR

R

azs' ( aoRaoy

) T; =

E Iyy-

Vay'

y + Jay R ) + E Iyy' ( K 0'0 y + JaR

(a00,)Ti

123 February, 1978 Third Order Asymptotic Efficiency of Maximum Likelihood Estimator 293

= s 1y8 V^b +^

s

I°« (Ka ',y +Jo'fl.y +J«'.y.p) UU .

Consequently,

EE ( i

3 z 00,

i

)

Ti T; Q. = E E E I°" (KQ'#y + Jpy•a') Up Uy aTfaT. y + 2EEIy6 VasUy 6y

which has the form required for minimizing solution of A. Thus if A* denotes the variance -covariance matrix of Q0, corresponding to the (extended) ML estimator and A be any other estimator in the set S , we have A - A* 2 0 ( non-negative definite). From this the following theorem is established Theorem 6 .1. Let 6* be the extended ML estimator in Sand 0 be any other estimator in Sand C by ane convex set in R° containing the origin . Then it holds that lim n - -

n[Pr{. (0* -0)e C } - Pr

{,/n (e -0)e C} ]

Z

0

We also have Theorem 6 .2. Suppose that L is bounded, negatively unimodal function around the origin, i.e., for any c the set { u : L (u) < c ) C R° is convex and contains the origin . Let 0* be the extended ML estimator in S and 0 be any other estimator in S. Then lim

n -^ m

n[E{L ('(0*-0))) -E{ L(/(0 -0)))] < 0

References

[1] Ghosh , J.K. and Subramanyam , K., "Second order efficiency of maximum likelihood estimators ," Sankhya Ser. A, 36, ( 1974). [21 Pfanzagl , J. and Wefelmeyer, W., "A third order optimum property of the maximum likelihood estimator ," Preprints in Statistics 26, University of Cologne ( 1977). [3] Takeuchi , K. and Akahira , M., "Third order asymptotic efficiency of maximum likelihood estimator in general case ," ( to appear).

124

Rep. Univ. Electro-Comm. 29-1, (Sci. & Tech. Sect.), pp. 37-45 August, 1978

Asymptotic Optimality of the Generalized Bayes Estimator* Kei TAKEUCHI** and Masafumi AKAHIRA***

Abstract For all symmetric loss functions, the generalized Bayes estimator is second order asymptotically efficient in the class A2 of the all second order asymptotically median unbiased (AMU) estimators and third order asymptotically efficient in the restricted class D of estimators. When the loss function is not symmetric, the generalized Bayes estimator is second order asymptotically efficient in the class A2 but not third order asymptotically efficient in the restricted wider class C than the class D.

1. Introduction The expansion of a generalized Bayes estimator with respect to a loss function of the type L(0)= I01 a(a>1 ) is obtained by Gusev ([4]). His result can be extended to all symmetric loss functions .

Strasser ([6]) also obtained asymptotic expansions of the distribution of the

generalized Bayes estimator.

It is shown in this paper that for all symmetric loss function the generalized Bayes estimator B„ is asymptotically expanded as (0)} +Z,(0)Z2 (0)-3J(0)+K(0)Z1(0)2 +op V -n (6n- 0)=Z1(0)+ 1 ao+13o {3J(0)+K I(0) V n 61( 0) 1(0)2 2I(0)3

where the symbols of the right-hand side are defined in the contents .

1 V n )'

And the asymptotic

distributions of the estimators are the same up to the order n-1 except for constant location shift.

Therefore if it is properly adjusted to be second order AMU and third order AMU,

it is second order asymptotically efficient in the class A2 of the all second order AMU estinators and also third order asymptotically efficient among the estimators belonging to the class D for c „=Vn, where D is the class whose element 6. is third order AMU and asymptotically expanded as cn(6n

-0)=

1(0) +c

n- 'Q(0)

and Q(0)=O,(1), E[Z,(0)Q (0)']=o(1) (i=1 , 2),

+op(c n -I)

where E denotes asymptotic expectation

and the distribution of cn(6n-0) admits Edgeworth expansion ([9], [10]). But when the loss function is not symmetric , the generalized Bayes estimator 0. has the asymptotic form :

V n (6n-0)=Z1(0)+ 1 [ao+Ro{3J(0)+K(0)} +Z,(0)Z2(0)-3J(0)+K(0)Z1(0)z] 1(0) V n 61(0) 1(0)2 2I(0)3 * Received on June 9, 1978 ** University of Tokyo *** Statistical Laboratory, University of Electro-Communications

125

Kei TAKEUCHI and Masafumi AKAHIRA

38

+I/n IZ2(0)-3J( BI(BK(BZ1(B)}+onl^n l and the existence of the last term excludes it from the class D, and it belongs to only the wider class C whose element satisfies all but E(Zt(8)Q( 6)') =o(1) (i=1,2) in the conditions of the class D ([9], [10]), and since the last term depends on the specific choice of the loss function , no uniform third order asymptotic efficiency property holds. Note that A2 D A3 D C D D.

2. Results Let (-', B) be a sample space. We consider a family of probability measures on B, 2= (Pe : Os9}, where the index set 9 is called a parameter space. We assume that 9 is an open set in a Euclidean 1-space R1. Consider n-fold direct products ('""), B'')) of (1, B) and corresponding product measures p6(n) of P6. An estimator of 0 is defined to be a sequence {On} of Bl")-measurable functions On on Xc") into 9 (n = 1, 2, •••).

For simplicity we

denote an estimator as Bn instead of {On}. For increasing sequence of positive numbers {cn} (Cn tending to infinity) an estimator On is called consistent with order 1c.1 (or 1C.1 -consistent for short) if for every c>0 and every 9s9 there exist a sufficiently small positive number 5 and a sufficiently large positive number L satisfying the following :

Pei "I {cn I On-0 I >L} <s lim sup e:le -al<8

([1]).

n-.oo

For each k = 1, 2, • • •, a {cn} -consistent estimator On is k-th order asymptotically median unbiased (or k-th order AMU) estimator if for any 9 E 9, there exists a positive number 6 such that lim sup C.-- 1 P'(-)[O.<01- 1 0 ; n-- 6 : 16-91<8 2 1= Jim sup Cnk-1

n-.oo 8:I6-91<8

P6<->{9.>B} - 2 2 1=0.

We have defined a k-th order AMU estimator to be k-th order asymptotically efficient in the class Ai, of the all k-th order AMU estimators if the k-th order asymptotic distribution of it attains uniformly the bound of the k-th order asymptotic distributions of the estimators in the class Ak ([2], [7]). If the class Ak is changed for the class C and the class D in the definition in the case when k=3, third order asymptotically efficient estmators in the class C and in the class D are defined, respectively ([9], [10]). It is genearally shown by Pfanzagl and Wefelmeyer ([5]) and Akahira and Takeuchi ([3], [8], [9], [10]) that there exist second order asymptotically efficient estimators in the class A2 but not third order asymptotically efficient estimators in the class A3. But it was also shown in [9] and [10] that if we restrict the class of estimators, appropriately, we have higher order asymptotically efficient estimators among the restricted class of estimators, and that the maximum likelihood estimator belongs to the class of higher order asymptotically efficient estimators.

We assume that for each 0 E 9 Pe is absolutely continuous with respect to a-finite measure n

Ec.

We denote a density dPe/d p by f (x, 0).

Then the joint density is given by f] f(x;, 0). i=1

n. Let 9=R1. Let In the subsequent discussion we shall deal with the case when c.=1 1Ln(u) be a bounded non-negative and monotone increasing function of Jul and z(0) be a

126

August, 1978 Asymptotic Optimality of the Generalized Bayes Estimator

39

non-negative function. Define a posterior density p.(8 I xn) and a posterior risk r(d8) by ill f (Xi, 0)1i(B)I Jifl

pn(B xn ) =

f (xi, 8)I ^(B)de^-1 ,

and rn(d I -t-) =L Ln(d-8)pn(O I x n )dO e respectively, where xn=(x,, X2, , xn).

Now suppose that lim Ln(uli/n)=L*(u) for all real n--

number u. We define

rn *(d I xn) =J L *(V n (d - B)) pn(B I xn)d8. L An estimator e n is called a generalized Bayes estimator with respect to a loss

function L*

and a prior density 7C if rn*(On Ixn)=infra*(d Ixn). deg

Then t n = V n (On - 0) may be also called a generalized Bayes estimator w. r. t. L* and 7C. Since lim inf n-.oo

Ln(d-B)p(B)dO-infS L*(Vn(d-8))p(8)d8 =0

de

9

deg

9

uniformly in every posterior density p(C), it follows that for a generalized Bayes estimator On lim n-.oo

inf rn(d I xn)-rn*(On I xn) deg

= 0.

Suppose that Xi, X2, • • •, X ., • • • is a sequence of i. i. d. random variables with a density f (x, 0) satisfying (i )-(iv).

(i) {x : f (x, 0)>0} does not depend on 0 ; (ii) For almost all x[,u], f (x, B) is three times continuously differentiable in 0; (iii) For each 0 E 6

2 02 0<1(0) =Ee logf (x, 0) } ]= -Ee [ ^ logf (x, 0)]
J(8)=Ee[ {a jlogf(X, 0)}{aelogf(X, B)}] and K(0) =EeI {aB log f (X, 0)}3] and the following holds : Ee[a03logf(X, 0)]=-3J(0)-K(8). By the following way we have shown in [7 ] that a maximum likelihood estimator (MLE) is second order asymptotically efficient. Let BML be an MLE. By Taylor expansion we have

0= E a logf (Xi, tML)

-1 ae

2 =

E

a log(Xi , i =tae

- 0) 0) + E { 06 a j logf (X i , 0)} (BML i =1

127

40 Kei TAKEUCHI and Masafumi AKAHIRA

R

' 82 logf(Xr,

0

*)}(8ML-0)2,

where Putting T„=V n (CML-8) we obtain

6

_l/n ;E aelogf(Xr,

0)+ n

^tE

aB21ogf(Xr, B)}Tn

1 Jn a 8 ae^ logf(X+, e * ) Tn 2 . +2n1^ n ll ^ Set Z1(8) =L

7

rE aBlogf(Xi,0);

Z2(0) =71, `E{ 2logf(Xi,0) +I(0) }; W(0) = 1

a 2 2 logf(X r, 8), n i=00'

I(0) =E9 alogf ( X,0)}2]

[

where

Hence the following

Then it follows that W(0) converges in probability to -3J(0)-K(0). theorem holds : Theorem 1 . ([4]). Under the conditions (i )-(iv) / V n (&ML

up to order n

-' 2

-8)= I(0))

+

2 V B) I(0))

3 321 n I(0))

Z1(0)2+0'

(1^

as n-,oo.

Put (/ML*=(ML+ K(BML) 6nI (8ML)2

Then 9ML* is second order AMU. From Theorem 1 we have established the following : Theorem 2. ([4]). Under the conditions (i)-(iv), OML* is second order asymptotically efficient in the class A2. It will be shown that the generalized Bayes estimator w. r. t. a loss function and a prior density is second order asymptotically efficient in the class A2. Let Bo be a true parameter of 0(E19). Further we assume the following : (v) i(0) is twice differentiable in 8. Then we have

pn(8 I X n)/ p n( Oo 1 Xn) =exp[log { pn(0 jX n) / pn (8o I Xn)} ] = exp E log { f (X i, 0)l f (X i, Bo) } + log {wc (0)l lc (Oo) } ]

[ =exp E logf(Xi, 0)-E logf(Xi, 0o) +logir(0)- 1og X (Oo)] [ j_1

0-G o )+ 1 { E a2 logf( Xr, Bo)}(0 -Oo)2 2 i1002

=expl l E logf(X i, 8o) }( l j_1 a0

128 August , 1978 Asymptotic Optimality of the Generalized Bayes Estimator

41

+ s 1_^ ae3log(x;, 8*)}(e-0o) 3+ ^(B) (0-e0-)+o(11 n )l =explv n {Z1(0o)}(0-Oo )+ 2 {V n Z2( Oo)-nI (Oo)}(0-Oo)3 + 6 W(0*)(0-00- )3+ 7r(e) (0-00-

)+0 V -n

where R=(XI, X2, • , Xn) and 10 *-Oo1510-00-1.

Since W(0) converges in probability to -3J(0)-K(0), it follows that P n(0 I k-)/p-(001 gn) r =exP {Zl (0o)}Vn (0-00)+ 1 v -I(6o)}{Vn ( 0-00)}2

61^ n Putting

{3J(0o)+K(0o)} {lam n (0-Bo)}3+V n(B(B0 ){1^n (0-00-)}+

t =11 n (0-Oo)

(1)

or(v nn lJ

we obtain

I .n) / p (0 1 9 n) 0

pn(0

=expl ZI(Bo)t+ 1 ^Z2(Bo)-I(Bo)^t2-3J(0o)+K(0o) t3+_ - (00) t+0-a L 2 n l/ ✓ n (Bo)ic V 6 Vn \ -n 1 /J =ex P[L

/ 1 11 Zl(B!2'2 Zl( 0o)2 -I(00-) j 7t' (0o) Z2(0o) 3J(Bo)+K(Bo) 3 2 Itn t + on1 ✓ I(Bo) J +2I(0) +1/n x(0o)t^ 2V n - t2+ 61^ nIJ

Z3 ( O3)3)[ pI 2 ( 0o) f t-Zi ( Oo)l2l ={exp I(0o) I 2I(0o) J LL ex

tc' (Oo) t+Z3(O0)t2-3J (0o)+K(0o ) t3+0-n 1 p 11n r(0o) 21/n 6 ✓ n (✓ n

={ex Z3(Bo)2J[ex

p 2I( 00-)

ll •1

I(8°)j t-Z10)12]

p 2

l

I(0)

Z2(00) _3J(0o)+ K(0o) I 1 1l + ✓ n tc( Oo)t+21/ n t2 61^n t3 + o'1 ✓ nJJ

=qn(t, 00- I fin) (say). Let

1 n = ✓ n (9n - Bo ).

Then the posterior risk is given by

( 2) rn* (en I xn) _ n pn( 0o I ±n) L* ( ln-t)gn (t, 0012n)dt. Further we assume the following :

(vi)

L*(u ) is a convex function ;

(vii)

L*(- u)gn(u + t, Go I zn ) du is partially differentiable with respect to t under the

integral sign. By (2) and the assumption (vi) it is shown that the generalized Bayes estimator i n w. r. t. L*(•) and 7r(•) is given by a solution u of the equation

(d/du)E L*(u-t)gn ( t, Oo I xn)dt=0. Since by (vii)

d °° du _-

L*(u-t)gn (t, Bo I xn)dt

129 Kei TAKEUCHI and Masafumi AKAHIRA

42

= d J L*(-t)gn(u+t, 0o l xn)dt du _.

_d °° dt L*(-u)gn( t+u, Bo l xn)du T I L*(-u)dq.n(t+u,BoIxn)}du, the generalized Bayes estimator (3)L*(- u)}dgn(tn +u, Since

ln

is obtained by a solution of the equation

0oIxn) }du=0.

t=1/ n (0-0o) and 1n=V n (Bn-0o), the solution On of (3) may be called to be the

generalized Bayes estimator. From (1) and (3) we have

(°° 0-

L*(-u)e

r.-Z,(eo l' -1(B°) +u r Zl(00) z r(Oo)} (-I(9o) jln +u- I Bo ll L ()

)2-3J(0o)+K(0o) •j1+ ^^(00) (1n+u)+Z2 (0o)(tn+u ll V n it(0o) 21/ n 6V n • (tn+u)3+oD

( ✓ n)}+ ✓ n(B(9o

)+ 11-n n+u)

n (B0)(tn+u )2+op(nn 1 )]du 3J(2VPutting

u=tn- j1(0o),

we obtain

{1

_L( )(u+&)' 1 ^c^(0o) Zl(Bo) (4) 0= L*(-u)e 2 I(0o)( u +u) +v n 7c (9o) (u+u + I(90) ) L l

[-

) 2 3J(Bo)+K(9o) u+u+Z1(0o) 3 +Zz(9o) (u+u+Z(9 2✓ n I(9o) 6 ✓ n ( I(00) ) +oy + tc'(0o) +Z2(0o) ( u + u+Z1(9o) 1^) } 1V n tc(0o) 11n I (Bo)) 2 • (u+u+ I(00))) +op()n)]du

3J(Bo)+K(9o) 2V n

_ L*(-u)e 2 { 1-I(Oo)u12v n

V-n [_I(e_1(oo)u-I(^(Bo

) (u+u)}u+u

+ 11(00))

1(9o)Z2( 0o) u+u u2+u2 +Z 1(00)2+2uu +2Zi(0o)u+2Zi(00)u 21, n ( ) I( 0o)2 I ( 00) I(00)

+I(9o) {3J6(V)n K(Bo)} ( u+u){u3+u3+

I!Be^33

+3u2u

1 +3u2Z1(B0) +3u2Z1(0o) +3u2u +3z1(00)2u+3Z1(0o)2u (00)2

1(90) I (90) 1

1(00)

Zi(9o)1 7c'(9o) Z2(00)Z2(00) Zl(00)Z2(00) +6uu I(00) J+ ✓ n r(9o)+ V n u+ V u+ I(0o)1,/ n 3J(0o)+K(9o)

Zio)2 221(00) 2Z1(00)

21^n u2+u2+ 1(00)2 +2uu+ I(Bo) u+ I(0) u}

130 August, 1978 Asymptotic Optimality of the Generalized Bayes Estimator 43

+ool V n )Idu. (I) Symmetric loss function. We assume the following :

L*(u) is a symmetric loss function about zero.

(viii) We define

-) U.

(5) co*= L*(u)uke 2 du.

(k=2j; j=0, 1, 2, ...).

I(Oo) uS

L*(u)u27+1e 2 du=0

Note that

(j=0, 1, 2, •••).

It follows from (4), (5) and the assumption (ix) that 0=-ISoou

2Z1(Bo) I(Bo)x' (6o) I(60) Z2(6o) + I(Bo) cpz+cpzu -V n it(Bo) (P2 2V n 2c92u

_____ 1 I(Bo){3J( ______6o)} ___2 Z1(6o2 60) +on(1) J + 1(00) 2 cpou + 6V n cP4+ 3cp2__ I(00) +6cpzu 1( Z_(6o) Z2(6o) Z1(eo)Z2(60) + Vn 1Pou+ I(6o)V n cQo + 1/ n X( 0o) Po 2Z1(00) l 1 l 3J(6o)+K(Oo) ( Z1( Bo)2 I(Bo) cpou }+I(Bo)2cpzu+ov V n 1. 21^ n i cpz+ I(6o)2 cpo+ Since I(6o){coo-I(6o)cpz}u =

3J(6o)+ K(Bo){I 7r/(00) {coo-I( (Bo)cp4-3cpz} Bo)cot }+ 6V ^C(6 o) 1^ n + Z1(00)Z2(eo)

1(6)1/

{ po-I(Bo)c02}

nn

I 36o)Z1(6o)2{cpo-I(6o)c02}+ov ( 2I (y n/ 6o )2 + n and

coo-I(Bo)cpz#0,

it follows that _ n'(6o) I(6o)co4-3coz 3J(6o)+K(6o) Z1 (0o)Z2(0o) (6) u 1(6o )ir(63)V n + coo-I( Bo)cpz 61(Oo )V n + 1(Bo)2V n 32100 V n6o)

Z1

n ). (6o)2+0'(1/'

Hence we have (7) in=Zl(60)+Zl,

I(Bo)

where u is given by (6). Since in=V n (On-Bo), we modify Bn to be second order AMU and denote it by On*. From Theorem 1, (6) and (7) it follows that the MLE BML* is asymptotically equivalent to the generalized Bayes estimator 0 . * up to order n-112. By Theorem 2 it is seen that On* is second order asymptotically efficient in the class A2. It follows from (6) and (7) that the generalized Bayes estimator On* belongs to the class D. Then the asymptotic distribution of B„* is equivalent to that of the MLE BML* up to order n-1. Since OML* is third order asymptotically efficient in the class D([9], [10]), 9n* is also so.

131

44 Kei TAKEUCHI and Masafumi AKAHIRA

Hence we have established the following : Theorem 3. Under the conditions (i )-(viii), the generalized Bayes estimator 6n* is second order asymptotically efficient in the class A2 and also third order asymptotically efficient in the class D. Remark : In the location parameter case,

L*(u)=u2, tr(9)=1,

the generalized Bayes

estimator is reduced to the Pitman estimator. It follows by Theorem 3 that the Pitman estimator is second order asymptotically efficient in the class A2 if it is properly adjusted to be second order asymptotically median unbiased.

( II) Asymmetric loss function. In the case when the loss function L*(•) is asymmetric we shall obtain a similar result as the symmetric case . Further we assume the following : (ix)

L*(u) is an asymmetric loss function ;

(x) There exists a c satisfying I(0,) ..

^ du=0. L*(c-u)ue 2 Then we define teo , u L*(c-u)u"e 2 du Put

u=

t„- ---

(k=0, 1, 2, •••).

and v=u+c.

In a similar way as the symmetric loss function we have from

(4) and the assumptions (ix)

and (x)

(8) v_

(eo) _+ I(0o)04-3q '2 .3J(Oo)+K(9o)+Z1(0o) Z2(Bo) I(9o)2V n 0o-I(0o)02 6I(6o)l/ n I(6o)_c (6o)V n _3J(6o)+K(9o)Z1(eo)2- 03 - Z2(90) 2I(00 ) 3 ✓ n 21011 (0.)021 1/ n _3J(9o)+K(Oo) Z1(eo)}+ov ( 1 nn ) I(eo)

Hence we have (9) tn=ZI(Bo)_c+v, I(6o) where v is given by (8). Since t„= ✓ n (6„-8o), we modify B„ to be second order AMU and denote it by B„*. Since there exists the fifth term of the right-hand side of (8), the generalized Bayes estimator B„* does not belong to D but to C ([9], [10]). However it follows from Theorem 1, (8) and (9) that the asymptotic distribution of B„* is equivalent to that of the MLE OML up to order n-'12 ([9], [10]). Since the fifth term of the right-hand side of (8) depends on the specific choice of the loss function, the modified generalized Bayes estimator B**(EA3) is not third order asymptotically efficient estimator in the class C ([9], [10]). Hence we have established the following : Theorem 4 . Under the conditions (i )(vii), (ix) and (x), the generalized Bayes estimator is second order asymptotically efficient in the class A2 but B„**(EA3) is not third order asymptotically efficient in the class C.

132 August, 1978 Asymptotic Optimality of the Generalized Bayes Estimator

45

References [ 1 ] Akahira, M.: "Asymptotic theory for estimation of location in non-regular cases, I : Order of convergence of consistent estimators , Rep. Stat. Appl. Res., JUSE , 22, 8-26 (1975). [ 2 ] Akahira, M. and Takeuchi, K.: "On the second order asymptotic efficiency of estimators in multiparameter cases ," Rep. Univ. Electro-Comm., 26, 261 -269 (1976).

[ 3 ] Akahira, M. and Takeuchi, K.: " Discretized likelihood methods-Asymptotic properties of discretized likelihood estimators (DLE's)," (to appear). [ 4 ] Gusev, S. I.: "Asymptotic expansions associated with some statistical estimators in the smooth case I. Expansions of random variables," Theory Prob. Applications, 20, 470-498 (1975).

[ 5 ] Pfanzagl , J. and Wefelmeyer, W.: "A third order optimum property of maximum likelihood estimator ," J. Multivariate Anal. 8, 1-29 (1978). [ 6 ] Strasser, H.: "Asymptotic expansions for Bayes procedures," In Recent Development in Statistics (J. R. Barra et al., Ed.) North-Holland 9-35, (1977). [ 7 ] Takeuchi, K. and Akahira, M.: " On the second order asymptotic efficiencies

of estimators,"

Proc. of the Third Japan-USSR Symp. on Prob. Theory, Lecture Notes in Math. 550, SpringerVerlag, 604-638 (1976). [ 8 ] Takeuchi, K. and Akahira, M.: " Third order asymptotic efficiency of maximum likelihood estimator for multiparameter exponential case," Rep. Univ. Electro-Comm. 28, 271-293 (1978). [ 9 ] Takeuchi, K. and Akahira, M.: " On the asymptotic efficiency of estimators ," (in Japanese), A report of the Symposium on Some Problems of Asymptotic Theory, Annual Meeting of the Mathematical Society of Japan, 1-24 (1978). [10] Takeuchi, K. and Akahira, M.: " Third order asymptotic efficiency of maximum likelihood estimator in general case," ( to appear).

133

Rep. Stat. Appl. Res., JUSE Vol. 26, No. 3, September, 1979

A Section

On the Second Order Asymptotic Efficiency of Unbiased Confidence Intervals MASAFUMI AKAHIRA* and KEI TAKEUCHI**

1. Introduction It has been known in [4] that the confidence interval based on maximum likelihood (or )}, On + [ua,2/-✓I(9n)}] is asympasymptotically efficient) estimator 6. given as [8n - [u12/'/ totically unbiased, and asymptotically efficient. The purpose of this paper is to discuss the problem of the second order asymptotic efficiency of confidence intervals, that is, to consider the asymptotic power of the confidence interval up to the order n- 1/2 in the neighborhood of the true value of the parameter. It will be shown that asymptotically unbiased, and second order asymptotically (most powerful) confidence interval at 0' = 9 + to 1/2 depends on the specified value of t, thus proving that there exists no uniformly second order asymptotically most powerful unbiased confidence interval. The situation is in contrast to the case of the point estimation where maximum likelihood (or usually BAN) estimators are uniformly second order asymptotically efficient ([5], [7]), and is related to the curvature of the parameter space as was discussed by Efron [3]. 2. Notations and Definitions Let X be an abstract sample space whose generic point is denoted by x, (B a o-field of subsets of 9C, and let 0 be a parameter space, which is assumed to be an open set in a Euclidean 1-space R1. We shall denote by (9C(n), (B(n)) the n-fold direct products of (9C, (B). For each n = 1, 2.... , the points of 9C (n) will be denoted by 5c. = (x1, .... xn) and the corresponding random variables by In = (XI, ... , Xn). We consider a sequence of classes of probability measures [P.,9: 0 e 0} (n = 1, 2, ...) each defined on (9C(n), G In)) such that for each n = 1, 2, . . . and each 0 E O the following holds: Pn o(B(n)) = P„+l.e(B(n) X 1t)

for all B( n) (=- (B(n). Let cn be the maximum order of convergence of consistent estimators ([1]). A subinterval of O, [Q (X3), 9n(Xn)] whose limit depends on the observed random variables Xn (properly measurable statistics) is called consistent of order _cn if for every t(# 0) lim lim Pn, o[9 + tcn I E [en(Xn). 9n(Xn)]) = 1. It I-- n- -

A consistent interval [9Xn), 9n(Xn)] is called an a-asymptotically unbiased confidence (oc-AUC) interval, 0 < a < 1, if for every E 0 and every t(# 0)

a

lim Pn, 9 [ 0

+ tCn 1 E

[9 (XZ), 9n(Xn)])

< 1 -

a,

a-

and for every 0 e 0 there exists a positive number 6 such that Received March, 1979. * Statistical Laboratory, University of Electro-Communications, Chofu, Tokyo 182, Japan ** Faculty of Economics, University of Tokyo, Hong8, Bunkyo-ku, Tokyo 113, Japan

99/29

134 M. AKAHIRA AND K. TAKEUCHI

30/100

l lm n-+oo

i nf e: 16-.'1 <6

Pn, e[ 9

E

L9n( Xn), 9 (1 )]) > 1 -

a.

Further an a-AUC interval [9n(X,), 9(Xn)] is called to be second order a-AUC if for every 0 e 0 and every t(:P& 0) 1 7 rim Cn[Pn, 6[9 + tcn 1 E Len(in), O (X,)]} - (1 - a)] ^ 0, n-,oo

and for every 0 E 0 there exists a positive number 6 such that lira inf Cn[Pn,g[9 E [en (Xn), O,,(1 ,)]) - ( 1 - a)] > 0. n-+oo

An a -AUC interval interval te,,(2,),

e: 1 a -s I <8

[Q*

(Z), Bn (Xn)] is called asymptotically efficient if for every a-AUC every 9 E O and every t(# 0)

lim [Pn, O(O + tc; n^oo

1

E Len (Xn), en (I,)]) - Pn,e[9 + tc; 1

E Len(in), en(X,)}1] < 0

A second order a-AUC interval [9,*(Xn), 0* (X1)] is called second order asymptotically efficient if for every second order a-AUC interval [On(XZ), 9n(Xn)], every 0 E O and every t (# 0) 77 lim Cn[Pn,6[0 + tc; 1 e [en ( XZ), On ( Xn)]} - Pn,e[9 + tc;1 a Len(in), 9n(Xn)])] < 0. X-Note that our definition of the asymptotically efficient confidence interval is different from that of Wald [10] (asymptotically shortest confidence interval in his terminology) in that a) he considered only exact confidence intervals, and b) he defined efficiency in terms of the supremum of the difference of powers of two intervals over all pairs of parameter values of 0, 0' which implies uniformity of convergence of the power function, which in turn implies the boundedness from below of the Fisher information; and both aspects limit the applicability of the theory. 3. Second Order Asymptotic Efficiency of Unbiased Confidence Intervals. We assume that for each 0 e 0 Pe is absolutely continuous with respect to a a-finite measure n

We denote a density dPe/d,u by f (x, 0). Then the joint density is given by n f (X,, 0). In the t= 1

subsequent discussion we shall deal with the case when cn =1/ n . Suppose that X1, X2, . . . , Xn, ... is a sequence of independently and identically distributed (i.i.d.) random variables with the density f(x, 0) satisfying (i)-'(iv).

(i) [x: f(x, 0) > 01 does not depend on 0; (ii) For almost all x[,u], f(x, 0) is three times continuously differentiable in 0; (iii) For each 0 e 0 0 < 1(0) = EaI / log.f(X, 9)}2] = -EBC2 logf(X, 0)] < co; (iv) There exist J

J(0) = Ee[Jd and

EB

0)}lag log fm 0)}]

r K (9)

and the following holds:

ff2 log f(X,

[d03

e[{a0 logf(X, 0)

=E

log f (X,

}3

]

9)] _ -3J(0) - K(0).

We put Z1(9) - nn M ae logf (X:, O);

Z2(0) = nn

l BZ logf(X1,

Rep. Stat. Appl. Res., JUSE. Vol. 26, No. 3, September, 1979

0) + 1(9);};

135 ON THE SECOND ORDER ASYMPTOTIC EFFICIENCY

W (0)

101/31

33 =

n r^'tl

log f (Xn 0).

Let 00 be arbitrary but fixed in 0. Consider the problem of testing hypothesis H: 0 = 0o against alternative K: 0 = 00 + to 1/2 (t # 0). By the fundamental lemma of Neyman and Pearson it follows that the most powerful locally unbiased test function has a rejection region of the following type :

II f( X,, 00 + to -1/2) =1 „ > A o + A ltn - 1/ 2

r

f(X, 00) r=1

a0 log f(X1, 0), (3.1)

where Ao and Al are certain real numbers , and if it is actually (grobally) unbiased it is the most powerful unbiased test function. By Taylor expansion we obtain

n f(X,, 0 + to -1/2)l H f( X:, 00)

t=

=

1

exp

(=1

{logf(Xi, 00 + to -1/2) -logf( XX, B0)}] CZ i- 1

= exp

^^ aB log Ax" 0o) + Zn t3

Yep log f(Xi, 00)

81

+ 6n.1 n s i x03 log

f(Xr, 0*)},

= exp [tzl (00) + t n [Z2(00) - ✓ n I(00)) + t n W (9*)I = exp {- 2 I(eo) + tZ1 (00) + 5.Z2(0) + Z;7 n_ W(0*)}, where

10*

-

01

< tn-1 /2. Since

W(0) converges in probability to -3J(0) - K(0), it follows that

II f( X,, 00 + tn -1 /2)/III f( Xi, r= 1

00)

r =1

(3.2)

=exP [- 2 I(eo) + tZ1 (00) + 5=Z2(Bo) - 65 n [3J(0o) + K(00)}/+ or\

✓

= e (t!/2)I(9o ). erz,(9o ) I 1 + t2 Z2 (0o) - t3 [3J(0o) + K(00)) + 0P \N ^J•

L 2V/n 6.,/n n First we consider the problem up to constant order. It follows by (3.1 ) and (3 .2) that there exists the solutions Z, and Zl (Z1 < Z1 ) of the following equation of Z1(00): e (t=/2u(00)etz,( e,) = A0 + A1tZ1(00). (3.3) Then it follows that e(t'/2) I(eo).etz 2(9o) >

Ao

+

A1tZ ,(00)

if and only if Z,(00) < Z ,, Z 1( 00) > Z 1, It can be easily seen that in order to make the oc-A UC interval we have to take Z1 = -u.12-✓I (02) and Z1 = ua/2-/I(0o) (See [4]), where ua/2 denotes upper 100x/2-percent point of the standard normal distribution. In order to determine Ao and Al we substitute Z, = -u,,,/2'✓I(Oo) and Z1 = ua/2,%/I(0o) in the equations

e-(0/2) I(eo).etzt =

Ao

+

A1tZ1.

e-a=/2),(eo).etz, = Ao + A1t21. Then we have e-W /2)I(9o).e- t„a1, ✓I(eo ) = e- (t /2)I(eo).etu-"'1 Leo ) =

AO + A 1tud /2N I(0o). Al + Altu. 12'✓I (00).

Hence we obtain A0 = e-(t=/2)ne,) cosh (tum/24/1(00)); Rep. Stat. Appl. Res., JUSE. Vol. 26, No. 3, September, 1979

136 M. AKAHIRA AND K . TAKEUCHI

32/102

1 e- (t=1 2)1(8=) Binh (tua /z✓I (6o)) •

21 =

ua/y./ I(Oo)

If there exist B„*(X„) and 6„*(X„) such that Z1(en (Xn)) = u.12,,/I(en X,,)) Z1(6n (Xn)) = -Ua/2_ ' / I (6a (X n)) ,

then the o6-AUC interval [6„*(2,), 6n (X,)] is asymptotically efficient . Hence we have established the following: Theorem 1. Under the conditions (i)-(iii), the tx-AUC interval is asymptotically efficient. We can actually construct the asymptotically efficient oc-AUC interval using the maximum likelihood estimator (MLE) 6ML. We put

en (X

n) = 6ML + n 7-

6n (XR)

= 6ML + -)^n

We have Z1(en)

=

Zl

6ML

7n

V

Y r1

1_

Z1 (6 ML)

'+'Oy(;/ n

= ,V / n Z2(6ML )

_ -

-

Y^I (6ML) + 0p

I

+

o

((

1

l

1 l \^/nl

Since Z1(en ) = u112/I(0'),

it follows that -Y/I(6ML) + 0p( 1

^n

Ua/2'`' I (6 ML) + 0, ( / n ) •

Hence we have

Y•

= -

ua/2 / - I(6ML)

By a way similar to the above we obtain U.//2 _ / I (V ML)

Hence we have established the following: Theorem 2. Under the conditions ( i) - (iii), the x -A UC interval 6ML - Ua/2 , OIL ML +

✓I(6ML)n

_ a/2 UU

✓I(6ML)ni is asymptotically efficient. Next we consider the case up to the order of n-1 /2, i.e., second order case ([6]). We put ZI(60) = ZI

+ n

in the following equation:

e- ( 9 1 / 2 ) 1 ( 0 9 ) ,erz,(e,> rl +

22^

Z2(60) -

6^

[3J(6o) +

We have [ Rep. Stat. App!. Res., JOSE. Vol. 26, No. 3, September, 1979

K(6o) )]

J

=

2 0 + 11tZ1(6o) (3.4)

137 ON THE SECOND ORDER ASYMPTOTIC EFFICIENCY

10 +Alt(2,+Ann1 = e a=iz>I,e :tz, +toi^n»

{1

+ -Z2 - 3J + K)}

n

Al {1 + Z2 -(3J+ K){. n n //

l

= e'(t

=/2) I,erz ,

(1 +

n

It follows by (3.3) and (3.4) that q A e-rz3+(t =/2)I =1+ t = + 1 + Alt

103/33

n

2

t

Z2 -

3

(3J + K).

t

1/n ,/n 21/n 67n

Hence we obtain 2

Z2 - (3J + K) .

t(1 - A l e-92, +0 712)I) 12

(3.5)

Putting Z1(90) = Z 1 + --AL in the equation (3.4), we similarly have n

2

3(

6

)

(3.6) Z2 3J + K }. t(1 - lel tz, +(t=i2>I) { 2 The rejection region like the type of (1) is given by _AL nn < Z1, Z1(00) - ^n > Z), (3.7) Z1(00) - _ where A and A' are given by (3.5) and (3.6), respectively. We have

Z1(0o) Zl(0o)

= Zl(0o) + .^ nn Z2(0o) - '

- nn - n

= Z1(00) + n Z2(00)

n - -1-

where Cl =

q

t

t2(3J + K)

2(1 - hle-tzi+(t /2)I)' c2 = 6(1 - Ale-tz .+(012)I)'

t2(3J + K)

t

Cl 2(1 - Ale tZ, +(t=/2)I)'

C2 -

6(1 - Ale-tZ^+tt=^2>I)

It follows from (3.7) that the rejection region like the type (3.1) is given by Z1(00) + n Zz(0o) < Zl + n

Z)(00) + n Z2(00) > 21 + C2 n

where cl, ci, c2 and c2 are given above. We put

Al = {: zi(eo) +.zz(oo)>2i + n A2 = f Xn : Z1(00) + Z2(00) < Zl + / .

In order to construct the second order a -A UC interval we have to determine Zl and 21 such that PP.eu(Al) + Pn.ea(A2) = a + o\ In '

(3.8)

d9Pn.eo(Al) + ^BPeo(A2) = of In

(3.9)

If XA denotes the indicator function of the set A, then (3.8) is given by En.6o(

XA)

+

En..0(XA =)

= a +

o(n

)-

(3.10)

Since /Pn. e(A) =

9)} du (n)

J

XA(n)

J

XA( n){- log tII AX I , 9) - II f (xi, 9) dju(n)

Id110

^ f(xt+

('

80 =1

i=1

Rep. Star. App!. Res., JUSE. Vol. 26, No. 3, September, 1979,

138 M. AKAHIRA AND K . TAKEUCHI

34/104

J

X A( Xn){

a9

log f(xt,

= En.B[XA(Xn)tE =1 ae

0)}tf[ f(x , 9) d ju(n)

logf(x1,

e)],

where p(n) denotes the n-fold direct product measure of 4 u, it follows that (3.9) is given by En,eo [ XA,(Xn) Z 1 (9o)l + En,eo[XA,(Xn)Zl(9o)l

(3.11)

= o(n) •

Putting Z1 (00) = Z1(00 ) +

-

=Z

2(90);

l Z*

*(90)

= Z1(00 ) + n- Z2(00),

we have from (3.10) l o( n [XA^(9,,){Z*(0o) + En,e,[XA ,(xn)

{Zl *(o o)

3n Z2(00)}]

(3.12)

5

- j n Z2(e0)}]

En,eo[XA,(. ,)ZI(e0)] - % En , eo[XA, (Xn)Z2(e0)]

+ EE,eo [XA,(Xn)Zl

*(00)]

En, Bo[XA ,( Xn)Z2(e0)l

- /

Since

En.e,[Z1(0o)Z2(eo)]

En,eo[Zi(0o)l = I(0o);

En,Bo [ Z2(00)]

= J(0o);

= M(00) - I2(00),

where

M(9) =

E0[

laez

log Ax, 0)} 2],

it follows that Vn,0o (Zl(00))

= EE.eo [ {Zt(00) +

=

Covn,eo(Zl(

En.e,[Zi(eo)]

00), Z2(00))

+

n Z2(0o)} 2] (3.13) n

n E ,0[Z1(90)Zz(0o)] + o

( .7-n

= EE,ea[Zi (eo)Z2(00)l - EE.eo[Zi (0o)]En.eo[Z2(90)l

=

En,eo [ Z1(00) +

nn

Z2(0o

(3.14)

} Zz(00) ]

n En.e,[Zi(Bo)] = En,eo[Z1(e0)Z2(00)] + c = J(90) + nn [M(00)

V12(00)1 + o(1 n

From (3.13) and (3.14) we have En,6o[Z2(00

) I Z1(00)] = Coyn,eo(Zl( 00), Z2(0o))Zi (eo) 1(e0)) VV.Bo(Z*

Hence we obtain _ En,eo[XA ,(Xn)Z2(e0)] = En,o, [XAi(Xn)En,eo(Z2(0o)) I Z1 (BO))l ` En , 6o[XA ,(Xn)Zj*(eo)] + °(^ n )

By a way similar to the above we obtain

j(9°

Rep. Stat. Appl. Res., JUSE. Vol. 26, No. 3, September, 1979

(3.15)

139 ON THE SECOND ORDER ASYMPTOTIC EFFICIENCY

Ee.eu[XA,(Xn)

E.eo[XA.(x,,.)Z1 *(90)] + o(J nn 1 •

Z2(e0)]

= I(00)E Form (3.12), (3.15) and (3.16) it follows that j1

105/35

(3.16)

/

I(eo(n E,,.e.[XA,(X»)Z*(o0)] + 1 - I(Bo), ✓ n En.e.[XA.(X.)Zi *(eo)] = o(n •

(3.17)

Itt follows by Edgeworth expansion that the probability density function f(x) of Z*(90)/Q, where a' = V,,,j.(Z*(9o))((3.13)), is given by / f(x) _ O(x) + n H3(x)O(x) + 01 n ) where O(x) = -L-e- x'/2 and Y3 is the third order cumulant of Z*/Q and H3 (x) is a Hermite polynomial , i.e. H3 (x) = x3 - 3x. Since f xH3(x)q(x) dx = f x(x3 - 3x)o(x) dx = f (x4 - 3x2)O(x) dx = f [(x4 - 6x2 + 3) + 3(x2 - 1)}O(x) dx = f [H4(x) + 3H2(x)} j(x) dx = f H4(x)((x) dx + 3 f H2(x)o(x) dx _ {-H3(x) - 3Ht(x)}O(x)

= (-x3 + 3x - 3x)o(x) = -x3/(x), where H1(x) (i = 1, 2, 3, 4) are Hermite polynomials, it follows that f xf(x) dx = f x {ct(x) + 5i.=H3(x*x)}dx + o(^ nn l

(3.18)

= f xo(x) dx - 6^3n f xH3(x)o(x) dx + of n = -O(x) -6 x30(x) + o 3(.1-n

Since by a way similar to the above , the probability density function g(x) of Z**(9o)/a'', where Q'2 = V 00(Z**(00)) = I(e0) + - nn J(Bo) + °\n)' is given by

where fl' is the third order cumulant of Z**(9o)/a ', it follows that (3.19)

f xg(x) dx = -O(x) - 6^3n x30(x) + o(- n ) •

Putting kt =Zt +^n;

ki =Zt +-/ -n

we have At = [Z* > kt}; A2 = {Z** < k;}. By (3.18) we obtain En.Ba[XA,(Z)Z*(B0)] _ [

(3.20) - 0 (a :0 /

6

P3

(Ok,

(L,

k

Rep. Stat. Appl. Res., JUSE. Vol. 26, No. 3, September, 1979

140 36/106 M. AKAHIRA AND K. TAKEUCHI

By (3.19) we also have EE,oQ[XAI (Xn)

Z **(Oo)J

= [_O(6r) c /3n (Qr )30(.L

(a, ^n Since

N3

= ri =

J

(3.21)

k'

k' )1. I

(Q)3' J

K(9o), it follows by (3.17), (3.20) and (3.21) that I117 )E^,eo(XAiZ*) + (1 - I")En .eo(XA1Zi * )

(3.22)

n){0(a)+ n 'Sal "Sa K

1.%I

(1 1../n)l^(Q)+6-✓ n \a')3^\o•' c2 =(1 -1c/n )

l +6. / n'Sa)3}"( Z l

-(1 Ian){ 1

a

a'`/n)

+61/n k,

From (3.22) we have Z1 +^n Z' +n = (1 -1.

( ^%n){1 +6.V/n ( (a')

3

(3.23) j I^ n) ll + 6 ✓n Ql)31 , 3 c'J K k, c1J - K (kl 3 1+I./n n+61/n(a'') Wn(Q)

^^

✓nWa^ 3-(a3} +n

=1 +cl./ n'J+

it follows from (3.23) that Since a2 = a'2 = I(90) + o r n// (3.24)

- 1 S (Zi + n) 2 - (Zi + n) 2} log {1 + (c1I I1

;_

+ 613/ k/ n (Z'

- Zi){ .

(c'I )J+C,61K (Z' +o ( ln) n

n

Putting Zl = ua/2

-

,/2 IV/

-✓

I(Bo) + n and Zl = -ua./2,^/I(Bo) + n, we have from (3.24) y^ uz/2 •I + ?I ' + C2 + c2

1/I -/n = (c1 n1)J + 613/K n

(I3 /2ua/2 + 13/2ua12)•

Hence we have + (c, - c)J (' u,^(/'2.^/ I 3

ua./2 - C2 - c 2 .

Since

J f( ) dx = J x

{O(x) + 6 K H3(x)O(x)} dx

_ fi(x)

-6

n

J

H3(x)o(x) dx

Rep. Stat. App/. Res., JUSE. Vol. 26, No. 3, September, 1979

(3.25)

141

ON THE SECOND ORDER ASYMPTOTIC EFFICIENCY

107/37

= (D(x) - n H2(x)O(x) = (D(x) - _ n (x2 - 1)0(x); f g(x) dx = O(x) - WK- (x2 - 1)O(x),

where fi(x) is a standard normal distribution function, it follows from (3.10) that a + 0(-n 1 = EE,Bo (XAi) + En, 90(XA,)

K + a / - W n (a12 - 110(at Hence we have Zl

Zl

.+ C2

CI

{

l r 1 -ac + °n ^( Q^n) - c` Q I n) (3.26)

+ 6,,/ n 1(QzZ - 1)o\Q' / - \a2 - 1/\ 61/} Since we put 21 = ua/2 .s/ I + 1 n

ZI = -ua/2✓ I +

1/n

we have from (3.26) ua/z / I +

(D

(

1/

Cz

n

+

-ua /2,

I

+

C2

+

^'

l

n )=1-OC+° (\n/. (3.27)

` Since

J

`

J

Q = ✓I(eo) c'I(9)n + o(^n); Q'=-✓I(9o)+ c1(8) +01^n^ it follows that

Q n

= (ua/z + \

CI

= u a/z - ua /2

n

^ j1

- I^ n + °\ %I-n)

C-J

I1/ n + c^/In + °n

-u./2'V I + C2 + 11

n ciJ c'2 + fj' 1 ^, _ ua/z + ua/2 I n + In +0 (^/

Hence we have \ua/z^,/I

-ua/2✓ I

+

+ c2 +_tJ l ( l In ^ )(-ua/z ^ c2 + ^/ \ In /

C2

+

17'

/

^( n ) = 2 + In O(ua/z) (U-1 I' J + c2 I') n + °1 ln

From (3.27) we obtain 2) - ua/2 (c1 + c'i)J + c2 -C'2 + 11-11 ' 1 0 (ua/ ,V -n ( I -'1-T / = ° \n

(3.28)

Rep. Stat. App!. Res., JUSE. Vol. 26, No. 3, September, 1979

142

M. AKAHIRA AND K. TAKEUCHI

38/108

Since by (3.28) C2 - c'2 + t7 - 17' = ua /z(c) + c'i)J ^/1 I

it follows that 17 - 11' = ua

/2(c) + ci)J - c2 + c'2. ./ I

(3.29)

From (3.25) and (3.29) we obtain ua/2(c, + c',)J - (c, - c,)J,/ I _ K,%/-I 2 6 UM/2 - c2; n i17 -T 2ua/2 tl' ua/2 (c, + c',)J - (c, - c',)J.%/ I - K,,./-I u,/2 - c2. 2^7 2ua/2 6

(3.30) (3.31)

where Cl = C

t __ t2(3J + K) 2(1 - ),l e luais ✓ ! +(6 /2)I)' C2 6(1 - ^le tua/s✓ 7 +(ts/2)!)> t t2(3J + K)

99 , 2(1 - h,elua:1i+(tsi2)I)' c2 = 6(1 - /I,,etua/s./!+(t>/z)!

with e (t'/2)'sinh (tua /2N I) + 0(1/,%/ n ).

^1 = 1 ua /2N

I(e0)

Hence we have Z, = ua/21/ I + nn ; Zl = -U./2N/ I + n where tJ and 17' are given by (3.30) and (3 . 31), respectively. We also obtain kl = ua /z ✓ I +- C2 + 17. k' = -ua / 2N I + C2 + t1'. ,,7 1/ n

We put

u(0; t) = c2 + tl; 1(0; t) = c2 + 17'. If there exist 0**(9„) and 0* *(.Xn) in the neighborhood of maximum likelihood estimator (MLE) 0ML such that

+ ^%Z2 (0 n *(X,)) = ua/z ✓I(en *(xn)) + u(0_**(nn); t) 1 (0 nn); t )n );t Z,(0 **(xn)) + 'n Z2 (0 *(xn)) = -ua / 2^✓I(#n *(xn)) + 1(0 Z1(Q* *(zn))

(3.32) (3.33)

([2]), then the second order VV a -AUC interval [B* *(2n), 0* *(. )] is second order-.asymptotically efficient . Hence we have established the following: Theorem 3. Under conditions (i) ' (iv), the second order a-AUC interval [B**(xn), 0* *(zn)] is second order asymptotically efficient. We shall actually construct the second order asymptotically efficient a-AUC interval using the MLE OML. We put e* *(xn) _ ua/2 + N I (0ML)n nr

ua/2 Bn *(xn) =BML +

+

S

N I( BML)n n

we have ZI(Bn *) + n Z2(en *)

=

Zl (

S'

p^ ua/2

ML O

+

n) .^/ n Z2(BML

" I ( ML)n

Rep. Stat. Appl. Res., JUSE. Vol. 26, No. 3, September, 1979

ud/2 N

I(0 ML)n

+8'I+vP(1)

143

ON THE SECOND ORDER ASYMPTOTIC EFFICIENCY

109/39

u^/2 + yr Zj(eML) + 1 (- ua/2 + ^r)2Z1(eML)

- \ N I( 6ML)n n 2 \ I✓I(eML)n n

+ - Z2(OML) + + r)Zy(eML) + 0,( W n ^7 ( - ^7I(OmL)n n n Since Z '1(eo) = Z2 (00) - N/ n I (eo); I'(eo) = 2J(eo) + K(eo), it follows that

Z'(eo) = Zz(e0) - 4/ n I r(eo) - -.v/ n J(90) - J n {2J(9o) + K(90)) _ -./ n {3J(eo) + K(eo)). We also obtain

Zi(e0)

n ri a 33 logf(X

" 90) + V, n Ir(6o)

0e3 logf(X, eo ) + N/ n {2J (eo) + K(e0)) n ri NI-W[-3J(00) - K(e0)) + N/ n {2J(e0) + K(00)) _ -J(eo)'1 n . Hence we have Z1(en

*) + /nz2(en *) 'V

r ua/2 +

( - `- Ice )

n

+ ua12

{-

✓2I (OML)n

(Z2(eML) - N n I (e ML)^

^/ n (3 J(9ML)

+ ua/2 (1nJ(OML)) + 01

".I(ML)fl = UM/2N I(OML) - na/2

+ K(OML))) + ' Z2(eML)

(

\✓ n

1nZ2(e ML) + / Z2(OML)

'I(e M /zJ(OML) + opN n ua/2 / (3J(OML) + K(GML)} + Clua 2I(OML)- n N146ML)

Since ✓I(B"*)= I +op\,In u(9 *, t) = u (OML; t) + 0p^

it follows by (3.32) that \1 1 us 1 22

Z2 (OML) J(OML) + _ = Z2(OML) - n.2 /2 _(3J(OML) + K(OML)J

vI( eMt)n V :V 2I(9ML)NI n Clua

+

/2J (

OML)

-ua/2(2J(OML) +

K(

OML))

2 I (#ML)n

II(9M nn

Then we have Vr = Cl

- -/ua./2 Z2 (OML)

+ Clua

/2

J(OML)

(3.34)

'. I(9ML)" J(VML) I(OML)3/2

Rep. Stat. App!. Res., JUSE. Vol. 26, No. 3, September, 1979

144 M. AKAHIRA AND K. TAKEUCHI

40/110

_ Ua/2 53J(OML)+' K(#.,,,) + 2J(OML) + K(OML) (p^ 2I(BML) I I(#+ V ML) ^I(BML)

In a similar way to the above we obtain u «/2 \

ML)

Z2Z(OML) _ J(e IUa/2 I(OML)3/2

/ l + /I(OML)) I(OML) C

f 3J(O ML) + K(OML) + 2J( OML) + K(ML) 2I(9ML) ^ I (OML) I (eML) ^ . Hence we have established the following: Theorem 4. Under conditions (i)- (iv), the second order a-AUC interval nS ,

(3 (3.35)

\\ Ua /2

CB

ML - Ua/2

+

, BML + "a/2

(3.36)

-✓I(eML)n n '✓ I(BML)n n

is second order asymptotically efficient, where 6' and 6 are given by (3.34) and (3.35), respectively. Remark. It should be noted that both of 6' and 6 depend on t and Z2(.). The second order CL-AUC interval (3.36) is most powerful but not uniformly most powerful because of the dependence on t. This testifies the fact that the asymptotically sufficient statistic up to the order n 1/2 is given by the pair (9ML, Z2(9ML)) ([9], also [8]), while in the case of point estimation 9ML is alone asymptotically efficien up to the order n l/2, i .e. second order. References [1] Akahira, M.: Asymptotic theory for estimation of location in non -regular cases, I: Order of convergence of consistent estimators, Rep. Stat. Appl. Res., JUSE, 22 (1975) 8-26. [2] Akahira, M. and Takeuchi, K.: Discretized likelihood methods-Asymptotic properties of discretized likelihood estimator (DLE's), Ann. Inst. Statist. Math., 31 (1979), Part A, 39-56. [3] Efron, B.: Defining the curvature of a statistical problem (with applications to second order efficiency), Ann. Statist., 3 (1975) 1189-1242. [4] Lehmann, E. L.: Testing Statistical Hypotheses, John Wiley (1959). [5] Pfanzagl, J.: Asymptotic expansions related to maximum contrast estimators, Ann. Statist., I (1973) 993-1026. [6] Takeuchi, K.: Asymptotic expansions for the critical limits of unbiased tests, Discussion paper No. 131, Kyoto Institute of Economics Research, Kyoto University. (1978). [7] Takeuchi, K. and Akahira, M.: On the second order asymptotic efficiencies of estimators, In Proceedings of the Third Japan-USSR Symposium on Probability Theory (G. Maruyama and J. V. Prokhorov Eds.) Lecture Notes in Mathematics 550 (1976) Springer-Verlag, Berlin, 604-638. [8] Michel, R.: Higher order asymptotic sufficiency, Sankhya, 40 (1978) 76-84. [9] Suzuki, T.: Asymptotic sufficiency up to higher orders and its applications to statistical tests and estimates, Osaka J, Math., 15 (1978) 575-588. [10] Wald, A.: Asymptotically shortest confidence intervals, Ann. Math. Statist., 13 (1942) 127-137.

Rep. Stat. Appl. Res., JUSE. Vol. 26, No. 3, September, 1979

145

Rep. Stat. App!. Res., JUSE. Vol. 26, No. 4, December, 1979

A Section

Remarks on the Asymptotic Efficiency and Inefficiency of Maximum Probability Estimators MASAFUMI AKAHIRA* AND KEi TAKEUCHI**

ABSTRACT Suppose that X1(i = 1, 2, . . . , n) are distributed according to a two-sided truncated normal distribution with unknown mean 0 . It is shown that the maximum probability estimator (MPE) of Weiss and Wolfowitz is asymptotically inadmissible and has smaller concentration of probability than the asymptotically efficient estimator BPT = (min X, + max X1)/2.

1. Introduction Weiss and Wolfowitz ([7]) defined the concept of the maximum probability estimator (MPE) which V MP is defined as the value of which maximizes B+rc, l n

II f(xi, O) dO

B-rc,. ' t=1

with appropriate choice of the sequence of constants (cn). And they proved that asymptotically PF,O(cn 1 Bn - 01
r - 01
for r'=Ar

and it is not ruled out that or may behave badly outside the fixed interval (-r, r). In regular cases, #r is asymptotically equivalent to the maximum likelihood estimator (MLE), and is asymptotically independent of r. But in non-regular cases OMP does depend on r (which is also emphasized by Weiss and Wolfowitz themselves), the asymptotic behavior of or is not well investigated. In (2], [3], [4] and [5] we defined one-sided and two-sided asymptotically efficient estimator in nonregular cases, as the one which maximizes lim P.,9[cn(Bn - 0) < a) for a > 0 n,w

or lim Pn,g[cn(On - 0) > -b)

for b > 0

and lim P e[cn 10n - 01 < c}

for c > 0

n-

Received 4 July, 1979 * Statistical Laboratory, University of Electro-Communications, Chofu, Tokyo 182, Japan ** Faculty of Economics, University of Tokyo, Hongo, Bunkyo-ku, Tokyo 113, Japan 22132

146 REMARKS ON THE ASYMPTOTIC EFFICIENCY

133/23

uniformly in 0 and a or b or c among asymptotically median unbiased (AMU) estimators which satisfy the condition lim Pn, 9[On <

0 1 = lim Pn,B[O& > 01 = 1 2

n-•.

uniformly in 0. Then the question is: if or is AMU, then is it asymptotically efficient in the sense above? The purpose of this paper is to show that the answer is not always yes, even in the case when asymptotically efficient estimator does exist, implying that the MPE may asymptotically inadmissible. The case discussed below is also covered by Weiss and Wolfowitz [8] and their Z. is basically asymptotically equivalent to 0, and at least as good as Or but they did not show explicitely the inadmissibility of the latter. 2. Results Let (S, (13) be a sample space. We consider a family of probability measures on CB, tP _ [PB: 0 E 0), where the index set 0 is called the parameter space. We assume that 0 is an open set in a Euclidean 1-space R1. Consider n-fold direct products (X(n), &n)) of (9C, (13) and the corresponding product measures Pn B of PB. An estimator of 0 is defined to be a sequence [On) of (Bcn1-measurable functions On on X(n) into 0 (n = 1, 2, ...). For simplicity we denote an estimator by Bn instead of [Vn}. For an increasing sequence of positive numbers f cn} (cn tending to infinity) an estimator On is called consistent with order f cn) (or (cn}-consistent for short) if for every e > 0 and every $ E 0 there exist a sufficiently small positive number (5 and a sufficiently large positive number L satisfying the following: lim sup n-+Po B: 18 -91
P., etc. 1

0n

-

01

> L} < E ([1]).

A [cn)-consistent estimator On is called asymptotically median unbiased (AMU) estimator if for any $ E 0 there exists a positive number 0 such that

O < 01 - I 0;

llm sup

P., Of n n-•^ B:I 8-91
lim

> 0 1 - I I = 0.

sup

I Pn,B10n X- 0:10-0 1<45 2

A distribution function FB(t) is called to be the asymptotic distribution of cn(On - 0)(or en for short) if for each real number t, Fe(t) is continuous in 0 and for any $ c 0 there exists a positive number d such that at any continuity point t of FO(t) lim sup n-o 0:I0-91
I Pn,B[Cn(

0 n - 0) S t) - FB(t) I = 0.

Let X = O = R1. Suppose that X1, X2, ... , Xn, ... is a sequence of independently and identically distributed random variables with a density f(x: 0). Suppose that 0 is a location parameter, i.e., f (x: 0) = f (x - 0). Further we assume that

Ice- (x'12) for l x i < 1; f(x) = 0 for l x i > 1, where c is some constant. In this case it is shown by Akahira [1] that there exists a consistent estimator with order [n), which is the largest possible order. Let R = (-r, r). The maximum probability estimator 61 of 0 is defined as d maximizing d+r/n n

II f (xi - 0)d0

d-r/n 1=1

(Weiss and Wolfowitz [7], [9]), and in this case O , is shown to be Rep. Stat. App!. Res., JUSE. Vol. 26, No. 4, December, 1979

147 MASAFUMI AKAHIRA AND KEI TAKEUCHI

24/184

-2rt; (min X1 + maxXX)formax X, - minXX > 2 min X, + 1- nfor X > min Xt+1- n ,max Xt -min Xt<2- nr; r MP =

maxXX-1 +nfor X < max X1 -1 +

n

, max X, - min Xt< 2 - nr ;

r

XformaxX,-1 } n<X<minX,+1-n , where ,k = - E Xt. n t., Remark. The definition of Or in the first case is not unique, but it does not affect the subsequent conclusions. First we note that the densities of the asymptotic distributions (asymptotic densities) of min Xi + 1 and max X, - 1 are given by u Ike ku for u > 0; 2.1 ) g,O = - t0 for u < 0; ( u _ lkekn for u < 0; 2.2) gz() - t0 for u > 0; ( respectively, where k = ce 112. Now we consider another estimator O. = (min Xi + max Xt)/2 which is asymptotically equivalent to the Pitman (best location invariant) estimator. Since min Xt + 1 and max Xt - 1 are asymptotically independent, the density of the asymptotic distribution of OPT = (min Xi + max X,) is given by g(u) = ke 2k " n i. (2.3) Next we shall obtain the asymptotic density of OMP. The asymptotic conditional density of (min Xt + max Xt) given max Xt - min Xi > 2 - 2r/n is obtained by

k 2ktai - ezkr) for Jul < r; hl(u) =I1 - K(r) (e Illl 0 for i ui > r; e-2kr. ) K(r) = (1 + 2kr where It can be shown that Xis asymptotically independent of min Xt and max Xt, or more precisely /X is asymptotically normally distributed independently of n(min Xt + 1) and n (max Xi - 1). Hence it follows that the asymptotic conditional density of min Xt + 1 - r/n given X > min X, + 1 - r/n and max Xt - min Xt < 2 - 2r/n is obtained by e-2kr

for I u I < r;

h4u) = k( n+r) K(r) a

for I u I > r.

Further the asymptotic conditional density of max Xi - 1 + r/n given k < max Xi - 1 + r/n and max Xt - min Xi < 2 - 2r/n is obtained by k -2kr K(r) e

h (u)= s k

K(r)

for I u I < r;

e k^' -0 for u < -r.

Since the order of the probability of max X, - 1 + r/n < X < min Xt + 1 - r /n is 1/n, the probability of X given max X, - 1 + r/n < X < min X, + 1 - r/n is negligible in this case. Hence the asymptotic density of #r is given by ke 2kl uI for Jul < r; g *(u) = I 2 e- k(lnH+r) for I uI > r. Rep. Slat. App!. Res., JUSE. Vol. 26, No. 4, December, 1979

148 REMARKS ON THE ASYMPTOTIC EFFICIENCY n

Let Pn,0 be the probability measures with the density

II f (XI r =1

-

135 /25

0).

Next we shall get the lower bound of Pn,tal n 1 9n =

-

0 1 > u}

Pn.B{ n(Un

-

0)

- u} +

^

ff

PP,o l n( Un

-

0) > u}

(2.5)

for all AMU estimators. Let u be arbitrarily positive but fixed and 6. be AMU. Let 00 be any fixed in O and put 01 = 00 - (u/n) and 9z = 90 + (u/n). Since for each real number t Pn,B{n(9n - 9) < t} converges uniformly in a neighborhood of 00, it follows that for sufficiently large n ^ u} '„ PP.90[n ( n - 9o) < u}; Pn.B,t9n :!z^ 001 = PP, ojn(Un - 91) F,,B,[Un ^

90}

=

Pn, e,1n(Un

O - 92) :< -u} - Pn,o,tn(Un - 90) :<

-u }.

In order to get the lower bound of (2.5) it is sufficient t find the maximum value of Pn,B, Un

<

90}

of

-

Pn , B,1 Un ^

00 1

(2.6)

for AMU estimators O. Let B. = 1xn • Un(xn ) S 90} and 0. be an indicator function of B. Then it follows that ...

J

^1n( n ) =II f(x, - 90) ^IJ dxr = EE , 90(On ) --^ (n -p co).

We also have 7 7 77 7n^' 1 f (Xi, 01) t i dxt 0n(xn ) 1 l jl f(xt, 02 ) t

Ir

(2.7) = En, B,(Y'n) - En.B,(^n)• In a similar way as the proof of the If the maximum value of (2.7) is obtained, then it is that of (2.6). fundamental lemma of Neyman ad Person, it is shown that 0. maximizing (2.7) is given by 11 f(Xi,

y'n ( n) = 1=1

0,

02)

II f (x ,, U2)

1=1

- 11 f(Xi,

9 1)

t=1

>

A

II f(xt, 90) ;

i=1

- II f(x r, 01) < A H f(x i, 90), 1=1

f=1

for some A. Since 91 and 02 are symmetric with respect to 90, it is clear that A = 0. Hence

1, if

II f(xi, 92) > [f f(x,, 01), that is,

i=1 i=1

f(xt, 02) > 0, II f(Xi, 01) = 0; (a) 11 i=1 1=1 or (b)

> 0, II f(xr, 0 1 ) II f(xt, 02)1-1

> 0 and x < c';

t=1

0, if II AXI, 02) < II A xi, 01), that is, r=1 r=1 (a') tH f(xr, 92) = 0, II f(Xi, 01) > 0; or 1, 02) > 0, II f(xi, 01) > 0 and x > c', (b') 1 IT f(x1=1 n

where z = - xt and c' is some constant n i=

Putting Rep. Stat. App!. Res., JUSE. Vol. 26, No. 4, December, 1979

149 26/136 MASAFUMI AKAHIRA AND KEI TAKEUCHI

A=

xnlmaxx;-1>O0- n,minxi+ 1>00

u n

B= {Xnlmax xi- 1
u u} C = {zn I max x -1 <00- ,minx;+1>Oo+ n n D={znlz

c')

we have 1, if x,, AU(CnD); )-^0, ifznEBU(CnD'). (2.8) Since X, max Xf - 1 and min X, + 1 are mutually asymptotically independent, it is seen that the conditional probability Pn,O{D I C) of D given C is given by for sufficiently large n Pn,g{D I C) = Pn,g{D}

In order that we get En,00(^n) -s 2

(n oo)

we must have PP,go{D } _,

2 (n -, co).

It follows from (2.1), (2.2) and (2.8) that for sufficiently large n En,91(^n ) -

Ea,0 (Yn )

Pn,O :(A)

+

2

Pn,o ,(C) -

PP,e,( A )

-

n,O,(C) 2 P

(2.9)

Pn,O:(A) I - e-2ku

From (2.5), (2.6), (2.7) and ( 2.9) it is shown that lim Pn, An I On

-

0 I ^!! u} ^i! e2ku

(2.10)

for all AMU estimators O. Hence the right-hand side of (10) is the desired lower bound. Therefore it is shown that the bound of the asymptotic distribution of n I On - 0I for AMU estimators 6. is given by lim P,,o(n n--

I On

-

0I

< u} < 1 - e 2k u.

(2.11).

Since both of the asymptotic densities of OPT and 6. 1p are symmetric, it is easily seen that these estimator are AMU. From (2.3) and (2.4) we have lim P,,o(n I OPT - 0 I < u} = 1 - e 2ku;

(2.12)

n-•oo

lim P,, O(n I OMP -

0I

< u} = 1 -e2ku for 0
(2.13)

Hence it is shown by (2.11), (2.12) and (2.13) that Opr is two-sided asymptotically efficient in the sense that the asymptotic distribution of OpT attains uniformly the bound given by the right-hand side of (2.11) but 0 , is not so. Remark that in this case the MLE is asymptotically (also in small sample) uniformly (in u) inadmissible which fact can be established in the following way. It is easily seen that Rep. Stat. Appl. Res., JUSE. Vol. 26, No. 4, December, 1979

150 REMARKS ON THE ASYMPTOTIC EFFICIENCY

137/27

1min X,+1 ifX>minX,+1, OML=

max Xi-1 ifX<maxXi-1,

X if max X,-1 <X<minX,+1 and asymptotically, it is equivalent to glmin X, + 1 with probability 1/2, n _ - tmax X, - 1 with probability 1/2.

Since the asymptotic density of J. is symmetric, it is easily shown that 6 ML is AMU. Thus the asymptotic distribution of n I OML - 0 1 is easily seen to be 1imPn,B{nIOML- 01 < u) = 1 -e k" n-_

which has twice as large as large scale as the asymptotic distribution of OPT, the asymptotic efficiency of 0ML compared with OPT is exactly 50 %! It should be also noted that OML is the limit of #i,, as r 0, and that O;K O. is inadmissible outside the interval -r < u < r, and that the interval converges to the null set when r goes to zero. On the order hand when r becomes large, the interval of u in which 0M , is admissible extends and the probability outside the interval also concentrates to the center, and as r -• oo, Ont, approaches to OPT which is uniformly efficient. Thus in this case the choice of larger r is always better than the choice of smaller r, which, although remarkable enough, does not hold in the other cases. It is also noted that the Pitman estimator is the generalized Bayes estimator with uniform prior and quadratic loss, but OPT is asymptotically equivalent not only to the Pitman estimator but also to any generalized Bayes estimators with symmetric loss function ([6]).

We have defined an AMU estimator 0,* to be one-sided asymptotically efficient in the sense that the asymptotic distribution of n(0* - 0) uniformly attains the bound of the asymptotic distributions of n(dn - 0) for AMU estimators 0n([3], [4], [5]). We have showned that the bound of the asymptotic distribution of n(On - 0) for AMU estimators On is given the following density: ke k l" 1 for J u I< k log 2; 1 0 for u I > - log 2, k (See the proof of Theorem 3.1 of [2]). Comparing (2.3) and (2.4), we may also conclude that OPT and 0MP are asymptotically equivalent for I u I < r but for I u I > r OPT is asymptotically better than o , in the sense that the asymptotic distribution of OPT is uniformly nearer to the bound than that of OMP. Though OM , and OPT are not one-sided asymptotically efficient in the sense that their asymptotic distributions uniformly attains the above bound ([3], [4], [5]), it is seen that OPT is asymptotically better than 9MP.

REFERENCES Akahira, M.: Asymptotic theory for estimation of location in non-regular cases, I :Order of convergence of consistent estimators. Rep. Stat. Appl. Res., JUSE 22, (1975) 8-26. Akahira, M.: Asymptotic theory for estimation of location in non-regular cases, II:Bounds of asymptotic distributions of consistent estimators. Rep. Stat. Appl. Res., JUSE 22, (1975) 99-115. Akahira, M.: On the asymptotic efficiency of estimators in an autoregressive process. Ann. Inst. Statist. Math. 28, (1976) 35-48. Takeuchi, K.: Tokei-teki Suitei no Zenkinriron (Asymptotic Theory of Statistical Estimation) (in Japanese), Kyoiku Shuppan, Tokyo. (1974) Takeuchi, K. and Akahira, M.: On the second order asymptotic efficiencies of estimators. Proc. of the 3rd Japan-USSR Symp, on Prob. Theory, Lecture Notes in Math. 550, (1976) 604-638, Springer Verlag. Rep. Stat. Appl. Res., JUSE. Vol. 26, No. 4, December, 1979

151

28/138 MASAFUMI AKAHIRA AND KEI TAKEUCHI [6] Takeuchi , K. and Akahira , M.: Asymptotic optimality of the generalized Bayes estimator . Rep. Univ. Electro-Comm ., 29, (1978) 37-45. [7] Weiss, L. and Wolfowitz , J.: Maximum probability estimators . Ann. Inst. Statist . Math ., 19, (1967) 193-206. [8] Weiss, L. and Wolfowitz , J.: Generalized maximum likelihood estimators in a particular case . Theor. Probability Appl., 13 , (1968) 622-627. [9] Weiss, L. and Wolfowitz , J.: Maximum Probability Estimators and Related Topics. Lecture Notes in Math. 424, (1974) Springer Verlag.

Rep. Stat. Appl. Res., JUSE. Vol. 26, No. 4, December, 1979

152 Ann. Inst. Statist. Math. 31 (1979), Part A, 39-56

DISCRETIZED LIKELIHOOD METHODS-ASYMPTOTIC PROPERTIES OF DISCRETIZED LIKELIHOOD ESTIMATORS (DLE'S)1' MASAFUMI AKAHIRA2) AND KEI TAKEUCHI

(Received Feb. 8, 1978; revised Feb. 13, 1979)

Abstract Suppose that X1, X2, • • •, Xn, • • • is a sequence of i.i . d. random variables with a density f (x, 0). Let cn be a maximum order of consistency. We consider a solution Bn of the discretized likelihood equation Z log f(Xi, On +rcn1)-^ log f(XX, On) = an(Bn , r) where an(0, r) is chosen so that On is asymptotically median unbiased (AMU). Then the solution On is called a discretized likelihood estimator (DLE). In this paper it is shown in comparison with DLE that a maximum likelihood estimator (MLE) is second order asymptotically efficient but not third order asymptotically efficient in the regular case . Further it is seen that the asymptotic efficiency (including higher order cases) may be systematically discussed by the discretized likelihood methods. 1. Introduction Recently second order asymptotic efficiency has been studied by Chibisov [4], [5], Pfanzagl [8], [9], Takeuchi and Akahira [3], [11], Efron [6], Ghosh and Subramanyam [7] and others. Furthermore third order asymptotic efficiency was discussed in Takeuchi and Akahira [11], [12], [13] and Pfanzagl and Wefelmeyer [10]. In this paper using the discretized likelihood method we consider the asymptotic efficiency of estimators including higher order cases. Suppose that X1, X2,. • •, Xn, • • • is a sequence of i.i.d. random variables with a density f (x, 0). Let cn be a maximum order of consistent estimator of 0. We have proposed a solution On of the discretized likeli1) The results of this paper have been presented by the first author at the meeting on "Asymptotic Methods of Statistics" at the Mathematical Institute in Oberwolfach of West Germany, November 1977. 2) Partially supported by Sakkokai Foundation. 39

153 40 MASAFUMI AKAHIRA AND KEI TAKEUCHI

hood equation n

n

logf(Xi, 0n+rcnl)-Z logf(Xi, On)=an(On, r) i=1 i =1 where an(0, r) is chosen so that Bn is asymptotically median unbiased (AMU) (the possibility of which will be shown in the context) [3]. Then the solution Bn is called a discretized likelihood estimator (DLE). If for each real number r, n

n

log f(Xi, 0+rcn1)-Y log f(Xi, 0) i=1

i=1

is locally monotone in 0, then the asymptotic distribution of the DLE On attains the bound of the asymptotic distributions (discussed below) of AMU estimators of 0 at r. It is easily seen that there is at least one estimator which attain the bound. In regular cases with cn= 4/ n the left-hand side of (1.1) is expanded as

(1.2)

e iZ log f(Xi,

en)

+ 2 ^r

a_

iZ

log f(Xi,

en)

+ ... =an(on, r) .

We derive from (1.2) the order to which the maximum likelihood estimator (MLE) is asymptotically efficient. In this paper it is shown that an MLE is second order asymptotically efficient but not third order asymptotically efficient. The motivation for the definition of the DLE is that ; when we test the hypothesis 0=00+rc; 1 against 0=00, the most powerful test is given by rejecting the hypothesis if n

n

E log f(X1, 00 +rcn1)-E log f(Xi, i=1 i=1

00)
hence if an estimator Cr is defined so that the event Br>00 is equivalent to the above inequality (at least asymptotically up to some order), then ©r is efficient (asymptotically up to some order) for specified choice of r. Therefore if ©r can be defined independently of r, then it is shown to be efficient (asymptotically up to the above mentioned order), and if not, we can establish that there does not exist any efficient (in the same sense) estimator. It is also seen that the asymptotic efficiency (including higher order cases) may be systematically discussed by the discretized likelihood method.

2. Notations and definitions Let (a', _B) be a sample space. We consider a family of probability measures on 9B, ' = { PO : 0 E 61, where the index set © is called the

154 DISCRETIZED LIKELIHOOD METHODS 41

parameter space. We assume that ® is an open set in a Euclidean 1space R'. Consider n-fold direct products (_'In), _B In,) of (X, _B) and the corresponding product measures P,,,0 of P0. An estimator of 0 is defined to be a sequence {Bn} of B`n)-measurable functions B. on _'In) into © (n=1, 2, ... ). For simplicity we denote an estimator as B„ instead of [On)

. For increasing sequence of positive numbers

infinity) an estimator

en

{ cn} (cn

is called consistent with order

tending to

{ cn}

(or

1c.1

-

consistent for short) if for every s >0 and every 9 E (9 there exist a sufficiently small positive number satisfying the following : lim sup n-> oo 0:

10 -81 < 5

a

and a sufficiently large number L

([1})

Pn,e {cnl0n-0I >L}
.

For each k=1,2,..., a {cn} -consistent estimator k is kth order asymptotically median unbiased (or kth order AMU) estimator if for any 9 E 0, there exists a positive number 8 such that lim sup n-->oo

0:

cn-'

10 -V1<5

Pn,0

=0;

{0„50} 2

lim sup cn-' Pn,l On >0} - I 0 2 n->O3 0:10-81
-lI pn

0{

cn ( 0n -0)
- G0(t,

0)- cn1(t, 0)- • • • -cn(k-1)Gk_l(t, 0 )1 =0

We note that Gti(t, 0) (i=1,-,k-1) may be generally absolute continuous functions, hence the asymptotic distributions for any fixed n may not be a distribution function. Suppose that en is AMU and has the kth order asymptotic distribution G0(t, 0)+c; 'G1(t, 0)+ • • • + c; (k-1 )Gk_1(t, 0). Letting 00 ( E 0) be arbi-

trary but fixed we consider the problem of testing hypothesis H+ : 0= (t> 0) against K: 0=00. Put p1/2 = { {l5 } : En, 0u +tcn1(cn) =1/2+ 0(c.; `k-1 )), 0 <0n(xn) < 1 for all xn E 'Cn) (n=1, 2, • • • )} . Putting Aa n , o0= 00 +tcn'

{cn(dn -00)St}, we have lim Pn, 0o+tcnl(Aen, 00) =1im Pn, 0o +tcnl {B

Oo +tcn'}

.

= 2

Hence it is seen that a sequence {XAan Bo} of the indicators (or characteristic functions) of Aen, o0 (n=1, 2,.. •) belongs to 01/2. If

155 42 MASAFUMI AKAHIRA AND KEI TAKEUCHI

sup lim c;-' {En,eo(¢n) -H+(t, B0)- c;,'H+ (t, Bo)( n) E P, /2 n-'°°

-cn (k-i>H+

i(t, Bo)} =0

then we have G0(t, Bo )< H+(t, Bo) ;

and for any positive integer j (;;5 k) if G(t, B0)=Hi+(t, 60) (i=1,• • •, j then

G,(t, e0)=H; (t, Bo) . Consider next the problem of testing hypothesis H- : 0=0o+tc;' (t
(on) E 01/ 2 n-'oo

m(k-1) Hk-i t, Bo)} =0

then we have G0(t, 0,)Z Ho (t, 0o) ;

and for any positive integer j (S k) if G(t, 90) = Hj-(t, 00) (i = 0, • • • , j then G;(t, 90)>H, (t, 00). 9n is called to be kth order asymptotically efficient if the kth order asymptotic distribution of it attains uniformly the bound of the kth order asymptotic distributions of kth order AMU estimators , that is, for each 0E H+(t, 0)

for t>0 ,

t Ht (t, 0)

for t<0 ,

G#, 0)=4 i=0, • • •, k-1 ([2], [11]). (Note that for t=0 we have G,(0, 0)=Hz+(0, 0) =Hj-(0, 0) (i=0,• • ., k-1) from the condition of kth order asymptotically median unbiasedness.) We assume that for each 0 E ® PO is absolutely continuous with respect to o-finite measure ,u. We denote a density dP5/da by f(x, 0). Let L(6; xn) be a likelihood function , that is, L(6 ;

_ f f (xt , 0), where .T~in = (x,, X2 ,** • , xn). For i=, each k=1, 2, • • • , a {cn } - consistent estimator Bn is called discretized likelixn )

hood estimator (DLE) if for each real number r, Bn satisfies the discretized likelihood equation (2.1) log L(Bn+rcn' ; xn) -log L(B,, ;

xn)=an (Bn,

r) ,

156 DISCRETIZED LIKELIHOOD METHODS 43 where an(0, r) is a function in 0 and r and it also depends on n. The function an(0, r) is not defined for the moment but will be determined in the sequel so that the solution obtained from the above equation be asymptotically median unbiased up to kth order . It should be noted that DLE Bn is required to be {cn }-consistent and we do not claim that the solution of the equation (2.1) be 1c. ) -consistent . We implicitely claim that there exists a solution of the equation in the 0(cn')-neighborhood of the true value. In the practical situation we have to obtain the DLE first by finding a {cn} -consistent estimator Bn in some way or another then find a solution of the equation in the neighborhood of On.

Suppose that for given function an(0, r),

(2.2) log L(0+ rcn' ;

xn)

- log L(O ;

xn)

- an(0, r)

is locally monotone in 0 with probability larger than 1- o(cn
cn-'IPn,B{en > 0-rcn' } - Pn,e[log L( 0, xn)

n -iC0 II: 1O-81
-log L(O -rcn',

'Cn) >an(e

-rcn', r)} 1 =0 .

Letting 00 (E 0) be arbitrary but fixed we consider the problem of testing hypothesis H : 0 = 00 - rcn,' (r> 0) against alternative K : 0=00. Putting Ae o= {cn(6n- 6)>-r} we have Pn ,o0 -rcn ' (Aen,o0)=1/2+o(cn'k-") Let QU k be the class of the all kth order AMU estimators . Set 012= ( {on }: En,eo- rc n ' (cn)= 1/2+o(cn`k-"), 0 o with the estimators Bn in `U k is contained in 0,/2. In SuffiPn,o0(Aen, o0) in CUk it is order to obtain the upper bound of lim ,

cient to find a sequence

{on }

of the tests which maximize HE E n,00 n -

(o

n)

It is shown by the Neyman-Pearson fundamental lemma that has the rejection Sn satisfying 01/2•

in

on

n log f( XX, 60-rcn')
where kn is some constant .

f(Xi, 00)

Then it follows from (2.3) that the upper

157 44 MASAFUMI AKAHIRA AND KEI TAKEUCHI

bound of lim Pn,BO(Aen,6O) in

V, n-.w

is given by lim En,6O n--

(on*).

Hence the kth

order asymptotic distribution of the DLE BDL attains the bound of the kth order asymptotic distributions of kth order AMU estimators at -r. In a similar way as the case when r>O, we also obtain for r<0 the upper bound of lim Pn,e0(Agn, e0) in V, of the same form. Hence the desired result also holds for the case r<0. In later sections it will be seen that BDL is asymptotically efficient up to second order. Note that the DLE usually depends on r in cases more than third order. In the subsequent discussion we shall deal with the case when cn =II/n.

3. Second order asymptotic efficiency Suppose that X,, X2i • • •, Xn, • • • is a sequence of i.i.d. random variables with a density f (x, 0) satisfying ( i) -(iv).

(i) {x : f (x, 0) > 0} does not depend on 0; (ii) For almost all x[p], f (x, 0) is three times continuously differentiable in 0; (iii) For each 0 E (9 0
{-j_-B - log .f (X, B)} 2]

E[

, 0)] < 0

( iv) There exist J(0)=Ee

[{ ae2-

log f(X, 0)} {

log AX, 0)} ]

and K(B) = EB L j aB

log .f (X, 0) I l

and the following holds : EB [_! log f(X, 0)] _ -3J(0)- K(B)

ao,

By the following way we have shown in [11] that an MLE is second order asymptotically efficient . Let B. be a maximum likelihood estimator. By Taylor expansion we have

O= tiZae log f(Xz , BML) =tE

0) ae log f(X^, +

{ae2 log f(Xi, B)} (BML-B)

158 DISCRETIZED LIKELIHOOD METHODS 45

E

a$

+1 2 ti=1 {ae2 log f(X£, o *)} (BML-B)2 , where 10*-015I0ML - 01. Putting T„=%/n(BML-o) we obtain U= 1 E a log Ax" 0)+ 1 {E aE log f(X" B)} T. t=1 ao n $=1 ao2 1 + 2n ^l n

a2 ae2

g f (X" , lo

o*) T„ .

Set Z1(o)= 1 E a log f(X" o) Vn =1 ao

Z2(o)=__L E { 1 n a2 log f(Xs, o) W(B)=- E n 1=1 aB Then it follows that W(o) converges in probability to -3J(6)-K(0)Hence the following theorem holds : THEOREM 1 ([11]).

Under conditions (i) ~ (iv)

) - 3J(o)+K(o) Zl(o)z+o^( 1 1 (9ML-0)= Zo(o) + f Z,(o)Z2(0 /nf I(0) N /W I(19)2 2,,/WI(O)-

(3.1)

up to order n- 1/2 as n-> oo. Put

BML

-BML+

K(&

)

6nI(0ML)2

Then 6ML is second order AMU. From Theorem 1 we have established the following : THEOREM 2 ([11]). Under conditions (i)-(iv), BML is second order asymptotically efficient.

In the sequel we obtain the same result for DLE. We further assume the following : (v) For given function a„ (o, r) log L(o+rn 1"2, z„)-log L(o,

is locally monotone in 0 with probability larger than 1-o(n 1). Remark.

In usual situation it is generally true since (1/n) (a2/ao2) .

159 MASAFUMI AKAHIRA AND KEI TAKEUCHI

46

log L(9, xn) is asymptotically equal to -I(9) (<0) and an(e, r) is smaller order than n-1, and is usually of constant order (0(1)) as is shown below. Let Bn be an DLE. Since n

n

Z log f(XX, en+rn-1/2)-j log f(Xi, en)=an ,

i=1

t=1

it follows by Taylor expansion that n

n Z ae

n

log f ( Xi , en)

2

+ 2n

Z

CF

log f(Xi, en)

a3 log f (Xi , e*) = E aeg

r2 n

+ 6n n

an

,

where

l0*-

enI <

.

r ,VW

Since (1/n) Z

{(a 2lae3) log f(Xi, 9)}

converges in probability to - 3J(0)-

i=1

K(0), it is seen that

(3.2) Z, (en)+ 2

Z2(en)} { I(an)+ ^n 6 r

{3J( Bn) +K(en)} +O ( 1 ) = an

On the other hand we have

(3.3) Z1(Bn)=Z1(0)+ 1 {Z2(0)-'%/ I(e)}Tn- 3J(0)+K(0) T7 +0P( 1 ../n 2.%/W ,V n) where T n=../ n (&-9).

Since

Z1(en)=Z1(e)+ 1 Z2(e)Tn- I(B)Tn./n

3J(6) +K(0)

2.,/W

T,^+op(

it follows that

(3.4)

Tn= I^B) {-Z1(en) +Z1(0)+

_n

Z2(0)Tn

_ 3J(6) +K(0) Tn +oP 1 2.v/-n ,/n Since I(en)=I(6)+ 1 12J(e)+K(6)}Tn-I-op( 1 4./n 4./n

1

),

160 DISCRETIZED LIKELIHOOD METHODS

47

J(&)=J(0)+ J'(B) T.+op( 1 )

K(BD) =K(8)+ 1 K'(6)T„ +oP( 1

)

Z2(e)=Z2(B)-J(B)T.+o'(1) it follows from (3.2) that

rn + 2

r {3rJ(B)+ rK(0)-3Z2(0)} +6 v/W

+ 2 / i {3J(0)+K(0)}T. +o, ) up to order n- 111.

From (3.4) we have

T.=_ an - r2(3J+K) + Z, + ZIZ2 - (3J+K)Z,2 +op r 1 ) rI 6I-1 n I I2.^/ n 21'.,/-n up to order n-112, where a„=-(rI/2)+an with an'=o(1/.,/n) so that B„ be AMU. In order to have second order asymptotic median unbiasedness of 9. we put r`(3J+K) _ rK +o\ 1 ) a`= 6-/n 61.,/W ^n and we denote by T,* the corresponding T. to this particular value of a..

Then we may also denote Then it is shown that

k is second order AMU. Hence we have established the following theorem. THEOREM 3. Under conditions (i) - (v), the DLE 6,* with a. defined above satisfies the following : T,* N/ n (en -0)

K + Z1 (0) + Z1(B)Z2(0) - 3J(0) +K(0) Z,(0)2 2I(0)a.,/ I(0)2. VW 6124,/n I(0) + r I Z2(0)- 3J(B)+K(B) Z1(0)} n +o( 1l ) n) 2I(0)^/ n I(0) ^/=

up to order n- 1/2 as n--> oo. Remark.

Since we have

EB [Z1(0) {z2(o) - 3J(0)+K(0) Zl(e)11=0 JJ I(0)

161 48 MASAFUMI AKAHIRA AND KEI TAKEUCHI

V 0 (T,*)= I(B) +o\.^l n l , and the third order cumulant is equal to that of the MLE up to the order n- '/2. Hence the asymptotic expansion of the distribution of T,* n- 1/2. Therefore the is equal to that of ^/n(9ML-B) up to the order DLE B,* is asymptotically equivalent to the MLE BML up to that order.

4. Third order asymptotic efficiency We proceed to the problem of third order asymptotic efficiency. We further assume the following : (vi) For almost all x[p] , f (x, 0) is four times continuously differentiable in 0; (vii) there exist L(O)=E, L as s log .f (X, 0)} aB log .f (X, B)}

a M(e)=EB I 1-66-F log

N(e) =E8

a 22

.f(X, }2]

log AX,

I

B)} ae log f(X, 0)}

and

f(X, 0)} 4] H(e) = EB 11ae log and the following holds : EB N log f (X, 0)] 4L(0) - 3M(O) - 6N(6) - H(0) O-14 Let 9n be an DLE. Since

E lo g f

(Xi, en+ / n ^ i log f (X4, en)=an ,

it follows that E a log f(Xi, B..)+ r E 82 log AX, Bn) 2n i=1 a82 ^/ n i=1 aB a4 as ^,3 n r2 +

an

r

E-

log f (Xi , Bn) +

E

6n./ n i=1 8B3 24n2 i=1 _FO

log f (Xi ,

6 *)

162 DISCRETIZED LIKELIHOOD METHODS 49 n

(1/n) Z (a'lao') log f(X;,, 0) converges in i=1 probability to -4L(0)-3M(0)-6N(6)-H(0), it is seen that

where

I0*-o1
Since

Z1(on)+ 2 { -I(B&)+ _ + 6n Z3(B

Z2(Bn)}

ban

{3J(9n) +K(on)}

4L(9n )+ 3M(&)+6N(Bn)+ H(Bn)}_ r , n)- 24n {

where Z3(B)= 1

' {iog f(x, 0)+ 3J(8)+K(0)}

Hence (4.1)

Z,(on)- rn + 2 I(Bn)- rn Z2(&)+ 6rn {3J(en)+K(&)} - 6n Zs(On)+- 24n {4L(Bn)+3M(Bn)+6N(Bn)+H(en)}

On the other hand we have

ZO(B )=Z1(o)+ 1 {Z2(Bn)-1/nI(0)}Tn- 3J(o)+K(o) Tn2 2 ^/n n VW 1 - 6n {4L(0)+3M(o)+6N(o)+H(0)}Tn +o^(n) where Tn=./n(B -o).

We obtain

(4.2) Tn= Z1(&)+Z1(e)+ 1 Z2(0)T.- 3J(o)+K(o) Tn I(o) ^/ n 2^/n - 6n {4L(o)+3M(o)+6N(o)+H(0)} Tn ] +oP(n) . Since

I(6n)=I(o)+ ^n {2J(0)+K(0)}TT -L) + 2n {2L(o )+ 2M(9)+5N(0)+H(0)}T„+ oP(ry1 J(en)=J(o)+ 1 {L(o)+M(o)+N(o)}Tn+op 1 ) ; ^/n K(en)=K(o)+ 1 n {3N(o)+H(o)}Tn+op( ) ; %/ 4/n

163 MASAFUMI AKAHIRA AND KEI TAKEUCHI

50

L(on)=L(O)+ 1 L'(e)T.+o,, 1 ) 4/n ^n

M(9n)= M(o)+JM'(o) TT

+o1, (^n)

N(6n)=N(o)d ^n N'(o)Tn+o^(_n) H(en)=H(e)+ 1 H'( o)TT+op(,V1n) Z2 (en)=Z#)-J(O)T. -

1

{2L(o)+M(o)+N(o)}T„+o p ( 1 ) 4/ n

and ZOn)=

Z3(B)Tn +op ( 1 )

it follows from (4.1) that Z,(9n)= an + I- r Z2+ r2(3J+K) _ r2 Z3+ r (3J+K)Tn r 2 2.,/W 6^/n 6n 24/n 3 24n (4L+3M+6N+H) r (4L+3M+6N+H)T„ +

+ 6n (3L+3M+6N+H)Tn+op(1) up to order n-1 as n--+oo. Under conditions (i)-(vii) we have from (4.2) (4.3)

Z1 +

rZ3 r3 11 rI 2 61.,/W I 21.,/W 61n 241n

T _ _ an

r r2(3J+K) +

rZ2 +

r (3J+K)Tn- r2L' Tn+ Z2 T.- 3J+K Tn 21.,/ n 21n I^/ n 21,^/ n r

2_

.LTn +o( 41n `rT n 61n p n =-an'+ K +Z,+ r (Z2_ 3J+KZ)+ Z,Z2 rI 6I .^/n I 21./n I I.,/n - (3JI3^ )Z2 + I L {Z1Z2 -(3J+K)Z12Z2- 6 41 + (3J+K)2Z1 _ (3J+K)Z1Z2 _ (3J+K)KZ1 + KZ2 212 21 61 6 _ r(3J+K) _ r2I(3J+K)Z2 + r2(3J+K)Z1 12 4 4

164

DISCRETIZED LIKELIHOOD METHODS 51

_ 3r(3J+K)Z1Z2 + r(3J+K)2Z2 + rIZ2 _ r2LE'Z1 2 41 2 2 +r(3J+K)2Z,2-r_fZI}+o(1) 21 4 _ n up to order n-1 as n--+oo, where a,=-(rI/2)-r3(3J+K)/61./n+rK/ 6I,/n +a;' with anl'=0(1/n) and 1=4L+3M-l-6N+H and £'=3L+3M +6N+H. We denote by T,* the corresponding T„ to this particular value of an. Then we may also denote T, * _ ,y/ n (9n* - 0). On the other hand under conditions (i) - (vii) we have obtained in [11] (4.4) /n(BML-0)= Il+I21

+ II { +

fz1z2-(3J2K)Z'2} ZiZ3 -

3J2jK Z

12Z2

(3J+K)2 3 _ 4L+3M+6N+H713} +o ( 1) Z, , n 61 2I

Let 0ML be a modified MLE so that it is third order AMU. Comparing (4.3) with (4.4), we see that .v/n(9ML-0) and T,* are essentially different in the order n-1. We put TML =.v/n(eML-0). Note that the difference of T L and T,* appears in the linear term of order n-1/2 and other terms of order n-1. It is shown in [12], [13] and [14] that the asymptotic cumulants are determined up to order n-1 by the terms of order up to n-1/2 if the first term is equal to Z(0)/I(0) ; and the fourth order cumulant is identical in the first term of order n-1 for all asymptotically efficient estimators. In the third order cumulants we have K3(T -E,(TL ))_ a(0) +

o(n)

K3(T,*-E, ( T.*))_ ^3 (B) + r3(B)+o(1) A/n n in) Therefore there is a difference of asymptotic distributions of "/ n (aL0) and T,* in the term of n-1 if r3(O) *0. If we denote by Theorem 3 and (4.3) T*= Z_ J+K + r L*+ 1 (Q- c)+ 6I2^/n / I ^/ n n R+op(nI) 1

L* 2I (Z2_ 3J +K K z)

165 52 MASAFUMI AKAHIRA AND KEI TAKEUCHI

Q= I2 (Z1Z2- 3J21K Z^2)

C=- J+K 212 We decompose that (4.5)

P3(B) + r3(e) +0(1) _ - 1 Eo (Z,3) + 3 Eo [Z1(T,*-Eo (T,*))2] .Vln n n 2I3 21 + 21n Eo [&(rL*+Q-c)2]+o(n )

Note that Eo (Q) = c. We have Eo (Z3) = K/4/n . For the second term of the right-hand side of (4.5) we use the following lemma (see [12], Lemma 4.1). LEMMA . Suppose that X1, X2, • • • , X,,, • • • is a sequence of i.i.d. random variables with a density f (x, B) which is differentiable in 0 for almost all x[u]. Suppose further that To is a .`"'-measurable function on 6''"' into R1 and differentiable function of 0 and that

B

EB (T6)=

J (") ao

IT, trr

f(xt, o)}

sT

du( x,)

holds. Then we have 1 a E, (To)- 1 Eo (aTo ) ^ n ao V/ n ao

Eo (Z1 To) =

By the lemma we obtain

(4.6)

Eo [Z1(T,*-Eo

(Tn*))2]

1 a Vo( Tn*)- 2 Eo [(T,*-Eo(T,*)) "/n ao ao

(T,*- Eo (Tn ))

ao (1) +n K i I z.,/+n +0 (n)

We also obtain (4.7)

Eo [Z,(rL*+Q-c)2] =2r Eo [Z^L*(Q-c)]+o(1)= r I M_ (3J+K)K l +0(1) 212 J I I

166 DISCRETIZED LIKELIHOOD METHODS 53

From (4.5), (4.6) and (4.7) we have r3(e) __ 3r [ M(0) - {3J(0) +K(0)} K(0) 21(0)2 L 1(0) 21(0)2 Since the third order asymptotic distribution of T,* attains the bound of the third order asymptotic distributions of third order AMU estimators at r, there is no third order AMU estimator which uniformly attains the bound, if 73(0) is not equal to zero. Hence we have established : THEOREM 4 . Under conditions (i) - (vii), BML is not third order asymptotically efficient if r3(0) *0.

5. Maximum log-likelihood estimator Instead of the equation (1.1) we may take a solution an of the discretized likelihood equation : n

n

(5.1)

Z 1ogfX,

i=1

Bn

+rcnl) -E lo gf( Xi, Bn- rcn1) =0

i=1

The solution an is 0 which maximizes rcnl n

Z log f(Xi, 0+t)dt=

J -rcnl 2 = 1

rcnl _rcml

log L(0+t)dt

where L(0) denotes the likelihood function. Then an is called maximum log-likelihood estimator (MLLE) ([3]). If log L(an+t) is locally linearized, an agrees with maximum probability estimator by Weiss and Wolfowitz ([15]). Now we shall consider a location parameter case when the density function f satisfies the following assumption : (viii) f(x, 0)=f(x-0) and f(x) is symmetric w.r.t. the origin. It follows by the symmetricity of f that the solution en of (5.1) is AMU. We also have

J(0) =K(0) = 0 . Let an be an MLLE. Then n

n

E logf(X,- an- (r/s/n))-Z logf(Xi- an +(r[/n)) =0 . i=1 i=1 Without loss of generality we assume that 00=0. Since n

n

Z log f(X -0n-(r/ '✓ n))-E log .f (Xi) i=1 i=1

167 54 MASAFUMI AKAHIRA AND KEI TAKEUCHI

I l =0 Z log f(Xi-on+(r/ ✓ n))-^ i=1 i=1 log f(X1), it follows by Taylor expansion around 0=0 that 2r :

[-L log f (Xt _ 0)

n

ao {

=1

J ,=0

{ 2r6,, .= n

a2

log .f (X' - o)

F L ae , Z

J 0=0

+ (_!1 t log .f (X _ 0)] e o ,9n' + 3nr3 n / L aB^ +lrren+ A lra' log AX,-0)]a_o~0 , 3 \^Pn n^/n /LaB' where 10* I S 10I . Putting T„ = .// n B,, , we obtain 2rZ,+2r (^n -I(0))Tn+{ Z3- /n(3J(0)+K(0))}

(.712 + 3 r )•

{4L(0)+3M(0 )+ 6N(0)+H(0)}

r T + r3 Tn0 ) . n

n

n

Hence it follows that Tn=

+ Z1Z2 ) +

r2 1%/ n I I2n 61n Z2 (Z1

Za+

1

21n

Zg

Z.'

I2

3 6In (4L + 3M+ 6N+ H) ( Ig

+r2 Il +op(n .

Under conditions (i) ~ (vii ) we have (5.2)

Tn=

Il + IZZ n + I n

{Z1Z2 + 2 Zi Z3- _I ( 4L+3M+6N+H)Z3}

+ rZ3 r2(4L+3M+6N + H) Z1+op 6In 612 n (in) up to order n' as n-> oo. As was stated preciously the difference in the order n-1 term between (4.4) and (5.2) does not affect the asymptotic distribution up to the order n1. Hence we have established : THEOREM 5.

Under conditions (i) ~ (iv) and (vi)-(viii), the MLLE

9n is asymptotically equivalent to the MLE 6ML up to order n1. It is shown in [12], [13] and [14] that BML maximizes the symmetric probability Pn,8 {4/nf BML-OI
168 DISCRETIZED LIKELIHOOD METHODS 55

up to the order n-1 among all regular AMU estimators. Therefore it is seen that

lira n[PP,g {,/ n ^BD1L -01 _0 n-oo

for all u, where 9DL denotes DLE. The asymptotic distribution of the MLLE OMLL is equal to that of BML up to the order n 1, but not in the n-3/2 and we can show that

lim n3'2[Pn,g { ,/n 1BMLL -01 _0 n_oo in general situations. Hence for symmetric intervals 0MLL is not fourth order asymptotically efficient.

6. Conclusion remarks on discretized likelihood methods" If we also define as asymptotic efficient estimator as

On

which max-

imize lim Pn,B {cnlOn -Oj
ing the following equation (6.1) is asymptotically efficient : (6.1) i

-1

f(Xi, Bn+rcnl) f(XX, On*-rcnl) -a f(XX, 0*) - n f(X1, Bn) t-1

where an is chosen so that B* is AMU. Then (6.1) is also expressed as (6.2) exp {log L(6,*+rcn1)-log L(B*)} -exp {log L(9*-rcn1)-log L(B*)} an . If exp (log L) is linearize , then (6.2) is reduced to the type of (5.1), where in this case the right-hand side of (5.1) is not always zero. Hence it is seen from the above that the asymptotic efficiency (including higher order cases) may be systematically discussed by the discretized likelihood methods. Further the discretized likelihood methods may be also applied to the statistical estimation theory of the fixed sample size. UNIVERSITY OF ELECTRO - COMMUNICATIONS UNIVERSITY OF TOKYO

REFERENCES [ 1 ] Akahira, M. (1975). Asymptotic theory for estimation of location in non -regular cases, I: Order of convergence of consistent estimators, Rep. Statist. Appl. Res., JUSE, 22, 8-26. [ 2 ] Akahira, M. and Takeuchi, K. (1976). On the second order asymptotic efficiency of 1) This section is based on Akahira and Takeuchi [3].

169 56 MASAFUMI AKAHIRA AND KEI TAKEUCHI estimators in multiparameter cases, Rep. Univ. Electro-Comm., 26, 261-269. [ 3 ] Akahira, M. and Takeuchi, K. (1977). Asymptotic properties of estimators obtained from discretized likelihood equations , Annual Meeting of the Mathematical Society of Japan. [ 4 ] Chibisov, D. M. (1972). On the normal approximation for a certain class of statistics, Proc. 6th Berkeley Symp. Math. Statist. Prob., 1, 153-174. [ 5 ] Chibisov, D. M. (1973). Asymptotic expansions for Neyman's C(a) tests, Proc. 2nd Japan-USSR Symp. on Prob. Theory, (Lecture Notes in Math. 330) Springer-Verlag, 16-45. [ 6 ] Efron, B. (1975). Defining the curvature of a statistical problem (with applications to second order efficiency), Ann. Statist., 3, 1189-1242. [ 7 ] Ghosh, J. K. and Subramanyam, K. (1974). Second order efficiency of maximum likelihood estimators, Sankhya, A, 36, 325-358. [ 8 ] Pfanzagl, J. (1973). Asymptotic expansions related to minimum contrast estimators, Ann. Statist., 1, 993-1026. [ 9 ] Pfanzagl, J. (1975). On asymptotically complete classes , Statistical Inference and Related Topics, Proc. of the Summer Research Institute on Statistical Inference for Stochastic Processes, 2, 1-43. [10] Pfanzagl, J. and Wefelmeyer, W. (1978). A third order optimum property of the maximum likelihood estimator, J. Multivariate Anal., 8, 1-29.

[11] Takeuchi, K. and Akahira, M. (1976). On the second order asymptotic efficiencies of estimators, Proc. of the 3rd Japan-USSR Symp. on Prob. Theory, (Lecture Notes in Math. 550) Springer-Verlag, 604-638. [12] Takeuchi, K. and Akahira, M. (1978). Third order asymptotic efficiency of maximum likelihood estimator for multiparameter exponential case, Rep. Univ. Electro-Comm., 28, 271-293. [13] Takeuchi, K. and Akahira, M. (1978). On the asymptotic efficiency of estimators, (in Japanese), A report of the Symposium on Some Problems of Asymptotic Theory, Annual Meeting of the Mathematical Society of Japan, 1-24. [14] Takeuchi, K. and Akahira, M. (1979). Third order asymptotic efficiency of maximum likelihood estimator in general case, to appear. [15] Weiss, L. and Wolfowitz, J. (1967). Maximum probability estimators, Ann. Inst. Statist. Math., 19, 193-206.

170 Ann. Inst. Statist. Math. 31 (1979), Part A, 403-415

ASYMPTOTIC OPTIMALITY OF THE GENERALIZED BAYES ESTIMATOR IN MULTIPARAMETER CASES KEI TAKEUCHI AND MASAFUMI AKAHIRA

(Received July 28, 1978; revised Oct. 24, 1979)

Abstract The higher order asymptotic efficiency of the generalized Bayes estimator is discussed in multiparameter cases. For all symmetric loss functions, the generalized Bayes estimator is second order asymptotically efficient in the class A2 of the all second order asymptotically median unbiased (AMU) estimators and third order asymptotically efficient in the restricted class D of estimators. 1. Introduction The expansion of a generalized Bayes estimator with respect to a loss function of the type L(B)=IOId (a>1) is obtained by Gusev [5]. His result can be extended to all symmetric loss functions. Strasser [8] also obtained asymptotic expansions of the distribution of the generalized Bayes estimator. In one parameter case the second order (or third order) asymptotic efficiency of the generalized Bayes estimator has been discussed by Takeuchi and Akahira [12]. It is shown in this paper that in multiparameter case for all symmetric loss function the generalized Bayes estimator d is asymptotically expanded as

^/n(e-B)=U-2^n I-IV+Vn I-IW+oP\^n where the symbols of the right -hand side are defined in the contexts. And the asymptotic distributions of the estimators are the same up to the order n-I except for constant location shift . Therefore if it is properly adjusted to be asymptotically median unbiased , it is third order asymptotically efficient among the estimators belonging to the class D ([3], [4], [11]) whose element d is third order AMU and is asymptotically expanded as 403

171

404 KEI TAKEUCHI AND MASAFUMI AKAHIRA

s/n(9-0 )=U+^n Q+op\"An l

and Q, = Op(1) (a = 1,---,p) and E [UQ' ]= o(1) (k=1 , 2) for all a, p=1, • • • , p, where E denotes asymptotic expectation and U= ( U1 , • • • , Up)' and /

Q= (Q, ,

... ' Qp)'.

2. Results Let (a', B) be a sample space . We consider a family of probability measures on B, ) = {P0 : 0 E 01, where the index set ® is called a parameter space . We assume that 0 is an open set in a Euclidean pspace R' with a norm denoted by II • II. Then an element 0 of 0 may be denoted by (01 i • • • , Bp). Consider n-fold direct products (1'`n), B`n)) of (X, B) and the corresponding product measures P' (n) of P0. An estimator of 0 is defined to be a sequence {B.} of _B (n)-measurable functions Bn on X(n) into ® (n =1, 2,. • • ). For simplicity we denote an estimator as d instead of {6n} . Then may be denoted by (61 f • • • , d,). For an increasing sequence of positive numbers {cn} (cn tending to infinity) an estimator is called consistent with order { cn} (or {cn}-consistent for short) if for every s>0 any every 9 E ® there exist a sufficiently small positive number 8 and a sufficiently large number L satisfying the following : lim sup P1n) {cnlle -0II _L} <s ([1]) . 0:10 -SI1<8

n-too

For each k=1,2,---, a {cn}-consistent estimator d is kth order asymptotically median unbiased (or kth order AMU) estimator if for each 9 E ® and each a =1, • • • , p, there exists a positive number 8 such that c,-1 Pan){B 0}-2

=0;

c,-ll Pen) {Ba>_BQ}- 1 lim sup n-» 0:110-s11<8 2

=0.

lim sup n--

0: 110-911<3

For B kth order AMU, Go(t, 0)+cnIG1(t, 0.)+ • • ' cn
lim c,-1I PBn){cn(Ba-ea)
cn 1G1( t, Ba)

... -Cn (k -1)Gk

-1(t,

0,,)I

=0

We note that G#, 0.) (i=1,..., k -1; a=1,---,p) may be generally

172

ASYMPTOTIC OPTIMALITY OF THE GENERALIZED BAYES ESTIMATOR 405

absolutely continuous functions , hence the asymptotic marginal distributions for any fixed n may not be a distribution function. Suppose that d is kth order AMU and has the kth order marginal asymptotic distribution G0(t, 0.)+cn'G1(t, 0.) + • • +cn ( k-')Gk_1 (t, O) (a=1, •, p) and the joint distribution of B admits asymptotic expansion up to kth order , i.e., the order of en ( "'. Letting Bo (E ®) be arbitrary but fixed. Denote Bo by (Bol, • • • , Bo,). Leta be arbitrary but fixed in 1, • • • , p. We consider the problem of testing hypothesis H+ : Ba=Boa+tcn' (t < > 0) against K : Oa = Boq . Put 01/2 { On} ; El0a+1,n1(0n) =1/2 + o(cnck-"), 0 _ ¢n(xn)51 for all x„ E _'`"' (n=1, 2,. • .)}. Putting Adq,00a={cn(Ba- Boa)St}, we have

lim Poa'+n 1(Aea,60.)=lim Poa+ i^n^ {BaSBoa-^ tcn'} = 2 Hence it is seen that a sequence {XA L, 0oQ } of the indicators (or characteristic functions ) of ABa, eoa (n=1, 2,. • •) belongs to 01/2 If sup lim cn -' {E(n) t, Boa) - cn 1H,+(t, Boa) - .. 80. (O n) - H+(

{^n}Eo ,a n-soo

n

(k-1)H+

i(t,

BOa)} -0

then we have

Go(t, Boa) s H+(t, Bo.) ; and for any positive integer j ( S k) if Gi(t, Boa ) = H,+(t, Boa ) (i =1, • • • , j-1) then G;(t, Boa)=Hj+(t, Boa) • Consider next the problem of testing hypothesis H- : BQ=Ba„+ tc-1 (t<0) against K : Ba = Boa . If sup Lim c'n -' {Edna (0n) -Ho (t, G0 )-cn'HC( t, Bon)- .. .

{!6n }Eo1

,2

n-->oo

-cn(k-I)Hk l(t , Boa)} =0

then we have Go(t, O..) Z Ho (t, Boa) ;

and for any positive integer j (< k) if Gilt, Boa) = H1 (t , Boa) (i =1, • • • , j-1), then G1(t, Boa)>HH (t, Boa). B is called to be kth order asymptotically efficient in the class Ak of the all kth order AMU estimators if the kth order asymptotic marginal distribution of it attains uniformly the bound of the kth order asymptotic marginal distributions of kth order AMU estimators, that

173

406 KEI TAKEUCHI AND MASAFUMI AKAHIRA

is, for each a =1, • • • , p

HI(t, O)

for t>O ,

Hz (t, O)

for t
i=0, • • •, k-1 ([2], [4], [9]). [Note that for t=0 and each a=1, • •, p we have Gi(0, 0.)=Hi+(0, O)=Hi- (0, 9Q) (i=0,• • •, k-1) from the condition of kth order asymptotically median unbiasedness.] B is called to be third order asymptotically efficient in the class D if the third order asymptotic marginal distribution of it attains uniformly the bound of the third order asymptotic marginal distributions of estimators in D. It is generally shown by Pfanzagl and Wefelmeyer [6], [7] and Akahira and Takeuchi [3], [4], [10], [11], [14] that there exist second order asymptotically efficient estimators but not third order asymptotically efficient estimators in the class A8. But it was also shown in [4], [10] and [11] that if we restrict the class of estimators appropriately, we have asymptotically efficient estimators among the restricted class of estimators and that the maximum likelihood estimator belongs to the class of higher order asymptotically efficient estimators. We assume that for each 0 E ® PB is absolutely continuous with respect to a-finite measure p. We denote a density dPB/dp by f(x, 0). n Then the joint density is given by T[ f (xi, 0). i=1

In the subsequent discussion we shall deal with the case when cn = ..l n . Let ®= RP. Let Ln(u) (u = (U1, • • • , Up) E RP) be a bounded nonnegative and quasi-convex function around the origin, i.e. for any c the set {u : L(u) S c} (c RP) is convex and contains the origin and ir(0) be a non-negative function. Define a posterior density pn(0I xn) and a posterior risk rn(d I xn) by

and

rn(d I xn) _ ^e Ln(d - 0)pn(0 I x n)d0 , respectively, where xn = (x, , ,

=L*(u) for all u E RP.

xn).

Now suppose that lim Ln(u /.,/ n )

We define

r*(d I x%) = J L *(-./ n (d- 0))pn(0 I xn)d0 . An estimator B is called a generalized Bayes estimator with respect to a loss function L* and a prior density it if

174 ASYMPTOTIC OPTIMALITY OF THE GENERALIZED BAYES ESTIMATOR 407

r:(BIxn)=inf rn(dIxn) dE8

Then i =../ n (9 - 0) may also be called a generalized Bayes estimator w.r.t. L* and a. Since Ln(d-0)5(0)d0-inf L*(%/n(d-0))5(0)d0l = 0

limlinf

dEB 8

n-- dE8 9

uniformly in every posterior density p(6), it follows that for a generalized Bayes estimator

limlinf rn(djxn)- r, (6 xn)j =0 n-•O

dE8

Suppose that X1, X2, • • • , Xn, • • • is a sequence of i.i.d. random variables with a density f (x, 0) satisfying (i)-(iv).

(i) {x: f(x, 0)>0} does not depend on 0. (ii) For almost all x[,u], f (x, 0) is three times continuously differentiable in 0. (a =1, • • • , p). (iii) For each a, R (a, j9 =1, • • • , p) 0
log f (X, 0) < 0 _ - EB I a2

aoaa6^

(iv) There exist Jad•r(6) =E8

LI

a2

a6Qaos

a log f(x, 6)} log f( x , 6)} {__

K(0)=E a L a log f(x, 6)} { a log f(x, 6)}

a6a a6^

• l a6, log

f(x, 6)} ]

and War(6

) = EB L a6sa6^a6,

log f (X, 0)]

and the following holds : Ee [ a9 log AX, 0)] _ -J ^.r(6)-J,.s(6)-J^r.a(6)-Kaar(6)

a6Qa6pa6r

It was shown by the same way in [9] that a maximum likelihood estimator (MLE) is second order asymptotically efficient. Let B be an MLE. By Taylor expansion we have

175

408 KEI TAKEUCHI AND MASAFUMI AKAHIRA

0=z E aea log.f(XX, B)

a2 log e) +E { :^ log f(X¢, o)} a aeaf(X1, a }z aoaaoo as oa)(Bo -oo + 1 E E E {E aoaao0aer log f(X=, o*)} (Ba-Ba)

(ea-

2 a a 7

(Ba-

'^

Ba)(Ba- 00)(87-B7)

where SI0*-0IISII9- 8!I. Putting T=./n(B-B) we obtain E {a2 log f(X{, e)} TTT, o= 1 {^ 8 log f(Xi, B)} Ta+1 ^/ n a a Boa n s s aoaaop + 2n ✓n

E E E {Z adaa'saor

where T=(T1f • • ., TT)'.

log f(X,, B*)} TTTfT7 ,

Set

Z-(8) =1/1n Z(O) 1

a '=1

log AX, B)

aBa

82 log f(XX, B)+Ia(B)

177 i=1 aeaae^

0) converges in probability to - {J p.r(B)+J 7.fl(B) Then it follows that W,,,( + Ja7•a(0) + Kaer(B )}. We Put Paa7(B) = J fi.7(0) + Jar-P(O) + Ja7.a( B) + K dr(0). Hence the following theorem holds : THEOREM 1.

Under conditions (i)-(iv) 2^n I-1V

+.^n I-1W+op(-V/1n) ,

where I= (I p) and P= (Pa,) are matrices and V= (Z pap, Up U), W= d r

(3^1, UZpr) and U= (U) are p- dimensional column vectors and U. = I "PZZ and Ia" denotes the (a, p)-element of the inverse matrix of the information matrix L Since the proof of the theorem is essentially same as that of one parameter case ([9]), it is omitted. Put

1 Y, where Y= (Z Up W . p,) is a column vector . d r

Then 9* is second order

176 ASYMPTOTIC OPTIMALITY OF THE GENERALIZED BAYES ESTIMATOR 409

AMU. From Theorem 1 we have established the following : THEOREM 2. Under conditions (i)-(iv) 9* is second order asymptotically efficient in the class A2.

Since the proof of the theorem is essentially same as that of one parameter case ([9]), it is omitted. It will be shown that the generalized Bayes estimator w.r.t. a loss function and a prior density is second order asymptotically efficient. Let Bo be a true parameter of B (E 9). Denote 0 and Bo by (B1, • • •, Bf,)' and (001f • • • , Bo,)' respectively. Further we assume the following : (v) For each a= 1, • • • , p, ir(B) is twice partially differentiable in Ba . Then we have pn(B I xn )/Pn(Bo I xn)

= exp [log {pn(B I xn)/pn(0 I FOX } = exp [ElOgif(X,,O)lf(X,,00 )1+10gl7r(0)17r(OO)I 1 i_1 J =exp

^

log f(xE, e) -Z 1

[, =eXP

I Y"

a i=1

Z=1

log f(x{, Bo)+log 2r(B)-log 2r(Bo)]

8 lo g f(xti, aea

Bo)}(Ba

-Boa)

2

a log f(x i, 0o)1 + 1 E E {E t=1 aoaaop

(0a-

2 a a

+6

E

{yE

aBaao^aB,

lo g

00 (00 -000

f(x" B*)}

EQ(B? • (Ba-Boa)(Ba-B(,p)(Br-B(r ) + Ea 7r(Bo) Boa)+o(1 )] -✓ n (oa-

=exp [v'i

E Z (Bo)(Ba -Boa) + 2 Ea E { ✓ nZ a(Bo) -nI a(00)}

• (Ba-BOa )(Ba-BOa )+ C / V

• (Br-Bor )

+E

E E E Waar( 0*)(0a-0Oa ) ( 0a -000

(Ba- BOa ) +oI

11

i(Bo) where 7C a(B)=air(B)/a0a (a =1,• • •, lp). It follows that pn(B I xn)/pn( 0 I xn) = eXp

E E { Zaa( Bo) _ Iaa(0 )} [Ea Z (BO) { ✓i (Ba -Boa )} + 1 2 a a ^/n {n(Ba-B0a )( Ba-Boa

)} - 6.,/n

E E Paar(Bo)

• {n^/n(Ba- Baa)(O -B°^)(B,-eo,>}

177

410 KEI TAKEUCHI AND MASAFUMI AKAHIRA 1

E ^a(B o){^/n(Ba-Boa )}+0,(,/n a

+ -✓ n7r (Bo)

Putting tQ = ,/-n (Ba - Boa ) (a =1, • • • , p) we obtain (1) pn(O I xn)/P..(eo I FO-) =exp [:p ZZ(O)t,+ 1 E E {Z-P(00) -IP( Oo)}tats 2 a P N/ n P tr r Pr 09o)tat

- 6.V/W - 5

+.Vln7r(Bo) ^a =eXp [-

2EEI

(eo)t.-FoP(1/n ) I

(O) (t.- U)(tP-UP)+

^a(Bo)ta +

+ 1

-,/n7c(Oo)

1

2 E IP(Bo)U UP

E E Z P(Bo) t atP

n P l1 E P-Pr(0O)tgtPtr+OP( 1 )] n a P T N/n 6.v/-

= (exp 2 E E I #o) U. UP) • [exp {-

E E I A00) (ta - U) (tp - UP)} ] __ 2 E 7r a(eo) ta+ 1 E E Z P(O0)t.tP • exp { 1 V/ -n ic(Oo) a n a P 6 1n E E E P.WOO)tatPtr + Op(

'^/lw ) }

_ (exp 2 E E I #o) U. UP)

• [cxp {- 2

E E IP(eo)(ta- Ua)(tP -UP)

E na(O0)t.+ 1 E E ZZ P(O0)ta tp n n(O0) a 2^/ n a P

{

E E E P-WOO )VA1 +Op( 1 )} 6,/ n a P r ^/ n =4'.(t, Bo 1x.)

where t = (t, , • • • , tp)'. given by (2) r. *(t I x.) _

(say)

Let t = ^/ n (6 - Bo).

^n

p.(Bo 1 xn)

5

Then the posterior risk is

L*(t-t)q„ (t, 60 I x„ )dt

178

ASYMPTOTIC OPTIMALITY OF THE GENERALIZED BAYES ESTIMATOR 411

Further we assume the following : (vi) L*(u) is a convex function ; (vii) For each a=1, . • • , p,

5

L*(-u)gn(u+t, Bo I xn)du is continuously

partially differentiable with respect to to under the integral sign. By (2) and the assumption (vi) it is shown that the generalized Bayes estimator t w.r.t. L*(.) and 7r(.) is given as a solution u of the equation

1

(d/dua) L*(u- t)gn(t, Bo I xn)dt = 0

(C'= 1'. - -, p) ,

Since by (vii)

d

dua

5

L*(u-t) gn(t, Bo I xn)dt d dua d

dua

5 5

L*(- t)gn(u +t, L*(- u)gn(t +u,

001

xn)dt

Bo 1 xn)du

ff +u, Bo^ x n) }du (a=1,..., p), -J L*(- u)Idtagn(t the generalized Bayes estimator t is obtained by a solution of the equation

(3) 5

L*(_v){_qn(t+u,0okn)}du__0 (a=1,..., p) dta ,

Since t=^/n(0-0o) and t=-/_n(0-00), t may be called to be the generalized Bayes estimator. From (1) and (3) we have

0

=5 L*(-u) exp [ -

[-

--

E IQa(Bo)(ta+ua -Ua)(ta+ua-UP)]

1 6(BO) ( ta +uo - Ue) 11+

1 ^/ n ^c (Bo)

LI ia(0o) (ta +

1 +2.,/n E Z-00)(ta+ua)(to+ua)

-6^n E + .^/n ^c(o)

-2^n

Pap ,(eo)( t a+ua)(to +UP)(t,+ur)}

n«(Bo ) + n Z(00) (tP up)

Z Z P-400) (to +ua ) (t r +u,)] du+ Or( `/n/

ua)

179 KEI TAKEUCHI AND MASAFUMI AKAHIRA

412

Putting i t , = to - U U (a =1, • • • , p) we obtain

0

=5

{- 2 Y Z I p(6.)( ua +ua)(up+it)}]

L*(-u)[exp

• [-LJ

+

Iap( u p +u a) {i+ , ✓ nn(Bo )

1

L

2,,/n a

Ll

p

a

7ra(60)(ua +ua+U

)

Zap(60)( ua +ua+U.)( u p +u p +Up)

+,k +U r (u , + it,+U) ua+uap + U )( up) 1a Zp papr( 60 r r )(

64./n

+ ^/n^c(6o)

7ra(6o)+ ^n

Zap(60)(up

+up +U)

-2 In E E pa460)(40 +up+U)(11r+ur+U,)]du+op( Further we assume the following. (viii) L*(u) is a symmetric loss function about the origin. We define

5

M= L*(-u) exp [-

I p(6o)u.up]du ;

2

pas L*(-u)uaup exp

E Ia0 o)u.up]du

[-i-

(a,= 1,..., A; Qapro=

J

L*(- u)uaupuru, exp [-

-i

I p(60)uaup]du (a, Q, r, 3=1,- --, p) •

Note that by (viii)

5

L*( -u)ua exp (2

Iap( 6o)u.up)du

L*(- u)uaupur exp (-

1

Iap(60 ) uaup)du=0

(a, j9, r=1,..., p) Then we have 0=J L*(-u)[exp

Ip(6o)uaup}] {1 -E E Ira(6o)urua}

• [-i] i] I p(60)up - E I p(6o)up - E 71

I p(6o)^r(6o) n xc(60)

180 ASYMPTOTIC OPTIMALITY OF THE GENERALIZED BAYES ESTIMATOR 413

( ud +u

p)(ur +ur +U')- 2 1/ i E E E II^(O0)Zra(O0) N

' (ud+ua)(urua+U1Ua+2uric,+2Ui,+2Urua)

n Ee Er Ea Ee I ft(oo )Pra e(oo)(u p +ua)(urusue +UrU'Ue

+ 1 6,/

+ 3urruaie + 3u,ur Ue + 3 Ur Uaue + 3 Ur. Uaue + 6urua Ue )

+ 1 na(Bo)+

E Z ZP(B0)ue+ 1

E Za0(9o)UU- 1

2%/ n l

.^l n 7r(Bo) P ,/ n A

• P.,'(0o)(upur+UpUr+2upiur+2Uur+2Usur)]du+oz,( 1 1 _ -ME Ip(oo)ue-

l E E Ia^(oo) r (oo) per-

I E E E IQa(eo)

P ^/ 7G P r 7r(Oo) 9 r a

• Zra(Bo)UrP 4+ 6 +

^a(oo) 7r(BO)

•

,V 1 n_

EE

E

E I p(Bo)

Prde(Bo)(Qa ra e +3UrUo Pae)

1 M+ 1 E UPZ P(oo) ME 17n a 2^/ n e

E P.P'(oo) 1

(PPr +UUU,M) + E

E I; Aoo)I 00)Pa 14N +01 ( ) B r a x/12

Using a matrix representation we obtain (IPI-MI)ic= ^n (PI-ME)7r*-{ 1n L- 2 1 (PI-ME)V + 1 (PI-ME)W+op(

1),

where E is an unit matrix , and L, V and W are column vectors with lr(oo) E E E I p(0o)Prae (Bo )Q9rae+ r 2

L= (- 16 V=

(zP r

Papr (o o)Pdr)

P, Nr(oo ) UpUr)

W=(E UAr(B0)) Since it is derived from (vi) that the matrix IPI-MI is positive definite, it follows that (4) A= 1

I-'7r* + 1 (IPI-MI )-IL- 2 1_ I-IV

181 414 KEI TAKEUCHI AND MASAFUMI AKAHIRA

Hence we have to=ua+U. (a=1,..., p) ,

(5)

where a=(u,, • • - , it,)' is given by (4). Since

Vn(B-Bo), we modify

d to be second order AMU and denote by B*. From Theorem 1, (4) and (5) it follows that the MLE 8ML is asymptotically equivalent to the generalized Bayes estimator 9* up to order n-'"2. By Theorem 2 it is seen that d* is second order asymptotically efficient in the class A2. We have defined in [4], [11] and [12] the class D as the set of the all third order AMU estimators B satisfying the following : (a) B is asymptotically expanded as

and Qa=OP(1) (a=1, , p) and E(UQk)=o(1) (k=1, 2) for all a, a=1, • • , p, where E denotes the asymptotic expectation of UUQ1 with U= 1 I°" 8 E log.f(Xi, B) ( = 1 , . . . , p ) ;

(b) The joint distribution of d admits Edgeworth expansion. It follows from (4) and (5) that the generalized Bayes estimator B* belongs to the class D. Then the asymptotic marginal distribution of B* is equivalent to that of the MLE 9ML up to order n-'.

Since BML is third order asymptotically efficient in the class D ([4], [10], [11]), 9* is also so. Hence we have established : THEOREM 3.

Under the assumptions (i )-(viii), the generalized Bayes

estimator d* is second order asymptotically efficient in the class A2 and also third order asymptotically efficient in the class D. UNIVERSITY OF TOKYO UNIVERSITY OF ELECTRO - COMMUNICATIONS

REFERENCES

[1]

Akahira, M. (1975). Asymptotic theory for estimation of location in non -regular cases, I: Order of convergence of consistent estimators , Rep. Statist. Appl. Res., JUSE, 22, 8-26. [2 ] Akahira, M. and Takeuchi, K. (1976). On the second order asymptotic efficiency of estimators in multiparameter cases, Rep. Univ. Electro-Comm ., 26, 261-269.

182

ASYMPTOTIC OPTIMALITY OF THE GENERALIZED BAYES ESTIMATOR 415 [3] Akahira , M. and Takeuchi , K. (1979). Discretized likelihood methods-Asymptotic properties of discretized likelihood estimators (DLE's ), Ann. Inst. Statist. Math., 31, A, 39-56. [4 ] Akahira , M. and Takeuchi , K. (1979). The Concept of Asymptotic Efficiency and Higher Order Asymptotic Efficiency in Statistical Estimation Theory, Lecture Note. [ 5 ] Gusev, S. I. (1975). Asymptotic expansions associated with some statistical estimators in the smooth case 1. Expansions of random variables , Theory Prob. Appl., 20, 470498. [ 6 ] Pfanzagl , J. and Wefelmeyer , W. (1978). A third-order optimum property of maximum likelihood estimator , J. Multivariate Anal., 8, 1-29. [ 7 ] Pfanzagl, J. and Wefelmeyer, W. (1979). Addendum to "A third -order optimum property of the maximum likelihood estimator ", J. Multivariate Anal., 9 , 179-182. [ 8 ] Strasser, H. (1977). Asymptotic expansions for Bayes procedures , Recent Development in Statistics (ed. J. R. Barra et al.), North -Holland, 9-35. [9 ] Takeuchi , K. and Akahira, M. (1976). On the second order asymptotic efficiencies of estimators , Proceedings of the Third Japan-USSR Symposium on Probability Theory (eds. G . Maruyama and J . V. Prokhorov), Lecture Notes in Mathematics 550, Springer-Verlag , Berlin, 604-638. [10] Takeuchi , K. and Akahira , M. (1978). Third order asymptotic efficiency of maximum likelihood estimator for multiparameter exponential case , Rep. Univ. ElectroComm ., 28, 271-293. [11] Takeuchi, K. and Akahira , M. (1978). On the asymptotic efficiency of estimators, (in Japanese), A report of the Symposium on Various Problems of Asymptotic Theory, Annual Meeting of the Mathematical Society of Japan, 1-24. [12] Takeuchi , K. and Akahira , M. (1978). Asymptotic optimality of the generalized Bayes estimator, Rep. Univ. Electro -Comm., 29, 37-45. [13] Takeuchi, K. and Akahira, M. (1979 ). Note on non-regular asymptotic estimationWhat " non-regularity " implies, Rep. Univ. Electro-Comm ., 30, 63-66. [14] Takeuchi , K. and Akahira, M. (1979). Third order asymptotic efficiency of maximum likelihood estimator in general case , (to appear).

183

Rep. Univ. Electro-Comm. 30-1, (Sci. & Tech. Sect.), pp. 63-66 August, 1979

Note on Non-Regular Asymptotic Estimation What " Non-Regularity" Implies* Kei TAKEUCHI ** and Masafumi AKAHIRA***

Abstract In the asymptotic theory of statistical estimation the authors tried to compare the regular versus non-regular situation to clarify the significance and implication of each of the regularity conditions. This paper summarizes the results far obtained.

1. Introduction Asymptotic theory of estimation is usually based on a set of regularity conditions. Regular case has been studied since Cramer [ 5 ] to Huber [ 6 ], and Pfanzagl [ 7 ] among others. Non-regular case has been studied by various authors at various occasions and Weiss and Wolfowitz [ 9 ] gave general results covering non-regular cases. The authors tried to compare the regular versus non-regular situation to clarify the significance and implication of each of the regularity conditions ([ 3 ]). In this paper we summarize the results thus far obtained.

2. Results Suppose that X1, X2, • • •, X,,, • • • are a sequence of i. i. d. random variables whose distribution Ft depends on an unknown parameter ^ESCR°.

A parameter to be estimated is

considered to be a real -valued function 9=0(^) of In the regular case it is assumed that i ) The distribution Ft is dominated by some measure p with the density f; (ii) The support , i. e. the set Ae= {x : f e(x)>0} is independent of e ; (iii)

f&) is continuous in ^ for almost all x Lit]; f t(x) is continuously differentiable (up to specified order ) in ^ for almost all x [It] ; lr ^ (v) The information matrix le = E log f t(x) }j log f t(x)} 1 exists. (iv)

Moreover it is usually assumed that ( vi) O(ff) is continuously differentiable (up to specified order) in ^. Then with some other set of regularity conditions it is shown that if the maximum likelihood estimator (m. 1. e.) is 0 =6(^) /whose ^/ is the m . 1. e. of 6, V n (6-8) is asymptotically normal with mean 0 and variance 100 )1I-' I a) and B is asymptotically efficient, and it is also shown to be second order asymptotically efficient. * Received on May 30, 1979. This paper was submitted to the Second Vilnius Conference on Probability Theory and Mathematical Statistics, 1977 to which the authors could not attend nor present it. University of Tokyo *** Statistical Laboratory, University of Electro-Communications

184 64 Kei TAKEUCHI and Masafumi AKAHIRA

The conclusion involves following statements : (a) The order of convergence of estimators is n''2 ; (b) Asymptotic bound for the asymptotically efficient estimators is given by the normal distribution. (c) The m. I. e. asymptotically attains uniformly that bound ;

(d) It also attains the bound up to the order n-112 (Pfanzagl [ 7 ], Takeuchi and Akahira [8]). If either of the above assumptions (i)-(v) fails to hold, some of the conclusions above may not necessarily hold true.

(1) In the undominated case, if the family of the probability distributions is decomposed into dominated subclasses such that E=UEa (a$9) a

and correspondingly the sample space X is also decomposed. a

and

Pr {X E 'a I B} =1

if

O E Ea ;

Pr {X E 'a I O} = 0

if

O Er Ea,

then each subclass Ea can be treated separately. For other undominated case little, if any, has been known.

(2) If the support depends on ^, the order of convergence of estimators is not necessarily to be n12. Assuming smoothness of O(ff) the order usually depends on the power a defined by

1- Pt- {At} = O(p ^ - ^' It ') as ^' approaches to ^, where PC, is the probability measure with the density f e'.

If f e(x)

is sufficiently regular, the maximum order of convergence is n'/" when 02 (Akahira [ 11. (3) Discontinuity of f E(x) in ^ plays similar role as the non-identical support (actually the later can be dealt with as the case when ft(x) = 0 for some x). (4) When the support depends on E, an (uniformly) asymptotically efficient estimator may not exist, and m. 1. e. may not be asymptotically good. In the case of convergence of order n'l"(a<2) m. 1. e. is not asymptotically efficient, nor even asymptotically admissible : Suppose that

f(x-B)-

cexpr-

2

(x-B)2}

if

Jx-BI <1 ;

0 t 1 otherwise, where c is some constant. Then the m. 1. e. is asymptotically equivalent to min X; +1 with probability 1/2 t

185

August, 1979 Note on Non- Regular Asymptotic Estimation

65

max X, -1 with probability 1/2 , i while the estimator 6* = (min Xi +max Xi )/2 is asymptotically uniformly more concentrated. (The asymptotic distributions of the two are equal except for the scale being twice as large for the m.1. e. as for 0*). In this case there exists one-sided asymptotically efficient estimator, but no estimator is asymptotically uniformly efficient. If we define two-sided efficiency by inf lim Pr { 10 -0 I
together with the asymptotically median unbiased (a. m. u.) condition of B (Takeuchi and Akahira E8]), then 0* above is uniformly (in t) asymptotically two-side efficient. Also Weiss and Wolfowitz's maximum probability estimator is uniformly inferior to 9* (Akahira and Takeuchi [3], [ 4 ]). When the order of convergence is (n log n)"2 and the bound of the asymptotic distributions is normal, then m. 1. e. is asymptotically efficient. (5) If the density is not smooth enough, while Fisher information is well defined, then in. 1. e. is asymptotically efficient but not second order (that is in the order n-"2) asymptotically efficient as in the regular case : Suppose that f(x-B)= 2 exp{- Ix-O } The asymptotic distribution of the in. I. e. 9 is given by 2

lim ✓ n Pr {V n (0-0)
where 0 (u)=Su

0(t) dt with 0(t)= Z e- t'/2, while the bound of the asymptotic distributions

of a. in . u. estimators is given by z 0(t)+ 6 n O(t)sgn(t).

which can not be uniformly attained (Takeuchi and Akahira E8]) but is attained for a specified to by an estimator B satisfying

-EIXi-8I+^ Xi-B+lo -c=0, n

where c is determined so that B be a. in. u. (Akahira and Takeuchi [ 2 ], [ 3 ]). (6) When the Fisher information is infinity, the asymptotic distribution can be obtained if the distribution of (8/aO)logf (X, 0) belongs to the attraction of some stable law, and m. 1. e. may or may not be asymptotically efficient : Suppose that Xi=(X,i, Xzi) (i =1, 2, •••) ; Xzi=6Xii+Ui (i=1, 2, ..) , where Ui's are i . i. d. with the density f (u) for which I= J {f1(.)} 2/f (u)du
186 66 Kei TAKEUCHI and Masafumi AKAHIRA

The above examples are chosen to illustrate the situation which usually happens in more general cases.

References [ 1 ] Akahira, M.: "Asymptotic theory for estimation of location in non-regular cases, I : Order of convergence of consistent estimators, Rep. Stat. Appl. Res., JUSE, 22, 8-26 (1975).

[ 2 ] Akahira, M. and Takeuchi, K.: "Discretized likelihood methods-Asymptotic properties of discretized likelihood estimators (DLE's)," Ann. Inst. Statist. Math. 31, Part A, 39-56 (1979). [ 3 ] Akahira, M. and Takeuchi, K.: "The Concept of Asymptotic Efficiency and Higher Order Asymptotic Efficiency in Statistical Estimation Theory," Lecture Note (1979).

[ 4 ] Akahira, M. and Takeuchi, K.: "Remarks on the asymptotic efficiency and inefficiency of maximum probability estimators," (to appear). 151 Cramer, H.: "Mathematical Methods of Statistics," Princeton University Press (1946).

[ 6 ] Huber, P.: "The behavior of maximum likelihood estimates under non-standard conditions," Proc. Fifth Berkeley Symp. on Math. Statist. Prob. 1, 221-233 (1967). [ 7 ] Pfanzagl, J.: "Asymptotic expansions related to minimum contrast estimators," Ann. Statist., 1, 993-1026 (1973).

[ 8 ] Takeuchi, K. and Akahira, M.: "On the second order asymptotic efficiencies of estimators," Proc. Third Japan-USSR Symp. on Probability Theory. Lecture Notes in Mathematics 550, Springer Verlag, 604-638 (1976). 191 Weiss, L. and Wolfowitz, J.: "Maximum Probability Estimators and Related Topics," Lecture Notes in Mathematics 424, Springer Verlag (1974).

187 Austral. J. Statist., 22 (3), 1980 , 332-335

A NOTE ON PREDICTION SUFFICIENCY (ADEQUACY) AND SUFFICIENCY' MASAFUMI AKAHIRA AND KEI TAKEUCHI

University of Electro- Communications and University of Tokyo Summary Let V, s$) be a measurable space and 9 a family of probability measures on s4. Let 98 and `C be sub cr-algebras of ,s4 and A0 a sub a--algebra of A. It is shown that if A is prediction sufficient (adequate) for A with respect to '6 and 90, and # is sufficient for 98'i' with respect to P then Y is sufficient for 98 V( with respect to 9a; that if 9$ is homogeneous and (980; 98, ') is Markov for 91, and 980 Ce is sufficient for 98"T with respect to 9A, then 980 is sufficient for 98 with respect to 9i'; and by example that the Markov property is necessary for the latter proposition to hold.

1. Introduction Prediction sufficiency (adequacy) was defined and discussed by, among others, Skibinsky (1967), Takeuchi & Akahira (1975), and Torgersen (1977). The purpose of this note is to show the relation with prediction sufficiency and the "usual" concept of sufficiency in the following situation. Suppose that the observations are given sequentially in time in two sets. Let (X, Y) be a set of observations where X is observed first and Y comes later. In such a situation one may be interested to know how much information in X should be reserved to be later combined with Y without any loss of information in the combined sample. Technically it amounts to giving the condition for the statistic T = t(X), that the combined statistic (T, Y) be sufficient for (X, Y). It is shown that it is sufficient that T be prediction sufficient for Y, and it is also necessary under further conditions. We may further add that in this formulation the distribution of Y can be dependent on X in a general manner so that we can include the sequential design case when the design (that is observation scheme) of Y is determined depending on X. We will, essentially, use the framework of Skibinsky (1967). The notion of adequacy in Skibinsky's paper is, however, replaced by the notion of prediction sufficiency. ' Manuscript received September 13, 1978; revised January 16, 1979.

188

A NOTE ON PREDICTION SUFFICIENCY 333

2. Results Let (T, 4) be a measurable space and 9a a family of probability measures on s4. We denote by 4; s42 the smallest o--algebra which contains each member of two subclasses sal, 42 of A Let XA denote the indicator function of a set A. 98 and T denote sub o--algebras of si, and 980 denotes a sub o--algebra of 90. Definition 1. 900 is said to be sufficient for 9o with respect to 95, symbolically

suf (9; 90),

oJ

if for each B E 90 there exists a 900-measurable function ¢B° such that for every p E9'

4s = EP"XB a.e. [p], where EP° XB denotes the conditional expectation under p of XB given moo. Definition 2. (Skibinsky, 1967; Takeuchi & Akahira, 1975). 910 is said to be prediction sufficient (adequate) for 90 with respect to ig and 9', symbolically

900 pred. suf (-60; 98,19) if Ro suf (-60; 90) and (980i 90, `e) is Markov for 9, that is, 98 and T are conditionally independent given 980 for each p E 9. Lemma. Let (d,°, A, p) be a probability space. (980; 90, W) is Markov for p if and only if for every B E 98 and every C E T

EPx'(X

B nc)

= Xc E P°( XB)

a.e.

[p].

Proof. Suppose that (980, 98, IC) is Markov for p. For every B E R, every C E ', every B0 E 900 and every C e ', ^o Ep

°nC'

XBnc dP = XBnc B„n

dp

C'

= Ep(XB„nBXc'nc) = Ep[EP°(XB„nBXc'nc)] = Ep[(EP°XB°nB)(EP°Xc,nc)1 = Ep[EP"(Xc'ncEP°XB„nB)]

Ep[Xc'nc EP"XB°nBJ = Ep[XB„nc'ncEP°XB XcEP°XB dp. B„nc'

Since {B0 fl C B0 E 980, C E '} generates the a--algebra 900vre, it follows that for every Be 90 and every C ET

EP°v'eXB = XcEp"XB

a.e. [p].

189 334 MASAFUMI AKAHIRA AND KEI TAKEUCHI

Conversely, if for every B E 98 and every C E W EP° XB = XcEP°XB a.e. [p], then for every B E 98 and every C E `' E',(XBnC) = EP°( "XBIC) = EP°(XcEP°XB) = EP°XBEP°Xc a.e. [p]. Hence (98,,;98, ') is Markov for p. Theorem 1. Let &' be a sub if-algebra of 980"`e. Suppose that 980 pred. suf (9; 98, W) and 9' suf (P; 980"`x). Then 9° suf (91; 98"<). Proof. For every B ER, every C E IC and every p E

9'

EPX B nC = EP ( Ep ° 'fXB nC) = EP(XcEP°XB)

(by Lemma)

= EP(Xc4s°) (by 98o suf (9; 98)) a.e. [p]. Since XcoOa ° is a 91 o Ig-measurable function and 9 suf (9; 980 w), it follows that there exists a 9-measurable function (Psnc such that for every pc9' 4snc= EP(Xc4s°) = EPXsnc

a.e. [p].

Since {B fl c I B E 98, C E W} generates 98"`x, it is seen that 9 suf (9 ; 98"x). For Theorem 2 we assume that 9 is homogeneous, that is, p and q are mutually absolutely continuous for every p and q in 91. Theorem 2. Suppose that (980; 98, W) is Markov for 9 and 980"'e suf (9; 98"`'). Then 980 suf (9'; 98). Proof . Since P is homogeneous and 980"` suf (-60; 98"') and (980; 98, W) is Markov for 9, it follows by the Lemma that for every B E 9V there exists a 9f 0-measurable function (AB'- such that for every pE9 EP°(XB) = EP°(EP°'XB) = EP°(4e°) = Os'

a.e. [p].

Hence 980 suf Remark. It is shown by the following counter-example that the homogeneity of 95 is necessary for Theorem 2. Suppose that 98 =If= {4, B, B`, X}, 98o = {4, '} and 9= {p, q} with p(B) = q(B`) = 1. Then (980; 98 , ') is Markov for 9' and 980"'C suf (9a; R"ig) but 980 is not sufficient for 98 with respect to 95.

190 A NOTE ON PREDICTION SUFFICIENCY 335

The following example shows that the Markov property is necessary for Theorem 2 to hold. Suppose that (X, Y, Z) are jointly normally distributed with means (0, 0, 0) where 0 is the unknown parameter and with known variances and convariances to be later specified. Let 910, y8 and' be the o -algebras induced by, respectively X, (X, Y) and Z. Let us assume that Y can be expressed as Y = X - Z + U, where U is independent of X and Z. We further assume that Cov (X, Z) = 0 and V(X) = V(Z) = V(U) = 1, where Cov and V denote covariance and variance, respectively. Then it follows that E(U) = 0, V(Y) = 3, Cov (Y, X) = 1 and Cov (Y, Z) _ -1. And given (X, Z), Y is conditionally normally distributed with mean X-Z and variance 1, hence its conditional distribution is independent of 0. Therefore (X, Z) is sufficient for (X, Y, Z) or equivalently 9"19 is sufficient for M"'. However, X is not sufficient for (X, Y), that is, 98o is not sufficient for 98, because given X, Y is conditionally normally distributed with mean X - 0 and variance 2, which is dependent on 0. Nor are Y and Z conditionally independent given X, hence (_qO, 98, ig) is not Markov. References SKIBINSKY , M. (1967). Adequate subfields and sufficiency. Ann. Math. Statist. 38, 155-161. TAKEucHI, K. & AKAHIRA , M. (1975). Characterizations of prediction sufficiency (adequacy) in terms of risk functions . Ann. Statist. 3, 1018-1024. ToRGEnsEN , E. N. (1977). Prediction sufficiency when the loss function does not depend on the unknown parameter. Ann. Statist. 5, 155-163.

191

Rep. Univ. Electro-Comm. 31-1, (Sci. & Tech. Sect.), pp. 89-96 August, 1980

Third Order Asymptotic Efficiency and Asymptotic Completeness of Estimators* Kei TAKEUCHI** and Masafumi AKAHIRA***

Abstract In the previous paper it was shown that the maximum likelihood estimator (MLE) was third order asymptotically efficient in multiparameter exponential cases. In this paper it is shown that the result is extended to more general cases. The concept of asymptotic completeness of an estimator

is introduced

and it is shown that the MLE is higher order asymptotically complete in the appropiate classes.

1. Introduction In the previous papers [ 6 ], [ 7 ] and [ 3 ] Takeuchi and Akahira have discussed the third order asymptotic efficiency of the maximum likelihood estimator (MLE) and the generalized Bayes estimator in multiparameter exponential cases.

In this parer we show that it is

possible to extend the result on the MLE to more general cases in a very straightforward manner by introducing appropriate classes of estimators. Further we introduce the concept of an estimator in the sense that it can beat asymptotically any other " regular " estimators by making appropriate transformations if necessary and show that the MLE is higher order asymptotically complete in the appropriate classes.

2. Third order asymptotic efficiency of maximum likelihood estimator in general cases In order to generalize the results in the previous paper E6], it is necessary to define appropriate classes of estimators among which asymptotic distribution is to be compared. As was shown in Akahira and Takeuchi [ 3 ] it is possible to establish the second order asymptotic efficiency of the modified MLE within the class of second order AMU estimators without any further restriction of the class of estimators to be considered. On the contrary in the previous paper [ 6 ] we restricted the class of estimators to be of a special type of functions of the sufficient statistic which we called the extended regular estimators. As will be shown later we can not usually have third order asymptotically efficient estimators without such restriction. In more general cases where sufficient statistics of finite dimension do not exist, the argument in the previous paper [ 6 ] can not be applied directly. We shall establish the third order asymptotic efficiency of the modified MLE and also of the modified generalized Bayes estimator within the classes C and D of estimators which are defined below. For the simplicity's sake * Received on June 10, 1980 ** University of Tokyo *** Statistical Laboratory, University of Electro-Communications

192 90 Kei TAKEUCHI and Masafumi AKAHIRA

we shall first consider the real parameter (one dimensional) case, and that the sample is independently and identically distributed with all necessary regularity conditions to be specified later. Let (1, B)be a sample space. We consider a family of probability measures on B, B= [Pe: 8 E 0} , where the index set 0 is called a parameter space.

We assume that 0 is an open

set in a Euclidean 1-space R'. Consider n-fold direct products (2C°' , B(n)) of (-T, B) and corresponding product measures Po,. of Pe.

For each n=1,,..., the points of _T(n) will be An estimator of 8 is defined to be a sequence {en} of B(n)-

denoted by .xn=(xi, . . . , xn).

measurable functions On on X(n) into (9 (n = 1, 2.... ).

For simplicity we denote an estimator as

On instead of {On}. For an increasing sequence of positive numbers {c.} (cn tending to infinity) an estimator On is called consistent with order {cn} (or {cn} -consistent for short) if for every E> 0 and every 9 E 0 there exists a sufficiently small positive number 6 and a sufficiently large positive number L satisfying the following :

lim sup n-+oo

8: 19-91<e

Pe. n {cn l8n-81 >L} <E

(Ell.

For each k=1, 2, . , a {cn} -consistent estimator On is called to be k-th order asymptotically median unbiased (or k-th order AMU) if for any 9 E 0 there exists a positive number S such that lim

sup

cnk-' Pe, n {6n <8} - 2

sup

cnk-' Pe,n

n-.oo e : 1e-91 <6

lim n-.oo 9 :

{B n >_8} -

1e-91<6

2

=0; =0.

For en k-th order AMU, Go(t, 8)+cn-'Gi(t, 8)+ ... +Cn-(k-')Gk-i(t, 8) is called to be the k-th order asymptotic distribution of cn(On-8)(or On for short) if Jim cnk-' I Pe,n {cn(en-8)
We denote that G1(t, 8) (i =1, ... , k-1) may be general absolutely continuous function, hence the asymptotic distribution for any fixed n may not be a distribution function. Suppose that On is k-th order AMU and has the k-th order asymptotic distribution Go(t, 8)+cn-'Gi(t, 8)+ . . . +Cn-(-')Gk-1(t, 0). Letting 8o(E(9) be arbitrary but fixed we consider the problem of testing hypothesis

H+: 8

= 8o+tcn -'( t>O) against K: 8=80. Put 01/2= { {/in} : Ee,+to„ ',n(On)=1/2+o(cn-(k-')), 055 n(.xn)<1 for all x.E_Ttn' (n=1, 2.... )}. Putting Adn,e,={cn(Bn-Bo)
lim Pe,+tC.-'> n(AB..e,)= lim Pe, +tcA ', n{On <8o +tcn-'}

n-»oo

Hence it is seen that a sequence

to

01/2.

n-+

2

{xAB. eo} of the indicators of AB., e, (n=1,2,...) belongs

If

sup lim cnk-' {Ee,,n(On)-Ho+(t, 8o)-cn-'Hl+( t, 80)- ... -cn-tk -'"Hk-1+(t, 80)} =0, (0.(E^ n-.oo

then we have Go(t, 8o)< Ho+(t, 8o) ; and for any positive integer j (
G;( t, 80)<_ H, +(t, 8 o).

193 August, 1980 Third Order Asymptotic Efficiency and Asymptotic Completeness of Estimators 91 Consider next problem of testing hypothesis H-: O=Oo+tcn-' (t<0) against K : 0=00.

If

inf lira cnk -' {Ee,.n(0n)-Ho-(t, Oo)-cn-'Hi-( t, Oo)- Oo)} =0, then we have Go(t, Oo)> Ho-(t, Oo) ; and for any poistive integer j(H[ (t, Oo). 0, is called to be k-th order asympotically efficient if the k-th order asymptotic distribution of it attains uniformly the bound of the k-th order asymptotic distributions of k-th order AMU estimators, that is, for each OEO

Gi(t, 0)- Hi+(t, 0) -{H;-(t, 0)

for t>0; for t<0,

2 ], [ 3 ], [ 4 ], [ 5 ]). [Note that for t=0 we have Gi(0, O)=Hi+(0, O)=Hi-(0, 0) (i =0, ... , k-1) from the condition of k-th order asymptotically median unbiasedness.] Thus if On* is k-th order asymptotically efficient in the above sense, then for any k-th order AMU estimator On we have

lim cnk '[Po, n {-a0.

(2. 1)

n_w

Note that the k-th order asymptotic efficiency defined in the above

for all a, b> 0 and 0 E 0.

gives a sufficient condition but not a necessary condition (2. 1). We assume that for each 0EO Pe is absolutely continuous with respect to a or-finite measure p.

We denote a density dPe/die by f (x, 0).

is given by

11 f (xi, 0). i=1

Then the joint density of (Xi, . . . , Xn)

In this section we shall deal with the case when cn=V n .

First we shall restrict our attention to the class of (first order) asymptotic efficient estimators which we shall call the. class A. It has been well established that any estimator On in the class A can be expressed as

Vn (9n-0)=I(B)1^ n a n+ op(1), n

where Ln= E log f (Xi, 0). i=1

Secondly, we call the class of estimators which are second AMU and second order asymptotically efficient as class B. Then any estimator On which is second order AMU and for which the distribution of 111 n (On-B) admits the Edgeworth expansion up to the order n-112 belongs to the class B. Now we generalize the concept into one step further. We call the class C the class of estimators On which are third order AMU and for which the distribution of ✓ n (On-0) admits the Edgeworth expansion up to the order n-' and

V n (Bn-B)I(0)V n 0 +l n-Q(0)+OP( nI where Q (0) is a quantity of stochastic order 1. We call the estimator O n belongs to the class D if in the above we have

E[IL.Q(6)^]=O'

194

92

Kei TAKEUCHI and Masafumi AKAHIRA

where E stands for the asymptotic mean. It is to be noted that Lemma 4. 1 in the previous paper [ 6 ] can be generalized straightforwardly as follows.

Lemma 2.1. Suppose that ZB is a function of X1, ... , X. and B and is differentiable in 0. Then E9(UZ9)=V n aBEe( Z9)-1 1 EB` 06 1 -W where U= V a log f (X, 0), provided that the differentiation under the integral sign of E,? I(0) 8B (Z9) is allowed. In the following discussions we assume that expectation of the relevant quantities always exist and the asymptotic value of the expectation and the asymptotic mean are the same at least up to the required order. Remark : The problem of non-existence of the relevant moments vis-a-vis asymptotic moments can be evaded in the following way. Suppose that we have a smooth and montone increasing transformation ij=0(6) so that 17 is bounded. Instead of 0 we may consider 27 as the parameter to be estimated. Since , is bounded we may assume that all the estimators are also bounded hence have moments of any order, and we can easily formulate the condition for the asymptotic moments and the moments of the asymptotic distribution being equal. If we consider the estimation of 77 instead of 0 the asymptotic theory based on moments is applicable in a straightforward way. Transforming back to the estimator of 0 by 9=0-'(fj) and considering the monotoniticy of the transformation of the distribution, the results are also applicable to the estimation of 0. Let us first consider the class C. Suppose that ✓ n (6„-B) is expanded as ✓ n (9„-B)= I(^j)+ 111 n Q(0)--On(n ),

where Q(B)=O,(1) and Zi(B)=- n

tZi - - log

f (X;, B).

Then we have the following lemma. Lemma 2.2. EB[Zi(B)Q(0)] =o(1) Proof. Define Te=V n (fin-9)- I (8))= .LQ(9)+oa(Vn). Then we have IA I,/-n Do V_n n aBE9[Q n). -0"(v 111

From which we have

(6)]+1-ynE^

+ov(n)

195 August, 1980 Third Order Asymptotic Efficiency and Asymptotic Completeness of Estimators 93 Ee[Zi(0)Q(0)]= ✓ n Ee[Zi(B)Te]+o,(1) =0,(1).

Thus we have the asymptotic cumulants of ✓ n (6n-0) up to the order n-' as Ee[V n (6n-0)]= nn iz1(0)+ /t2(B)+o(n) ;

Ve[✓n (6n- B)] = 1

+

n

v 2(B)+o (n) ;

n n)

K3[V n (6n-e)]=Tn P3(0)+ - 73(0)+O

K4[l/ n (6n-0)]= n P4(0)+0(n ). We can show in exactly similar way as in the previous papers [ 5 ] and [ 6 ] that ^3(B) = - 3J(9) +2K(6) .

I(0)3 P'(B)=-I8)•{3H*(0)+12N*(0)+4L(0)} +IB)s{2J(0)+K(0)} {J(9)+K(8)} where

L(O)= E ^{ a 303 log f (X, 0)}{aB log .f (X, 0)}] N*(O)=

E9[ f

logf(X, 0)+I(6)}{a log f(X, B)

}2]

H *(B)= Ee[{a8 log f(X, 0))4]-3I(o)2. Since the odd order cumulants affects only the asymmetric aspects of the asymptotic distribution, the asymptotic expansion of Ps,. {V n 1 6n - 01
is independent of them. Hence we have

the following theorem. Theorem 2. 1. Let OML* be the modified MLE in the class C and On be any other estimator in the class C. Then it holds that limn[Pe,n{Vn I6ML*-0j 0 n-*oo

for all a> 0 and all 6 E 6. For the class D we further have that r3(0)=0 and p1i(0)=/33(0)/6, hence all the asymptotic moments are determined up to the order n-' except V2. Since V2 is minimized for the modified MLE we have the following theorem.

Theorem 2.2. Let OML** be the modified MLE in the class D and On be any other estimator in the class D. Then it holds that lim n[Pe,n {-a
for all a>0, all b>0 and all BE 6.

3. The concept of asymptotic completeness of an estimator We consider first the case of a real-valued parameter. The concept of asymptotic efficiency and higher order asymptotic efficiency gives a very strong condition for the "goodness" of an estimator in that it uniformly maximizes asymptotically the probability of its lying within a

196

94 Kei TAKEUCHI and Masafumi AKAHIRA

neighborhood of the value irrespective of the parameter value and the choice of neighborhood. But only one somewhat artificial restriction is that the estimators considered should be (higher order) asymptotically median unbiased. It is, however, easily seen from the context that the only requirement is that the class of estimators to be compared must have the same asymptotic "location" whatever the exact meaning of the word may be, without which nothing can be derived. Therefore one may consider different types of the definition of " location ", and accordingly different type of "adjustment" of the estimator to satisfy the condition. Thus we may consider in the following way. Suppose that we want to establish asymptotic optimality of an estimator One, e. g. MLE, in some class of estimators.

What has been virtually established is that for any estimator On in the class if both On and Ono are adjusted like O n*=On+

1 g(On) ;

n

Ono*=On0+ 1 h(OnO) ; n so that On* and On0* are both (higher order) asymptotically median unbiased then 0n°* is (higher order) asymptotically uniformly "better " than On*. If we change the definition of " location " the way of "adjustment " will be changed, but the same results hold for adjusted estimators On" and On*.

This fact will lead to the following question : Instead of comparing

two adjusted estimators On* and On°*, isn't it possible to compare On and the adjusted On* ? Naturally, the way On* is adjusted depends on On to be compared, but is independent of the parameter. And we introduce the following concept. An estimator On* is called to be asymptotically complete with respect to a class M of estimators if for any On EM there is an "adjusted" estimator On **=On* +gn(On *)

so that for any a>0, b>0 and 0 we have lim [P9,- {-a0.

(3.1)

Now let L be a loss function such that L(0)=0 and L(u) is an increasing function for u>0 and a decreasing function for u<0 and it is bounded. If On* is asymptotically complete in the above sense, then for any estimator On in M there is On** which is a function of On* for which lim [ Ee, n( L( cn( On**-B)))- E9,n(L(cn( On- 0)))]<0. (3. 2) n-co

Higher order asymptoic completeness can be defined in a similar way. For each k=2, 3, ... , an estimator On* is called to be k-th order asymptotically complete with respect to a class M of estimators if for any On E M there is an "adjusted" estimator On**=On*+gn(On*) so that for any a>0, b>0 and 0 we have

lim cnA -'[Pe,n {-a0.

(3.3)

n-'oo

Further On* is called to be k-th order asymptotically symmetrically complete with respect to a class M of estimators when (3. 3) holds when a= b.

If On* is k-th order asymptotically complete

with respect to M, then for any estimator On in M there is On** which is a function of On'e`

197

August, 1980 Third Order Asymptotic Efficiency and Asymptotic Completeness of Estimators 95

for which limcnk '[ Ee, n( L( cn( Bn * *-6)))- E9,n(L(cn(Bn- 0)))]S0.

(3.4)

n-.oo

If 6n* is k-th order asymptotically symmetrically complete with respect to M, then (3. 4) holds for all symmetric loss function around the origin. Unfortunately the above definition is meaningless as such, since no estimator can asymptotically complete, which can be seen below. Let On* be an asymptotically efficient estimator (e. g. MLE) and 1/ n (0n*-0) be asymptotically distributed with mean 0 and variance 1/I, where I is the Fisher information. Let On be an AMU estimator with asymptotic variance larger than 1/I. Obviously On is not asymptotically efficient and inferior to On*. A naturally adjusted estimator of On* corresponding to On will be On**=On*+c/v n

which has the same

asymptotic mean with On. Now take (0<)b(0) large enough. Then we have lim [Pe,n{-a
which contradicts the condition of asymptotic completeness. However we may adjusted On* we can not get uniformity irrespective of a and b.

It is, of

course, possible to get the reverse inequality if we adjust On* depending on a and b, but the nice uniformity would be lost. There is nothing unnatural in this problem : If there is a bias in the estimators, the estimator with large variance may hit upon the true value by chance but one with small variance has little chance to get close to the true value. Therefore we must modify the concept of the asymptotic completeness and call On* asymptotically symmetrically complete if (3. 1) holds when a=b, then (3. 2) holds for all symmetric loss function. The class F of estimators here is more specifically defined as the class of estimators On such that cn(On-6) has the asymptotic distribution in the sense that lim Pe, n {cn(On-0)
where the convergence is locally uniform in 0 and Fo(a) is absolutely continuous in a and continuous in 6, and we further assume that cn is equal to the maximum order of consistency. Now we have the following theorem, the proof of which is omitted. Theorem 3. 1. Let On* be asymptotically efficient and the density of the asymptotic distribution of cn(On-B) is symmetric around the origin. Then On* is asymptotically symmetrically complete with respect to the class F.

In this case it is seen that On* need not be modified. Higher order asymptotic compleness has, however, more meaningful implication. Now we consider the regular case with cn=Vn and we define the classes B*, C* and D* which are defined in the same way as the class B, C and D, respectively but omitting the condition of higher order asymptotic median unbiasedness. Then the following theorems hold : Theorem 3.2. The MLE is second order asymptically complete with respect to the class B*. Theorem 3.3. The MLE is third order asymptotically symmetrically complete with respect to the class C*.

198 96 Kei TAKEUCHI and Masafumi AKAHIRA

Theorem 3.4. The MLE is third order asymptotically complete with respect to the class D*. The outline of the proof of Theorem 3. 4 is as follows :

Let 65ED* and the asymptotic bias of Bn be Ee.n(Un(0n-6))_nµi(B)+o( n and also of Bn* E9,n(V n (Bn*-0))=n Ftl*(B) 1 In ).

Let the modified estimator fi0n** be defined as 11 (7n**.e,,*+ i {1ai*(en*)- f.Li*(en*)} n

Then we can prove exactly as of Theorem 6. 1 in the previous paper [ 6 ] that

lim n[Pe, n

a < I/ n (O n **-0)0

n-+oo

for all a, b>0 and 0, because the asymptotic moments of V _n (6n-0) and V n (en**-B) are the same up to the order n-' except the second term in the variance which is smaller for On*. The proofs of Theorem 3. 2 and 3. 3 are similar and even simpler. The classes C* and D* are much natural classes than classes C and D and Theorems 3. 2, 3. 3 and 3. 4 may be nearly the final conclusion of the asymptotic derivability of the MLE because we have the problem of the choice of the way of modification which should be based on the consideration of the asymptotic bias versus asymptotic variance, which can not be decided once and for all, but what is established in the above implies that we only need to consider the estimators which are modified MLE but it is not necessary to take any other (regular) estimators into consideration.

4. Conclusion remark The conclusion of this paper can be easily generalized to the multiparameter cases and the discussion is very much similar to that in the multiparameter exponential case. So we omit the details of the derivation.

References Akahira, M.: Asymptotic theory for estimation of location in non-regular cases, I : Order of convergence of consistent estimators , Rep. Stat. Appl. Res., JUSE, 22, 8-26 (1975). [ 2 ] Akahira, M.: A note on the order asymptotic efficiency of estimators in an autoregressive process, Rep. Univ. Electro-Comm., 26, 143-149 (1975). [ 3 ] Akahira, M. and Takeuchi, K.: The Concept of Asymptotic Efficiency and Higher Order

[1]

Asymptotic Efficiency in Statistical Estimation Theory, Lecture Note (1979) ; Revised (1980). [ 4 ] Akahira, M. and Takeuchi, K.: Discretized likelihood methods : Asymptotic properties of discretized likelihood estimators (DLE's), Ann. Inst. Statist. Math., 31, Part A, 39-56 (1979). [ 5 ] Takeuchi, K. and Akahira, M.: On the second order asymptotic efficiencies of estimators, Proceedings of the Third Japan-USSR Symposium on Probability Theory, Lecture Notes in Mathematics, 550, Springer-Verlag, 604-638 (1976). [ 6 ] Takeuchi, K. and Akahira, M.: Third order asymptotic efficiency of maximum likelihood estimator for multiparameter exponential case, Rep. Univ. Electro-Comm., 28, 271-293 (1978). [ 7 ] Takeuchi, K. and Akahira, M.: Asymptotic optimality of the generalized Bayes estimator in multiparameter cases, Ann. Inst. Statist. Math., 31, Part A, 403-415 (1979).

199

Statistics & Decisions 1, 17-38 (1982) © Akademische Verlagsgesellschaft 1982

ON ASYMPTOTIC DEFICIENCY OF ESTIMATORS IN POOLED SAMPLES IN THE PRESENCE OF NUISANCE PARAMETERS Masafumi Akahira and Kei Takeuchi

Received : revised version : May 4, 1982

Abstract The concept of asymptotic deficiency is extended to the case when a common parameter 6 is estimated from m sets of independent samples each of size n , and the asymptotic deficiencies of some asymptotically efficient estimators relative to the maximum likelihood estimator based on the pooled sample are discussed in the presence of nuisance parameters.

1. Introduction. The third order asymptotic efficiency of the maximum likelihood estimator (MLE) in the case of independently and identically distributed

( i.i.d.)

samples has been established by Pfanzagl and Wefelmeyer ( 1978 ), Akahira and Takeuchi ( 1978, 1981a ) and Ghosh , Sinha and Wieand ( 1980), and it can be formulated in terms of asymptotic deficiency introduced by Hodges and Lehmann ( 1970) ( Akahira, 1981 ; Akahira and Takeuchi 1981a, 1981b). Let X1, X2, Xn , ...

be i.i.d. random variables according to

the distribution with a density function f(x,6) where 6 is a real

AMS Subject Classification ( 1980): 62F12, 62F10. Key words and phrases :

Asymptotic deficiency , asymptotically efficient

estimator , maximum likelihood estimator , pooled samples, nuisance parameter, loss of information , asymptotic unbiasedness , ancillary statistics.

200 18

AKAHIRA-TAKEUCHI

parameter . Under a suitable set of regularity conditions it can be proved that the MLE 0n of 0 can be asymptotically expanded into the form

^(6n- 0) = I Z + 1 Q* + opI 1 1 , T \Jn/ n CC where Z 1 G a0 log f(xi,0) and I = E0[{ae log f(x,0)}2] and that vrn- i=l which admits the same for any asymptotically efficient estimator

60

type of expansion x(00 0)

1 Z+ 1 Q+ op\ 1 1 n t An- 0

we have * V(Q0) > V(Q )

where V designates variance. This implies that we can construct a "modified " estimator ( relative to 0n) of the form

0** = 6* + h(0*) n n n n so that 0n* has the same asymptotic bias as

6n up to the order n-1

and that

lim n[P0 n{rI 6*n *-0 < a} - PO n n ,

{,rI 0n-0

< all > 0

for all 0 and all a > 0. Moreover, if n1 is defined so that the n (with Q**) based on the sample of size n1 has modified estimator 0 ** the same asymptotic distribution as that of the estimator

6n

(with Q0)

based on the sample of size n(>nI) up to the order n ' , we have

lim(n-n1 ) = I{V(Q0)- V(Q**)} n-o (Akahira, 1981). We call the above limit the asymptotic deficiency of 6n relative to the MLE. Note that the right-hand side of the above

201 AKAHIRA-TAKEUCHI

19

equation does not necessarily mean the difference of the variances of estimators but of the asymptotic variances , hence we do not need to bother about the remainder terms ( Akahira and Takeuchi , 1981a). In this paper we extend this concept to the case when a common parameter 0 is estimated from m independent sets of samples and discuss the deficiencies of some asymptotically efficient estimators relative to the MLE based on the pooled sample from the distribution with different nuisance parameters.

2. Asymptotic Deficiencies Of Estimators For Samples From The Distributions With Different Nuisance Parameters. Suppose that it is required to estimate an unknown real-valued parameter 0 based on m samples of size n whose values are

Xis (i = 1,...,m;

j = 1,...,n) from the distributions with different nuisance parameters Now we assume that for each i Xil, Xi2, ...' Xin

Ei (i = 1,...,m ).

are independently and identically distributed according to a density function

f(x,0,Ei)

with respect to a a- finite measure p.

We assume further regularity conditions: (A.1) For each i the set {xlf(x,0,Ei) > 0} does not depend on 0 and Ei. (A.2) For each i , for almost all x[ul, f (x,0,^i) is three times continuously partially differentiable in 0 and Ci. (A.3) For each 0 c 0 and each i

0 < I80 = E [{ae log f(Xij,0 )}2l 2 -E[a2 log f (Xij,Oi)l

< -o;

0 < Iii = E[{---L log f(X1^e ,^i)}2l i

202 AKAHIRA-TAKEUCHI

20

2 -E[a 2 log f ( X.j6,Ei)) i < CO

(A.4) The parameters are defined to be "orthogonal" in the sense that 2 E[862 log f i

(Xi^,O,Ei)^ = 0 (i

Remark that this assumption is not necessarily restrictive, since 901Y ( i = 1,...,m) so

otherwise we can redefine the parameters that they have the above orthogonality. (A.5) There exist

2 -log f (Xij,e, i)}{se log f ( Xij,0,^i)}] 000 - E[ { De 2 2 se2 log f ( Xij,6,^i)}{ai log f ( Xij,6 ,Ei)}] JeeE = E[{ 2 JUF = E[{808 1 log f

Je - E[{

(Xii ,O, i)}{a6 log f ( Xij,O,Ei)}]

2 868 1 log f(Xij,O , i)}{a i log f (Xij,e ,ci)}I

2 Je = E[ { a 2 log f (Xij,6,i)}{se log f (Xij,U,Fj)}} i

K000

= E[

{-

Ke0 = El {-56

log f(Xi3,U, i)}3 ]

log f (Xis , e , Ei)

,

}2 {al

log f ( Xis , e , i) } ]

2 2 E[{a 2 log f ( Xij,6,Ei)} 2l - 160 M0066 = 30 2 Me0 = E[{8A8ki log f(Xij,U, i)}2] , and the following hold:

203 AKAHIRA-TAKEUCHI

21

3 E[ -2

log f (Xij,A,Ei) l = -3J600-K B00 1 3 3 log ,B, i)l = -ji E[ 2 f (Xij

aA ai E[ 2 log f(Xij,B, i)J _ -Je aAa^i for i = 1, .

., M.

From the assumption ( A.4) it is noted that K88 Denote the log likelihood functions

= JBcA - JBA ' Li(8,Ci) ( i = 1,...,m) and

L(A,El,...,Em) by n c Li(8,Ci) = G log f(xij,8,^i) (i = 1,...,m) j=1 m m n L(8,^1,..., m) _ L.( 8, log f (xij,0,Ei) i=1 i=l j=l

I E

When are known, we denote by 80 the MLE of 0 based on L(0, 1,..., m).

For each i = 1, ..., m let

of 0 and based on

Ai and be the MLE's

L i,E1 (0 ) , respectively .

Let

Em

be the MLE ' s of 8 , Cl, ..., ^m based on L(0 , 1,..., m) , respectively. ( i) The MLE 80 when

are known.

Since A0 is the MLE of 0 for known parameters

^l' ..Em '

it follows that

ae L(BO,l,...,m) = 0

Then it is seen by a similar way as Akahira and Takeuchi ( 1981a, page 92) that the expansion of 0 0 is given by

'(e0 A)

+I Ze + 1 *2 z00Z0 - 31 *3 Ze2 + 0 1 I VI 2^ fn zA+ QO+0 (say) ✓n p (i)

where

204 AKAHIRA-TAKEUCHI

22

m

in

Ze, Zee

Ze

I

zee , it

=

m

i

in

i

i=1 1A6> J i=1 JeBA

m and K* _ Ke66 with i=1

Ze 1 An-

a6

Li(,Ei) 2

zee =

1 {a 2 L. ( A, i) + nIeA} n ae

Hence it is easily seen that 1 * * *2 (J*+K*) 2 (2.1) V(QO) = I *4 (M I - J ) + 21 *4 * m

where M =

I i=1

Me000*

(ii) The MLE 's 61, ..., . Since a * * aA L(8 1'..., m) = 0

0 (1 = 1,....m)

it follows by a similar way as the case (i) that the stochastic expansion of 6 is given by

r(e*-A)

* * 1 * 1 * 3J +K *2 2iT 1*3 ze 1* z0 + ,r I* z6ez0 1 m

1

i

i_

1

Z

+ YrnI* i =11 ii z6EZ I*2

i=1 T

1 6i Z ( + op l 2 \ 1 / 2i I i=1 I1 Ze + I

1

QO + 1 vrn-

Q1 r

+ o 1 p

* * Z + 1 Q + 0 1 ( say) I ,T p (^n::)

1

(say)

205 23

AKAHIRA-TAKEUCHI

where

z = 1

Li (e

a

a2

zed

Yrn-

aeaE1

Li(O, i)

(2.2) V(Q*) = V(QO) + V(Q1)

with i2 i2 1 mc 1 i Jere JO V(Q1) = I*2 1 = 1 1 i Mee - Ii - Ii

1 CmC 00 1 1

i2 1

Jere

i2 1CIl Jed

+ *2 iLl Ii I* Ii + 2I*2 i-1 Ii2 (iii) The weighted estimator by ^^a = 2/ae2)

We consider the estimator m C i

1 `fee ei

e _ i= mC

gee

iLl According to the remark by R. A. Fisher (1925) that the second order derivative of the loglikelihood function gives the "actual" information of the MLE , we may compute. By a similar way as in case ( i) we have * (2.3)

ze

+

*

In 3J1 +K1 * * 6861 He ze z e I 2 i=1

1

1I * 2 zeeze -

Iee

+

1 me 3Jieee Keee i i2 + 2v I*2 i=l l

ze ee

206 AKAHIRA-TAKEUCHI

24

+

1

,T

i f 1 Zi Zi - I m J8 Zi z*

m CC

I*

i11

I

i 1 JOSE

8 12

I*2

+ op

i=l

IU

^

r 11

=1 i I Z 21 I* _ *Z 8 I

+1Q 0 1 1 2 +p + 1 Q lr 1 Q2 0 r

z+ I

/rn- An-

1

Q + 0 1

(say)

p

vrn-

where m

Q2 =

2 21

1I 1

(3J688

*

+K660)

i

2

z 8 i Z8

(I lee

Since Cov(QO,Q2) = Cov(Q1,Q2) = 0 it follows that (2.4) V(Q) = V(QO) + V(Q1) + V(Q2) with m (3J i 888+K i888) 2 _ (3J*+K*)2 V(Q2) = 21 *2 i=l let 21 *4 (iv) The weighted estimator by Ie8-I68
m = i=1 C i i=1 188 Then the stochastic expansion of is given by

207

AKAHIRA-TAKEUCHI

1

^(e-e)

*

Ze

1

+ * ✓n I

I

m

1

i

i zeez e

i=1 186

I*2 i= 1

IN

+

i i Jeee +Ke6e

*

i2

z i2 e

Ziz

6 6

6

1 1 zi zi T I* i=1 I e

in

2^ I i=1 Iee

1 m 2J N i e+K ei66

+

1

25

i

i

+ 1 J6 e+K66E zizi v I* i=1 Ii I 00

M Ti

i 2Je^e+ KOe zZ*+o( 1 i=1 I e p

0 Z it 1 C' * L. 12 P T T 2V I i=1 I 1

Z e + 1 QO+ I

1

QI+

✓n

,r

*

ze + 1 + 0

Q3+

1

f

Q4+op 1

r

1 1 (say),

p

I

1

where in Q3 I

i i i 1 in Jeee+Keee 12 Z Z +Z e8 e IN e 21 i=1 IN

1

i=1

1 C 2Je0e+Kei i e6 i * _ 1 * * 3J*+K* *2 zeze 1 *2 zeeze + 2I*3 Ze I *2 i=1 Ii

ee

i i i in J6 6+K6e ze Ze = * Q4 I i=1 I

* 66

z

We divide Q3 into two parts Q5 and Q6 as follows: 1 m C^

Q3

I iLl

* 1 in Jeee i* zeez 6 1* i=l Ii Z6z6 ee ee

1

i

i

1

Ii z66z e

*

I*

1 me i *2 1 mC i i z e + I* icl J66eze + 2 I* ix (Jeee+Ke0e) Ii _ 1 m

ze

I* i=1 I1

ee

m

- Ze i _ J8 Z

I*

6

I

z1

Z

+ 21* ixi (Jeee + K6ee) 16 - I

es = Q5 + Q6 (say) .

z

208 26

AKAHIRA-TAKEUCHI

Since Cov(Qi,Q.) = 0 (i i j; i,j = 0, 1, 4, 5, 6) , it follows that (2.5) v(Q) = V(QO) + V(Q1) + V(Q4) + v(Q5) + v(Q6)

(v) The weighted estimator by I08_= Ie ( e,Ei) (i We consider the estimator me L 160 Ii

9* = 1=1 m

Then we have

r(ee) _

Ze +

1

QO +

1 Q1

+

1

Q4 +

1

I

Q7 + 0

P(1 An-

1 z * + 1 Q* + oP (say) I where i C^ Jeee i * J *2 __ 1 m 1 i i - 1 * 1 Cm ZeeZe I* i11 Ii Z 0Ze + I* Ze ZeeZe Q7 1* ae

1* iLl Ii

i

ee i

i

i

i2 1 m 3J66e+K6e6 i 1 m 3J66e+Ke6e X Z Z X Z +*3 21*2 i=1 i2 e I i=1 Iee e e

* *

+KZe *2 _ 3J 21*4

Since 2

m Q7 = Q5 21 *2 iLl(3J000+K000) Ze 1 ( Il e

Ze

=Q5-Q2 and Cov(Qi,Qj) = 0 (i # j; i,j = 0,1,2,4,5), it follows that (2.6) V(Q*) = V(QO) + V(Q1) + V(Q2) + V(Q4) + V(Q5) .

209

AKAHIRA-TAKEUCHI

27

(vi) The asymptotically best estimator based on 6. andi (i = 1, ,m). The estimators S and S have the property of being functions of 8i and

i (i = 1,..., m) alone. Now let us construct an estimator

6 = f61t...,6m,

1,..., m)

which is asymptotically best among the estimators which depend only on 6i and

i (i = 1,...,m).

We assume that f is twice continuously

partially differentiable in 6i and

i (i

Assuming the

condition of consistency

Denoting of

Z1 =C19 .... m Em ;

e1 b = of

k J

0 6l 61e,...,6m =6, El Ell...l Em Em 81e,...'0m 0 , El Ell..., EM =E M

a2f eQk aZlD^k for i, j, k, 2, = 1, ..., m , we have from the consistency condition that m ai = 1; bk = 0 i=1 and also

( k = 1,...,m)

210 AKAHIRA-TAKEUCHI

28

m

aai m m c = ; L c.. = 0 ; j=1 iJ ae i=1 j=l 1J C aai mm iLl dik 0 ' dik aEk ' e Qk = 0 for i, k, Q = 1, ..., in. Taylor's expansion of 0** yields that

e** = 0

in m m + ai (01 0) + 2 I cij( ei- e)(0 cci=1 i=1 j=1

j -0)

+ G X dik(el e) (Ekk) + op(n) i=l k=l **

,r(e -0 )

The asymptotic variance of

is given by

2 n 2 mCC ai n ai V(0i) L i i=1 i=1 00 m of which minimum value under the condition I ai = 1 is attained when i=1 i i ai = m.00 = 180 C i I

(i = 1 ,...,m) .

i==1 00 Then we have aai dik

aCk

2Je0+K00 s ik

Ie0(2Je^0+Kee^) *2

for i, k = 1, ..., m , where 6ik denotes Kronecker ' s delta. It follows from these conditions that m

m

m

,T(e Iie ,r(si-e) + 1 cijn(ei-e)(0.-0) I

i=1

2vrn-

i=l

j=1

m Cm n(Oi-0)(^k i=l k=l dik vrn-

k) + oplYrn)

211

AKAHIRA-TAKEUCHI

29

and substituting the stochastic expansion of 0i we get m

r(0 **-0) Z I

O+

1 *

0

Ze

Z1

Ji m Zee _ e9e + 1

i =1 7i 00 109

I

Zi * Zi

V I i

=1 I^

0

m cc i i i 1 * L Jed Z12 + 1 * C J09+K90 ZiZi n 37n I i=1 I 7 I * i== 1 I80ICc 0 i i 1 m 2JeE0+Ke0 Zi I*2

i=1

I

E

Z 0

i - d J OO+Ke0 ZiZi CC CC + 1 L L 2Vn I* i=1 j=1 Ieeie9 ij I*Iee e 9

1

Ze + 1 Q0 + 1 Ql + opl 1 / (say) , I \\ ^/ where 1

Q1

ci

J000+K990 i ^- - d . Z Z Z i 21 i=1 j=1 1601e0 ^ I*Iee 0 0

Since we have Cov(QO,Ql) = 0 we should minimize V(Ql) in order to minimize the asymptotic deficiency of 0** relative to 6* under the condition

m ta2J K I(2J*+K*) i 00000 00 + (2.7) ,l =c ij = a6 = I* - 1*2

for i = 1, ..., m. It is expressed by

V(Q*) _ * c - d J099*K000 1 2I i=1 j=1 ii i^ I 109190 and noting that cij = cji we get by applying Lagrange 's method of

212 30

AKAHIRA-TAKEUCHI

constrained minimization i

c ij

8i3

Jeea *K eee

+ 10000

^`i+Aj) (i,j = 1,...^m)

where ai (i = 1,...,m ) are Lagrangean multipliers.

From (2.7) we have

m m 0= I I ci. =-J*K +2I A1IA0 . i=1 j=l I i=1 Hence C i = - J*+K* i=1 i I 0e21 2 Again from (2.7) it follows that i

i

ci = J006*K000 + A1Ie0I* + a.Iee 160 j=1 ;1 2J6ee+Ke0e Iee(2J*+K*)

i _ J000 180(3J*+K*) ^i10e

I

21

*2

for i = 1, ..., in. * Consequently the minimum of V(Q1)

is attained when

I1 Ji +I3 Ji Ii Ii (3J*+K*) ii +Ki + a eee eee ee eee 00 000 ee 00 cij = Z* 1*2 ij 1*

and * __ 1 * m J ie00 1 3J*+K* *2 Z e Z Q1 1*2 0 i=1 Zi 0 21 *3 0 ee Hence the asymptotically best estimator a is expanded , after some rearrangements of terms into the form

/_(9**-e) Z* + 1 Q0 + 1 Q1 + 1 Q4 + 1 Q5 + 0 1 ✓n p (^n::) YST Z fn F

213

AKAHIRA-TAKEUCHI

31

and the asymptotic deficiency of 0 relative to 0 is given by I{V(Q4) + V(Q5)}. Comparing the above with the stochastic expansion of 0 , we get

0 - = - n Q6 + op (1) n m

1* (J000+K000 )( 0i-0)2 + op(n) . 21 i=1

Hence an estimator which is asymptotically equivalent to 0** up to the order n-1 is given m

0*** = 0 _ 1*

2I i=l

(3600+Ke00 )(

0i e)2

where I*, 3000 and K000 are estimators of I*, J000 and K000 respectively, obtained substituting 0 and Ci for

0 and

Ei (i = 1,

...,m). The adjusting term m 1* X (J000+K000)(0i 2I i=1

-0)2

may be considered as the estimated non-linear effect in the best combination of 0.'s. i From the above we have established the following theorem. THEOREM 2.1. Assume that (A.1) - (A.5 ) hold. The stochastic expansions of the estimators 00, 0*, 0, 0, s and 0** are given by

,T(e0-0) z0 + 1 QO + o 1 ; ,T P (,' ) Z0 + 1 Q0 + 1 Ql + op\1/ Yrnvrn✓n(6-0) = * Z0 + 1 QO + 1 Q1 + 1 Q2 + 0 1 p F ,r ,T

/

^(80) _ * Z* +1Q + 1 Q +1Q + 1 Q +1Q +o C1); 0 F 0 Fn 1 p F 5 v 6 + vrnFn 4

r(e*-0)

1

QO + 1 Ql - 1 Q2 + 1 Q4 + 1 Q5+opl 1 ) Fn F vrnFn r `vln-

Z*

I

214 32

AKAHIRA-TAKEUCHI

* Z0 + 1 QO + 1 Ql + 1 Q4 + 1 Q5 + o l rn r dn` r p rn respectively , where

Q5 and Q6 are given in the above Q0' Q1' Q2' Q4' and Cov (Qi,Q.) = 0 for i ¢ j (i,j = 0,1,2,4,5,6) except for i = 2 and j = 6. The asymptotic deficiencies of 0, e , 6* and A** relative to 0 are given in the table below.

Estimator Tn

Asymptotic deficiency of Tn relative to n I*V(Q2)

I*{V(Q4) + v(Q5) + v(Q6)} I*{V(Q2) + v(Q4) + v(Q5)}

I*{v(Q4) + v(Q5)}

6**

From Theorem 2.1 it is seen that 0 is asymptotically better than ** 0, 8, S and 0 up to order n . Using the estimator 0 we may construct an estimator 0* asymptotically equivalent to 0 as follows: in + 0* = 0 - ml ( 3J600 Ke00)(6 0)2 C i 1=1 l l `^00 2 i where J606

= J600 ( 6.) and K660 - K600 (

0i) (see Akahira and Takeuchi,

1981b). We can formulate still another estimator , i.e., the linear estimator 0 given by (m i* L 3AA 04

e* = =1m

,

i I AAA

i=1 where for each i = 1, ..., m the weight `966 is defined by 2

`^00 n a 2 Li(0'Y ae

215 AKAHIRA-TAKEUCHI

33

(0 may be replaced by and i). Then we get

Z0 + 1 QO + 1 Ql - 1 Q2 + op(1 j r Therefore the asymptotic deficiency of 0* relative to 0* is equal to that of 0. Moreover , we have from the above, i i* `^66 + `_00 c i me me

+6 1 m 2 2 iLl

iLl `^0A

iLl `'00 Thus the estimator

* 0i = 0 + op(n)

( 0* + 0)/2 is asymptotically equivalent to 0* up

to the third order, i.e., order n - l , that is , the asymptotic deficiency * * is zero , which can also be shown by the of (0 +0 / 2 relative to 0 more direct way as follows. Let 0 be the MLE .

Then we have

1ma Li(0, 0 = n 1 a0

- e )2

k.)(e*-e) + 1 1Gm a3 L i Z)(6*i = n 1 a2 L (0 (0 i' ae2 i i' i i 2n i a03

+0 (1) p n

Similarly we have 1m 0 = n 1 a6 Li(0i,Ei) 3 2 Li(0*. i 0*) +2n I a 3 Li(0*, *) (0 6*) 2+0p(1 ) = n a 2 a0 1 a0 And noting that 1 n 1

I 1

m

a3

1

m

0 3 Li(0i.^i) = T aA m

a2

*

1

n

in

a3

E

3

Li(0 * ,Z.) + op(1 )

1 a0

a2

n 1 a02 Ll(0 , i) = n ae2 Li(0 , i) + op(1) we have m 2 2 I 2 Li(0i,^i) + a 2 Li (0, n a i) (0*-0i) = 0 (n) * Z*)l

l t ae

Thus we get

ae

216 34

AKAHIRA-TAKEUCHI

m

i

i*

^

E `(Jee + use) ei i

o

ee + 00

Hence we have established the following theoremTHEOREM 2 . 2. Under the assumptions (A.1) - (A.5), the estimator a is asymptotically equivalent to the MLE 6 up to the order n -^ and its * asymptotic deficiency relative to 6* is zero , and the estimator (e +e)/2 is asymptotically equivalent to 6 up to the order n and its asymptotic deficiency relative to 6 is zero.

3. Discussion. V(Qi) (i = 0,1,...,6)

can be interpreted in the following way. V(QO)

consists of two parts, i.e., * * 2 ) and (J +K)) *4 (M*I* - J*2 I 21 the first being the "loss of information " of the MLE and the second due to the condition of (higher order asymptotic ) unbiasedness ( Amara, 1980). These two factors are unavoidable even when we know the true values of the nuisance parameter. Then for the MLE 6 we have three additional terms i2 i2 1 1 1 i _ J6c0 _ J0

rr (MeeE

I*2 i1l Ii

I1

Il

i2 1 C Je 2I*2

iY

i2 I

and i2 1 t° 1 _ 1 Je;6 1*2 i11 Ii I Ii

ee

of which the first is the "loss of information " due to the presence of

217

AKAHIRA-TAKEUCHI

35

unknown nuisance parameters , the second is the unbiasedness effect and the third represents the loss due to pooling. V(Q2) can be considered as representing the effect of the nonlinearity of the optimum combination of the MLE ' s to get the MLE based on the whole sample. Comparing 0 with 0 , it is shown that the information contained 60 (i = 1, ..., m) are lost when we use I66 (i = 1, '3 ... ,m) in its stead, and V(Q5) represents the loss with respect to 0

in the statistics

and V(Q4) with respect to Ci (i = 1,...,m). Comparison of 0 and 0 does not lead to any uniform conclusion, which may seem strange since in 0 a better estimator 0 of 0 is used instead of 0 (i = 1,...,m) , but it can be interpreted that 0 could be better since I00(0i'Ei) may be closer to J 60 than 160O'Ei). Comparing 0** with 0* , it is shown that the loss of information caused by restricting the estimators to be functions of 0i and Ci (i = 1,...,m) alone is expressed by V(Q4) + V(Q5). Of the two terms Q4 and Q5 the latter is the weighted mean of the asymptotically ancillary statistics JI

ZAe - 000 Zei

(i = 1,...,m)

lee and is easy to interpret , but the former is a weighted sample covariance between

( Z/I e0 ) - ( Z/I ) and Zi/Iii , but its meaning is harder to

grasp. In case of the symmetric location -scale problem in which the density function g(x,0,Ei )

is expressed as

1 f for each i = 1, ..., m and where f is an even function, we have for each i = 1, ..., m ,

218 AKAHIRA-TAKEUCHI

36

I00 2, 2 ; Ie = EI{ae (x, e, )}{a g(X,0, )}^ = 0 i

i

L

where A and B are constants determined as

A = with

f(x)d,; B = J x2{^' ( x)}2 f(x)dx

fi(x) = log f ( x). Further we obtain for each i = 1, ..., m

K600 = - 0 i -=0; i J600 JeO^ = 2 i3 (3A+C) Jere = 2 13 (-A+C) ; K60 _ i3 (-A-C) where C = I x{^'(x)}3 f(x)dx . And for each i = 1, ..., m , JO - 0; JO = 0 It is further calculated that for each i = 1, ..., m ,

where

D = J{4"(x)}2 f(x)dx and F = J x2{c" ( x)}2 f(x)dx . Therefore , in the estimation of the common location parameter of different populations of the same shape but with different scale parameters, we have Q2 = 0 and Q6 - 0 , thus we have 6 = 8 and 0 = 0 . Especially , when the underlying distribution is normal, we have for each i = 1, ..., m ,

J06^ 2E3 {31

x2 f(x)dx - J x4 f ( x)dx} = 0

and Je,O + Klan = 0 , which implies that Q4 0 and also Z60 = 0 hence Q1 = Q5 = 0. Then we see that 0*, 0 and 0 ** asymptotically equivalent , and the most commonly used estimator

are all

219 AKAHIRA-TAKEUCHI

37

6i/j 8=i=1 j./Z2 11 n n 0. = i = n X Xij and 1 = nll (Xij-% i2 (1 = 1,...,m) is, j=1 j=1 although not identical with the MLE , asymptotically equivalent to it up

with

to the order n 1.

Acknowledgements. This paper was written while the first author was at the Limburgs Universitair Centrum in Belgium as a guest professor .

The visit was sup-

ported by a grant of the Belgian National Scientific Foundation (N.F.W.O.). He is grateful to Professor H. Callaert of the LUC for inviting him and for his interest in this and related work.

References Akahira, M .: On asymptotic deficiency of estimators .

Austral. J . Statist.

23, 67-72 (1981). Akahira, M . and Takeuchi , K.: Asymptotic Efficiency of Statistical Estimators: Concepts and Higher Order Asymptotic Efficiency , Lecture Notes in Statistics 7, New York-Heidelberg -Berlin, Springer ( 1981a). Akahira, M . and Takeuchi , K.: On asymptotic deficiency of estimators in pooled samples .

Technical Report of the Limburgs Universitair

Centrum, Belgium ( 1981b). Amari, S. : Theory of information space : A differential -geometrical foundation of statistics. RAAG Report , 106 (1980). Fisher, R. A .: Theory of statistical estimation .

Proc. Camb . Phi.. Soc.

22, 700-725 ( 1925). Ghosh, J. K., Sinha , B. K. and Wieand , H. S.: Second order efficiency of the mle with respect to any bounded bowl-shaped loss function. Ann. Statist. 8 , 506-521 ( 1980). Hodges, J. L . and Lehmann , E. L.: Deficiency . 783-801 ( 1970).

Ann. Math. Statist. 41,

220 38

AKAHIRA-TAKEUCHI

Pfanzagl , J. and Wefelmeyer , W.: A third order optimum property of the maximum likelihood estimator. J. Multivariate Anal. 8, 1-29

( 1978).

Takeuchi , K. and Akahira , M.: Third order asymptotic efficiency of maximum likelihood estimator for multiparameter exponential case. Rep. Univ. Electro -Comm. 28, 271 -293 (1978).

Masafumi Akahira Kei Takeuchi Department of Mathematics Faculty of Economics University of Electro- Communications University of Tokyo Chofu, Tokyo 182 Hongo , Bunkyo-ku Japan Tokyo 113 Japan

221

Ann. Inst . Statist. Math. 37 (1985), Part A, 17-26

ESTIMATION OF A COMMON PARAMETER FOR POOLED SAMPLES FROM THE UNIFORM DISTRIBUTIONS MASAFUMI AKAHIRA AND KEI TAKEUCHI (Received Sept. 6, 1983; revised Mar. 12, 1984)

Summary The problem to estimate a common parameter for the pooled sample from the uniform distributions is discussed in the presence of nuisance parameters. The maximum likelihood estimator (MLE) and others are compared and it is shown that the MLE based on the pooled sample is not (asymptotically) efficient. 1. Introduction In regular cases the asymptotic deficiencies of asymptotically efficient estimators were calculated in pooled sample from the same distribution (Akahira [5]) and in the presence of nuisance parameters (Akahira and Takeuchi [7]). In non-regular cases the asymptotic optimality of estimators was discussed in Akahira [4], Akahira and Takeuchi [6], Ibragimov and Has'minskii [10], Jureckova [11] and others, and also recently a Monte Carlo study on the estimator considered in Akahira [1]-[3] has been done by Antoch [9]. In this paper we consider the problem to estimate an unknown real-valued parameter 0 based on m samples of size n from the uniform with different distributions on the interval (0-, B + ^1) nuisance parameters which is treated as a typical example in non-regular cases. In some cases the MLE and other estimators will be compared and it will be shown that the MLE based on the pooled sample is not better for both a sample of a fixed size and a large sample. Related results can be found in Akai [8].

2. Results Suppose that it is required to estimate an unknown real-valued Key words and phrases: Maximum likelihood estimator, weighted estimator, uniform distributions. 17

222 18 MASAFUMI AKAHIRA AND KEI TAKEUCHI

parameter 0 based on m samples of size n whose values are Xif (i=1, • • , m ; j =1, • • • , n) from the uniform distributions on the interval (0 0+^i ) with different nuisance parameters ^i (i=1,• • •, m). For each i let X(l)<X(2)<• .. <XX(,,) be order statistics from Xi1, XXE, • • •, Xi,,. We consider some cases. C a s e I. ^i = E (i =1, • • • , m) are unknown.

The MLE 0ML of 0 based on the pooled sample {X1} is given by d^L = 2 (min Xi( 3)+max Xi(n)) • lsism 15i$m

An estimator B„ of 0 based on the sample of size n is called to be (asymptotically ) median unbiased if Pr {&n 0} = Pr {dnB} = 2

^lim IPr {d&S0} - 1 I =lira Pr {dn 2

B} -

n--

2

=0 uniformly in some neigh-

borhood of B) (e.g. see Akahira and Takeuchi [6]). Then it is shown that dML is one-sided asymptotically efficient in the sense that for any asymptotically median unbiased estimator d„ lim [Pr {n(&IL-0)S t} - Pr {n(dn-0)St}]? 0

for all t>0 ;

{n(dn- 0) St}]S0

for all t<0 ,

or lim [Pr {n(#ML-0) St} -Pr n-.m

since in this case d,L is actually the MLE from the sample of size mn (see Akahira and Takeuchi [6]). Case II.

^i (i =1, • • • , m) are known. The MLE d ML of 0 based on the pooled sample {X f} is given by 1

dML= 1 {max(X(n)2

^i)+ min 15i;sm 1;ism

(

XX (

) +Si)}

Then it can be shown in a similar way as in [4] and [6] that 0ML is two-sided asymptotically efficient in the sense that for any asymptotically median unbiased estimator do

Lim [Pr {nIdML - 0^
for all t>0 .

n--

(For the definition see Akahira and Takeuchi [6], page 72 and Akahira [4l).

223 ESTIMATION OF A COMMON PARAMETER FOR POOLED SAMPLES 19

C a s e III.

Ei (i =1, • • • , m) are unknown. For each i the MLE's 9i and ^i of B and Ei based on the sample Xil, • • • , Xi,, is given by i=

2

(X,(1)+Xi(n )

i=

,

(Xi ( n)-Xi(l,) ,

respectively . Let &IL be the MLE of 0 based on the pooled sample {Xij} . Then it will be shown that BML is not two -sided asymptotically efficient . We define 9,*= 1 {max (9i+ Ei -E°)+ min (9i- Ei+Et)} 2 15i5m

Then it can be shown from Case II that 9n*, is asymptotically locally best estimator of 0 at Ei = EOt (i =1, • • • , m) in the sense that for any asymptotically median unbiased estimator 6. lim[ Pe{ n1,*e1
{n19n-

0I t }] >_0

n-w

for all t>0. First we shall obtain the MLE 9ML based on the pooled sample {Xij}. Let fi(x, 0) be a density function of the uniform distribution on the interval (0 - Ei, 0 +E). Since the likelihood function L(0 ; E l, • • , E m) is given by m

L(0; E 1,..., Em ) =TT

n

TT f

i(

i=1 j=1

x j, 0)

1 1 2n

n

for

xi(n

TT E i

)- E i;50SXi<1, + E i (i=1,•••,m);

i=1

0 otherwise . In order to obtain the MLE 9ML it is enough to find 0 minimizing if[ Ei i=1

under the condition Ei _ max {0 - xi(1) , xi(n,-0} for all i. Let 9* be some estimator of 0 based on the pooled sample {Xij} . For each i we put E*=max {B*- xi(l), xi,n) -B* }. Then we have for each i

E* =max {9 * - 9i+Ei, Hence the MLE

9$ L

ei

+Ei -B*}

=

Ei +I

B i -d *1 .

is given by 9* minimizing

TT (gi +I -9 *I) 9i

i=1

that is, such an estimator 9* is given by either of the estimators (i=1,•••,m). Since

224

20 MASAFUMI AKAHIRA AND KEI TAKEUCHI

TT (^i +1Bi-B *D=TT TT (1+ for sufficiently large n it is asymptotically equivalent to

TI si(1 +E

lei-e*^ 1 .

Hence it is seen that for sufficiently large n the MLE eML is asymptotically equivalent to a weighted median by the weights 1/^i (i = 1, • • • , m). Next we shall discuss the comparison among B, L, the weighted estimator and other estimators . We consider the case when m=2. Then for each i=1, 2, Xi,, • . • , Xi„ are independently , identically and uniformly distributed random variables on (0-^i , 8+ei). Then for each i=1, 2, the joint density function fn(x, y; 0, ei) of Bi and ^i is given by

n(2^n

(2.1)

fn(x,

1)

y; 8, ^i) =^

n -z

for 05y5^i and

B-^ i +y5x58+

i- 2J

;

0 otherwise .

For each i=1, 2 the density function fn(x; 0, si) of 0, is given by (

(2.2)

n n-1 (;i-Ix-BI )

for B - ^7i<x<8+^;i ;

fn(x ; 0, s^i)= 2^i 0

otherwise .

Also for each i = 1, 2 the conditional density function f.(x I y ; 0, $i) of

Bi given ti is given by (2.3)

fn(x

I y ; 0, e i ) =

1 2(e -y)

for B -^:i } y5x
0

otherwise ,

that is, the conditional distribution of Bi given ^i is uniform distribution on the interval (0-(^i-gi), 8+(e,-gi))• For two (asymptotically) median unbiased estimators B' and B2 of 0, B' is called to be (asymptotically) better than e2 if Pr {nIB'-0l tI&„ 2} _ Pr {nIB2-8I5tI^,, 2} a.e. for all t>0 (lim [Pr

{n1 B' -Oast} -Pr {nIB2- 1 _0

for all t>0), and then for

n-.Co

simplicity we denote it symbolically by B' } B2 (B' } 62), where Pr {A as.

^2} denotes the conditional probability of A given e, and ^2. From (2.3) we see that the conditional density of 6,-0 and #,-0 given ^, and 2

225

ESTIMATION OF A COMMON PARAMETER FOR POOLED SAMPLES 21

is given by

fn(

x l,

x21 ^1, 2) _

1 4T1z2

for

I x il
0 otherwise , where Ty =e2-c , (i = 1, 2). If c1 7*1 > c2T2, then the conditional density function f.(yI g1, ^2) of Bo-0 With Bo= C1B1+C2B2 given ^1 and 2 is given by 1

for y< - c 1T1+c2T2 ;

(C1z1+c2z2+y) 4clC2T1T2

1 2c1z1

(2.4) fn.(y I ^1, ^2) _

for

1

for y>c1r1-c2z2.

(c1T1+c2z2-y)

4c1c2T1z2

I

Iy
If c1z1
1

(c1Tl+c2z2 - y)

4c1C2T1z2 (2.5)

fn(yI ^ 1,

2)=

for y> - C1z1+c2z2

1 2c2z2

for

1

IyI<-C1z1+c2z2;

for y
(Clzl+C2T2 +y) 4C1C2T1T2

If c`=

^5/(^1 +&)

=c% (say)

(i j; i, j =1, 2) ,

then

(2.6)

e c C2T2 51 ( 52-^2)

('lzl 2(51-e1)

If e`=g>/( 1+^2)= C'' (say)

(i*j; i, j=1, 2) ,

then (2.7)

1 _ iG-_ _s2 i'Tl ^2 ( S1-S1) ' C2 'z2 ^1(E2-2)

Hence we have the following :

(2.8) 22 1 if and only if

1

= ^ 1 +52

Cizl 2

z

'.C 1-2

=1

2 1 ( ^l - J 2(L l - t1) C1 71

226 22 MASAFUMI AKAHIRA AND KEI TAKEUCHI

+^2 ^i '-2

(2.9)EZ1 if and only if

ee (CC tt S1 s2-S2) t1(S2-S2) C2'T2

c 2

On the other hand as is seen from the above discussion on the MLE, the MLE BML based on the pooled sample {X,5} is given by

0ML =j

& if 1<2 ,

02 if e1 > e2

The conditional density of the MLE B,^L given g, is given by (2.3). Now we consider two cases, i.e., ^1= e2 and S 1 # e2. Note that the first Case III, was also mentioned in the Case I with another optimum criterion. Case IIII .

e.

= In the case when ci,=ci=^5/(&1+^2) (i j; i, j=1, 2), S1=S 2

e1<>g2 if and only if ,(^-g 2) >S2(S - ,) ,

i.e. ,

C2r2>C,r 1.

Hence we have rl

(2.10)

e1

<> ^2 if and only if

cir1 +C27.2 < r2

We also obtain 2 CCe1e2C[(^1 t-^2) Z0 Ci'T1+C2 r2-(civ1+ C2r2) =

From (2.4) to (2.10) we have established the following theorem.

(i)

0

ri C ( r 1+ C2 r2

r

C l r 1+ C 2 r 2

Fig. 2.1. Comparison of the conditional densities of c161+c262-0 given & and S2 with (i) cz=c,1=^j/(&+t2) (i#j; i, j=1, 2) and (ii) ci=c'' =tj2/(ei+ ea) (i4j; i, j=1, 2) and of (iii) the MLE Bart given &s. They are given by (2.5) and (2.3), respectively.

227

ESTIMATION OF A COMMON PARAMETER FOR POOLED SAMPLES

THEOREM 2 .1.

23

If e1=^2 and S1<s2, then ^2B1+ ie2 2BI+b 102 >'BbiLB1 ee S1+S2 ^1+^2

if

e+1=52

and

e1 > e2,

then c2B1 +cz }, S2B1 +cie2 } ^1+ee le 22 52 1+ 2

B ML -B2

We assume that c1=c2=1/2 and ^1=^2=e. By (2.2) it follows that for each i=1, 2 the asymptotic density of n(&-B) is given by

.fi(x)= 1 e -1xW

(2.11)

Then the characteristic function qS(t) of the asymptotic density function of n[{(BI+B2)/2}-B] is given by (1+e2t2/4)z We may represent fi(t) as follows : 2 /4

^ t2

(2.12) t)-

1 + 2(1 +e 2t2/4)

Since

f - 1 exp

(- 2 IYI) exp (ity)dy = ^ exp (- Ix I) exp (i_x) dx 2 -1 1+e2t2/4 '

3 i i exp (--- iYl ) exp (ity)dy= 2

s:

IxI exp (- Ix I) exp (i

_ x)dx

Fig. 2. 2. Comparison of the asymptotic densities of n(61-8) and n[((61+ 62)/2) -0] given by (2.11) and (2.13), respectively.

228 24 MASAFUMI AKAHIRA AND KEI TAKEUCHI

1- Q t2/4 (1+S2t2/4)2

it follows from (2.12) that the asymptotic density of n[{(&+02)/2}

-e}

is given by (2.13)

Lx -1 exp (- fixi)

From (2.11) and (2.13) we have established the following theorem. THEOREM 2 .2.

If

e1 = e2=^,

then

Case III2 • $1#s2• We consider only the case when

e1

if &1«2;

02

if ccl>2

S,

<$ ,.

We assume that c1^1 * e2S 2•

By (2.2) it follows that for each i=1, 2 the asymptotic density of n(0, - 0) is given by

(2.14)

fi(x)= 2 t exp (-IxI/^{) ,

and also its characteristic function ¢,(t) is given by

01(t) = 1

1+Vt2

Hence the characteristic function ¢*(t) of the asymptotic density of n(0o-B )=n(c101+c202-0) is given by

*(t) = 1

(1 +cieit2) (1 +cz^at2)

Fig. 2 .3. Comparison of the asymptotic densities of n(9^ - B), n(9o-B')= n(c191+c292-B) and n(42-0) given by (2.14), (2.15) and (2.14), respectively, when F,«2.

229

ESTIMATION OF A COMMON PARAMETER FOR POOLED SAMPLES

25

Since (c'V C 'Ie2) 1 +c;eit2 1 + c2^2t2 ) it follows that the asymptotic density f(x) of n(Bo-0) is given by (2.15) f(x)= 2(c,e11 1 clea) {Cii exp (- c 1) -c2^2 exp (__!L ) There exists a positive number to such that

5°

50 f()d

f(x)dx ,

where f1(x) and f (x) are given by (2.14) and (2.15), respectively. Hence we have established the following theorem. THEOREM 2 .3.

Suppose that

and c1`s 1 *c2^2.

S1<S 2

Then

OML-01>- B0 as.

in some neighborhood of 0 in the sense that lim [Pr {nIBML-9ISt}-Pr {njBO-9I5t}]z 0

for all t<-t0.

noo

Further 00 >-OML =B1 IS.

in far away from 0 in the sense that lim [Pr {nI&-0jSt} -Pr {nI&L-0ISt}]>_O for all t>t0. Remark.

From (2.14) and (2.15) it is easily seen that

0O> 02

and

as.

0ML -01 > 02 . as.

The case e1>2 may be treated quite similarly. UNIVERSITY OF ELECTRO - COMMUNICATIONS UNIVERSITY OF TOKYO

REFERENCES Akahira, M. (1975). Asymptotic theory for estimation of location in non-regular cases, I: Order of convergence of consistent estimators , Rep. Statist. Appl. Res., JUSE, 22, 8-26. [ 2 ] Akahira, M. (1975). Asymptotic theory for estimation of location in non- regular cases, II: Bounds of asymptotic distributions of consistent estimators , Rep. Statist. Appl. Res., JUSE, 22, 99-115.

[1]

230 MASAFUMI AKAHIRA AND KEI TAKEUCHI [3] Akahira , M. (1976). A remark on asymptotic sufficiency of statistics in non -regular cases , Rep. Univ. Electro-Comm ., 27, 125-128. [ 4 ] Akahira , M. (1982). Asymptotic optimality of estimators in non-regular cases, Ann. Inst. Statist. Math., A, 34, 69-82. [ 5] Akahira, M. (1982). Asymptotic deficiencies of estimators for pooled samples from the same distribution , Probability Theory and Mathematical Statistics , Lecture Notes in Mathematics, 1021 , 6-14, Springer , Berlin. [6 ] Akahira , M. and Takeuchi, K. (1981 ). Asymptotic Efficiency of Statistical Estimators : Concepts and Higher Order Asymptotic Efficiency , Lecture Notes in Statistics 7, Springer, New York. [ 7 ] Akahira , M. and Takeuchi , K. (1982). On asymptotic deficiency of estimators in pooled samples in the presence of nuisance parameters , Statistics and Decisions, 1, 1738, Akademische Verlagsgesellschaft. [8 ] Akai , T. (1982). A combined estimator of a common parameter, Keio Science and Technology Reports, 35, 93-104. [ 9 ] Antoch, J. (1984). Behaviour of estimators of location in non-regular cases: A Monte Carlo study , Asymptotic Statistics 2, the Proceedings of the 3rd Prague Symposium on Asymptotic Statistics , 185-195 , North -Holland, Amsterdam. [10] Ibragimov, I. A. and Has'minskii, R. Z. (1981 ). Statistical Estimation : Asymptotic Theory, Springer, New York. [11] JureMova, J. (1981). Tail-behaviour of location estimators in non -regular cases, Commentationes Mathematicae Universitatis Carolinae , 22, 365-375.

231

Rep. Star. Appl. Res., JUSE Vol. 32, No. 3, Sept., 1985 p.p. 17-22

A-Section

A Note on the Minimum Variance Unbiased Estimation When the Fisher Information is Infinity Masafumi AKAHIRA* and Kei TAKEUCHI**

Abstract In the case when the Fisher information is infinity, it is shown that the locally minimum variance of unbiased estimators is equal to zero. Some examples are also given.

1. Introduction In non-regular cases the minimum variance unbiased estimation has been studied by Chapman and Robbins [6] , Kiefer [7], Polfeldt [8], Vincze [101, Akahira [11 , Akahira and Takeuchi [21, [31, [41, [9] , Akahira, Puri and Takeuchi [5] and others. In the previous paper by Akahira and Takeuchi [2] they introduced the concept of the one-directional distribution and discuss the locally minimum variance unbiased estimation for its family. In this paper, from another point of view we shall treat the case when the Fisher information is infinity and show that the locally minimum variance of unbiased estimators is equal to zero. Further we shall give two examples.

2. Results Let (X, Y) be a pair of random variables defined on some product space x x I of ?I and with a joint probability density function (j.p.d.f.) f0(x, y) with respect to a a-finite measure y, where 0 is a real -valued parameter. Then it follows that for almost all (x, y) [µx l1x Y] f0 (x, Y) = f0 (x I Y) f0 (Y), Received September 11, 1985 * Department of Mathematics, University of Electro-Communications. 1-5-1, Chofugaoka, Chofu-shi, Tokyo 182, Japan. (The author is visiting Queen's University in Canada from April to July 1985.) * * Faculty of Economics, University of Tokyo. 7-3-1, Hongo, Bunkyo-ku, Tokyo 113, Japan. (Key words) Locally minimum variance, Unbiased estimator, Fisher information.

232 M. Akahira and K. Takeuchi where f0 (x I y) and f0 (y) denote a conditional p.d.f. of x given y and a marginal p.d.f. of y, respectively . We assume that for almost all (x, y) [µx yJ , f0 (x,y), fe(x I y) and f0 (y) are continuously differentiable in 0. Since log f0(x, y) = log fe(x I y) + log fe(y) a.e. µx y it follows that XIY1Y 1X, (0)Y =B E[1 (0)] + (0)

where IX,Y(o), IXIY(o) and IY(0) denote the amounts of Fisher information with respect to f0(x,y), f0(xly) and f0(y), respectively, e.g.,

alo aee (x,

IX Y(0) JJ 1

y) cZ fo(x,y)dµx,y

3€xV

and E0Y(.) is the expectation w. r. t. f0(y). We denote J0(Y) = IXIY(o). Here we assume the following conditions (A.1) to (A.3). (A.1) O<J0(y) <° for almost ally [µy] and all0, and E00Y[J00(Y)]= oo and IX,Y(00) for some Oo. (A.2) There exists a sequence I Bn I of measurable sets of the domain of Y such that bn E0Y[J00(Y) XBn(Y)]
VXIy(6 (}{)) = 1 for all 0, 0 y J0 (y) where EX B ly(.) and V0 '(.) denote the conditional expectation and the conditional variance ofXgiven y. We shall show that the locally minimum variance of unbiased estimators of 0 is equal to zero. Theorem : Assume that the conditions (A.1) to (A.3) hold. Then inf V00(O(X,Y)) = 0. 6(X,Y) : unbiased Proof : Let an estimator 6* = 6 *(X,Y) be 1 o* = bn J00^')XBn(Y)

16y(X) - 0 0 ^

+ 00 .

233

Minimum Variance Unbiased Estimation By the assumptions (A.2) and (A.3) we have

EB 'Y(0 *) = bn EB [JB0(Y)XBn (Y) I EB IY (0Y(X)) - 0017+ 00 = 0

for all 0, hence 0* is an unbiased estimator of 0, where EB 'Y(.) denotes the expectation w.r.t. f0(x,y). We also obtain by (A.2) and (A.3) EX, Y (0*2) = bnz E, Y [J002 (Y)X Bn (Y) 10Y(X) - 00 } 2 ] 00 60 20 0 E0 X, Y [J00 (Y)XB (Y) {6 Y(X) - 00 11 + 002 + b n 0 n bnz E 0 [J80 (Y)xBn(Y) EX'Y (0Y(X) - 00)2 ] 00 20 + b0 EB [J00(Y)XB (Y) I E0 [Y (0Y(X)) - 00 + 002 n 0 n 0 bnz E0o [J00(Y)XBn(1')1 + 002

Then it follows that the variance of 0 at 00 is given by V X,Y 1 00 ( 6* ) = b n which tends to zero as n --^

because of (A.2). Thus we complete the proof.

Next we shall show some examples. Example 1 . Let Xi = 0Yi + Ui (i = 1, 2, ... , n), where (Yi,Ui) (i = 1, ... , n) are independently and identically distributed (i.i.d.) random vectors and for each i, Yi and Ui are mutually independent. Let f(u) be a known density function of Ui. Assume that lU= f(u)12 du< 00 J f(u) Putting Y = (YI, ... , Yn ) we have IY(0)=0 since the distribution of Y is independent of 0, and EY(J0 (Y)) = nE(Y 12)IU.

234 M. Akahira and K. Takeuchi Assume that E(Y12) _ 00 and f(u) is a density of the standard normal distribution. For any K > 0 and each i = 1, ... , n we define Yi for IYiI S K ; Yi,K =

1

0 for IYi I> K .

Since E(Y12) = 00, it is easily seen that E(Y1, 2) tends to infinity as K ^. We also define an estimator n i E YiK Xi OK= nE(Y1,K ) It is clear that 0K is an unbiased estimator of 0 . Since the variance of OK at 0 = 0 is given by n E(E YiK2) i-1 VO(0K) = E0(0K2) _ l _ n2f E(Y1 K2)I2

1 nE(Y1,K2)

it follows that lim VO(OK) = 0, K- 00 hence inf V0(0) = 0. 6 : unbiased Example 2 . Let Y be an unobservable random variable according to the chi-square distribution with one degree of freedom. Assume that X1 ... , Xn are independently, identically, normally and conditionally distributed random variables with mean 0 and variance Y given Y , where n > 5. Then we have I Y I(X1 _0 )2] ^ = nEe ( Y ) _ 00. 1X,Y(e) nE i Y I( XlY 0 ) 21 = nEe [ - EX, For any K > 0 we define an estimator

for E (Xi -X)2 > 1 ; i=1

n for E (Xi -X)2 < K i=1

235 Minimum Variance Unbiased Estimation where cK is a constant such that eK is an unbiased estimator of 6. Then we shall show that K VO(OK) = 0. n _

First we easily see that E (X1-X)2 is expressed as a product of two independent random i=1 variables Y and Z, where Z has the chi-square distribution with n-1 degrees of freedom. Let f1(y) and fn_1(z)be the density functions of Y and Z, respectively. By the unbiasedness condition of eK we obtain 1 1 f 1 y )f

1

If > yz=K We also have the variance of eK at 0 = 0 2

1 (z

(2 . 1)

)dydz cK

f l (y)f n_l (z) dydz = vK (say). nyz2

f fZ> y Putting

00 1 y fl(y) dy,

F(Kz) = f 1 Kz we have 0 F(Kz) = c' f

3 y y 2e 2 dy Kz

,

1 Kz where c and c' are constants. By (2.1) and (2.3) we obtain 00 1 = cK J F

(Kz) fn-l (z) dz = A V KCK ,

0 which implies cK = O(K - 1 /2), where A is some constant. By (2.2) and (2. 3) we also have 0 vK = c2 F^2) fn-1( z)dz = B ^ cg = O (K-1 /2 nz 0

(2.2)

236 M. Akahira and K. Takeuchi where B is some constant. Then we obtain li V0(OK) = lim^"K = 0.

Hence we have inf VO(O) = 0. 0 : unbiased

Acknowledgements The authors wish to thank the referees for their comments. The paper was written while the first author was at Queen's University in Canada as a visiting professor. The visit was supported by a grant of the Natural Sciences and Engineering Research Council of Canada. He is grateful to Professor Colin R. Blyth for inviting him. REFERENCES [1] Akahira, M. (1984): "On the Bhattacharyya inequality in non-regular case," Surikaisekikenkyusho Kbkyuroku (Prot. Symps., Res., Inst. Math. Sci, Kyoto Univ.) 538, 65-80. [2] Akahira, M. and Takeuchi, K.: "The lower bound of the variance of unbiased estimators for one-directional distributions" (Submitted for publication). [3] Akahira, M. and Takeuchi, K.: "Locally minimum variance unbiased estimator in a discontinuous density function" to appear in the Metrika. [4] Akahira, M. and Takeuchi, K. (1985): Non Regular Statistical Estimation, Monograph. [5] Akahira, M., Puri, M.L. and Takeuchi, K.: "Bhattacharyya bound of variances of unbiased estimators in non-regular cases" to appear in the Annals of the Institute of Statistical Mathematics. [6] Chapman, D.G. and Robbins, H. (1951): "Minimum variance estimation without regularity assumptions", Ann. Math. Statist., 22, 581-586. [7] Kiefer, J. (1952): "On minimum variance in non-regular estimation",Ann. Math. Statist., 23, 627-630. [8] Polfeldt, T. (1970): "The order of the minimum variance in a non-regular case", Ann. Math. Statist., 41, 667-672. [9] Takeuchi, K. and Akahira, M.: "A note on minimum variance" to appear in the Metrika. [10] Vincze, I. (1979): "On the Cramer-frechet-Rao inequality in the non-regular case", In: Contributions to Statistics. The Jaroslav Hajek Memorial Volume, Academia, Prague, 253-263.

237 Ann. Inst. Statist. Math. 38 (1986), Part A, 35-44

BHATTACHARYYA BOUND OF VARIANCES OF UNBIASED ESTIMATORS IN NONREGULAR CASES MASAFUMI AKAHIRA, MADAN L. PURI* AND KEI TAKEUCHI

(Received May 19, 1984; revised Jan. 16, 1985)

Summary Bhattacharyya bound is generalized to nonregular cases when the support of the density depends on the parameter, while it is differentiable several times with respect to the parameter within the support. Some example is discussed, where it is shown that the bound is sharp. 1. Introduction It is well known that the Cramer-Rao and the Bhattacharyya bounds are most important and very useful for the variances of unbiased estimators. They are, however, not applicable to the non-regular cases when the support of the distribution is dependent on the parameter. Same is true about more general and simpler bounds by Hammersley [6], Chapman and Robbins [2], Kiefer [8], Fraser and Guttman [5], Fend [4] and Chatterji [3], among others. (For an exposition of some of this work along with extensions in different directions, see Polfeldt [10], [11] and the recent papers of Vincze [15], Khatri [7] and Mori [9], among others). In his paper, Polfeldt [10] discussed the lower bound of the variances of the unbiased estimators when the class of probability measures is one-sided, that is, when P, is absolutely continuous with respect to P02 (symbolically, P,,02. In this note, our main interest is to obtain the Bhattacharyya bound when for any 01, 02, with 010 02, neither PBI << PB2 nor P82 << P01. 2. Results Let ' be an abstract sample space with x as its generic point, A a o-field of subsets of ', and let ® be a parameter space assumed to be an open set in the real line. Let 9= {P0 : 0 E ®} be a class of prob* Research supported by NSF Grant MCS-8301409. Key words and phrases: Cramdr-Rao bound, Bhattacharyya bound, unbiased estimator. 35

238 36 MASAFUMI AKAHIRA, MADAN L. PURI AND KEI TAKEUCHI

ability measures on (X, mil). We assume that for each 0 E 0, P0(•) is absolutely continuous with respect to a c-finite measure p. We denote dPe/d,u by f(x, 0). For each 0 E 0, we denote by A(0) the set of points in ' for which f (x, 0)>0. We shall consider the Bhattacharyya bound of variances of unbiased estimators at some specified point 0o in 0. We make the following assumptions :

(A.1) p((n ( U A(so+h)))DA(so))=0 c>0 Ihl f(x, 0), and for every 0 E (0o-s, 0o+s),

1

I Ax) I Ax, 0) dp < oo implies A(0)

U

r(x) I p(x)dp < oo •

A(B)

(A.3) For some positive integer k lim sup n-o x E U A(8o +jh) - A(80)

>-1 (_V (

j)f(x, so

+jh) <00 ,

I h I 'p(x)

J=1

i=1,•••,k. First we prove the following lemma. LEMMA.

Assume that (A.1), (A.2) and (A.3) hold. If ¢(x) is any

measurable function for which

I ¢(x) I p(x)dp < oo, then X

lim 1 h-.o h' i

( -1)' ^ ) (x)f(x0

0o + jh)dfe=0

.

U A(6o+jh)-A(go)

J=1

PROOF. By (A.2) and (A.3) it follows that for every i=1,. • • , k and every 0o E 0, there exist positive numbers s and K, such that IhH(_1)1

() f(x, 0o +jh) < KKp(x) for l ihl <e and x E A(0o)c ,

where A(0o)c denotes the complement of the set A(0o). Also, it follows from (A.1) that for every j=1,• • •, i, there exist a sequence {s1„} of positive numbers converging to zero as n-+ oo and a monotone nonincreasing sequence {S1„} of measurable sets such that l jh l < s p„ implies

239

BHATTACHARYYA BOUND OF VARIANCES OF UNBIASED ESTIMATORS 37

A(Bo+jh)-A(Bo)cSin and p ( fl Sin)= 0.

If for each j =1,• • •, i, I jhI<

n=1

e in, then

hi

i (-1)i(j)O(x)f(x, 00+jh)dp i U A(Oo+jh )- A(Oo) J=1 Ki

I qS( x ) I P( x)dp

S

{

K{I cb(x)I P(x)dp

U Sin

U A(9o + jh)-A(Oo)

i=1

1=1

Ki I cb( x ) I P( x)dp , sE J=1 J Sin

which tends to zero as n->oo . Remark.

The proof follows.

The assumption (A.2) together with the condition in the

above Lemma is satisfied with p(x) _ Z cif (x, 0) when the following i=1

holds : For each Bo and s > 0, there exist countable points 01 , 02, • • • and positive constants c1, c2,• • • such that U A(6i)DA(B) for all 0 E (Boi=1

s, 00+s ) and that

and F, cif(x, Bi)>f(x, 0) for all 0 E (Bo-s, Bo

ci
i=1

+ s) and almost all x[P].

We assume the following : (A.4) For each x E A(00), f (x, 0) is k-times continuously differentiable in 0 at 0=0g. (A.5) For each i=1,• •, k, x, 00+h) 8Bi f( lim sup <00 , h-•0 XEA (80)

where p(x) is defined in (A.2).

P(x)

We now prove our main theorem on the Bhattacharyya bound of the variances of unbiased estimators. THEOREM 2.1. Assume that (A.1) to (A.5) hold. Let g(0) be an estimable function which is k-times differentiable over ®. Let g(x) be an unbiased estimator of g(0) satisfying

5

I 9(x) I P( x )dp < oo .

A(B0)

Further, let A be a k x k non- negative definite matrix whose elements are

240

38 MASAFUMI AKAHIRA, MADAN L. PURI AND KEI TAKEUCHI

A£f= ( 1 J a£f(x, 00) of f(x, 00) d ^` J Ax, eo) 1. ae£ aef

i, j=1,•••,k.

A(80)

Assume that 2££, i =1, • • • , k are finite. If d is nonsingular at 00, then (2.1) Var (9) Z (gu)(0 o), ... , g(k)(00 ))A-i(g(i)(0 0), ... , g(x)(0o))' , 00

where g(£)( 0) is the i-th order derivative of g(0). PROOF. Denote

c(x)f(x, B)dp

go(0)= J

J f(x, B)=f(x, 0+h) -f(x, 0) .

and

A(00)

Then, by (A.5), we have (2.2)

1-

£ Wg0(0)]

'&'(00) = li m

'= 6a =lhm h£

I-0 h

g(x)dkf( x , 0o) dp

£ A(60)

= lim

9'(x ) -£ dhf(x, Bo)dp A(60)

( !j f(x, die =1m J 9(x ){ _Bo+h) 9

A(B)

9(x)

[ ae £ f( x ,

0) ] d p

A(B0)

0

where dhg(0)=dh 1(dhg(B)), i=1,• • •, k and 0<<1. Since W) = f 9(x) A(B)

• f(x, 0)d1 , we obtain for each i=1, • • •, k

(2.3)

d n(g(00) - go(00)) =E

f

=1

(-1)1 (

{g(0o+jh)-g o( Bo +jh)}

fE (-1)J^ j I l

-

=

5

J

4(x)f(x, 9o +jh)dte

A(60+fh)

9(x)f(x, eo +jh)dq}

A(B0)

£ E(-1 )f(j)

(5 - A(60+fh)

• g'(x)f(x,

A(80)f1 A (90+fh)

A(00)-A (00+fh)^

8o +j h)dp}

= J=l (-1)f (j) l^A(80+fh) J

=f^ (- 1)f (j) S

J )9(x)flx, A(B0) n A (Oo+f h)

A(60+fh)- A(80)

i(x)f(x, Bo+jh)dp

0o

+jh)dp}

241

BHATTACHARYYA BOUND OF VARIANCES OF UNBIASED ESTIMATORS 39

=fE

(-1)'(j )

t

1

gr(x)f(x, Bo+jh)dp

U A(Oo+kh)-A(Oo) k=1

f J 1)J >

l /9(x)f(x, Bo+jh)du

U A(Bo+kh)-A(eo) k=1

By (2.3) and Lemma, we have for each i=1, • •, k, [-j.g(o)]

(2.4)

=lim hi d Lg(eo) =lim h; dn(g(60)-g0(00))+li V¢ dng(eo) h-o h-0

^ =lim ht dhg(O0)= Bf go(B) e_e

0

From (2.2) and (2.4) we obtain for each i=1, • • •, k, (2.5)

L 8B' g(B)]

a-BO & A(Bp)

)[-8Bi .f (x, 0)] a_ odf,

Proceeding now as in the regular case (see e.g. Zacks [16], page 190), one can show that the Bhattacharyya bound of the variances of the unbiased estimators of g(0) is given by (2.1).

3. Example We consider the location parameter case, i.e., f(x, 0)=f(x-0), and unbiased estimators of 0.

We assume that for any pZ1, the density function f(x) is given by c(1 - x2)p-1

f(x)=

f 0

if

if

Ixk< 1 IxIz1

where c=1/B(1/2, p) with B(a, p)=sax 1(1-x)0-1dx (a>0, p>0).

Case ( i ) : Let p = 1. Then the distribution is uniform, and it is easy to check that min Vareo (0)=0 for any specific value 0o. (See Takeuchi B: unbiased

[14]). Case ( ii): Let p=2. In this case , it is easy to check that the Fisher

information

51

[f'(x)/f(x)]2f(x)dx=00.

For any s > 0 , we define an estimator B. which satisfies

242 40 MASAFUMI AKAHIRA, MADAN L. PURI AND KEI TAKEUCHI

if IxIs1 -6 ,

- c,f'(x) /f(x)

0 if 1-6<jxj51 , where c, is a constant determined from the equations

51

(3.1)

1 B,(x) f (x)dx = 0

5' 1

and

B,(x) f'(x)dx = -1 .

We shall determine B,(x) for x outside the interval [-1, 1] from the unbiasedness condition (11'1 +9

(3.2)

B,(x)f(x- 6)dx=6. J -1 +9

First consider the case 0 < 0:!9 1, and define g(6) = f 1 1+8 #,(x) f (x - 6)dx .

(3.3)

Since B,(x) and f'(x) are bounded, g(O) is differentiable and g'(6)= e,(x)f'(x-6)dx. _ -1+9

If we assume that B,(x) is bounded for 1<x52, the right hand side of (3.3) is also differentiable, and we have by (3.2) and (3.3) 1+9

(3.4) 1-g'(B)=-J

6,(x)f'(x-6)dx .

Differentiating (3.4) again, and noting that lim f'(x)= -3/2, we have x-.1-0 1+9

g"(6) 2 0,(1+0)- f 1

(3.5)

B,(x)f"(x - 6)dx

If B. satisfies (3.5), then it also satisfies (3.4) since lim g'(6) _ -1; it 9-.0

also satisfies (3.3) since lim g(6)=0 by (3.1). 9-.0

Since the integral equation (3.5) is of Volterra's second type, it follows that the solution B,(x) exists and is bounded. Repeating the same process, we have the solution e,(x) for all x>1. Similarly, we can construct 9,(x) for x<-1. On the other hand,

(3.6) Vargo (e,) = c' ,

E +&

[f'(x)lf(x)]Ef(x)dx

while from (3.1), we have (3.7) c, 1x1 1 J

f'(x)dx = 0 -1+.

Hence Vargo (9,)=c,.

and J

c, 1x1 1

-1+.

[ f'(x)/ f (x)]'f (x)dx =1 .

243

BHATTACHARYYA BOUND OF VARIANCES OF UNBIASED ESTIMATORS 41

[ f'(x)/ f (x)]2 f (x)dx = oo, we have from (3.7),

Now, since lim r 1 .moo

inf Vare, (B)=0 for any specified value Bo.

lim c,=0. Consequently , `4

9:

unbiased

Before proceeding on to the next cases, we note the following : (i=1, .. • , k) given in Theorem 2.1 are finite,

If k < p/2, then A since

5

1 -1 (1-

X 2)p-2k-l dx < oo

Also, 1 aif(x -e) aff(x-e) dx

c6+1

(3.8) 9

-1 f(x -B)

ae f

ae{

= f' (-1)t+> f(x) f"'(

x)fcf'(x)dx ; i, j = 1,..., k,

We also obtain for Ixl<1, f<1'(x)= -2c(p-1)x(1- x2)P-2 ; fc2>(x)= -2c(p-1){(1-

(3.9)

x2)P-

2 - 2(p-2)x2(1- x2)P -2]

f(2)(x) = 4c(p-1) ( p - 2) {3x(1- x2)P-2 - 2(p - 3)x2 (1- x2)P-1) ; .. .

If i+ j is an odd number, if follows by (3.8) and (3.9) that 2 = 0 since f"'(x) f 1>'(x) is an odd function. From (3.8) and (3.9), we have 211=

4c(p -1)2B(

3 , p-2)

Ail=8c(p-1)2(p-2) {2(p-3)B(2 , p-4) -3B(2 , p-3)}

222=

4c(p-1)2{B (2 , p-2)-4(p-2)B(

+4(p-2)2B(5, p

2,

p-3)

-4)}

233 = 16c(p-1)2(p-2)2 {9B (3 , p-4) -12(p-3)B(2 , p-5)

+4(p-3)2B 2 , p_6)} ;... ( Case (iii) : Let p=3,4. Then, we have, for any unbiased estimator B(X) of 0,

(3.10)

511

O(x)f(x)dx=0

244

42 MASAFUMI AKAHIRA, MADAN L. PURI AND KEI TAKEUCHI

11

(3.11)

B(x)f'(x)dx =-1

(3.12) 5 1

^(x)f(x)dx=0, k=2,..., p-1. 2

p-1

Noting that ^- [ {

ck f (k)(x) (

/f (

1 k=i

x)] dx <

implies c2= • • = c,-, = 0, we

have by Takeuchi and Akahira [13], that the infimum of Varo (B)= B2(x)f(x)dx under (3.10), (3.11) and (3.12) is given by inf Varo (B) ^11

= 1/A11 , where 21,=

5[

any 8>0 there exists

and

5'

9 : ( 3 .1 0 )-( 3 .12)

f'(x)/f(x)]2f(x)dx=(p-1 )(2p-1)/(p - 2) and for

0,(x) in (-1, 1) satisfying (3.10), (3.11) and (3.12),

B;(x) f (x)dx < (1/A,,) -{- s.

We can extend B,(x) for x outside (-1, 1+e

1) from the unbiasedness condition

0,(x) f (x - B)dx = B.

First, we

-1+B

consider the case when 0< 0<1, and define g(O) (0) =

5 1'

9,(x) f (x - B)dx. 1 +B

In a similar way to the case (ii), we have for k=2,• • •, p-1, 1+0 „

(3.13)

(- 1)k +lg(k +1)(O)= BkO,(1+0)- 0,(x)f ( k +1)(x-B)dx l Bk= lim f( k)(x)

Since the integral equation (3.13) is again of Volterra's second type, it follows that the solution B,(x) exists. Repeating the process, we can construct an unbiased estimator B,(x) for all values of x. Then, it follows that inf Var8, (B)=1/A,, for any specified value 00. 9: unbiased

Case (iv) : Let p=5, 6. Note that

J 11 [f'(x)f"(x)/f (x)]dx = 0 J11

[f"(x)/f(x)]2f(x)dx=

(p-1)(2p-1)(2p-3) (2p'-7p+8) (p- 2) (p- 3) (p-4)

= 222 (say)

and J1 imply c,= • .. =cp_1=0.

1 Y, ck f(k)(x)/f (x)I 2 f(x)dx < oo

1 k=i

Then we see that

Vargo (B)Z(1, 0)( '111 0 0 222

0 = Ali /_(1) 1

245

BHATTACHARYYA BOUND OF VARIANCES OF UNBIASED ESTIMATORS 43

for any specified 00, where 211 is defined above (3.8). Here again, as in the previous case inf Var,0 (8)=1/211 for any specific 00. 6: unbiased

Case (v) : Let p=7. In this case, we see that k=3. Using Theorem 2.1, (3.8) and (3.9), we obtain

Vargo (e)> ICI

3

1 ill = E('-_211233

)]_1

where

0

213

222

0

0

283

with 2ii=144cB(2, 5) ; 2ia=1440c {8B(2, 3 )-3B(3, 4)} 2E2 =144c {B (2 , 5) - 20B (2 , 4) +100B (2 , 3) } 233=144O0c {9B(3, 3)-48B (2, 2)+64B(2, 1)} We also obtain 2ia = {8B(5/2, 3)-3B(3/2, 4)}2 2112aa B(3/2, 5){9B(3/2, 3)-48B(5/2, 2)+64B(7/2, 1)} Here again inf Vargo (B)=[211(1-(213/211213)]-1 for any specific 0 0 , i.e. 9: unbiased

we have a sharp bound. Case (vi) : For pZ 8 we can continue in a similar manner by choosing k=[(p-1)/2], where [s] denotes the largest integer less than or equal to s. The above discussion establishes that here the bound is sharp but generally it is not attainable.

Acknowledgement The authors are indebted to Professor Michael Perlman for the very careful and critical examination of the first version of the paper.

246 44 MASAFUMI AKAHIRA, MADAN L. PURI AND KEI TAKEUCHI

His constructive comments and suggestions for improvements are gratefully acknowledged. They are also grateful to the editor and the referee for their helpful comments. UNIVERSITY OF ELECTRO - COMMUNICATIONS INDIANA UNIVERSITY UNIVERSITY OF TOKYO

REFERENCES [ 1 ] Bhattacharyya, A. (1946). On some analogues of the amount of information and their use in statistical estimation , Sankhya, 8, 1-32. [ 2 ] Chapman, D. G. and Robbins, H. (1951). Minimum variance estimation without regularity assumptions, Ann. Math. Statist., 22, 581-586. [3 ] Chatterji, S. D. (1982). A remark on the Cramer-Rao inequality, in Statistics and Probability: Essays in Honor of C. R. Rao, North Holland Publishing Company, 193196. [ 4] Fend, A. V. (1959). On the attainment of the Cramer-Rao and Bhattacharyya bounds for the variance of an estimate, Ann Math. Statist., 30, 381-388. [ 51 Fraser, D. A. S. and Guttman, I. (1951). Bhattacharyya bounds without regularity assumptions , Ann. Math. Statist., 23, 629-632. [ 61 Hammersley, J. M. (1950). On estimating restricted parameters, J. Roy. Statist. Soc., B, 12, 192-240. [ 7 ] Khatri, C. G. (1980). Unified treatment of Cramer-Rao bound for the non-regular density functions , J. Statist. Plann. Inf., 4, 75-79. [ 8 ] Kiefer, J. (1952). On minimum variance in non-regular estimation , Ann. Math. Statist., 23, 627-630. [ 9 ] M6ri, T. F. (1983). Note on the Cramer-Rao inequality in the non -regular case: The family of uniform distributions, J. Statist. Plann. Inf., 7, 353-358. [10] Polfeldt, T. (1970). Asymptotic results in non-regular estimation, Skand. Akt. Tidskr. Suppl., 1-2. [11] Polfeldt, T. (1970). The order of the minimum variance in a non -regular case, Ann. Math. Statist ., 41, 667-672. [12] Rao , C. R. (1973). Linear Statistical Inference and Its Applications, Wiley, New York. [13] Takeuchi, K. and Akahira, M. (1986). A note on minimum variance, in press in Metrika. [14] Takeuchi, K. (1961). On the fallacy of a theory of Gunner Blom, Rep. Stat. Appl. Res., JUSE, 9, 34-35. [15] Vfncze, I. (1979). On the Cram€r-Fr€chet-Rao inequality in the non- regular case, in Contributions to Statistics, The Jaroslav HAjek Memorial Volume, Academia, Prague, 253-262. [16] Zacks, S. (1971). Theory of Statistical Inference, John Wiley.

247

Metrika, Volume 33, 1986 , page 85-91

A Note on Minimum Variance By K. Takeuchi1 and M. Akahira2

Summary: Minimizing f {o(x)}2f (x)dp is discussed under the unbiasedness condition : f o(x)fr(x)di. = ci (i = 1 , ..., p) and the condition (A): fi (x) (i = 1, ..., p) are linearly independent, P f {ft(x)}2/f(x)d,
1 Introduction Let (x, B) be a sample space. We consider a family P= {P0 : 0 E 8} of probability measures on B, where the index set O is called a parameter space. In minimum variance unbiased estimation theory, the locally best unbiased estimator,y* of y = g(O) at 0 = 00 is obtained by minimizing f {'t(x)}2dFB0(x) x under the condition that f y(x)dP0(x) =g(0) x

for all 0 E

0.

(1.1)

When the parameter space 0 is a finite set {00, 01, ..., Op}, the condition (1.1) is reduced to

f y(x)dPei (x) = g(Oi) = c i (say)

x

(i = 0, 1, ..., p)

And it is easily derived that the minimizing solution y* is obtained as P

y *(x) = E

i=o

kfi (x),

1 Kei Takeuchi, Faculty of Economics , University of Tokyo, Hongo, Bunkyo-ku, Tokyo 113, Japan. 2 Masafumi Akahira, Statistical Laboratory , Department of Mathematics , University of ElectroCommunications , Chofu, Tokyo 182 , Japan. 0026-1335/86/020085-091 $2.50 © 1986 Physica-Verlag, Vienna

248 86 K. Takeuchi and M. Akahira where dei

A (x) = dPe (x), 0 provided that For each i = 1, ..., p, Pei is absolutely continuous with respect to Poo.

dPoi f x

(C.1)

2

dPeo

dPo o (x) <°° (i = 1, ...,p)•

f(x ) (i = 1 , ..., p) are linearly independent.

(C.2)

(C.3)

The purpose of this note is to investigate the situation where for some i the first and/ or the second condition does not hold. Remark that the third condition is obviously necessary. First we consider the case when we assume the first condition but not the second. To deal with the case we need a more subtle formulation of the condition in place of (C.2) which is given in the next section.

2 Results Let f(x) and fi(x) Q = 1, ...,p) be measurable functions. Then we consider to minimize

f {9(x) }2f(x)dp x under the unbiasedness conditions: f B(x)f(x)dµ = ci (i = 1, ...,p) x and the condition (A): fi(x) (i = 1, ..., p) are linearly independent, {fi (x)}2

AX) (X)

du< °°

(i= 1, ...,k;k
fx

and P { affi(x)}2 i=1

f(x) x

dp
249

A Note on Minimum Variance 87

implies ak+1=...=ap=0.

Note that in the above f and f (i = 1, ..., p) may not necessarily be density functions. In fact , in some applications we may take the derivatives of the density with respect to the parameter 0 as f and obtain some generalization of Bhattacharyya inequality. Also generally f is equal to one of fi's but it is not essential for the subsequent discussion. Let A11 be a k x k non-negative definite matrix whose elements are

__ tj X

f, (x)f.(x) dµ f(x

(i , 1= 1,...,k).

Theorem: Under the condition (A), inf f {9(x)}2 f(x)du = c(k)A11 c(k) 8: unbiased x

holds, where c(k) = (c1, ..., ck) ' is given in the above. The equality is not usually attained. Proof. We put

A n= x : 1 m a ^ x P

.f,•(x)I 1

x)
We define the class C„ of estimators such that for any B„ E Cn, en (x) = 0 for x Ef An . We also put

^^()

n An

fi(x)f (x) f(x)

dµ (i , l = 1, ..., p)

and denote the matrix (J4^")) by An . Then we have min Vare (9n) =c'An

OnE Cn: unbiased

1c,

where c = (c1, ..., cp)' is given in the condition (A). On the other hand we may make a partition of the matrix An into blocks as

A=

A(n) Ai21

n 1 A21 A22

where 1A(n) and A2Z are k x k and (p - k) x (p - k) matrixes , respectively.

250 88 K. Takeuchi and M. Akahira

Let c = (ci, c2)' with cl E Rk and c2 E Rp-k. Then the condition (A) implies that

(2.1)

lim c'An c = n-

for every vector c = (e 1, c2 )' such that c2 : 0. To prove the theorem it is necessary and sufficient to show that

as n - oo

and since the matrix An is non negative definite it is enough to show that

(A22 - A211111 ' A12 )-' - 0

as n -> °°,

that is, the minimum characteristic roots of the matrix An = A22 - AiiAii -1Aiz diverges to infinity as n tends to be large. Assume otherwise , then we have a sequence {cn } of vectors such that cn = ciw ^2n)' with 11 4n 11 = 1 and cn Ancn -C2nAne2n =pn (say)

n -> -. and p, 1' p* Then there exists a subsequence { cnr } of {.E*} } such that cnj = (c ink, c2Q' with II cinj II = 1 and as j - °°,

where c* (ci, c2 )' with Il ci II = 1. We can show that c*'Anc* -> p*

as n -) °°,

which contradicts (2.1). Assume otherwise, then there exist e > 0 and no such that

c*'Ano c* > p* + C. Since An is monotone increasing in n in the sense that c'An c is so for any vector c, it follows that for n > no cn'Anocn G cn'Ancn 5 p*.

251

A Note on Minimum Variance 89

Hence p*+e
f {6(x)}2.f(x)dµ S under the unbiasedness condition f O(x)f (x)dk = (f + f) O(x)ft (x)dp = c; (i = 1, ..., P), x S SC

where f (x) (i = 1, ..., p) are linearly independent. Assume that we have a condition similar to (A) above:

{f (x)}2 f f(x) dµ
i = 1, ..., k),

and P { Fi aif{(x)}2 i=1

S f(x)

dµ < 00

implies ak+1 = ... = aP = O,

and further we assume that fi (x) (i = 1, ..., p) are linearly independent within S. Let Ai 1 be a k x k non-negative matrix whose elements are _ .f1(x)f ( x) du (i,1= 1, ..., k). s f(x)

For any estimator 9(x) we define a vector d = (d 1, ..., d,)' by di = f 6(x)f(x)du Se

(i = 1, ...,P)•

252 90 K. Takeuchi and M . Akahira

Suppose that d is given , then the unbiasedness condition is reduced to f O(x)f(x)dµ = c, - di (i = 1, ..., p)

S

and under this condition we have from above theorem that inf

f

6: unbiased with S

{6(x ) }2f(x)dµ =(Cl-dl)'A 11 '(cl-dl),

a fixed d

where c = (C1i..., CP)' _ (C1, C2 )' and d = (d 1, d2)' W1thCl, d1 E Rk andc2, d2 E Rp-k It is obvious that the set of possible vectors dl is a linear space which is denoted by V. Then we have

inf 6: unbiased

f

{6(x) }2 f(x)dp = min (c1- dl)'Aii'(cl -dl). d lE D1

S

If the dimension of Dl is 1, we have a basis d*, ..., d7 of Dl and any d 1 E Dl can be expressed as

dl =bldi +...+bidr =D*b, where D* _ (d 1 , ..., dl ') is a k x 1 matrix and b = (b 1, ..., bl)'. Then it is straightforward to show that min (c1 -di )'A i '(c l

-dl)

dIEDi

= min (cl -D*b)'Aii'( cl -D*b) b

= ciA,* iD*'(D*'Aii iD*)-iD*Aii lci and the minimum is attained by b* = (D*'Aii'D*)-'D*Aii lcl. If, moreover, fi (i = 1, ..., p) are not linearly independent in S, the vector c -d is restricted to a linear space E and inf f {B(x)}2f(x)dji=(co-do)'Ao-'(co-do), 6: unbiased with d; S c-dEE where Ao is the maximum rank principal minor of Ai 1 and co and do are corresponding subvectors of c and d. Then do is on a hyperplane H of some dimensionality and

inf

f {6(x)}Zf(x)dp= min

9: unbiased S

d0EH

(co- do)'A0 -1(co -do).

253

A Note on Minimum Variance 91

Note that even when the parameter space © is infinite , minimizing the variance with a finite number of conditions is often applied to obtain a lower bound of the variances of unbiased estimators , as is the case of Cramer -Rao and Bhattacharyya . (See Hammersley; Chapman/Robbins ; Kiefer ; Fraser/Guttman; Fend; Sen/Ghosh ; Chatterji ; Akahira/ Puri/Takeuchi and also Isii with relation to mathematical programming)

References Akahira M , Puri ML, Takeuchi K (1986) Bhattacharyya bound of variances of unbiased estimators in non-regular cases . To appear in the Ann Inst Statist Math 38 Chapman DG, Robbins H (1951 ) Minimum variance estimation without regularity assumptions. Ann Math Statist 22:581-586 Chatterji SD (1982) A remark on the Cramer-Rao inequality. In: Kallianpur, Krishnaiah, Ghosh (eds) Statistics and probability : Essays in honor of C R Rao. North Holland , New York, pp 193-196 Fend AV (1959 ) Bounds for the variance of an estimate. Ann Math Statist 30:381-388 Fraser DAS, Guttman I (1952) Bhattacharyya bounds without regularity assumptions . Ann Math Statist 23 :629-632 Hammersley JM (1950) On estimating restricted parameters . J Roy Statist Soc (B ) 12:192-240 Isii K (1964) Inequalities of the types of Chebyshev and Cramer -Rao and mathematical programming . Ann Inst Statist Math 16:277-293 Kiefer J (1952) On minimum variance in non-regular estimation . Ann Math Statist 23:627-629 Sen PK, Ghosh BK ( 1976) Comparison of some bounds in estimation theory . Ann Statist 4: 755 -765 ; Correction. Ann Statist 5:593

Received August 11, 1983

254 Metrika, Volume 33, 1986, page 217-222

A Note on Optimum Spacing of Observations from a Continuous Time Simple Markov Process By K. Takeuchi' and M. Akahira2

Summary : Assume that X(r) is a continuous time simple Markov process with a parameter 0. The problem is to choose observation points to < rl < ... < rT which provide with the maximum possible information on 0. Suppose that the observation points are equally spaced , that is, for t = 1, ..., T, rt - rt-1 = s is constant. Then the optimum value for s is obtained.

1 Introduction In various practical situations especially in engineering applications of time series analysis, it often happens that the process itself is continuous in time, but measurements can be made only at discrete time points. Then the problem of choosing sample points arises, and its most natural formulations would be "what is the best set of sample points in continuous time points which would give the largest possible amount of information with respect to some unknown parameter given the size of the sample?" In this paper we deal with the problem in nearly the simplest possible case, that is, the underlying process is simple Markov and the measurement points are equally spaced, and calculate the optimum spacing. It is conjectured that the solution given is also optimum even when the sample points can be arbitrarily spaced, but the authors have been unable to prove it yet. The model can be also generalized in various ways, which will be discussed in subsequent papers. As far as the authors are aware, there seems to be little literature which has dealt with this problem, and we mention Taga (1966) who dealt with a similar problem in connection with the least squares prediction.

2 Optimum Spacing of Observations Suppose that X(r) is a stationary Gaussian process with a continuous time parameter r, and also suppose that E(X(r)) = 0, Var (X(-r)) = a2 and Cov (X(rl),X(r2 )) = 0 1T2 - Tl1a2, that is, X(r) is a simple Markov process with a parameter 0, where 0 < 0 < 1. We as-

1 Kei Takeuchi, Faculty of Economics , University of Tokyo, Hongo, Bunkyo-ku, Tokyo 113, Japan. 2 Masafumi Akahira, Department of Mathematics , University of Electro- Communications , Chofu, Tokyo 182, Japan. 0026 - 1335/86/03040217 - 222 $2.50 © 1986 Physica-Verlag , Vienna

255

218 K. Takeuchi and M. Akahira

sume that 0 and a2 are unknown , and it is required to estimate 0 based on T+ 1 measurements on X(T), which are denoted as X(ro ), X(Ti), ..., X(TT). The problem is to choose such observation points To < 7-1 < ... < TT that provide with the maximum possible information on 0. For simplicity 's sake we assume that for t = 1 , ..., T, Tt - Tt_ i = s is constant, that is, the observation points are equally spaced , and we want to get the optimum value for s . In order to simplify the notation we denote Xt = X(Tt) (t = 0, 1, ..., T) and also put p = 0s. Then we have the following simple Markov process Xt = pXt_1 + Ut (t = 1, 2, ...), where {Ut} is a sequence of independently, identically and normally distributed random variables with mean 0 and variance 02 ( 1 - p2), and X0 is a normal random variable with mean 0 and variance a2 and for each t, Xo is independent of Ut. The log-likelihood function of p and a2 based on the sample (Xe, Xi, ..., XT) is given by I logL(p , a2)=-

T-1 T XO+4T +(1+p2) xt -2p E xtxi_1

2 2 I 2(1p )0 t=1 t=1

T T+1 T+1 - 2log(1-P 2) - 2 loga2- 2 log21r.

The first and second order partial derivatives with respect to p and a2 are also given by T-1 T P 2 xo + XT +(I + p2) E x2 - 2p T xtxt-1 8 log L(p, a2) ap (1-p) a2 t=1 t=1 T 2 p 1 xt -

T P Z xtxt_1 + T

12)0, (1-p t1 t=1 1-p 2;

a a02 log

L(p, a2) I

T-1 JX2+xT +(1 +p2

2(1-p2) a4

T T+1 xtxt_1 E x2 -2p 2a2 t=1 t=1 T-1

a2 1 + 3p2

T

- Ti log L (P, u2) _ (I -p2) 302 xo +xT t (1 + p2) t El x2 - 2p t El xtxt_1 I 4p T-1 2 T (P -t=ixtxt t=i xt +(I-P2)2a2 + I T I 1 2 - T( (1-p2)a2 t=1 t

1 +p2).

( 1-p2)2'

(1)

256 A Note on Optimum Spacing of Observations 219

a2 8(a2 )2 log

L(P, a2 )

T-1 2 T 1 T+1 {x^x+(1+P2) i I1 At - 2p t E1 xtxt-1 2a4 (1-p2)a6

a2 apaa log

_L j L( P, a2) = Q 2

a

1 aP

log L (P, a2) +

Tp 1- 2

(2)

(3)

P j.

It follows by (1), (2) and (3) that the information amounts on p and a2 are obtained by

a2 l T(1 + p2 ) Ipp =-ELape logL(P, 0 2 ) = (1 _p2)2

1

a'

.[,,2,,2 = - E

a(a2

)2 log L(P, a2)

T+ 1 2a4

a2 2 1 TP .1p,2 E apaa2 logL(p, a ) _ (1 _p2)a2 .

Hence the inverse of the information matrix is given by

IPP

Ipa2

Y

1

T(l + p2) Tp (1 -p2)2

(1 - p2)a2

Tp T+ 1 I Ipa2 Ia2°2 )

\(1-p2)a2

2a4

where T 2p2 J(P)- 1-p2 {1+(T+1)( 1-p2)}•

We can regard J(p) as the amount of information obtainable from the sample with respect to the parameter p, and when T is large , for any best asymptotically normal estimator A (which may be the maximum likelihood estimator, the least squares estimator, etc.). T+ 1 (,3-p) is asymptotically normal with mean 0 and variance (T+ 1)J(p)-1. And for the initial parameter 0 with which we are concerned, the in-

257 220 K. Takeuchi and M. Akahira

formation is given by - I ap `z s202s - 2 ( 2T 02s I(8) - J(0s) = T + ) aB 1-Bzs T+1 1-Bzs

(4)

since p = O. Now we shall obtain such value of s which maximizes 1(0). Since (T+ 2T pz ) (logp)pz z 0 B z (log )2 I(0 T+1 1- p z ) 1 -p Putting t = ( 1 - pz)/pz we have

402(log 0 )2I(0) _ T { log (1 + )}2 + 0 (1) =It + 0( 1) (say). In order to get s maximizing 1(0) given by (4), it is enough to take the solution to of dItldt = 0, i.e., log (1 + t) = 21;/(1 +:;) which is about 3.926. It is also noted that for 1; =:;o, p _ 0.451. Hence s maximizing 1(0) is given by s =-{log (1 + to)}/(2 log 0) -0.797/log 0. Even when T is small, the amount of the information provided by the sample may have some value, and it is interesting to calculate the value of 0 which maximizes

T

z Bz(log B )zI(B)= en

{1+(T+1)(1 -e')},

(5)

where (3=2slog0. Then the optimum solution is given by the negative root of the equation 2 1 2 -- + =0 Q 1-ceQ 1-eQ which is independent of 0, where c = (T- 1)/(T + 1). Table 1 provides us with the values of Q which maximize (5) for some values of T. Note that for T = 1, i.e. the case of two observations , the optimum is given by s = 0. For comparison if T = 3 we have c = 1/2, hence 0 =-1.151, while for T - 00, i.e. c = 1, we have R = -1.594 which says that for small T, the optimum spacing is smaller than for large, but is at least 73 % of the asymptotically optimum spacing if T>3. The optimum value of s depends on the unknown parameter 0, hence we can not apply the result to the practical case but the autocorrelation of the successive measurements for the optimum spacing is independent of 0, and we may get some rough idea about the optimum spacing since in general cases we may have some prior information about the possible speed of decreasing of Cov (X(t), X(t + s)).

258 221

A Note on Optimum Spacing of Observations Table 1 T

2

3

4

5

9

-0.926

-1.151

-1.261 -1.327 -1.371 -1.402

T

9

10

20

0

-1.444

-1.459

-1.526 -1.566 -1.580 -1.594

50

6

7

8 -1.426

100

Otherwise we may resort to a two -stage procedure by first observing Tt values based on an initial guess and then observing T2 (= T - TI) values corresponding to the estimated optimum spacing based on the first sample. Now let us consider the problem of testing the hypothesis 0 = 0 o against the alternative 0 * 00 . Then asymptotically , the locally most powerful test and the locally most powerful unbiased test are equivalent to the one -sided and the two-sided tests respectively of the maximum likelihood estimator (e.g. Rao 1965). When the sample spacing is given , the hypothesis is transformed into H : p = po = 00'. Then if T is large, the tests based on the maximum likelihood estimator are given by the procedure: One-sided case; Reject H iff

T + 1 (P - po) > ua/ J(p0) (or

Two-sided case ; Reject H iff

T+1 (,a-po)
T+1 _ I^ - po I > u«/2 /

J(po),

where un denotes the a-upper quantile of the standard normal distribution. Under the contiguous alternative K : p = po + r/ T +I- the asymptotic powers of the above tests are represented as monotone increasing functions of r J(p0) , where r is a real number. Transforming back to the original parameter, the asymptotic power for the alternative 0 = Oo + t/\/7T1 is given as a function of t I dp/d0 I J = t \/i. Hence the power is maximized when I(00) is maximized. Therefore the optimum spacing is also given by s =-0.797/log 00. Remark: In the above discussion we excluded the possibility of 0 being equal to zero. It may be of some importance, however, in practice to test the hypothesis 0 = 0 against 0 > 0. Then it can be shown that the local power is larger when s is smaller and the maximum of the local power can not be attained, which can be also seen from the fact thats-0as0- 0. Acknowledgements: The authors are indebted to the referees for their helpful comments which resulted in improvements at various points of the paper . They also wish to thank Mr . H. Osawa of the University of Electro-Communications for making the numerical calculation of Table 1.

259

222

K. Takeuchi and M . Akahira: A Note on Optimum Spacing of Observations

References Rao CR (1965) Linear statistical inference and its applications. Wiley, New York Taga Y (1966) Optimum time sampling in stochastic processes (in Japanese). Proc Inst Statist Math 14:59-61

Received February 8, 1984 (Revised version July 10, 1984)

260

ON THE BOUND OF THE ASYMPTOTIC DISTRIBUTION OF ESTIMATORS WHEN THE MAXIMUM ORDER OF CONSISTENCY DEPENDS ON THE PARAMETER *

Masafumi Akahira(*) and Kei Takeuchi(**)

Abstract In this papar, the case when the order of consistency depends on the parameter is discussed, and in the simple unstable process the asymptotic means and variances of the log-likelihood ratio test statistic are obtained under the null and the alternative hypotheses. Further its asymptotic distribution is also discussed.

(*) Department of Mathematics, University of Electro-Communications, Chofu, Tokyo 182, Japan. The author is on leave and visiting Queen's University from April to July 1985. Faculty of Economics, University of Tokyo, Hongo Bunkyo-ku, Tokyo 113, Japan.

AMS Subject Classification (1980). 62F11, 62M10. Key words and phrases: Order of consistency, Asymptotically median unbiased estimator, Autoregressive process, Log-likelihood ratio, Asymptotic distribution.

This paper is retyped with the correction of typographical errors.

261

1. INTRODUCTION In the regular case it is known that the order of consistency is equal to V^n_, but in the non-regular case it is not always so, e.g.

nl/a (0 < a < 2 ),

n log n

etc., which are independent of the unknown parameter (e.g. see Akahira , 1975a; Akahira and Takeuchi, 1981; Vostrikova, 1984 ).

However , the order of consistency

may depend on it in the case of the unstable process. Here a discussion on that will be done. In the autoregressive process { Xt} with Xt = 9Xt_1+ Ut (t = 1, 2, ...), where { Ut} is a sequence of independently and identically distributed random variables and X0 = 0, it is known that the asymptotic distribution of the least squares estimator of 9 is normal for 101 < 1, Cauchy for 101 > 1 (e.g . White , 1958 ; Anderson, 1959) and some one for 101 = 1 (Rao , 1978 ). Further, in the case when 101 < 1 the asymptotic efficiency of estimators was studied by Akahira ( 1976 ) and Kabaila (1983) for the ARMA process , and their higher order asymptotic efficiency has been discussed by Akahira ( 1975b, 1979, 1982 , 1984 ) and Taniguchi ( 1983) for the ARMA process . In this paper in the first order AR process with 10 1 > 1 the asymptotic means and variances of the loglikelihood ratio test statistic are obtained under the null and the alternative hypothesis.

262

Further its asymptotic distribution is also discussed. 2. RESULTS Let (X, B) be a sample space and 0 be a parameter space, which is assumed to be an open set in an Euclidean 1-space R'. We shall denote by (X(T), 13(T)) the T-fold direct product of (X,B). For each T = 1, 2, ..., the points of X(T) will be denoted by ;VT = (x1, ... , XT). We consider a sequence of classes of probability measures {PB,T : 0 E O} (T = 1, 2, ...) each defined on (X (T), B(T)) such that for each T = 1, 2, ... and each 0 E 4 the following holds: PO,T (B(T)) = PB,T+1 (B(T) X X) for all B(T) E B(T). An estimator of 0 is defined to be a sequence {9T} of B(T)-measurable functions. For simplicity we may denote an estimator 9T instead of {9T}. For an increasing sequence {CT} (CT tending to infinity) an estimator 0T is called consistent with order {cT} (or CT-consistent for short) if for every e > 0 and every i9 E O there exists a positive number L such that

lira lim sup

L-9oo T-* oo O:18-i9! < 6

(Akahira, 1975a).

PO,T

{CTIT

-

01

_> L I = 0

263

It is known that the order of convergence of consistent estimators is equal to VTn_ is the regular case, but not always so, e.g . nl/a (0 < a < 2), n log n etc., in the non-regular case when the support of the density depends on the parameter 9. In both cases the order of consistency is independent of 9. If CT can not be decided independently of 9, we may change cT(9) instead of cT in the above definition. However, in such a definition we shall not be able to determine uniquely the value of cT (9) at Oo (Takeuchi , 1974). Then a similar phenomenon to "superefficiency" happens . Indeed , if 0T has an order cT of consistency independent of 0, then for a specified value 90 we define an estimator 9T as { d _ 1 (OT - 90 )

+"

9T

for 16T - 901 _< CT1/2 for 16T - 001 > CT

1/2

where f o r each T = 1, 2, ..., dT is a constant with dT > 1. We define 4 (0) as follows:

4(00)

= CTdT

264

and for any 0 90 CT

for

4(6) _ CT{1- 10 } for Case (i)

0

=

10 - Sol 10 -901

< cT-1 /2 >CT1

/2

90. Since IBT - 9o l < CT1 1 2 implies CTdT 1eT-

901 = CT I9T - 901, it follows that POo,T

{ 4 (oo)16


- eoI

IOT - 90 I

> L}

>Ll

+ Poo , T { I9T - 001

Case (ii) 0 90 . In the case when

001


T1 /2

OT - 00 1 >

10 - OoI

<

> CT1/2 CT1

}

/2, I9T -

implies 4 (9)IOT -9I < 4(9)I9T-9o I < 1, and CT1

/2 implies 4 (0) 16T -

91

= CT (O) IOT - 91 <

CTI9T - 91. Then we have for L > 1 PO,T

{GT(e) IOT

-91

>L}
-e1

>L

}.

-

10 - 001 > CT 1 / 2, IeT - 90 1 < CT-1 /2 implies 4 (0)IBT - e1 < 4(e)I9T - 001 < CTI9T - 91, and 001 > CT1 /2 implies I eT In the case when

- 1/2

CT(e)OT-e1 =CT

1- 10-901

IeT-e0I
265

Then we have for L > 1 PO,T {c

(9)M -91 > L }
> L}.

Hence in both cases (i) and ( ii) we obtain for L > 1

PO,T {(o)IoT -

01

> L}

L} + Peo,T {cTIOT - 0.1 > 4(2} Letting n -+ oo and L -* oo, we see that the righthand side of the above inequality tends to zero locally uniformly. Since {dT} can be a sequence tending to infinity as T -* oo and for any 9 h 00 , 4(9)/ cT -4 1 as T -p oo, it is possible to make the order of convergence arbitrarily large at 9 = 90. Hence we can not decide uniquely the value cT(9) at the point.

A CT-consistent estimator 9T is defined to be asymptotically median unbiased (AMU) if for any 19 E O there exists a positive number b such that lim sup T->oo 0: 10-191<6

lim sup

T-+oo 0:10-a91
PO,T{OT<9}-2 = 0;

IPB,T {9T >

9}

-2I

=0.

266

Let OT be an AMU estimator satisfying the following: (2.0) lim [ Po,T{cT

(OT_e )

< t} - 2

> 0 for some t > 0;

T->oo

lim __ [ PO,T {CT

(OT

T-4oo

- o)


If for any sequence {c' } of order of consistency with (2.0), lim 4/cT < oo, then

{CT}

is called maximum or-

T-+oo

der of consistency. There are most cases when the maximum order is uniquely determined, but there may not exist an estimator which satisfies (2.0) and uniformly attain the bound of PB,T

{CT(OT

-9)


in the class of AMU estimators with ( 2.0). It is noted that the condition ( 2.0) means non-constant of the concentration probability of the estimator in some interval involving the origin. We consider a simple autoregressive (AR) process {Xt} which is defined by Xt = 9Xt_1 + Ut ( t = 1, 2, ...), where { Ut} is a sequence of independently, identically and normally distributed random variables with mean 0 and variance 1 and X0 = 0.

267

In the case when 191 < 1, the bound of the asymptotic distribution of the all AMU estimators is obtained up to the n-1/2 and it is shown that a (modified) least squares estimator is (second order) asymptotically efficient (e.g., see Akahira, 1975b, 1976, 1979, 1982, 1984). In the subsequent discussion we shall treat the case when 101 > 1. Then it is known that the order {cT(B)} of consistency is given by CT

(0)

- 191T for1 01 >1; = 1, I T fot 101

(e.g. see Anderson, 1959; Rao, 1978). Letting 100 1 > 1, we deal with the problem of testing the hypothesis H : 0 = Oo + (u/cT(9o)) against the alternative K : 0 = 00. Putting 01 = e0 + (u/cT(90)) we consider the log-likelihood ratio LT given by T

(2.1) LT = (90 - 0 ) ( E XtXt1 t=2

- 00+ 2

01

E t-1 t=2

Then we shall obtain the asymptotic mean and variance of LT under H:0=0 iandK:0=90.

268

For each k = 0, 1,.. . , t we have (2.2) 2 9kVe ( Xt -k) = 9kQtk

EB (Xtxt - k) = 0E9 (Xt- l Xt -k) Since for each t = 1, 2, ...,

V0 (Xt) = e2Ve (Xt-1) + V (Ut) , it follows that o. = e2ot 1+1, (t=1,2,...),

(2.3)

where o,02 = 0. First we obtain

(2.4) T

VB (LT)

(BO - el)2

{ v9

E xtxt-1 t=1

T - (Oo

T 2

+ 61) Cove

Xtxt-1, Xt -1 t=2 t=2

+ (0o+01 )2Ve TL 4 L, t 1 t=2

By (2.2 ) and (2 . 3) we have (2.5) T

Ve xtxt - 1 t=2 T

_ E Ve (Xtxt-1) + 2 Y Cove (Xtxt-1, Xt'Xt'-1) t=2 t
_ i (tee o 1 + 62a 1) + 4 t=2 t
Qt

^t le2(t'-t).

(Say)

269

We also obtain by (2.2) and (2.3) (2.6) T

V0

T E

= 2 E of 1 + 4

Xi 1 t=2

t=2

of le2(t'-t) ;

t
(2.7) T Cove

T^

T

Xtxt -1,> Xt 1

t=2

t=2 T

= 2 Ut ort le2(t,-t

)-1 + ^t le2(t'-t)+1 + Qt le

t
t 1. Since by (2.2) T

T

E0 Xtxt -1

Ee

T

T-1

= EB (XtXt-1) = e = Qt 1 = 0

t=1 t=2 T-1 T = 2 Ut E X2 2 1 t=1 t=2

t=2

t=1

it follows from ( 2.1) that T-1

T-1 2

(2.8) EB (LT ) = (60

- el)

01 6 E vt - 2 Qt 00+

t=1

t=1 T-1

=(e0 -01 )B0 (e_ 2ell ^vt

t_1

Since by (2.3) ^2= 1+e2+...+

02(t -1)

-

T-1 2 92 (T-1)

t=1

-

8

it follows that

2-1

2

Ort =

(92

-1

T - 1

02 - 1)2

- 1.

1

^

Ort

270

By (2.8) and (2.9) we have ( 2 . 10 )

Eeo (L T ) = u - 1) 22 + o(1) ;

( 2 . 11 )

EB 1 (L T ) = -

290 (00

u

200 (00 _ 1)2

+0(1) .

Since 00 - 01 = -u90 T and 00 + 01 = 200 + u00 T = 200 + o ( 1), it follows from (2.3) to (2.7) and (2.9) that

(2.12) Veo (

LT) = u200-2T

T

(Qt

at 1

- 0oot i) + o(1)

t=2

T =u2002TEO.t 1

+o(1)

t=2

z

00 (00 - 1)

a + o(1).

Similarly we have

(2.13)

u VB, (LT) _ 00 (00 _ 1)2 + o(1).

In the case when 00 < -1, we have similar asymptotic means and variances of LT under H and K to those of this case.

271

Case (II): Oo = 1. Since by (2.2) and (2.3) T T-1 T-1 (T -1)T E El (XtXt-1) _ E E1 (Xt) _ t =

t=2

t=1

t=1

2

it follows from (2.1) that z E1(LT) = 4 + o(1).

(2.14)

In a similar way as the case (I), we obtain under H 0=61=1+(u/T) (2.15) u2

E01(LT)=-2T2

1- 01(T -1)

(T -1)61 2

(1- 6-2)(1- 61)

1 -6-2

u2 61T _ 02 T - 1 2T2 (61_1)2 61-1 Since 1 _ T 61-1 2u+T^ it follows from ( 2.15) that

(2.16) l2 2 r l E91 (LT) = -2T2 (1+TI - (1+TJ } T 'T z 2 ( 2u 2

2T

+ T)

8 (e2u - 1) + 4 + 0(1).

272

We also have the asymptotic variances as (2.17) u2

Vl(LT)

T

u2

T-1

=T21: ct 1=71: t=2 t=1

u2

t= 2 +o(1);

(2.18) Ve i (L T )

u2 e2(T-1) - 1 T - 1

2

2T

=^a Zt -1 =

t=2 u2

T 2 2_ 2 2 (e1 1) 61

U ) 2(T-1 )

T2 -f T

_ 2 _ 1 _ 2e2"-1

U

4u2 2u

T2

( 2u -}-

(T

u2 2 T)

-

2u + T

( ) +0 1

= 1 (e2u _ 1) _ 2 + o(1)

In the case when Bo = - 1, we have similar asymptotic means and variances of LT under H and K to those of this case.

Hence we established the following theorem.

1)T

273

Theorem . In the AR process the asymptotic means and variances of the log-likelihood ratio LT under H : 0 = 01 and K : 0 = Oo for 1001 > 1 are given by the following:

1001=1

1001>1 Eo.(LT )

4 + 0(1)

+ 0(1)

^ aeo(e

Eo,(LT )

^( B + 0 ( 1) -2O

-8(e2u - 1 ) + 4 + 0(1)

Veo(LT )

eo(00 12 + 0(1)

2 + 0(1)

Vol (LT)

o eo(e

2

+ 0(1)

4(e2u - 1) - z + 0(1)

Table 2.1

Remark. By the Taylor expansion it is shown that for

I0o1

=1 1 2u u2 Eol(LT)=- 8(e -1)+2+0(1 )=-4 +... u2

Vey (LT) =

2 + ... ,

274

in which the leading terms are equal to those by E9o(LT) and V90(LT), respectively. Next we shall consider the asymptotic distribution of LT under H and K. Using the discussion by Basawa and Brockwell (1984, page 165), we see that LT converges in distribution to {ao (u) Y + be;(u)Z} Y (i = 0, 1) under H and K, respectively, where for each i = 0, 1, ae; (u) and b92 (u) are constants, and where Y and Z are mutually, independently and normally distributed random variables with mean 0 and variance 1. By the fact we may obtain the bound of the asymptotic distribution of the all AMU estimators, but there might not exist an AMU estimator whose asymptotic distribution uniformly attains it.

ACKNOWLEDGEMENTS The paper was written while the first author was at Queen's University in Canada as a visiting professor. The visit was supported by a grant of the Natural Sciences and Engineering Council of Canada. He is grateful to Professor Colin R. Blyth for inviting him.

275

REFERENCES Akahira, M. (1975a). Asymptotic theory for estimation of location in nonregular cases, I: Order of convergence of consistent estimators. Rep. Stat. Appl. Res., JUSE, 22, 8-26. Akahira, M. (1975b). A note on the second order asymptotic efficiency of estimators in an autoregressive process. Rep. Univ. Electro-Comm., 26, 143-149. Akahira, M. (1976). On the asymptotic efficiency of estimators in an autoregressive process. Ann. Inst. Statist. Math., 28, 35-48. Akahira, M. (1979 ). On the second order asymptotic optimality of estimators in an autoregressive process. Rep. Univ. Electro-Comm ., 29, 213-218. Akahira, M. (1982). Second order asymptotic optimality of estimators in an autoregressive process with unknown mean. Selecta Statistica Canadiana VI, 19-36. Akahira, M. (1984). Asymptotic deficiency of the estimator of a parameter of an autoregressive process with the missing observation. Rep. Stat. Appl. Res., JUSE, 31, 1-13. Akahira, M. and Takeuchi, K. (1981 ). Asymptotic Efficiency of Statistical Estimators : Concepts and Higher Order Asymptotic Efficiency. Lecture Notes in Statistics 7, Springer-Verlag, New York. Anderson , T. W. (1959). On asymptotic distributions of estimates of stochastic difference equations. Ann. Math. Statist., 30, 676-687. Basawa, I. V. and Brockwell, P. J. (1984). Asymptotic conditional inference for regular nonergodic models with an application to autoregressive processes . Ann. Statist., 12, 161-171. Kabaila, P. (1983 ). On the asymptotic efficiency of the estimators of the parameters of an ARMA process. Journal of Time Series Analysis, 4, 37-47. Rao, M . M. (1978 ). Asymptotic distribution of an estimator of the boundary parameter of an unstable process. Ann. Statist., 6, 185-190. Takeuchi , K. (1974). Tokeiteki suitei no Zenkinriron (Asymptotic Theory of Statistical Estimation ). (In Japanese ) Kyoiku-Shuppan , Tokyo. Taniguchi, M. (1983 ). On the second order asymptotic efficiency of estimators of Gaussian ARMA processes . Ann. Statist., 11, 157-169. Vostrikova, L. Ju. (1984). On criteria for ca-consistency of estimators. Stochastics 11, 265-290. White, J. S. (1958). The limiting distribution of the serial correlation coefficient in the explosive case. Ann. Math. Statist., 29, 1188-1197. Requ en Mai 1985

276 Masafumi Akahira 1 and Kei Takeuchi 2

ON THE DEFINITION OF ASYMPTOTIC EXPECTATION ABSTRACT In this paper a definition of asymptotic expectation is given and its fundamental properties are discussed. 1. INTRODUCTION In the asymptotic theory of estimation, the concept of asymptotic expectation is widely used (e.g., Akahira and Takeuchi , 1981 ; Ibragimov and Has'minskii, 1981 ; Lehmann , 1982), and it is usually remarked that it can be different from the asymptotic value of expectation . It is, however , not sufficiently accurately defined in the literature, especially when the asymptotic distribution does not exist. In the paper we shall give a definition of the asymptotic expectation and show its properties , e.g., its linearity and a Markov type inequality. We shall also obtain the necessary and sufficient conditions for the convergences in probability and distribution.

2. RESULTS Let {X,a} be a sequence of non-negative random variables . For any sequence {A„} of positive numbers we define an A„-censored sequence as X,,(A,,) = min{X,,, A„},

n = 1, 2,....

For {X„} we denote by A(X„) a set of all the sequences {A„} of positive 1 Department of Mathematics , University of Electro-Communications , Chofu, Tokyo 182, Japan 2 Faculty of Economics , University of Tokyo, Hongo, Bunkyo-ku , Tokyo 113, Japan. 199 I. B. MacNeill and G. J. Umphrey (eds.), Foundations of Statistical Inference, 199-208. Q 1987 by D. Reidel Publishing Company.

277

200 M. AKAHIRA AND K. TAKEUCHI

numbers satisfying as n -^ oo.

Pr{Xn>An}-+ 0

We define the upper and the lower asymptotic expectations of {X,,} as As.E(Xn) =

inf

limsupE (Xn*(An))

{An}EA ( Xn)

n-+oo

and As.E(Xn) =

inf lim inf E(X,,(An)),

{An}EA( Xn) n-+oo

respectively. Note that they can be infinity. Definition 2.1. If As.E(Xn) = As.E(Xn) < oo, then we call it the asymptotic expectation of {Xn} and denote it by As.E(Xn). The following theorem establishes that the above definition is reduced to the usual concept when Xn has an asymptotic distribution. Theorem 2 .1. If Xn has a proper asymptotic distribution F, that is Pr{Xn < x} -. F(x)

as n -' oo

for every continuity point x of F, then

As.E(Xn) =

J0 O xdF(x),

provided that the right-hand side is finite. Proof. Denote by Fn(x) the distribution of Xn. We put µ = xdF(x). 0^ Since for any fixed positive number A

I E(X„(A)) - E (X*(A))I

1

0

A xdFn(x) -

J0 A xdF(x)

+AIFn(A) - F(A)I,

we have

I E(Xn*(A)) - E(X *(A))I -' 0 as

n -+ oo. (2.1)

278 ON THE DEFINITION OF ASYMPTOTIC EXPECTATION 201 Since µ is finite, it follows that I E(X*(A)) - µI -^ 0

as A -+ oo. (2.2)

Then it follows from (2.1) and (2.2) that for any sequence {E,n} of positive numbers there exists a sequence {A.,,,,} of positive numbers such that I E(X{(Am)) - pI < Em

(m = 1, 2, ...). (2.3)

For the sequence {A,,,,} there is a monotone increasing sequence {N(m)} of positive integers such that for n > N(m) I E(Xn(A,n)) - E(X *(Am))I < E,n.

(2.4)

For n satisfying N(m) < n < N(m + 1) we define m = N-1(n). Denoting AN-i(n) and EN-1(n) by An and en, respectively, we have by (2.3) and (2.4) I E(Xn(An)) -.01 < 2En for sufficiently large n.

Letting En -^ 0 as n -^ oo, we obtain

I E(Xn(An)) - pj --^ 0 as

n

-t

oo.

This completes the proof. In order to obtain some properties on the asymptotic expectation we have to prove some lemmas.

Lemma 2 .1. If As. E(Xn) < oo, then for any sequence {An} of positive numbers tending to infinity as n -^ oo, we have that {An} E A(Xn). Proof. Assume that there exists a sequence {A;,} of positive numbers such that limn---o. A'„ = oo and {A;,} 0 A(Xn). For some c > 0 there exists a subsequence {A, . } of {An) such that Pr{Xn, > A;,^ } > E. Since for any {An} E A(Xn), Pr{Xn > An} --^ 0 as n -^ oo, it follows that for sufficiently large j Pr{Xn, > And } < E,

279

202

M . AKAHIRA AND K. TAKEUCHI

hence An, > A'.. And we have for sufficiently large j E(Xn.(Ani)) 1 E(Xn,(An.)) > A'. Pr{Xn, > And } i > And

hence the last term tends to infinity as j - ^ oo . Hence we have lim sup E(Xn(An)) = oo, n- oo

which contradicts the condition of the lemma. This completes the proof. We denote by A,.(Xn) a subset of A(Xn) whose element { An} satisfies limn---oo An = 00.

Lemma 2 .2. The following hold: As.E(Xn)

l nm up E(X X (An)); inf liminfE(X;,(An)).

_ {An} inf (X..)

As.E(X,) =

{A„}EA„(X,.) n- ^ oo

Proof. If the left-hand side of either of the two equalities above is infinity, it is obvious that the right -hand side is also infinity, hence we may assume that the right-hand side is finite. For any {An } E A(Xn) we put Pn = Pr{Xn > An}. Since pn -+ 0 as n -+ oo, putting A' = max{An , pn112} we have An -+ oo as n -^ oo, hence {An} E A„( Xn). Since An > An, we obtain E(Xn(A,)) ^ E(Xn*( An)). (2.5)

Since E(Xn(A' )) - E(Xn(An))

< (An - An)Pr{Xn

> An}

1/7 - pn r

it follows from ( 2.5) that

lim {E(Xn(An)) - E(Xn*(An))} = 0. Hence for any (An) E A(Xn) there exists a sequence {A„} E .4(X) such that n mo{E(Xn(An)) - E(Xn(An))} = 0.

280 ON THE DEFINITION OF ASYMPTOTIC EXPECTATION 203 This fact leads to the conclusion of the lemma. Lemma 2.3. If As .E(X,,) < oo, then there exists a sequence {A,,} E !100(X,, ) such that

limsupE(X,,(A;,)) = As.E(X,,); n-+oo

lim inf E(X„(A* )) = As.E(Xn). n-+oo

Proof. For a sequence { en) of positive numbers, such that en 10, there exists a sequence {An ,n} of positive numbers such that

lim sup E(Xn* (A n,,. )) < Ai.E(Xn)

+ en.

n-+oo

Without loss of generality we assume that, for each n, An,, is monotone decreasing in m. We Put Pn,,n = Pr{Xn > An ,n}. Then for each n, Pn,,n is monotone increasing in m and for each m, limn^oo Pn,m = 0 . For any m there exists n(m) such that Pn,,n < en for n > n(m). For any n let m = m-1(n) be a maximum value satisfying n(m) > n. Since m-1(n) -* 00 as n -+ oo, putting A** = An,,n-1( n) we have Pr{Xn > A;,*} <

of which the right-hand side tends to 0 as n - ^ oo. For any fixed no we obtain limsupE(Xn*(A;,*)) < limsupE(Xn*(An,,n- l(no))) n-'oo n-+oo

< As.E(Xn) + ern- 1 (no) . Since e,n-1(no) - ^ 0 as no .-' oo, we have

limsupE(Xn*(A**)) < As.E(Xn).

(2.6)

n-+oo

On the other hand it follows from the definition of As.E(Xn) that the inverse inequality of (2.6) holds . Hence it is seen that the equality in (2.6) holds. In a way similar to the above, it is shown that there exists a sequence {An***} E 6100 (X,,) such that

iim inf E(Xn* (An***)) = As.E(Xn). n-+oo

We put An* = min{A;,*, A;,**}. Then we have Pr{Xn>An)-+ 0 and

as n -^ oo

281 204

M . AKAHIRA AND K. TAKEUCHI

limsupE(X„(A;,)) < limsupE(X,,(A,,*)) = As.E(X,,); (2.7) n-oo

n-+oo

lim of E(Xn(An*)) < lim of E(X,,(A;,**)) = As.E(Xn). (2.8) Returning to the definitions of As.E(X„) and As.E(Xn) we see that the inverse inequalities of (2.7) and (2 .8) hold. This completes the proof. Lemma 2.4. Assume that As.E(Xn) < oo. Let {An*} be a sequence satisfying the condition of Lemma 2.3. Let {Bn*} be any sequence such that B„ -p oo as n -+ oo and B;, < An* for all n > no with some no. Then {B„} satisfies the condition of Lemma 2.3. Proof. By Lemmas 2.1 and 2 . 2 we have {B,,} E A(Xn) and lim sup E(X „(B * )) < lim sup E(Xn* (An*)); n^oo

(2.9)

n-+oo

lim inf E(Xn(Bn* )) < lim inf E(Xn* (An*)). n-boo n-'oo

(2.10)

From the definitions of As.E(Xn) and As.E(Xn) we see that the inverse inequalities of (2.9) and (2.10) hold. This completes the proof. Lemma 2.5. Assume that As.E(Xn) < oo and As.E(Yn) < oo. Then (i) As.E(Xn) + As.E(Yn) > As.E(Xn +Yn);

(ii) As.E(Xn) + As.E(Yn) < As.E(Xn +Yn); (iii) As.E(Xn + Yn) < As.E(Xn) + As.E(Yn) < As.E(Xn + Yn). Further if Xn > Yn, then As.E(Xn - Y,,):5 As.E(Xn) - As.E(Y,,); As.E(Xn - Yn) > As.E(Xn) - As.E(Yn).

Proof. (i) By Lemma 2 . 3 we take {An} E Aoo(Xn) and {Bn} E Ao(Y,,) such that

As.E(Xn) = limsupE(Xn*(An)); n-ioo

As.E(Yn) = limsupE(Y„(Bn)). n^oo

Let cn = min{An, Bn}. Since

E((Xn +Yn)*(cn)) < E(Xn*(An)) + E(Y„ ( Bn)),

282

ON THE DEFINITION OF ASYMPTOTIC EXPECTATION 205 it follows from Lemmas 2.3 and 2.4 that

lim sup E((Xn + Yn) * (cn)) < As.E(X,.a) + As.E(YY). n-.oo

By Lemma 2.2 we obtain

As.E(Xn + Yn) < As.E(Xn) + As.E(Yn). (ii) By Lemma 2.3 we take a sequence {cn} E Aoo (Xn + Yn) such that Xs-.E(Xn +Yn) = lim sup E((Xn +Yn)*(cn)). n-oo

Since E(Xn(2))+E(Yn (2)) -5 E((Xn+Yn

)*(cn)),

it follows from Lemmas 2.3 and 2.4 that lim inf E (Xn* (2)) + lnm of E (Y„ (2)) As. E(Xn + Yn). n-ooo

By Lemma 2 . 2 we have

As.E(Xn) + As.E(Yn) < As.E(Xn + Yn). (iii) By Lemma 2.3 we take { cn} E Aoo (Xn + Yn) such that Xs-.E(Xn + Yn) = limsup E((Xn +Yn)*( cn)). n-'oo

Since

E((Xn+Yn)*( cn))>_E(Xn ( 2))+E(Y: (2)) it follows that

lim sup E((Xn + Yn)*( cn)) n-+oo

lim sup E (Xn* (c")) + lim inf E (Y„ (c")) . n- oo

2

n-+oo

2

By Lemmas 2.3 and 2 .4 we have

Xs-.E(Xn + Yn) > As.E(Xn) + As.E(Yn). Similarly we obtain As.E(Xn + Yn) < As.E(Xn) + As.E(Yn).

283 206 M. AKAHIRA AND K. TAKEUCHI The proof of the rest follows directly from (iii). This completes the proof. From the above lemmas we have the following theorems. Theorem 2.2. Assume that As.E(X,a) and As.E(Y„) exist. Then As.E(X,, + Yn) = As.E(Xn) + As.E(YY). Further if Xn > Yn, then

As.E(Xn - Yn) = As.E(X,,) - As.E(Yn). The proof of the theorem is directly derived from Lemma 2.5. Theorem 2.3. (Markov type inequality .) If As. E(X,,) < oo, then for any c>0 As.E(X„) limsup Pr{Xn > c} < n-.oo C lim inf Pr{Xn > c} < As.E(Xn) n-+oo C

Proof. There exists a sequence {An} such that As.E(Xn) = lim sup E (XZ(An)) n-oo

and An -+ oo as n -^ oo. Then we can take no such that for n > no, An > c and Pr{Xn > c} = Pr{XX(An) > c} < E (X. An c By Lemma 2.3 we have lim sup Pr{Xn > c} < n-. oo

As.E(Xn) C

The other inequality is obtained similarly. This completes the proof. Theorem 2.4. Xn converges in probability to zero if and only if As.E(Xn) = 0. Proof. The proof of necessity is clear . If Xn does not converge in probability to zero, then there exists a positive number c such that limn,oo Pr{Xn > c} > 0. It follows from Theorem 2.3 that this contradicts the condition. Thus the proof is completed.

284 ON THE DEFINITION OF ASYMPTOTIC EXPECTATION 207 Theorem 2.5. Assume that random variables X„ and X have the distribution functions F„ and F, respectively, and the moment generating function

(m.g.f.) of X, 9(0) =

J

eDxdF(x),

exists for all B in some open interval I including the origin . Then X„ converges in distribution to X as n --^ oo, i.e ., X, -' X if and only if As.E(e°X^) = g(O) for all 0 E I. Proof. If Xn D+X, then e6X^ D,e'X, hence from Theorem 2.1 As.E(e°X^) = g(0)

for all

0 E I.

If As.E(e°X^) = g(0) for all 0 E I, then it follows from Theorem 2.3 that the sequence {Fn} is tight, hence for any e > 0 there exist no > 0 and K > 0 such that for n > no Pr{Xn > K} < E. Then there exists a subsequence { n;} of {n} such that Fn; converges in distribution to some distribution Fo as i -+ co and

As.E(Xn;) = lim

$-.00

eDxdFn;(x) = g(0)

for all

B E I.

If a subsequence {F,,,,,} of {Fn} does not converge to Fo, then there is some positive number b such that a set A6 = {n : A(Fn, Fo) > S} is infinite, where A(F, G) denotes the Levy distance between two one-dimensional distributions. We also have a subsequence {Fm.} of {F,,,,,} such that rn E A6 (i = 1, 2, ...) and Fm' converges in distribution to some distribution Fo as i - ^ oo. By the first part of the proof we have Fo = F0, which contradicts the assumption. This completes the proof. Further, if Yn converges in probability to zero as n -, oo and a function f (x) y) is continuous in y, then it may be shown that As.E(f (Xn, Yn)) _ As.E(f (Xn, 0)). For general real random variables Xn we have Xn = X,+, - X;, where X,4 = max(Xn, 0) and Xn = max(-Xn, 0). Then we define the upper and the lower asymptotic expectations of {X,,) as

As.E(Xn) = As.E(Xf1) - As.E(Xn ); As.E(Xn) = As.E(Xn) - As.E(X. ), respectively, and if -oo < As.E(Xn) = As.E(Xn) < oo we call it the asymptotic expectation of {Xn} and denote it by As.E(Xn).

285 208 M. AKAHIRA AND K. TAKEUCHI In a way similar to the case of the non-negative random variable it may be shown that

As.E(-X„) _ -As.E(X„); As.E(-X,,) _ -Xs.E(X,,); As.E(X„ + Y„) < As.E(X,,) + As.E(Y,,); As.E(X„ + Y„) > As.E(X„) + As.E(Y„), provided that As.E(IX„ I) < oo and As . E(IY„ I) < oo. Hence As.E(X„ +Y,,) = As.E(X„) + As.E(Y„), provided that As.E(X,,) and Ae .E(Y„) exist.

ACKNOWLEDGMENTS The paper was written while the first author was at Queen's University in Canada as a visiting Professor. The visit was supported by a grant of the Natural Sciences and Engineering Research Council of Canada. He is grateful to Professor Colin R. Blyth for inviting him.

REFERENCES Akahira, M., and K. Takeuchi (1981). Asymptotic Efficiency of Statistical Estimators: Concepts and Higher Order Asymptotic Efficiency. Lecture Notes in Statistics 7. New York: Springer.

Ibragimov, I. A., and R. Z. Has 'minskii (1981). Statistical Estimation : Asymptotic Theory. New York: Springer. Lehmann, E. L. (1982). Theory of Point Estimation. New York: Wiley and Sons.

286 Metrika, Volume 34, 1987 , page 1-15

Locally Minimum Variance Unbiased Estimator in a Discontinuous Density Function By M. Akahiral and K. Takeuchi2

Abstract: The exact forms of the locally minimum variance unbiased estimators and their variances are given in the case of a discontinuous density function. Key words and phrases: Locally minimum variance unbiased estimator , Non-regular cases, Discontinuous density function.

1 Introduction In the theory of minimum variance unbiased estimation in regular cases , it is usually assumed that the density function is continuous with respect to the unknown parameter 0 to be estimated. In non-regular cases the minimum variance unbiased estimation was studied by Chapman and Robbins (1951), Kiefer (1952), Polfeldt (1970), Vincze (1979), Takeuchi and Akahira (1986), Akahira, Puri and Takeuchi (1986) and others. Though the Chapman-Robbins-Kiefer type discussion does not require the continuity of the density function in 0, their bounds are not usually sharp and the precise form of the locally minimum variance unbiased (LMVU) estimator is not given. In this paper we shall give an example where the density function is discontinuous in 0, and obtain the exact forms of the LMVU estimators of the parameter and their variances , and illustrate some aspects of the LMVU estimators in such a situation.

2 Results Suppose that the density function f(x, 0) (with respect to a a-finite measure p) is not continuous in 0, while the support A(0) _ {x I f(x, 0) > 0} can be defined to be independent of 0. Note that this condition does not affect the existence of an LMVU

1 M. Akahira, Statistical Laboratory, Department of Mathematics , University of Electro-Communications, Chofu, Tokyo 182 , Japan. 2 K. Takeuchi, Faculty of Economics , University of Tokyo, Hongo, Bunkyo-ku , Tokyo 113 , Japan. 0026 - 1335 /87/1/1 - 15 $2.50 © 1987 Physica-Verlag , Heidelberg

287 2 M. Akahira and K . Takeuchi

estimator , because it can be shown that an LMVU estimator at 0 = 00 exists if f ff(x, 0)}2dp(x)< o0 A(e0) f(x, 00) (e.g. see Barankin 1949 and Stein 1950). But the LMVU estimator sometimes behaves very strangely. Let X1, ..., X„ be independently and identically distributed random variables according to the following density function w.r.t. the Lebesgue measure

p

for 0<x<0 and 0+1<x<2;

f(x, 0) = q

for 0 < x < 0 + 1;

0 otherwise, where the parameter has the range 0 < 0 < 1, and p and q with 0

q

P1 0

0 1 6+1 2

Fig. 2.1 The density function x f(x, 0) given by (2.1)

Then the LMVU estimator „* = 6n(x1 , ..., xn) at 0 = 00 has the form

6n(xl ,...,xn)=f n f(x"tt) dGn(?7), 0 1=1 f(xi' 00) where Gn (ri) is some signed measure over the closed interval [0, 1] (e.g. see Stein 1950). First we shall construct the LMVU estimator at 0 = 00 given by the form (2.2) in the case when n = 1, i.e., x 1 = x. We put 2

Ke0(0 , r?) = f ire0,n(x)f(x, O)dx 0 with

_ f(x, tr) 1ro0 (x) f(x, 00)

(2.3)

288 Locally Minimum Variance Unbiased Estimator in a Discontinuous Density Function 3

In order to obtain the LMVU estimator 6i(x) at 0 = 00 it is enough to take some signed measure G over [0, 11 such that

Ee[0i (X)l = f j f f(x, "' dG(n)} f(x, 0)dx 0 o f(x, 00) 1

= o K00(0, r7)dGn) =0 for all 0 E [0, 1]. For rt > Bo we have for 00<x
for 00+1x<

+1;

(2.4)

p otherwise. For 77 < 00 we also obtain

forri<x<00; p IrBO,»(x) p q

for + 1 <x < 00 + 1;

1 otherwise. Note that for = 00, ae0,,1(x) = 1. In order to calculate K00(0, ri) we consider two cases 0 > 00 and 0 < 00. Note that for 0 = 00,Ke0(00,17) = 0.

Case 1 : 0 > 00. If 17 < 00 , then we have by (2.5) 2

K00(0,77 ) =O f ire0,,.(x )f(x, 0)dx 77 00 0 71+1 00+1 0+1 2 = f pdx + f qdx + f pdx + f qdx + f pdx + f qdx + f pdx

0 = 1.

n

00

0

77+1 80+1 0+1

289 4

M. Akahira and K. Takeuchi

If 00 < ri < 0 , then we obtain by (2.4) 2

ireo ,,7(x)f(x, 9)dx

Ke0(B,7?) = of

2 2 00 17 z A+iqdx +fl _ pdx +f p dx +f pdx + 4 dx 1 + f qdx + f pdx f 00+1 p T7+1 0+1 o 00 q 0

= 1 +c(r7-00), where 1 c=--4 (>0). pq

If 0
i7 00+1 0+1 q2 77+1 2

= f pdx + f - dx + f pdx + f qdx + f - dx + f qdx + f pdx 0 e0 q 0 n e0+1 p 0+1 77+1

=1+c(6-00). Hence we obtain for 0 > 00

I for 7? <00; Kee(O,r7) 1+c(r7-60) 1 +c(0 -00 )

for 00
(2.6)

for 9 <'q,

where 1 c=--4. pq Case 2: 0 < 90. In a similar way to the Case 1 we have by (2.4) and (2.5) 1

for B0
K00(0,17)= l+c(00-r7) 1+c(80-0 )

for 6<17<0o;

(2.7)

for r7<0.

We take a signed measure G* over [ 0, 1 ] satisfying G*({0}) _ -1/c, G*({ 1 }) = 1/c, G*({Bo }) = 0 and G*((0, 0 ) U (0 0 , 1)) = 0.

290 Locally Minimum Variance Unbiased Estimator in a Discontinuous Density Function

Since by (2.6) and (2.7) KOO(0, 0) =

1

for 0 > 00;

1 +c(00 -0)

for 0 <00;

1 +c(0 -00 )

for 0 >00;

1

for 0 <00,

and

K90(0, 1) = we obtain

E0[Bi(X)]

=o 1o

1rBO ,,7(x)dG *(1t)j

f(x, 0)dx

= o Ke0(0, n)dG*(i?) 1

= c {KBO(0, 1) -K00(0, 0)} +K00(0, 00)00

=0 for all 0 E [0, 1], which shows the unbiasedness. Hence the LMVU estimator 6 *(x) at 0 = 00 is given by 1 = f ir00,,7(x)dG*(rl) 6i (x) 0

1 ^ {'TOO,,W-'TBO,o(x)}+00, where q

for 0<x<00;

for 00<x<1;

7T00,1 (x)-1T0o,o(x) = for 1<x<00+1;

for 00+1<x<2.

5

291 6 M. Akahira and K. Takeuchi

80+1 2 x

P-1 I - - ---4

Fig. 2.2 . The values of 1re0 1(x) - ae p 0(x) for all x E [0, 2]

Note that 6i(x) can take values outside of the interval [0, 1 ] which is the range of the parameter 0. Since for 0 > 00,

Ee [ire0 ,i(X) - ir80, 0(X)] =C(0 -00), Ee[{ire0 ,i(X)-ireo,0(X) }21=cif+(p-4 l p4

)z

( 0 -00)

1 I

we obtain for 0 > 00 V0($i(X))= -+(0-00 ){ 1-(0-00)}, where Ve designates the variance at 0. In a similar way to the case when 0 > 00, we have for 0 < 00

Ve(6i(X )) = ± +(00 -0) { I -(00 -0)1. Hence the variance of the LMVU estimator ei(x) is given by

Vo(ei(X))

_ 1 +10 - 00(1- 10 -001).

Note that

V00 i(X )) > - = Ve0($i(X )) for all 0 E [0, 1].

(2.8)

292 Locally Minimum Variance Unbiased Estimator in a Discontinuous Density Function 7

n V.(8*(X)) c+4

c +00(1 -e

1 c

0

1 e0 2

e 01

e+i 0

Fig. 2.3 . The variance of the LMVU estimator 0^j(x) at 0 = 00 given by (2.8) when 00 > 1/2

Next we shall construct the LMVU estimator at 0 = Bo given by the form (2.2) in the case when n > 1. From (2.3) we have 2 2( n f (X, , 77) 1 H 0 ...0

1 1=1 f(xi, B0)

1

.f(x,, 0) dX i ... dxp i=1

_ {Kg0(O,r7)}" =KgO(9,?7)

(say)

(2.9)

By (2.6) and (2.7) we obtain for 0 > BO 1 K00(0,r7)

for ri < 00

{1+c(r7-00)} n for 0
(2.10)

for 9
and for 0 < 00 1 for t00
for 9
{1 +c(00 -0 )} n for r7 <0, where 1 c--4. P4 Note that for 0 = 00 , KOO(00, r7) = 1.

(2.11)

293

8 M. Akahira and K. Takeuchi

We take a signed measure

dGn

(

1\

over [0, 11 satisfying

Gn

1

8n(17) dr7 n (1 +clr1901)" sgn (rl -eo)

(Say)

for 0
Gn({1})=

(2.12)

1 +C9o)n-1' I

nc{1 +c(1 -00)}n -I ,

Gn({00 }) = 0 0 . Note that Gi is consistent with the G* given in above case n = 1. Then the estimator

6n

1 n

Bn(x1,...,xn )

f

11

f(

1,

(2.13)

f(xt,BO )dG*(17)

is unbiased. Indeed, we have by (2.9), (2.10) and (2.11) and for 0 > B0 2 1 xi' 1 EB($n) = f ... f i f II f( 7) dGn(rl)} n f(xi, 6)dx1 ... dxn

0 0 0 i =1 f(x;,00)

=1

1

f

Ke0(0, n)dGn(17)

=Gn({0})Ke0(O,0)+G n( {1 })xeo(e,1)+Gn( {eo })Ke0(B,eo) Bo

0

1

+fgn(ri)dri+ f { 1 + c(77-00)}"Sn(77)dr?+ { 1 +c(0 -0o )} "fgn(17)dr1 0

1

00

{1 +c(0 -90)}"

nc(1 +c90 )n-1

0

1 0

1

drl nc { 1 +c(1 -00 )}n-1 n ^11f {1 +c(00 -,)} n

l e 1 1 1 1 +(1-- fo d7l+(1 -n {1+c(0-Bo)}" n f {1+c(7-eo )}"d^ +00

1

=0. Similary we obtain for 0 < 00 E0 ($n) = 0

294

Locally Minimum Variance Unbiased Estimator in a Discontinuous Density Function 9 and for 0 = 00

Eep(S*) = Bp. Hence the estimator On is the LMVU estimator. Next we shall construct the estimator. Putting

" f(x;,17 n 718"0 ,+7() =fn1 fH iro0 ,'n(xi) f(xi, eo)

with x = (x 1, ..., xn), we have from (2.13) 1

6*`)=o rr00(x)dG *(i7) = 7fg0 p (x)G*({0}) + fg0,1(s)G*({ 1 }) + e0 00

1

+O f IrBp,7(x)gn(17)d17 + f TreO ,,70C)Sn(r7)d4-

0 For t = 0, 1 we have

7roo 0 (,1) =tlll lrBo,O(xi) = \pI a \q )b = (p)a- b

and a"^Q)b" ,TBp,l (X) =i n17T Bp ,l(xi) =

1

R P 4 1P

with a=#{x110<x;<0o}, b=#{x111<x1<80+1}, a'=#{x;IBO<xi<1} and b'=#{x;190+1<xi<2}, where # { } designates the number of the elements of the set { }.

Note that 0
'TBO,»(x)-\P1S14/r-1P^J r'

where s=#{xelrl<xi<0p}andt=#{x;lri+1<x,<0p+1}.

(2.14)

295 10 M. Akahira and K. Takeuchi

If00
P = ^

where s '=#{x1I00<xi
(q )a

I a'-b'

b

Gn( {0})+

-

I

00 1s - t

+o \Pq/

Gn( {1

\ 1

}) +Op

s-t

Sn ('^7)d'^'7 +ef q) Sn(77)d7? 0

Pa-b 14^a,-b,

nc(1 +c00 ) n- +nc{1 +c( 1 -00)1`1

+Bp

pq s-t 1 1P s'-t

+pf 1P

gn(17)d17 Bf

/

- gn(17)d1?,

(2.15)

where 1

gn(17)= 1

1

n)(1+c117-00I)

nsgn (17-00)

(0
For n > 2, it may be difficult to calculate directly the variance of the LMVU estimator 6n given by (2.15), but it may be possible to do it by (2.13). Putting 2 K00(0, f lre,n,n'(x)f(x, 0)dx 0 with

'Tep,n ,n'(x) = 7rep ,n(x)'r0p ,7j'(x)1 we have

296 Locally Minimum Variance Unbiased Estimator in a Discontinuous Density Function 11 2 2 1 1 n

f(x=, 1))

Ee(6n2) = o ... o 1f o i 111 f(xi, Bo)

rl f(xi, ?I') dGn(ii)dGn(iiI ) II f(xi, 0) H dxi i=1 i=1 i=1 f(xi, 00)

1 1 2 2 n f(xfi, n)f(xi, n ') 0 0 ...0 i =1

( (xi,B0))2

17)f(x,1 = f f 1 f2 f(x, 1)f(xi, 9)dx1ndGn (rl)dGn(??') o0o (f(x, 00)) 11

=oo

Keo (0, n,

Here we shall obtain the variance of the estimator

have for r?' <71 < 00 1 4

(2.16)

n')dGn(17)dGn(??')

$n at 0 = O. By (2.4) and (2.5) we

for 0 <x
P for n <x <00; for 00 <x
(2.17)

for 1?'+1<x
for 17+l<x<00+1;

1

for 00+1<x<2.

Since rre0 ,n,n'(x) = 7re0,n,,n(x), it is easy from (2.17) to obtain ire0,n,n'(x) for 17

<1?'

<0o.

From (2.17) we have

(l+c(Oo-'0) if ??'
(2.18)

297 12 M. Akahira and K . Takeuchi

By (2.4) and (2.5) we have for 77 < 00 -<-,q' (- 1 for 0 <x <17; 4 for 77 <x <00; p p for 00 <x <77'; q for 77'<x<77+1;

7re0,,7 17'(x) 1

p for 77+1<x<00+1; q 4 for 00+1<x<77'+1; p l l for 77 '+1<x<2. Then we obtain for 77 < 00 < 77' (2.19)

K0o(0o , 77, 77') = 1. From (2.18) and (2.19) we have

Ke0(0o , 7i,7i')=

1+c(00-77)

if 77'<77<00;

1+c(00-77')

if 77<77'<00;

1

if 77 < 00 <77'.

(2.20)

In a similar way to the above we have

Ke0(0o , 77,77')=

1

if 77 ' < 00 <77;

1+c(r7 '- 00)

if 00<77'<71;

1 +c(77-00)

if 00 <17<77'.

In order to calculate E00(6*2) we obtain from (2.16) i Eeo(6*2) = G*({O}) f KB0( oo, 0, r7 ')dG*(77') 0 i +G*({1})f Ke0(0o, 1,77')dG*(77') 0 i + G*({00 }) f K0(00,00 , 77')dG*(77') 0

(2.21)

298 Locally Minimum Variance Unbiased Estimator in a Discontinuous Density Function 13 1 ep

+pf o K

(0, n7 ,?7')dGn('R)dGn(T7')

1 1

(2.22)

+f B K"ep(0 , 17, r7')dGn(17)dGn (?7') 0 0 By (2.20) and (2.21) we have 1+c(eo-"I') Kep(B0 , 0, o) = 1

for t'
(2.23)

for 00
' 1 for r7 <9p; Kep(Bo, 1 ,77') =

(2.24)

1+c(r7'-00) for 0 0 <77'.

It is also clear that Ke p(B0 , Bp, r7') = 1 for all r7' E [0, 1]. (2.25) By (2.12), (2.23), (2.24) and (2.25) we have 1

(2.26)

Gn({0}) f Ko0(Oo , 0,77')dGn(I7') = 0; 0

1 Gn({1}) fKep(ep, 1,r7' )dGn(71') o

I cn{1

+c ( 1 - e o )}n-1

(2.28)

Ke"p(B0, 0o, ?7')dGn(1?')=Bo.

Gn({9p}) of

By (2.12) and (2.25) we also have for 0 < 17 < 00 1

Kgp(Bp, 77, 0)Gn({0})+ Knop(ep , 77,1)Gn ( { 1 ep

1

+(f +f +f)Kep(Bo,7l,'n')dGn(n') 0 ' ep

=77.

(2.27)

+Kep(ep,T? , ep)G n( {ep))

299 14 M. Akahira and K . Takeuchi

Then we have for n > 2 1 0° f0f

KB0(

00,rl,rl' )dGn(n)dGn(rl')

00 1

=f o Ke"0(00,17, r7 ')dGn(r7')dGn(n) 0

_ 0f r1dGn(11) 00 1 1+CO0) for n=2; -+2C21og( 2c 00

1

(2.29)

1

cn +c2n (n - 2) c2n(n - 2) (1 +c00)n

for n > 2.

In a similar way to the above we obtain by (2.12) and (2.21) 1 1

f e K00 (00, r7, ?7')dGn(rl)dGn(17') 0 1 2c 00

1 1 { 1+c(1-0o )}j for n=2; 1+c(1-0o)+-log

(2.30) 1 cn 00

1

1

{ 1 +c(1 -00)}n - 1

1

1 1

c2(n -2) 1 (1 +c(1 -0o))n

11

for n > 2. From (2.22), (2.26) - (2.30) we have 1 + 2c2 log {(1 +c00)(1 +c(1 -00))} for n = 2; EB0(en2) _ I 2 1 1 00 +c2n (n -2) 2-(1 +CO0 )n-2

1 (1 +c(1 -00))n-2

for n > 2. Hence for n > 2 the variance of the LMVU estimator 6n is given by log {(1 +cOo )( 1 +c(1 -00 ))} for n = 2; V0o($n) =

}

1 1 1 - 2- for n > ( I + cO0) n --2 (1 +c(I -00))n-2 -2)

1

2.

300 Locally Minimum Variance Unbiased Estimator in a Discontinuous Density Function

15

Note that for sufficiently large n

Vg0(e„)

2 +OIn I , c2

i.e., the variance of the LMVU estimator 6* is the order of n-2. The calculation of the variance V0(6*) at any B is more complicate than that of VOO(S*).

References Akahira M, Puri ML, Takeuchi K (1986) Bhattacharyya bound of variances of unbiased estimators in non-regular cases. Ann Inst Statist Math 38A:35-44 Barankin EW (1949) Locally best unbiased estimates . Ann Math Statist 20:477-501 Chapman DG, Robbins H (1951 ) Minimum variance estimation without regularity assumptions. Ann Math Statist 22:581-586 Kiefer J ( 1952) On minimum variance in non -regular estimation . Ann Math Statist 23:627--630 Polfeldt T (1970) The order of the minimum variance in a non-regular case. Ann Math Statist 41:667-672 Stein C ( 1950) Unbiased estimates of minimum variance. Ann Math Statist 21:406-415 Takeuchi K, Akahira M (1986 ) A note on minimum variance. Metrika 33:85-91 Vincze I (1979) On the Cramer-Frichet-Rao inequality in the non-regular case . In: Contributions to statistics . The Jaroslav Hdjek Memorial Volume . Academia, Prague , 253-263

Received June 12, 1984

301 Ann. Inst. Statist. Math. 39 (1987), Part A, 593-610

THE LOWER BOUND FOR THE VARIANCE OF UNBIASED ESTIMATORS FOR ONE-DIRECTIONAL FAMILY OF DISTRIBUTIONS MASAFUMI AKAHIRA AND KEI TAKEUCHI

(Received Sept. 13, 1985; revised Feb. 15, 1986)

Summary In this paper we introduce the concept of one-directionality which includes both cases of location (and scale) parameter and selection parameter and also other cases , and establish some theorems for sharp lower bounds and for the existence of zero variance unbiased estimator for this class of non-regular distributions. 1. Introduction For the lower bound for the variance of unbiased estimators, the most famous is the so-called Cramer-Rao bound. But the Cramer-Rao bound and its Bhattacharyya extension assume a set of regularity conditions. Chapman and Robbins [2], Kiefer [4] and Fraser and Guttman [3] obtained bounds with much less stringent assumptions, but they still require the independence of the support of the parameter 0 or almost equivalently that the distribution with 0#0 0 is absolutely continuous with respect to that with 0=0 0 when 0 is the specified parameter value at which the variance is evaluated. Recently in the non-regular cases the Cramer-Rao bound has been discussed by Vincze [8], Mori [5] and others. In the previous paper, Akahira, Puri and Takeuchi [1] get the Bhattacharyya type bound for the variance of unbiased estimators in non-regular cases. In this paper we introduce the concept of one-directionality which includes both cases of location (and scale) parameter and selection parameter and also other cases, and show that the bound for the variance of unbiased estimators is sharp in the sense that the actual infimum of the variance of unbiased estimators is equal to the bound for a specified 00, for this class of non-regular distributions. We also estabKey words and phrases: Cramer-Rao bound, Bhattacharyya bound, unbiased estimator, one-directional family of distributions, sharp lower bound. 593

302 594 MASAFUMI AKAHIRA AND KEI TAKEUCHI

lish that for a wide class of the non-regular distributions the infimum of the variance of unbiased estimators can be zero when the sample size is not smaller than 2. A simple but rather general case of the class of distributions of which none dominates another is that it is characterized by a real parameter 0 and the distribution shifts monotonically as 0 changes. For one dimensional random variables the case can be visualized by the following example.

01

01<02<03 Mathematical definition for such cases in a rather general set-up is given in Section 2 and is termed as one-directional family of distributions.

2. Definition of the one- directional family of distributions We assume that we are given a model consisting of a sample space (X, p) and a family '={PB: 0 E ®} of probability measures , where a parameter space ® is an open subset in a Euclidean 1-space R1. Throughout the subsequent discussion we shall assume the following : (A.2.1) For each 0 E ®, PB is absolutely continuous with respect to a o-finite measure p and the corresponding density w.r.t. p is f(x, 0). Let A(0) be a support of f (x, 0), that is, A(0) = [x: f (x, 0) > 0}. The determination of A(0) is not unique so far as any null set may be added to it, but in the sequel we take one and fixed determination of A(0) for every 0 E ® which satisfies the following : (A.2.2) For any disjoint points 01 and 02 in ©, neither A(01)DA(02) nor A(01) c A(02). For 01<02<03, (A.2.3)

A(01) n A(02) c A(01) n A(02) A(01) n A(03) c A(02) n A(08) . (A.2.4)

If 0„ tends to 0 as n --+ oo, then

303 THE LOWER BOUND FOR THE VARIANCE OF UNBIASED ESTIMATORS 595

ft ((DUI inn A(et) GA(B)) =,u ( (x(11 tiun A(Bt )) 4A(0)) = 0 , where EGF denotes the symmetric difference of /two sets E and F. (A.2.5) For any two points 0, and 02 in ® with 0,<02i there exists a finite number of ^t (i =1, • • • , k) such that 01 _ e0 < el< • • • <$1=02 and p(A(st)nA(e,))>0 (i=1,..., k). Then IT is called to be a one-directional family of distributions if the conditions (A.2.1) to (A.2.5) hold. 3. The lower bound for the variance of unbiased estimators when X is a real random variable Now suppose that X is a real random variable with a density function f (x, 0) whose support is an open interval (a(0), b(0)), then the condition of one-directionality means that a(0) and b(0) are both monotone and continuous functions. Without loss of generality we assume that a(0) and b(0) are monotone increasing functions. Therefore we can formulate the following problem. Let X=®=R1. Let X be a real random variable with a density function f (x, 0) (with respect to the Lebesgue measure p) satisfying the following conditions (A.3.1) to (A.3.7) : (A.3.1)

f(x, 0)>0

for

a(6)<x
f(x, 0)=0

for

x5a(0), x>_b(0) ,

where f(x, 0) is continuous in x and 0 for which a(0)<x0, b'(O)>O for all 0. (A.3.2) lim f (X, 0) = lim f (X, 0) = 0 , x-a(6)+0 x-+ b(B)-0

and for some positive integer p lim at f(x,

0) =

lim

at f(x, 0)=0 (i=1, • • •, p-1) ,

x-•a(6)+O aet x-b (8)- O aet

lim ap f(x, 0)=A,(0) , x-.a(6)+0

lim

aep

ap

x -+ b(6)-0 aep

f(x, 0)=Bp(0) ,

where Ap (0) and BP (0) are non- zero, finite and continuous in 0. (A.3.3) (atlaet)f(x, 0) (i=1, • • •, p) are linearly independent. (A.3.4) For some 00 E 0 {f( x, 0O)}2

00) du(x)

0 a(60 )

f (x,

304 596 MASAFUMI AKAHIRA AND KEI TAKEUCHI

is finite for each i=1,• • •, k, and (,X, at

ri __ n

)z

(11 b(°o)

f(x ,

a(00)

dp 0o)

is infinite for each i=k+1, • • •, p unless ck+1= • • • =c,=0. (A.3.5) For Bb E ® there exists a positive number a and a positive-

valued measurable function p(x) such that for every x E A(B) and every

0 E (e o- e, Bo+ s), p(x) > f(x, 0), and for every 0 E ( B o- e, B o+ Z),

I

r(x) I

A(B)

I r(x) I p(x)dp < oo .

x f (x, 0)d p < oo implies 6E(Oo BO+,)ACB)

(A.3.6) For each i=1, • • •, p+1, ^Z(-1)f \^)f(x, 9o+i h)

sup

Tim

I h I `p(x)

A-0 xE

U A(BO+1h )- A(60) J=1

< oo .

(A.3.7) For each i=1,. • •, p+1,

It f (x, eo+h) TM sup < oo . h-. 0

xE

p(x)

A (eo)

Note that the conditions (A.3.5), (A.3.6) and (A.3.7) are assumed to obtain the Bhattacharyya bound for the variance of unbiased estimators (see Akahira et al. [1]). First we consider the special case when p=0, then we have to modify slightly the condition (A.3.2) as follows : (A.3.2)' lim f(x, 0)=Ao(0)>0 , x-a(8)+0

lim f(x, 0)=B0(0)>0.

x-b(B)-o

In the following theorem we shall have a lower bound. THEOREM 3.1. Let g(0) be continuously differentiable over ®. Let g(x) be an unbiased estimator of g(0). If for p=0 and a fixed Bo, the conditions (A.3.1), (A.3.2)', (A.3.5), (A.3.6) and (A.3.7) hold, then min 4: unbiased

where

Vgo

Vgo

(g') = 0 ,

(g) denotes the variance of g at 0= 00 .

PROOF. We first define gg(x)=g(00)

for

a(00)<x
Since a(0o)00, it follows from the unbiasedness condition that

305 THE LOWER BOUND FOR THE VARIANCE OF UNBIASED ESTIMATORS 597 b(o) r b(B0) r b(o)

(3.1)

g(8) = g(x) f (x, O)d p = a(O)

f

g(x) f (x, a(B)

O)d p +

f

g(x) f (x, O)d p .

b(80)

Putting b(6)

h (B) = J a(B) g(x)f (x, O)dp , we have by (3.1) r b(e) (3.2)

f

g(x)f(x, O)d ,a = g(O) -h(O) .

b(00)

Differentiating both sides of (3.2) with respect to 0, we obtain

(3.3)

b(B)

b '(0)Bo(0)9'(b(0))+

b(OO)

I

g(x) a o

A x, B)Jda=g'(B) -h'(B)

Differentiation under the integral sign is admitted because of (A.3.7) with p=0. If g(x) satisfies (3.3), then it also satisfies (3.1) since g(Bo) =h(00). Since by (A.3.1) and (A.3.2)', b'(e)Bo(B)>0, it follows that the integral equation (3.3) is of Volterra's second type, hence the solution g(x) exists for b(B)>x>_b(Bo). Similarly we can construct g(x) for x< a(00). Repeating the same process we can define g(x) for all x. Hence we have min

V00()

=0.

¢: unbiased

Thus we complete the proof. The following useful lemma is a special case of the result by Takeuchi and Akahira [7]. LEMMA 3 .1. Let g(0) be p-times differentiable over 0. Suppose that the conditions (A.3.3) and (A.3.4) hold and G is the class of the all estimators g(x) of g(6) for which

5

b(OO) A

a(BO) a(BO) 0

g(x)f(x, Bo)du=g(00)

g(x ) {

a9(

f (x, Bo)} du = g (i) (Bo) (i= 1, ... , P),

where g(`)( 0) is the i-th order derivative of g(0) with respect to 0. Then inf VB (g) _ (g(i) (0o), ... , g(k) (0o))A-' ( g(i) (eo), ... , g ( k) (eo))'

$EG

where A is a k x k matrix whose elements are .i

2Z>

(

b(BO)

a(BO) U

f (x,

Bo)

B7 P x,

A x, 690)

00) du (i, 9 =1, ... , k).

306 598 MASAFUMI AKAHIRA AND KEI TAKEUCHI

The proof is omitted. In the following theorem we shall get a sharp lower bound. THEOREM 3.2. Let g(O) be (p+1)-times differentiable over ®. Let g(x) be an unbiased estimator of g(0). If for p> 1 and a fixed 00, the conditions (A.3.1) to (A.3.7) hold, then inf VOD(9) = vk (0o) ¢: unbiased

that is, the bound vk (00) is sharp,( where vk (Bo ) =

(g (1) (00)

, ... , g(k)

(00))A- 1(g(,) (00), ...

g(k)

(00))^

with a k x k matrix A given in Lemma 3.1. PROOF. From the unbiasedness condition of g(x), (A.3.2) and (A.3.7) we have D(0) ^

(3.4) a(80)

(3.5)

D

'(x)f (x, 0o)dp = g(00) , g(x)

J a(B)

I a{

Ax, 0o)

J

(i=1,..., p)

dp =g(s) (eo)

j^

By Lemma 3.1 it follows that the sharp lower bound of

(1' D(B0)

V9D( il)= Ja(00)

{g(x) -g(00 )}2 f (x, 00)d p(x) under (3.4) and (3.5) is given by (g(')( 00)'...' 1(g(1)(0o) y ..., g(k)(B0 )),, i.e., g(k)(eo))Ag(k) (00)) A-' (g(1 ) (00), ... , g (k) (3.6) inf VBo ( 9) = (g((00) , . .. , (00))l 0- (3.4), (3.5)

= v k (0) (say) Note that the right-hand side of (3.6) is the Bhattacharyya bound for the variance of unbiased estimators at 0=0 (see Akahira et al. [1]). From (3.6) it follows that for any e>O there exists gr,(x) in the interval (a(00), b(60)) satisfying (3.4), (3.5) and

VBD(g, )Svk( 0o)+e

.

We can extend g, (x) for x outside (a(00), b(00)) from the unbiasedness condition b(0)

(3.7)

J g, (x) f (x, O)d p(x) = g(0) a(B)

for all

For B>Bo, i.e., b(B)zb(00) we put b(BD) ^

h(0) = J b(e) r, (x)f (x, 0)d.(x) .

OE

®

.

307 THE LOWER BOUND FOR THE VARIANCE OF UNBIASED ESTIMATORS 599 By (3.7) we obtain b(6)

(3.8)

g, (x) f (x, O)d p(x) = g(O) - h(B) . b(BO)

Differentiating (p+1)-times both sides of (3.8) with respect to 0, recursively, we have by (A.3.1), (A.3.2), (A.3.5), (A.3.6) and (A.3.7) 1

0(0) o)

b( 80) b(B)

^,(x){ _

f( x , B ) dp(x)=g(')(8)-hu)(e)

(J

^b(O) g, ){ - f(x, 0)}dp(x)=g(P)(0)-h(P)(0) , 1

(3.9)

BP(0 )b(1)(9),,(b(0)) +Sb(8))9'.(x)1 0

86P+ i

f(x, 0) }dp= g(P +1)(6)-h +1)(6) . (P

Differentiation under the integral sign is admitted because of (A.3.7). If g,(x) satisfies (3.9), then it also satisfies (3.8) since g")(00)=h(()(00) (i=1,---,p) and g(00)=h(60). Note that h(P+') (B) is determined by the values of g, (x) for a(B) <x x z b(00). Similarly we can construct &(x) for x
inf

6: unbiased

Vao(g)= V,E(8o)

i.e., v, (90) is a sharp bound. Thus we have completed the proof. We shall give one example corresponding to each of the situations where the conditions (A.3.1), (A.3.2)/(A.3.2)' to (A.3.7) are assumed. Example 3.1. Let X be a real random variable with a density function f (x, 0) (with respect to the Lebesgue measure p) satisfying each case. (i) Location parameter case. The density function is of the form f (x - 9) and satisfies the following : f(x)>0

for

a<x
f(x)=0

for

x5a, xZb ,

and lim f(x)>O, lim f(x)>O and f (x) is continuously differentiable in the open interval (a, b).

(ii) The case on estimation of g(O)= O with a density function

308 600 MASAFUMI AKAHIRA AND KEI TAKEUCHI

c(1-(x-8)2)11-1

for Ix-0l<1 ,

f(x-0)= {

0

for

I X-0 1 >_1 ,

is discussed in Akahira et al. [1], where q>1 and c is some constant. (iii) Scale parameter case. The density function of the form f (x / 0) / B satisfies the following :

f(x)>0

for

f (x) = 0

otherwise ,

0
and satisfies the same condition in (i). (iv) Selection parameter case (e.g., Morimoto and Sibuya [6]). Consider a family of density functions whose supporting intervals depend on a selection parameter 0 and are of the form (0, b(0)), where - oo <0
AX, 0) =

p(x) F(0)

for

0 xSb(0) ,

l 0 otherwise , b(8)

where p(x)>0 a.e. and F(0)=JB p(x)dp(x).

Note that the cases (i),

(iii) and (iv) correspond to the case p=0, and (ii) corresponds to the case p=q-1, where q is an integer.

4. The lower bound for the variance of unbiased estimators for a sample of size n of real-valued observations Now suppose that we have a sample of size n, (X1, • • • , Xn) of which Xi's are independently and identically distributed according to the distribution characterized in the previous section. Then we can define statistics Y=max XX+min X, , 1gign

1^t:in

Z=max Xi-min Xi 15isn

1;5i$n

and we may concentrate our attention on the estimators depending only on Y and Z. Since they are not sufficient statistics, we may lose some information by doing so. More generally, for a sample of size n, (X1, • • • , Xn) from a population in a one-directional family of densities with a support A(0), we can define two statistics 0=sup{01XiEA(0) (i=1,•••,n)}, 2=inf{0IXXEA(0) (i=1,•••,n)}

309 THE LOWER BOUND FOR THE VARIANCE OF UNBIASED ESTIMATORS 601

and also define Y= 2 (0-+#) , Z= 2 (B - 9) . There are various ways of defining the pair of statistics Y and Z, but disregarding their construction we assume that there exists a pair (Y, Z) which satisfies the following. Let Y and Z be real-valued statistics based on a sample (X,, • • • , X..) of size n for n>_2. We assume that (Y, Z) has a joint probability density function f0(y, z) (with respect to the Lebesgue measure p ,,,$) satisfying

a.e.,

fe(y, z)=fa(y I z)ha(z) ,

where fg (y I z) is a conditional density function of y given z with respect to the Lebesgue measure pv and he (z) is a density function of z with respect to the Lebesgue measure a,. Note that if Z is ancillary, h0 (z) is independent of 0. We assume the following condition : (A.4.1) For almost all z [,as]

f0(yl z)>0

for

a$(e)
fe(yl z)= 0

for

y5a.(0), y >b$(8) ,

where as (0) and b, (6) are strictly monotone increasing functions of 9 for almost all z [,az] which depend on z, and h0(z)>0

for

c
h®(z)=0

for

zSc, zzd ,

where c and d are constants independent of 0. We also assume that for almost all z [p.], fa (y I z) instead of f (x, 0) satisfies the conditions (A.3.2) to (A.3.7), and we call the corresponding conditions (A.4.2) to (A.4.7). Let q(y, z) be any unbiased estimator of g(8). We define byte)

(4.1) ^y (B) = 9'(y, az(e)

z)fe (y l z)dpv

for a.a. z [ ,a,] .

Further we assume the following condition : (A.4.8)

0y(0o)=g(0o)

for a.a. z [py]

OV)(B )=0 for a.a. z [t ] (i= y 1,•••, k) o y where ¢()(8) is the k-th order derivative of ¢y(O) with respect to 0.

In the following theorem we shall show that the sharp bound is equal to zero.

310 602 MASAFUMI AKAHIRA AND KEI TAKEUCHI

THEOREM 4 .1. Let g(0) be (p+1)-times differentiable over 0. Let #(X1, be an unbiased estimator of g(0). If n>_ 2 and for a fixed ( X 1 , . ,. .X„) , 00, the conditions (A.4.1) to (A.4.8) hold, then, inf V00(g)=0. 6: unbiased

PROOF. From the unbiasedness condition we obtain

(4.2) 5 q5z(0) he(z)dpz=g(o)

for all

0E®.

First we assume that OJO) is given, then under (A.4.1) to (A.4.7) we have by Lemma 3.1 (4.3) inf (bZ(0o' J {g(y, z)-0a(Bo)}E.fOb(y l z)dpv o: (4.1) aa(O0)

_ (^a' (e0), • • ,

=vk(00 1 z)

zk' (Bo)) A '

(qa' ( Bo), • • . , }I•zk' (BO))'

(say) ,

where 0(t) (B) is the i-th order derivative of oz (O) with respect to 0 and d is a k x k matrix whose elements are

(y I z) {f60( y I z)} a0i fab(y I z)} d1i 2`' JaZ( o) fB A From (4.3) it follows that for any s > 0 there exists g, (y, z) such that bz(0)

(4. 4) (4.5)

z d B

a,(0)

J

bz(60) az(00)

for all B E©,

{g(y, z) -0z (00)}2f0b(y I z)dpv
Since by (4.2)

5

d by(0) A c .) ay(0)

9.(y,

z)f0(yI

z) h6(z)df vdf s=g(0)

for all 0 E ®, it follows from (4.5) that

(4.6)

Q (%)

J1

z (O o )

{g.(Y, z) - 0. (00) }2 f0b (y I z) h0o(z)d av d p. < ^ d vk (eo l z) h0o(z)dp,+ s

By the condition (A.4.8) vk(9oIz)=0 for a.a. z [,az] . From (4.6) we obtain

311 THE LOWER BOUND FOR. THE VARIANCE OF UNBIASED ESTIMATORS 603

Jd

(4.7)

(e0)

c Q(80)

{g. (?!, z) - Bo }2fab(y I z) heu(z) dp v d,ua < e

Putting g, (x,, • • • , x„) = g , (y, z), we have by (4.4) and (4.7) EB(g,)=g(8)

V00.) < e

for all 8 E ® ,

.

Letting a -> 0 we obtain

inf

V00(9)

=0.

unbiased

Thus we complete the proof. Now we shall find a function ¢a(8) satisfying the condition (A.4.8) when 8=R1, g(8)=8 and as is a Lebesgue measure. Without loss of generality, we put 00=0. We define (8)= Msgn 8I0I1+ 2(z-c)k ho ( z) {(d - z) k +2 + I 0 Ik +2 (z - c)k +2}

(4.8)

where M is a constant and k is a positive integer. Then we have

J

B h z dz=M

d ,( ) B( )

Ir

sgn 0l8Ik + 2(z-c)k

Jdc (d- z) k +2+ IBIk +2 ( z-C) k+2 dz

\\ uk _ M s gn e I 0 Ik+2 du d-c Jb 1 +I0Ik + 2ukt2

(after transformation u= z-c d-z M v MK sgn BIBI dvo 1 + vk+2 d-c d-c 81 where K=

Jb

1+vk +2 dv is a constant . If we put M=(d-c)/K, then

5zohe zdz=o And it is easily seen that ¢$(0)=0 for a.a. z, gZt>(0)=0 for a.a.

z (i=1, • • •, k).

Thus it is shown that 0,(0) given by (4.8) satisfies the condition (A.4.8). We consider the estimation on the location parameter 0. Let X, and X2 be independently and identically distributed with a density function f (x, 0) of the form f (x - 8) which satisfies the following :

312

604 MASAFUMI AKAHIRA AND KEI TAKEUCHI

f(x)>0

for

a<x
f(x)=0

for

xSa, x>b,

and lim f(x)>O, lim f(x)>O and f (x) is continuously differentiable in x-.a+ 0 x-.b-0

the open interval (a, b). We define Y=2(Xi+X2) Z=2(X,-X2). Then if the conditions (A.4.1) to (A.4.8) are assumed, we have

inf

Veo(B)=0.

d: unbiased

Note that Z is an ancillary statistic, but that (Y, Z) is not sufficient unless f(x) is constant for a<x
5. A second type approach to obtain the lower bound for the variance of unbiased estimators Suppose that (X, Y) is a pair of real random variables according to a joint density function f(x, y, 0) (with respect to the Lebesgue measure p) which has the product set (0, a(O)) x (0, b(O)) of two open intervals as its support A(B), where a(O) is a monotone increasing function and b(O) is a monotone decreasing function. We assume the condition

(A.5.1) inf

f(x, y, B)>0 .

(x.Y) a A(B)

Let the marginal density functions of X and Y be f, (x, 0) and f2 (y, 0) with respect to the Lebesgue measures Px and py, respectively. We further make the following assumption : (A.5.2) The density functions f,(x, 0) and f2(y, 0) are continuously differentiable in 0 and satisfy the conditions (A.3.5), (A.3.6) and (A.3.7) for k=1 when fl (x, 0) and f2 (y, 0) are substituted instead of f (x, 0) in them. In the following theorem we shall show that the sharp bound is equal to zero. THEOREM 5 .1. Let ®=R1. Suppose that X and Y are random variables with a joint density function f (x, y, 0) (with respect to a a -finite measure u) satisfying (A.5.1) and (A.5.2) for a fixed 0o. Let g(0) be continuously differentiable over 4. Let q(x, y) be an unbiased estimator of g(0). Then

inf ¢: unbiased

V90(g)

=0

313 THE LOWER BOUND FOR THE VARIANCE OF UNBIASED ESTIMATORS 605

PROOF. We first define gr(x, y)=g(B0)

for

(x, y) E A(Oo) .

In order to extend g(x, y) for (x, y) outside A(80) using the unbiasedness condition , we consider unbiased estimators #j x) and g2 (y) of g(8) with respect to fl (x, 0) and f2 (y, 0), respectively , such that g1(x)=g(80) for 0<x60 , i.e., a(O)> a(80) we put

a(00)

h, (0) = g(Bo)

0

.f1(x, B)dpx

and also for 0<00, i.e., b(B) >b(80) b(00)

h2(0) =g(80)

J0

f2(y, 0)dpy

Since #,(X) and g2 (Y) are unbiased estimator of g(8), it follows that a(B)

g,(x)fl(x, 6)dpx= g(6) -h,(8)

for all

8zO ,

92(y)f2(y , 0)dpy=g(B)-h2(8)

for all

e s eo .

a(B0)

(5.1) b(O)

b(00)

Since the supports of the density functions f, (x, 0) and f2 (y, 0) are open intervals (0, a(8)) and (0, b(0)), respectively, it follows from (A.5.1) that 0< lim

f,(x,

0) =a , (0) (say),

x-.a(6)-0

(5.2) 0< lim f2(y, 0) y-.b(O)-0

Differentiating both sides of (5.1) we have by (A.5.2) (5.3)

a

a(B)

gi(a(B))a,(0)-F Ja(B) 9''(x) ^ B b

f,(x, B)}df^x=g'(B)-h;(B)

for all B>_ Bb . Since the equation (5.3) is of Volterra's second type, it follows that the solution g,(x) exists for all x>a(00). If g,(x) satisfies (5.3), then it also satisfies (5.1) since g(00)=h,(B0). Similarly we can construct the unbiased estimator g2(y) for all y>b(B0). We define an estimator

(5.4)

g (x,

Y)=.

g(80)

for

0<x
#I (x)

for

a(B0)<x, 0
g2 (y)

for

0<x
Then g(X, Y) is an unbiased estimator of g(0) with variance 0 at 0=00.

314 606 MASAFUMI AKAHIRA AND KEI TAKEUCHI

Indeed, we have from (5.1) and (5.4) EO [9(X, Y)l

( =J

ff a(BO)

b(8) 0

f y, 9)d p + J \ g 1( x )f (x, y, 9)d p b(0) a(0)

g(B0)f0 (x,

0

a(00)

a o (Ob ) =

g(BO)

f 1(x, 0)d px+

\

J a(80)0 ) #1(x)fl (x, 0)dpx ,

for all 0 Z 00. Similarly we have that EO [9(X, Y )] = g(0) for all 0:5.- 00. Hence we see that g(X, Y) is an unbiased estimator of g(0). We also have, for all 0 Z 90

VB(s^(X, Y))= g2(0 0) ^o(BO)f1(x, B)dp -

a(80)

9'1(x)f1( x, B)dp - g2(0)

When 0=00, we obtain VOO(4(X, Y))=0 . Thus we complete the proof . We can give the following example. Example 5.1 . Let X , ,--- , X „ and Y , ,--- , Y„ be independently and identically distributed random variables with the uniform distributions U(0, 0) and U(0, 1 / 0), respectively. Put T1=max Xi ,

T2=max Y{ .

1sti5n

15LSn

Then the unbiased estimator d (T1i T2) of 0 with variance 0 at 0=1 is given by

(5.5)

B(t1, t2)=

#1(t1)

for

15t1, 0
#2(t2)

for

1;5t2, 0
1 for t1<1, t2<1 , where B1(t1)=

(1 + 1 1t1 for 15t1 ,

92(12)=(1- 1)t2 n

for 15t2.

Indeed, we can easily see that the estimator B(T1, T2) We also have, for 0<0S1

VO(B(T^, T2))= We also have, for 0>-1

{1-

(n-1)2 n(n-2)

}(On

__.02)

315 THE LOWER BOUND FOR THE VARIANCE OF UNBIASED ESTIMATORS 607

(5.7) V0(0(T^, T2))= {1- (n+1)2 1(9_"_ 02) . n(n-2) From (5.6) and (5.7), we obtain

V1(9(T1, T2))=0 . Thus we see that the unbiased estimator B(T1, T2) given by (5.5) with variance 0 at 0 =1. As a further case of this situation, let X and Y be independent real random variables according to density functions (1/0)f(x/0) and Of(By) (with respect to the Lebesgue measure p) with a positive valued parameter 0, respectively, which satisfy the following : (A.5.3)

f(x)>0

for 0<x<1 ,

f (x) = 0

otherwise ,

and f(x) is (p+1 )-times continuously differentiable in the open interval (0, 1) and for each i=0, 1,..., p ca>( )

0< lim f x < oo , 0< lim x-•0+0 xP-f

x-.1-0

fcs) (x)

(1 - x)P-t


By Theorem 4.1 we have the following : THEOREM 5.2. Let g(0) be an estimable function which is (p+1)times differentiable over R'. Let $(X, Y) be an unbiased estimator of g(0). If the conditions (A.5.3) and (A.4.8) on pe(xjt) hold, then

inf V90(g) = 0 , 6: unbiased where p0 (x It) denotes the conditional density function of X given XY= t.

PROOF. Letting T=XY, we have the conditional density function pe (x l t) of X given T= t : x (5.8)

p0 (x l t) _

for

Bt<x<0 ,

50 xf(0)f(xt)dx 0 otherwise ,

for almost all t [p].

Since for 0=1 and almost all t [p]

p1(x l t) =

( x

l0

-L

f(x)f ( X )

for

t<x<1,

otherwise ,

316 MASAFUMI AKAHIRA AND KEI TAKEUCHI

608

we obtain from (5.8) pB (x l t) p, ( (t)

for all

6, a.a. t [,u]

where ct=

(5t x f( )f x

(x )dx)-1.

Putting

gt (x) _

xf(x)

for

t<x<1 ,

0 otherwise , for almost all t [p], we have (5.9)

a.a.

p,(xI t)=f(x)gt(x) ,

t [f^]

hence the same conditions on p, (x l t) as (A.4.1) and (A.4.2) hold. Indeed, we have by (A.5.3) and (5.9) limo p, (x l

t)=f(t + 0)gt(t +o)= tt f(t+0)f(1+0)=0

(5.10) lim p,(xlt)=f(1- 0)g' (1 -0)=0

x-. 1 -0

for almost all t [p]. (5.11)

By the Leibniz's formula, we obtain

(j)f"' (x)g('t -1)(x) atpaxx l t) = E J=O =E (2 )fci_i)(x) g1)(x),

a.a.

t

>=a j

By (A.5.3) we have for i=1, • • •, p-1 and a.a.

(5.12) lim ai p, (xIt) =0, x-.1-0

a x,

[p]

t [tt]

lim ap p 1 (xIt) =ct * 0

x-.1-0 axp

where ct is finite. Since g
t i 1 8f-i )=ct E (-1)f x^+1 axf-i f( x

it follows by (A.5.3) and (5.11) that for almost all t [,u],

lim x-•t+0

(5.13) lim x-.t+0

ai p, (xl t) =0

(i=1,...,p-1),

ax,

app1(xl t) =D,00 , axp

.

317 THE LOWER BOUND FOR THE VARIANCE OF UNBIASED ESTIMATORS 609

where D, is finite. It is seen by (5.9), (5.10), (5.12) and (5.13) that the same condition on p, (x I t) as (A.4.2) holds. For i=1,. ., [p/2]

{P

I {

0<Jc

,(

x 10

1

2

oo ,

a.a.

8 P I (xI t) dx<

t

[p]

and

LI{ i- E +,c; aLt

pI (x I t)} 2 ax dx ,

[P,2]

t

PI (x 1 0

a.a.

t [p ]

is infinite unless CEPi2]+1 = • • . =cP, where [s] denotes the largest integer less than or equal to s, since when x-^ 0+0 or x--).t-0 the numerator of the integrand approaches to a polynomial in x or t-x of the degree p-i* if ci.*0 and cia+,= • • . =cP=O and the denominator tends to that of the degree p. Hence the same condition on p, (x I t) as (A.4.4) holds for k = [p/2]. And also from (A.5.1) it is seen that when x-0+0 or x--).t-0, {(8i/ ax`)p, (x I t)} l pI (x I t) (i=0, 1,. • • , k) approaches to polynomials of different degrees, hence they are linearly independent. Putting PI (x) =

sup

p, (x' 10

x': IX'-XI
for appropriate c> 0, the same conditions on pI (x I t) as (A.4.5) to (A.4.7) hold. By Theorem 4.1 we obtain the conclusion of Theorem 5.2. When g(O)=O, Example 5.1 can be also an example of Theorem 5.2.

Acknowledgements The authors wish to thank the associate editor and the referee for useful comments. UNIVERSITY OF ELECTRO - COMMUNICATIONS , TOKYO* UNIVERSITY OF TOKYO

REFERENCES Akahira, M., Puri, M. L. and Takeuchi, K. (1984). Bhattacharyya bound of variances of unbiased estimators in non-regular cases, Ann. Inst. Statist. Math., 38, 35-44. Chapman, D. G. and Robbins, H. (1951). Minimum variance estimation without regularity assumptions, Ann. Math. Statist., 22, 581-586. Fraser, D. A. S. and Guttman, I. (1952). Bhattacharyya bounds without regularity assumptions, Ann. Math. Statist., 23, 629-632. * New at University of Tsukuba.

318 610 MASAFUMI AKAHIRA AND KEI TAKEUCHI Kiefer, J. (1952). On minimum variance in non -regular estimation , Ann. Math . Statist., 23, 627-629. M6ri , T. F. (1983). Note on the Cramer -Rao inequality in the non -regular cases: The family of uniform distributions, J. Statist. Plann. Inference, 7, 353-358. Morimoto , H. and Sibuya , M. (1967). Sufficient statistics and unbiased estimation of restricted selection parameter , Sankhyd, A27, 15-40. Takeuchi, K. and Akahira, M. (1983). A note on minimum variance, Metrika, 33, 85-91. Vfncze, I . (1979). On the Cram€r-Frdchet-Rao inequality in the non -regular case, In Contributions to Statistics , the Jaroslav H'ajek Memorial Volume , Academia, Prague, 253-262.

319 STATISTICAL THEORY AND DATA ANALYSIS II K. Matusita (Editor) ® Elsevier Science Publishers B.V. (North-Holland), 1988

SECOND ORDER ASYMPTOTIC EFFICIENCY IN TERMS

191

OF

ASYMPTOTIC VARIANCES OF THE SEQUENTIAL MAXIMUM LIKELIHOOD ESTIMATION PROCEDURES

Kei TAKEUCHI and Masafumi AKAHIRA

Under suitable regularity conditions, the Bhattacharyya type bound for asymptotic variances of estimation procedures is obtained. It is also shown that the modified maximum likelihood estimation procedure attains the bound if the stopping rule is properly determined.

1. INTRODUCTION Sequential estimation procedures can be defined in two stages (a) definition of a stopping rule, and (b) definition of the estimation procedure once the stopping rule is determined. In this paper , we restrict ourselves to the class of estimation procedures related with a sequence of sequential sampling procedures where the sample size tends stochastically to infinity and is asymptotically constant in the sense that its coefficient of variation tends to zero . Then we can show that the asymptotic variance of the estimator must satisfy the Bhattacharyya type bound , and for one parameter case, the modified maximum likelihood (ML) estimation procedure attains the bound, if the stopping rule is properly defined. The above results can be interpreted to mean that the asymptotic loss (deficiency) of the ML estimation procedure can be reduced to zero when we apply an appropriate sequential estimation procedure . This fact can be established also for the asymptotic distribution of the sequential ML estimation procedure which will be dealt with in a subsequent paper by the same authors.

2. BHATTACHARYYA BOUND AND MAXIMUM LIKELIHOOD ESTIMATION PROCEDURE Let X1, X2, •••, Xn, ••• be a sequence of independent and identically distributed random variables with a density function f(x, 0) with respect to a a -finite measure p, where 0 is a real-valued parameter. Suppose that the sample size n is determined according to some sequential rule. Actually, we consider a sequence of sequential estimation procedures {IIQ : a=1 ,2, ••• } such that Ee,Q(n) =v. (0) becomes large uniformly in 0 as a-- , where for each a we define a sto oping rule and estimators based on it and consider the asymptotic distribution of V (9a-9 ) as a--. For simplicity, we denote v0(9) by v. In order to consider the second order asymptotic efficiency, we assume the following conditions.

(A.1) Eo(n) = v+o(1), Vo(n)/v = O(1), Ee(nk/vk) = O( 1) (k=2,3,4), {(a/119)va(9)}/va(9 )= O(1) and { (a2/a92)va(9)}/va(0)=O(1).

320 192

K. Takeuchi and M. Akahira

(A.2) {x : f(x,O) > 0} does not depend on 0. (A.3) For almost all x[p], f(x, 0) is four times continuously differentiable in 0. In the Taylor expansion t

108 /(x,e+h) - h e(i)( 9,x)+h4R(x,h), /(x,e) . 1 i!

R(x,h) is uniformly bounded by a function G(x) which has moments up to the fourth order, where f(i)(O,x) _ (ai/aei)€(e,x) (i = 1 ,2,3,4) and €(O,x) = log f(x,0). (A.4) For each 0, 0
J(O)=Ee[f(1)(e,X)f(2)(O,X)], K(0)=Ee[(f( 1)(O,X)}3], L(0) = Ee[f(1)(O,X)f(3)(O,X)], M(0) =Ee[{f(2)(O,X)}2] -12(0), N(0) = Eg[{f(1)(O,X)}2€(2)(6,X)] +12(0), H(0) =Ee[{f(1)(O,X)}4] - 3I2(O),

and both J(0) and K(0) are differentiable in 0, and Ee[f(3)(O,X)] _ -3J (0) - K(0) and Ee[f(4)(O,X)]=4L( O)+3M(0 )+ 6N(0)+H(0). We put

Z,"= ¢1 and

)(o,x ), Z2, = 1- Y {(2)(e, x.)+ r(e)), vi=1 -Vvi=1 Z 3'V 1_ {e«'(e, X)-3J(e)-K(e)}. s,^ Vv;=1

The following lemma is very useful for calculations ofcumulants. LEMMA 2. 1. Suppose that Y0 is a function of X1, ---, Xn and 0 and is differentiable in 0. Then Ee(Z1VYe)=

and

ay -= Ee( ) E d e(Ye)- 1_ -/vim 1/v de

az aY 'v)- 1-Ee(Z1 ). Ee(Zi Ys)= 1_ - Ee(Z1vYe)- 1- Ee(Y .v vv -Vv

provided that differentiation under the integral signs of Ee(YO ) and Ee(Z1,vYe) is allowed. The proof is omitted since the lemma is similar to Lemmas 5.1.1 and 5.1.2 in Akahira and Takeuchi [21(see also Lemmas 2.1.1 and 2.1.2 in Akahira [1]).

In the following theorem, we have the Bhattacharyya type bound for asymptotic variances. THEOREM 2 . 1. Assume thatthe conditions (A.1) to (A.4) hold. Then for any asymptotically unbiased estimator On of 0, i.e., Ee(On)=0+o(1/v), it holds

321

Second Order Asymptotic Efficiency

ve(V/v ("en-9))? 1 + 1 (J(e

193

)+K(9) + 2v_ (e)12+o( 1 ),

1(0) 2v (0)12(8 ) i 1(e) v(0) v(e)

where Ee(•) and Vo(•) designate the asymptotic mean and the asymptotic variance, respectively , and v'(e) =(a/ae)v(e). REMARK . In the above theorem , for each k= 0,1,2, ••• we ngcan by the term asymptotic mean of Zv is equal to pv up to the order v-k if Z„= Zv+op(v-k) and E(Zv) = pv, and note that the above lemma also holds the asymptotic mean.

PROOF . Since E0 (0n)=0+o(1/v), VvE0( 6nZ1,v)= land n a2 2log/(x.,0) _VvZZv-n1, i=tae

it follows that Ee[V v (fn-0) Z1 v1=o (1),

Ee[Vv(fn-ex,V vZ1v2-n_I+Z2. )1=o(1), Vv

where v , v' and I denote v(0), v'=v'(O)=(d/dO)v(O) and I(0), respectively. Similarly, J and K below stand for J( e) and K(0). Then we define the extended information matrix B as is given by Eu 1v(1/VZ1vtZ2v-Vvn1

B-^ E6(Z1v) Ee[Zlv(VvZly+ Z2v- n _I)1 1^ v

EelVvZly+Z2v- n_n21 ^/v

Since by Lemma 2.1 and Wald 's identity [31

Ee(Z1)=1(0) V(-V EB[Z" v(v' v Z2 v+Z2v-

n- n 1= J(0)+ K(0)+2I - +o (1) , V

v

v

Ee[(-./VZ2 +Z2v - 1/n_ I)2] v

=EB[vZiv-21nZ1v+

n2 21n 12+2VvZ2 Z2v- vZ2v+Z1)

=3y12-2v12+V12+o(1)

=2v12+o(1), it follows that B- 1\ (B11 * * ** ),

where

(J+K+21- )21 ff f v B11 = {1_ tl 2v12

1

+0(1)

322 194

K. Takeuchi and M. Akahira 1+1

(J+K+

I 2v1^2 /

2v')2+o(1). I V

Hence we have Ve ( V V(Bn -0)) z B11 . This completes the proof. In order to show that the modified ML estimation procedure attains the Bhattacharyya type bound given in the above theorem, we need the following lemma.

LEMMA 2.2. Assume that the condition (A.1) holds. If 1 1 1 ^v(0n-0)=1(0)Z'•vVQ+ P( Jv

then V0(V v (6n- 0)) =

+ V0Q+o 1 (0) v (-)

where Q =Op(1).

Putting T= Y V (en-0), we have

PROOF.

2 1 g 2 ^ 2) =E0((T_Z_L._1 )2 +2 V0(Vv (0n-0))=E0(T Z' T- Zl,r)

C0(Z1.) V EO(Q)+ 1( ,v7) = 1 2 0)E0(Zl 1 (0)

Since by Lemma 2.1 E0(Z1 vT

1 a

1 dT

)= v' 0 E0(n- V E°(a ) 1-E0[ !-(e„-e) +Vv) Vv 2Vv

=1+0(1),

the result follows. Denote by 8 L the maximum likelihood estimator of 0 baled on the sample (X1, , Xn). Let O * ML be the ML estimator modified so that Ee(0*ML)=0+o(v-1), and generally it automatically ensures that E0(0*ML )= 0+o(v-3/2). THEOREM 2 .2. If the stopping rule is so determined that sampling is stopped at n and satisfies n• ^

- L. e N0iyc,X. )= v(en,L)[(e;d+c(OfL)+e with

(2.1)

i=1

_ J(0)v'(0 ) + L(0) v"(0) 1 C(0) 1(0)v (0) 1(0) 2v (0)

21(0 ){

2L(0)+M(0)+N(0)}

and some random variable e with E0(e)=o(1), then the asymptotic variance of O*ML is given by (0)+K(0)2v_ ( 0) 2+o( 1 ), V (v v(8 - 0))= 1+ 1 J V . ML I(0) 2v(0)12(0) 1 1(0) v(0) I v(0)

which is equal to the Bhattacharyya type bound, that is, the modified maximum

323 Second Order Asymptotic Efficiency

195

likelihood estimation procedure has the second order asymptotic efficiency property in terms of asymptotic variance.

PROOF.

Since by (2.1)

- ei2)(eML,Xi)=v(eML)1( Mt)+d^ML), i=1

it follows by the Taylor expansion that - e 2)(e,X.)- l I e9)(9,Xi)J(0ML-e)- 1 l ¢4)(e,Xi)J(0ML-e)2 !=1 ,=1 2 1i=1

(2.2)

=vl+(v'I+vr)(eML- 0)+ 2 (v"I+2v'I+vr'XoML- e)2+c(O)+oo(1),

where I' and v" are defined by I'=I'(e)=(d/de)I(O) and v"=v"(e)=(a2/ae2)v(e). Since I'=2J+K, we obtain from (2.2) nI-V v Z2v+{ n-(3J+K)+Z3 V V v (O ML-e) %/ v

+ 2v

(4L+3M+6N+H)v(9ML-e)2

=v1+J1-{v'I+v(2J+K)}Vv(SML-e) v + i (v"1+2vr+vrlv(0ML-e)2+c(e)+ p(1), 2v

which implies J 3J+K n-v v' ^/v Zl °+ - Zt° (n-vu='Vv (Z2.,- ZI ,^-

1 I Z1 Z3 v

4L+3M+6N+H

•nZl v

212v

+ 2I2v {v"I+2v (2J+K)+v(2L+2M+5N+H)}Z2 V+c(e)+oo(1), (2.3) since N/v(6ML-e)=(Zi ^/1)+Opp(1/^), J'=(d/de)J(9)=L(e)+M(9)+N(e)=L+M+N and K'=(d/de)K(O)=H(e)+3N(9)=H+3N. In order to determine n so as to satisfy Eo(n)=v+o( 1), we have from (2.3) _ Jv' L v" 1 do) since

Iv + I 2v + 21(2L+M+M,

=v%'Vv andEe(nZi=vl+o(v). L = L(e)=Ee(Z1 Z3 ) +o(1),Ee(nZ1

We also obtain from (2.3) Z

nivvZ1

.v 121/ v(Z2v

1Z1'r1+12vV' 1v+oP(V).

On the other hand we have

V i==1

=

-1

^vi =1

!f i=1

e(2)(e, X i)J v (;ML -e)

(2.4)

324 196

K. Takeuchi and M. Akahlra

e,3)(

+V

=l

3rl

-. o,x,)}v(ewML-o )z +o = VV

1 nI =Zlv+(^VZ2v

=zl "+

(

,. )J v(oMt -o)-2w(3J+K)v (OMt-o )z+ap(N/ V

Z1 , , _,_

I(n-v)1

-VV

-

1

1

Jv(eMt- o)-- (3J +K)v (oML -o)z +o (-)

v

2Vv

a Jv

which implies - .. 1 n - v 1 Jv (oMt- o)= I Z1 ,, ;F-= V V Z 1/2.v Iv Zl v+ - 3J+K Zi "+ 213Jv Jv

P(

1-).

(2.5)

From (2 .2), (2.4) and (2.5) we obtain 1 J+K Z J v (o Mt-o)= Zlv- 213J v Zl'"

y' 2 1 IzvV v Zl."+o"(J v)

hence , by Lemma 2.2, 1 J+K 2v 1 ' ) 2+o( 1 . V0(Jv (9Mt-8))= I+ z( v v) 2v7 I + w

w

Since the asymptotic variances of 9ML and O * ML are equal up to the order of v-1, the conclusion of the theorem follows.

REFERENCES [1] Akahira , M. (1986). The Structure of Asymptotic Deficiency of Estimators. Queen's Papers in Pure and Applied Mathematics o.75, Queen's University Press, Kingston, Ontario, Canada. [2] Akahira, M. and Takeuchi, K. (1981). Asymptotic Efficiency of Statistical Estimators : Concepts and Hi her Order Asymptotic Efficiency, Lecture Notes in Statistics 7, Springer- er ag, N ew Y ork.

[3] Wald, A. (1973). Sequential Analysis. Dover Publications , Inc., New York.

University of Tokyo University of Tsukuba

325

SECOND AND THIRD ORDER ASYMPTOTIC COMPLETENESS OF THE CLASS OF ESTIMATORS* Masafumi Akahira, Fumiko Hirakawa, and Kei Takeuchi

1. Introduction

The concept of higher order asymptotic efficiency of estimator 9n of a parameter 0 in an open subset a of RP is usually defined by the property that for any other estimator On satisfying some condition we have nlimoyt (k -1)/2 LP.,n {vim (9n -

0)

E C} - Pe,n

{v^

(9n - 0) E C}] > 0

for all 0 E 8 and any convex set C containing the origin. And usually the condition imposed is k-th order asymptotically median unbiased and the like, and 6n is usually "modified" maximum likelihood estimator (MLE) (see [4], [6], [7] and [8]). However, the condition of the asymptotic median unbiasedness is rather arbitrary, and a more meaningful property will be that of higher order "asymptotic completeness" which is defined as follows: O is called k-th order asymptotically complete if for any estimator On within some class of estimators we can construct Bxi'=Bn+h, (0 * ) ,

{hn} being a sequence of functions depending on On so that we have nlim n(k

1)/2 IP9,n {v' (Bn*

- 0) E C} - PB,n {/ (On - 0) E C}] > 0

for all 0 E e and any convex set C containing the origin. Actually, the definition was given in the monograph [4] by Akahira and Takeuchi. But there the discussion was limited to the class of asymptotically median unbiased estimators in its application . The concept is also discussed in [10], but again the class of estimators was limited to that of regular functions of sufficient statistics. The purpose of this paper is to show that , for any class of estimators which admit Edgeworth expansions but are not necessarily asymptotically median unbiased , we get the second order asymptotic completeness of the MLE, and the third order asymptotic completeness of the MLE BML together with the second order derivative of the log-likelihood function evaluated at the MLE which is denoted by Z2 OML) later. ( This paper is retyped with the correction of typographical errors.

326 12

It follows from this higher order asymptotic completeness property that , in any "decision theoretic" set-up , where we consider a sequence of loss functions of the type Ln (u) - L*(/ u) for a sufficiently large n , for any Bayes decision rule 6n with smooth prior we can construct a 6;a = OML + hn ( OML , Z2 (9ML ) ), { hn} being a sequence of functions depending on O, , such that nhlt n (E [Ln(6n - 9)] - E [Ln(bn - 0)]) >- 0 -oa for all 0 E A. The concept of asymptotic completeness is related to that of asymptotic sufficiency. A sequence {Tn} of statistics is called to be higher order asymptotically sufficient if for any sequence {Tn} we can construct Tn' = gn(Tn, Un), {g.} being a sequence of functions depending on Tn, where Un is a random variable independent of 9, such that the asymptotic distribution of Tn' coincides with that of Tn up to the order n -ik'1i/2. (The concept of higher order asymptotic sufficiency may be given in other ways but basically it amounts to the above.) Asymptotic sufficiency, however , has generally been derived from the statement that we have Tn

-

Tn

= op (n

,

_k/2)

which is actually much stronger than the equivalence of asymptotic distributions up to the order n-(k-1)/2 where Xn = op(Yn) means that Xn/Yn converges to zero in probability as n -f oo. For example in Takeuchi [9], Bickel, Gotze and van Zwet [5], it was discussed that for any Bayes decision procedure 6n it is possible to get 6n = gn (0,,,) such that 6n - 6n = o, (n-3/2) only if the loss function is symmetric. However, when the loss is not symmetric, it was shown in [9] that even with Z2 we can not construct 6n = gn (BML, Z2 (9ML)) such that 6n -6n = op (n-3/2). But, in order that 6n and 6n have asymptotically equivalent distributions up to the same order, hence have asymptotically the same risk, it is not necessary that 6n and 6n are stochastically equivalent up to the same order. As a simple illustration of this fact, let us assume that 1 n

6n= On + -En,

where en has the property that E(EnIBn) = o(1). Then 6n-On= nOp (E,) =O,(n

),

327 13

but E [exp(itbn)] =E exp it

[

=E [exp

on +

(iton)

n15n)

}]

] + n E [en exp (ito ), + 0

(

-_ )

=E [exp (it6n)] + o Cn/ , which means that the distributions of 9n and 5n are asymptotically equivalent up to the order n'1. Actually, from what is derived in the paper, it is shown that, under a sequence of decision problems with a proper set of regularity conditions, for any Bayes decision rule bn with some smooth prior there exists a bn (OML, Z2 (6ML)) such that P9,n{8n EA} =Pe,n{bn EA}+o(1) n for all measurable set A in the decision space. The main purpose of this paper is to establish that 9ML is generally second order asymptotically complete, and BML together with Z2 (OML) is third order asymptotically complete. Although this result is given in the framework of "point" estimation theory, it is also applicable to other problems of inference including testing hypothesis and interval estimation, which will be discussed in subsequent papers.

2. Preliminaries

Let X be an abstract space , elements of which are denoted by x, and let B be a v-field of subsets of X. Let e be a parameter space , which is assumed to be an open subset of the Euclidean p-space RP with the usual norm denoted by II - 11. We shall denote by (X('), 13(n)) the n-fold direct product of (X, B). We consider a sequence of classes of probability measures {Po,n : 0 E 6} (n = 1, 2, ...), each defined on (X(n), B(n)), such that for each n and each 0 E ® the following holds:

PB,n (B(n)) = P0,n+1 (B(n) X X)

for all B(n) E B(n). An estimator of 0 is defined to be a sequence {9n} where on is a Bin)-measurable function from X(n) into 9 (n = 1, 2,. ..). For simplicity we denote an estimator as on instead of {on}. For an increasing sequence of positive numbers {cn} (cn tending to infinity) an estimator 9n is called to be cn-consistent if for every 77 of 6, there exists a sufficiently small positive number 6 such that

328 14

> L} = 0 Jim 71m sup Pe ,n {c,,, L-.oo n-•oo 0: 110-+111 <6 Ilea - III

(Akahira [1]).

3. One parameter case

Suppose that B is an open subset of R1. Let X1, X2, ... , Xn, ... be a sequence of independently and identically distributed (i.i.d.) real random variables with a density function f (x, 0) with respect to a a-finite measure µ, where 0 E B. In the subsequent discussion it is enough to consider only the case cn = V , ' n - . F o r each k = 1, 2, ..., 0 < a(0) < 1 and a continuously differentiable function a(I) of 0, a cn-consistent estimator On is called k-th order asymptotically (a, a)-biased (as. (a, a)-biased) estimator if, for any 77 E O, there exists a positive number b such that lim sup nik-11/2

n-.oo 0:10 -,11 <6

IPe,n

{

nI (I) (In - I) < a (I)} - a(0)I = 0,

Jim sup n (k-1)/2 I Pe,n { nI(I) ( In - I) > a (0)} - 1 + a(I)I = 0, -.oo 0:10-1<6

where 1(0) denotes the Fisher information of f where a(I) and a(0) are determined from the outset. Now we assume that a(I) is a constant a. Alternatively, we may fix a(0) to be constant letting a(0) depend on I, and the subsequent discussions apply exactly in a similar way. For each k = 1, 2.... we denote by Bk(a, a) the class of k-th order as. (a, a)-biased estimators for which the distribution of

nI(0) (On - I) - a (0) admits the Edgeworth expansion up to the

k-th order. If an estimator On belongs to the class Bk (a, a), then we call it a Bk (a, a)-estimator. F o r each k = 1, 2, ... , a Bk (a, a)-estimator On* is called k-th order asymptotically efficient in the class Bk(a, a) if for any Bk(a, a)-estimator fin Pe,n -tl < nl(0) B;, - 0 - a(0) < t2 > P9,n -tl < nI(0) On - B - a(0) < t2

+o n- (k-1)/2

for alltlit2>0and0EB. We assume the following conditions. (A.1) {x : f (x, 0) > 0} does not depend on I. (A.2)k For almost all x[µ], f (x, I) is k times continuously differentiable in I. expansion

log

k f (x, 0 + h) = . h ' 0) (I, x) + hkR(x, h)

f (x, 0) 8_ 1 Z.

R(x, h) is uniformly hounded by a function G(x) which has

In the Taylor

329

15

moments up to the k-th order, where

0l(9, x) = (ai/aOi)e(9, x) (i =1, ... , k) with e(9, x) = log f(x, 9). (A.3) For each 9 E O

0 < I(9) = E9 [{t1(9,X)} 2 ] =

l -E6 [e(2)(0'X)J

< oo,

and 1 (0) is differentiable in 9.

(A.4) There exist = E9

J(9)

[e(l) (0, X)e(2) (0, X)]

{

K(9) =E9 {')(ox)}3] , and both J(9) and K(9) are differentiable in 9, and EB [e(3)(0,X), = -3J(O) - K(9). In the following theorem we obtain the asymptotic distribution of a B2 (a, a)-estimator up to the second, i.e., the order n-1/2. Theorem 3.1. Assume that the conditions (A.1), (A.2)3, (A.3), and (A.4) hold and that the asymptotic cumulants of a biased best asymptotically normal (BAN) estimator On are given as follows: For Tn = Vrn (On - 9) Co(9) + C'(

E9(Tn )

) +o \ n= 1 VB(7n ) = I(0 ) + C2 (0) +0 \ ^/ \Tn ) C,)+a\\

where Ci(9) (i = 0, 1, 2, 3) are differentiable functions of 9. Then the second order asymptotic distribution of the second order as. (a, a)-biased BAN estimator 9n, of the form 9n = On ( 9n) - n-1u2 (9n), is given by

(3.1)

P1,n {

nI ( 9) (On'

-n1/2u

- e) < t + a(9)}

1 =4,(t + va ) +

-q(t

+ V10 {_

I3/2 (9) C3(9)t2

I(9)C2(9) a'(0) 1I3/2 (9)C3(9)vat + t I\Co( B ) - 2 -3 I(9) (

+ 2J(O) K 2I/2 9 )

(O)

(a( 9) -

va))

} +0

(

✓n

`

,

where 4)(u) = f.O(x)dx with O(x) = (1/ 27r)e-x2/2 and 4(va) = a.

330 16

Remark: In the above theorem, ul(0) and u2 (0) are given as follows: u1(0) -Co ( u2(0)

0) - a(0) - v«. I(0) ,

=C 1 (0)

- Co(0)ui (0) +

2 I (e) { C2(0) - 2i4 () } [a(0)

TT) 1CO (0) - u, (0) }] + 1I (0 ) C3(0)[ {a(0)

O

I(0)(Co(0) - u,(0))}2 -1]. In the following theorem we shall obtain the bound for the asymptotic distributions of B2(a, a)estimators up to the order n-1/2. Theorem 3 .2. Assume that the conditions (A.1), (A.2)3, (A.3), and (A.4) hold. Then the bound F* (t, 0) for the asymptotic distributions of

n1(0) (in - 0) for the all B2 (a, a)-estimators

On

is

given by F* (t, 0) =41(t + va) + 1 0(t + va) f

{3J(61 /2 (O)}t 2

t a(0)(2JL(0) + K(0)) vaK(0) 1 I(0) {a'(0) 21(0) 61(0) }J

in the sense that for any B2 (a, a)-estimator On Pe,n { nI ( 0) (On - 0) < t + a(0) } < F* (t, 0) for all t>0and all 0E9, PB,n { nI ( 0) (On - 0) < t + a(0) } > F* (t, 0) for all t<0and all 0E6. From Theorems 3.1 and 3.2 we have the following: Theorem 3.3. Under the conditions (A.1), (A.2)3, (A.3), and (A.4) it holds that for the biased BAN estimator 0n (0) I (0)C2(0) C3(0) _ - 3J( 0) 2K (0) Co J3(0) 2

for all 0E9,

and further if Co(0) = I(0)C2(0)/2, the second order as. (a, a)-biased BAN estimator

On

is second

order asymptotically efficient in the class B2 (a, a). Sketches of the proofs of Theorems 3.1, 3.2 and 3.3 are given below, since their detailed ones are done in Akahira [2]. Sketch of the proof of Theorem 3.1. T 0* - 9 U1 (0n) u2 (0n) et n- n-

331 17

where ul (0) is continuously differentiable in 9 and U2(0) is a function of 9. Then we have the following asymptotic cumulants of Tn =

nI (o) (Bn - 0):

(3.2) EB(Tnn) (Co - u1) + I/n(C1 - Cou'1 - u2) + o \ n= , (3.3)

VB(Tn)=1+ C2-211

+ol ✓n I,

(3.4) #c3,9 (Tn) = 7n= 13/2''3 +0

(=), where I, Co, U1, ui, u2, C2 and C3 denote 1(9), Co (0), u1(0), ui (0) = du1(9)/d9, U2 (0), C2 (0) and C3(9). From (3.2) to (3.4) it follows that the Edgeworth expansion of the distribution of

/I(9) (o;, -

is given by (3.5)

Pe,n { nI (o, - e)) < t + f(C0 - u1)} ='P(t) - 1 !b(t) {/i(Ci - COU4 - U2) + 2 (C2 - jUl t + 6I3/2C3(

t2 - 1) 1

(71). By the second order as. (a, a)-biased condition we have the following u1 and U2: a -vim

(3.6)

u1 =Co -

.

(3.7) U2 =C1 ( c'2 - Cou1 + - {a - vfI(Co - u1)} / 2 2 1 + 163[{a- VL

(Co-

UI) }2-

11,

where ' (v0) = a. Since I'(0) = 2J(9) + K(0), it follows from (3.5), (3.6) and (3.7) that (3.1) holds. This completes the proof. Sketch of the proof of Theorem 3.2. Let 0o be arbitrary but fixed in O. Consider the problem of testing the hypothesis H : 9 = 01 against alternative A : 0 = 00, where o1 = oo + O(1/vrn-). In order to obtain the upper bound of Pe,,,n { nI (9o) (Bn - 0o ) < t + a(9) }

for each t > 0 and all B2 ( a, a)-estimators Bn, that is, under the second order as . ( a, a)-biased condition Pe,,n{

n7 ( 0 I )(B n

-

o I )
a }=+o( 1 ), n

332 18 we first take a(2J + K) l^ 01 eo + r1 - 1IJ a , - 21 J nI -vfnwhere a, a', I, J and K denote a(Oo), a'(0o), I(Oo ), J(0o) and K(0o). Setting f (xi, Bo) Tn = n log - J (Xi, 81)

by the Edgeworth expansion of the distribution of Tn we take the following an so that the test with a rejection region {Tn > an} has the level a + o(1/ f): t2 t (, a ( 2J + k)1 ,

-'i 6I3/2

1 { (3J + 2K ) t2 -I-3(J + K) -a t + K(112 - 1 ) } + o (=)

.

In a similar way to the above we have the asymptotic power of the test with the rejection region {Tn ^ an } as follows:

(3.8)

Peon{Tn > an} =4^(t + va ) + 7)(t + va)

(3J + 2K )t2 - t2 { , a(2J + K) 1 a 21 613/2

+0 (1) = F*(t, Bo ) (say). By the fundamental lemma of Neyman and Pearson it is seen that the asymptotic power series of the most powerful test of the level a + o(1//) is given by (3.8). Since 0o is arbitrary, we have the desired result for all t > 0. In a similar way to the case t > 0, we also obtain the conclusion for all t < 0. Sketch of the proof of Theorem 3.3. By Theorems 3.1 and 3 . 2 we have F* (t , 0) - Pe,n {

(3.9) _

1

qi(t + va) r

nI (B) (On* - 0) < t + a(0) }

3J(B) + 2K(0) () I3C3 (0) (

t2 + 2vat ) - (Co - I2 2) t }

+OI

If one assumes that 3J ( Oo)+2K (Bo)+I3 ( 0o)C3(0o) 54 0 for some Bo E O , it leads to a contradiction to the bound F* (t, 0). Hence it follows that

333 19

C3(9) _ -3J(0) (^)K(0)

(3.10)

for all 0 E ®.

Since F* (t, 0) is the bound for the asymptotic distributions of

nI (B) (Bn - 9) for all B2 (a, a)-

estimators O , it follows from (3.9) and (3.10) that

CC(0) < I (0)C2(0) /2

for all 0 E

e.

If CC (0) - I(O)C2(0)/2, then it is seen that the asymptotic distribution of

nI (B) (Bn - 0) attains

the bound F* (t, 0) for all t and all 0 E a Hence the desired result follows. Let BML be the maximum likelihood estimator (MLE) of 0. Then the second order asymptotic efficiency of the MLE is given as follows: Theorem 3.4. Assume that the conditions (A.1), (A.2)3, (A.3), and (A.4) hold. Then the estimator BML modified from the MLE to be in B2(a, a) is second order asymptotically efficient in the class B2(a, a). The proof follows from Theorems 3.1 and 3.3 since Co(0) = C2(9) - 0 and C3(0) - -{3J(0) + 2K(0)}/I3(9) in the asymptotic cumulants of the MLE. (See Akahira and Takeuchi [4], page 90). Theorem 3.5. Assume that the conditions (A.1), (A.2)3, (A.3), and (A.4) hold. Suppose that

B„ is any BAN estimator of which the Edgeworth expansion up to the order n-1/2 is valid and the coefficients CC(0) (i = 0, 1, 2, 3) in its asymptotic cumulants as in Theorem 3.1 are continuously differentiable in 0. Then the MLE is second order asymptotically complete in the sense that there exists a modified MLE BML, of the form BML = OML + 1 91 (BML) + 192 (BML) ,

such that Pg,n {-t1 < >PO,n

nI(B) (BML - B) - a(0) < t2}

{ -t1 < nI (B) (in - B) - a(9) < t2 } + o

for allt1it2 > 0andall0E9. Sketch of the proof. For fixed a, define an(9) by

(

1 n

/

PB,n { nI ( B) (Bn - B) < an (B) } = a + o 1 1 ) , Po,n{ nI (B)(Bn-0)>an(0)}=1 -a+o(z I, n/ locally uniformly in B, then an (B) can be expanded as an(0) =

334 20

a(0) + n-1/2b(B) + o (n-1/2), where a(O) and b(0) are continuously differentiable. Consider the class B2(a, an) with an(0) thus defined. Analogously to the proof of Theorem 3.4 we can show that within the class there exists a modified MLE BML, of the form BML = BML + 1 91 ( OML) + 192 (OML)

which is second order asymptotically efficient in the class B2(a, a). This implies the desired result. For the third order asymptotic completeness we further assume the following: (A.5) There exist Ee [Pl1i(0,X)tl3i(9,X)], EB [{e(2)(0,X)

}2],

E0 [{€(')(0,X)}2e(2)(9,X)] and Ee [{0)(0,X)}4]. Theorem 3.6. Assume that the conditions (A.1), (A.2)4, (A.3), (A.4), and (A.5) hold. Suppose that On is second order asymptotically efficient in B2 (a, a), of which the Edgeworth expansion up to the order n-1 is valid. Then the pair of statistics (BML, Z2 (BML)) is third order asymptotically complete in the sense that there exists a modified MLE °ML, of the form BML = BML + 791 (BML) + n92 (BML, Z2 (OML )) + n703 (BML , Z2 (OML)),

such that /n-I(O) (BML - B) - a(O) < t2}

Pe,n {-t1 <

>Po,n {-t1 < nI(0) (On - B) - a(0) < t2} + o I 1 I n for all t1, t2 > 0 and 0 E e, where Z2(0) = E

f(2)(0,X,)/f + fI(0).

Sketch of the proof. The asymptotic cumulants ni (i = 1, 2, 3, 4) of Tn = f (0n - 0) have the following form: i1

=Ee (TT) = µ1o( 9) +

e(TT) =

K2 =1"

k3 =ic3 ,e (T)

µ1 1(B)

+ µ l2(0) + o

(1)

n ( n 1 + µ2 ) + µ22n(9) + 0 :)' I(0) -7n f

= Ee [{T- Eo(T)}3] = A319 ) +µ32(g) + 0(1),

335 21

r-4 =K4 ,8(Tn)

= Eg [ {Tn - E (Tn)}4] - 3 {Ue(T)}z

_ A4ng)

O +0 (

i

) .

Then it is seen that the coefficients µ1o(0), µ11(0)), µ210), µ31(0) and µ42(0) in the above are determined from the second order asymptotic efficiency of in, but µ22(0) and µ32(6) are not so (e.g. see Akahira and Takeuchi, [4]). And it follows that, in the Edgeworth expansion of the distribution PO ,n { I/rnI(O) (in - B ) - a(O) < t}, the only undecided term has the form 31U32(0) + {µ22(0)/ I(B)}w(t), where w (t) is some linear function of t. It follows by the discretized likelihood method in Akahira and Takeuchi [3], [4] that for any Bn E B2(a, a) with µ32(B) and µ22 (9) in its asymptotic cumulants, any fixed 0o E 6 and each real t, there exists a second order as. (a, a)-biased estimator Bt, with µ32(•) and µ22(•) in its asymptotic cumulants, such that (3.11) 31L32(00) +µ22(0o) mo(t) < 3µ32(0o) + µ22( 0o) w(t) I(0o) I(0o)

For any given Bn, let µZ2(0) be the corresponding µ22(0), then it is shown that there exists to = to (0o) such that /22(00) = µ22(0o). Since, by (3.8), for any /032(') 3µ32(9O) +

µ22( BO)w(to ) < 3P32(Bo) +

OWN

µ22(B O)00),

I(0o)

it follows that (3.12)

µ'32(B0) :5 P3200)•

From (3.11) and (3.12) we have 3µt3`z (0o) + µoz2 ( 0o) w(t

µ2z(eo) w(t) ) < 3µ32(0o) + I(B-) I(0o)

for all real t. Hence, for any second order asymptotically efficient estimator Bn in B2 (a, a) (3.13)

Peon {-t1 < rn- (0t° - Bo) - a(Oo) < t2} >P00,n{-t1
for allt1,t2>0.

336 22

Now define B

ML = BML + 1 h

1 (BML)

+ n1 n h2 (BML ) +

n =h3 (BML)

such that Ell (OML ) is asymptotically equal to Eo (9n) up to the order n-1, then BML and Bn have the same values for µ10(0), µ110), µ21(0), µ31(0 ) and µ42 ( B), but the coefficient 632(B) of the _ order n 1 in the third order cumulant for OOML is identically equal to zero. By the discretized likelihood method in Akahira and Takeuchi [ 3], [4] it follows that ( l ( 1 60 to (00)5 +op I n^ j Z2(00) - J Bt° I (Bo) 21 + 2n1(Bo) Now define Bt° = BML +

t0 Z 2 (BML) , 2nI (BML)

where io = to (BML)

Then it can be shown that ( l B1o = Bto + n^W + op \nf /

(3.14)

where W is some stochastic quantity of magnitude of order 1 and EBo(W) = o(1). Since Bt° can be stochastically expanded as 1 f (Bt0 - Bo ) = 1(00) Z1(Bo)

1

+ 7Q +

n

+o

p (1) n (1)

it follows from (3.14) that / i (01° - Bo) = I (80) 21(0.) + ^Q + r1 (R + W) + op

(n

I

where both Q and R are certain stochastic quantities of magnitude of order 1. Then it is shown in Akahira and Takeuchi [4] that the asymptotic distributions of the above two estimators Bt° and Bt° coincide up to the order n-1 if E90 (W) = 0. Hence it follows from (3.13) that for any second order asymptotically efficient estimator Bn in B2(a, a) 1 Peon {-t1 < f (Bt° - BO) - a(Bo) < t2J >Peo,n {-t1 < (On - 90) - a(Bo) < t2J + o (n) for allt1,t2>0.

337 23

Since Bt° is independent of 0o and 0o is arbitrary, we have the conclusion of the theorem.

4. Multiparameter case

Assume that 9 is an open subset of RP. Let X1i X2, ... , Xn, ... be a sequence of i.i.d. real random variables with a density function f (x, 0) with respect to a a-finite measure ji, where 0 E 9. Denote by Bn = (Bnl) ... , 9np), an estimator of 0 = (01, . . . , Op)' E 9. We assume that the distribution of f (Bn - 0) admits the Edgeworth expansion up to the order n-1, and Bn has the stochastic expansion 1 ^(0na-Ba)=Ua+ 2rn-

with U,,

=

E

Iap(O)

p-1

1 (1 nWQ+Op ) n

1(a/i9O,3) logf(Xi, O) /VIn,

Q.

=

(a=l,...,p),

Op( 1)

and W, = OP(1) (a =

1,.. . , p), whose asymptotic cumulants are the following: for Y.

=

,r (On

a

- 0Q) (a= 1,...,p)

Eo(YY)=µo,(O)+µ1 )+µ2n(0)+o(-); VrnCov(YQ, Y3) =I (0) + CQ!) + Ca(0) -1 n + 0 (); ^(1) (0) x(2) (0) 1 K3,0(Y.,Yfl,Yy)= K4,0 (Y., Yfl, Yy, Y6) =rash

+ Qpn (0) +0 (

+O^n)

n)

for a,,O, ry, b = 1,... , p, where I' (O) is an ( a„ Q)-element of the inverse matrix of the Fisher information matrix. For each k = 1, 2, ..., an estimator O;, is called k- th order asymptotically efficient in the class Ak of the estimators On with the same bias k-1

Ee (0na ) = 0a + E µia ( O)n-(i+l)/2 + 0 (n-k/2 ) (a = 1, ... , p) i-o up to the order n-k/2 if for any estimator Bn E Ak PB,n {vi ( 0n - O) - µo (0) E C} > Po,n {I/n ( en - O) - µo (0) E C} + O ( n-(k-1)/2) for any convex set C of RP containing the origin and all 0 E 9.

We shall use the following notations : for a,,3, ry, 6 = 1 , ... ) P

338 24

EB

Ja1w

[1 oaeap _ 1o

K«Rry =E9 [{ P

J aR ry = a'=1 P

P

P

log f (X, B) } {

_ 1og f(x,o)}J;

0 1og f(x,o)} f(xo)} {_ 1og f(x e)} {_

]

Iaa'Ipp'Iryry' 7 R ry'>

p' =1 ry' = 1 P

P

Kct fl-t = E E I aa IRR 'I77 Ka'p'ry'; a'=1 0'=1 -r'=1

M.0.-Y6 =E9 M',6'''5

=

a2 log log B) f (X, } { ae7aea f (X, B) }, - 1.0 45; {{ ao o R a

> > E >a'I aa' a' p'

Ipp ' Iryry' Idb

' Ma'p' •- 'o' ;

7'

J7R=^^Ia«IRR' Ja,R ry

Ja.7=ETIaa' IryryJa'p7'; a'

ry'

K,y R = > I«^ IRR' Ka' p,7. a' R'

Theorem 4.1. Assume that Bn is any asymptotically efficient estimator in the class of the estimators with the same bias µoa(B) (a = 1,. .. ,p) up to the order n-112 in the above, which admits the Edgeworth expansion of its distribution up to the order n-1/2. Then the MLE OML is second order asymptotically complete in the sense that there exists a modified MLE BML, with the salve asymptotic bias as On up to the order n-1, of the form BML = OML + =y1 (BML) + -^R92 (BML) ,

such PO,- j Vrn ( 41L

-

that

/

8) - µo(8) E C} > PO , n { ✓rt (On - B) - A' (0) E C} + o

for any convex set C of RP containing the origin and all 0 E 6, where 1Lo(8)

\

(

^ 1 /

_ (µo1(8), ... , µoP(O))'. The proof is omitted since it is done in a similar way to that of Theorem 3.4. Theorem 4.2. Assume that On is second order asymptotically efficient in the class of the estimators with the same bias up to the order n-1 in the above, which admits the Edgeworth expansion of its distribution up to the order n-1. Then the pair of statistics (OML, Z2 (BML)) is third order asymptotically complete in the sense that there exists a modified MLE BML, with the same asymptotic bias as On up to the order n-3/2, of the form

339 25

OML = OML + _q1 (OML) + np (BAIL , Z2 (BAIL)) + n7q3 (BM L, Z2 (OML))

such that P9,n {s/ i(OML

- 9)- µo(8)

( ^1 {v4 (On-B)-a. (B)EC +O EC}1POn I

for any convex set C of RP containing the origin and all B E

Z2(8) =

7ne0 ,

6, where

}

n

logf(xs, O) + ^j(B)

with the Fisher information matrix I(9). Sketch of the proof. First it is noted that the coefficients µ0a(8), µ1a(9), C,(,p(9), r.(1),(0) and 00 xao.ya (O) (a, /Q, ry, 8 = 1, ... , p) in the above asymptotic cumulants of On are determined from the second order asymptotic efficiency of On- In order to obtain BAIL with more concentration probability than On up to the order n-1 by the Edgeworth expansion of the distribution of On, it is enough to find an estimator On = B;, which minimizes CQR (0) = Cove (Q,, Q,), given ^cap7(0). Putting V a p = Ea=1 Ia0 (8/88.y)U0 with Iao = 1`0 (0) (a,# = 1, ... , p), we have for a, /3, y = 1,...,P

(4.1) Kap•y(8) = E9 (Q«V, ) + E0 (QoV7«) + Ee (Q. V«o) We also have for a, /3, ry, 8 = 1.... )P1 (4.2) E9 (UOU-1Q,) =2 (µlai17 - Ka07 - JO"*') + o(1), P

(4.3)

ll E9 (UOVy6Qa) =2 Ala (K-Y60 + J7O.6) + (Kb o" + Jb a ) 6'=1 P

(Kryb"6 + Jry6 '6 - JaaJ76"6 + Moa76 6'=1

+ o(1), where for each a = 1,...,p, µ1a denotes µia(O). From the second order asymptotic efficiency of On we obtain (4.4) Eo (UaQa) = o(1) (a = 1, ... ,P)• Then it follows by the Lagrange method that, under the conditions (4.1) to (4.4) and E9 (Qa) _ 2µ1a + 0(1), Q. = Qa minimizing Cove (Qa, Qo) have the following: for a = 1, ... , p

340 26

(4.5) Q« =2µ1. + 710-f (0) Vp7 - p7(0)U. p=17=1 p=1Y=1 +^^Aa7(0)(upU7-Ip7)+EEEAary6(0) /3=17=1

0=1'y=16=1

(UpV76 + Kap7 + Jn7•0) , where rl«7(0), µ0"7(0), A17(0) and Apry6(9) (a,0,-y,6 = 1,. .. , p) are Lagrangian multipliers determined so that the conditions are satisfied . Hence it is seen the desired estimator 9 has Qa in its stochastic expansion. Substituting 9ML for 0 in (4.5 ) and making it to have the same asymptotic bias as On up to the order n-312, we obtain the desired result.

References [1] M. Akahira: Asymptotic theory for estimation of location in non-regular cases, I: Order of convergence of consistent estimators , Rep. Stat. Appl. Res., JUSE 22 (1975 ), 8-26. [2] M. Akahira: The structure of asymptotic deficiency of estimators , to appear in Queen 's papers in Pure and Applied Mathematics.

[3] M. Akahira and K. Takeuchi: Discretized likelihood methods-Asymptotic properties of discretized likelihood estimators (DLE's), Ann. Inst. Statist . Math., 31 (1979), 39-56. [4] M. Akahira and K. Takeuchi: Asymptotic Efficiency of Statistical Estimators : Concepts and Higher Order Asymptotic Efficiency , Lecture Notes in Statistics 7, Springer-Verlag, New York- Heidelberg- Berlin ( 1981). [5] P. J. Bickel , F. Gotze and W. R. van Zwet: A simple analysis of third-order efficiency of estimates , Proceedings of the Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer , Vol. II (L. M. LeCam and R. A. Olshen eds.), Wadsworth ( 1985 ), 749-768. [6] J. K. Ghosh , B. K. Sinha and H . S. Wieand : Second order efficiency of the mle with respect to any bounded bowl-shaped loss function , Ann. Statist., 8 (1980), 506-521. [7] J. Pfanzagl and W. Wefehneyer : A third order optimum property of the maximum likelihood estimator , J. Multivariate Anal., 8 ( 1978), 1-29. [8] J. Pfanzagl and W. Wefehneyer: Asymptotic Expansions for General Statistical Models, Lecture Notes in Statistics 31, Springer -Verlag, Berlin-Heidelberg-New York-Tokyo (1985).

341 27

[9] K. Takeuchi : Higher order asymptotic efficiency of estimators in decision procedures, Proc. III Purdue Symp. on Decision Theory and Related Topics , Vol. II , ( J. Berger and S. Gupta eds.), Academic Press , New York ( 1982), 351-361. [10] K. Takeuchi and K. Moriinune: Third-order efficiency of the extended maximum likelihood estimators in a simultaneous equation system, Econometrica 53 (1985), 177-200.

M. Akahira F. Hirakawa Department of Mathematics Department of Mathematics University of Electro- Science University of Tokyo Communications Yainazaki , Noda-shi Chofu, Tokyo 182 Chiba 278, Japan Japan K. Takeuchi Faculty of Economics University of Tokyo Hongo, Bunkyo-ku Tokyo 113, Japan

342

Ann. Inst. Statist. Math. Vol. 41, No. 4, 725-752 (1989)

HIGHER ORDER ASYMPTOTICS IN ESTIMATION FOR TWO-SIDED WEIBULL TYPE DISTRIBUTIONS MASAFUMI AKAHIRA' AND KEI TAKEUCHI2 'Institute of Mathematics, University of Tsukuba, Ibaraki 305, Japan 'Research Center for Advanced Science and Technology , University of Tokyo, Komaba, Meguro-ku, Tokyo 156, Japan

(Received May 18, 1988; revised January 25, 1989)

Abstract . We consider the estimation problem of a location parameter 0 on a sample of size n from a two-sided Weibull type density f (x - 0) = C(a)exp(- Ix-0Ia) for -oo<x
Key words and phrases : 2a-th order asymptotically median unbiased estimator , 2a-th order asymptotic distribution , 2a-th order asymptotic efficiency, Edgeworth expansion , maximum likelihood estimator.

1. Introduction Higher order asymptotics have been studied by Pfanzagl and Wefelmeyer (1978, 1985 ), Ghosh et al. ( 1980) and Akahira and Takeuchi (1981), Akahira ( 1986), Akahira et al. ( 1988), among others , under suitable regularity conditions. In non-regular cases when the regularity conditions do not necessarily hold, higher order asymptotics were discussed by Akahira and Takeuchi (1981), Akahira ( 1987, 1988a , 1988b), Pfanzagl and Wefelmeyer (1985), Sugiura and Naing (1987) and others. In this paper we consider the estimation problem of a location parameter 0 on a sample of size n from a two-sided Weibull type density f (x - 0) = C( a) exp (- Ix- O I a) for - oo < x < oo, - oo < 0 < oo and 1 < a < 3/2, where C(a) = a/ {2F(1 / a) }. It is noted that there is a Fisher information amount and a first order derivative of f (x) at x = 0, but there is no second order one of f(x) at x = 0. It is also seen in Akahira (1975)

725

343 726 MASAFUMI AKAHIRA AND KEI TAKEUCHI

that the order of consistency is equal ton-'12 in this situation. Then we shall obtain the bound for the distribution of asymptotically median n(2a 1)12. We unbiased estimators of 0 up to the 2a-th order, i.e., the order shall also get the asymptotic distribution of the maximum likelihood estimator (MLE) of 0 up to the 2a-th order and see that the MLE is not generally 2a-th order asymptotically efficient . Further, we shall obtain the amount of the loss of asymptotic information of the MLE. 2. The 2a-th order asymptotic bound for the distribution of second order AMU estimators Let X1,..., Xn,... be a sequence of independent and identically distributed (i.i.d.) random variables with a two -sided Weibull type density f (x - 0) = C( a) exp { - I x - 01 a} for - oo < x < 00 where 0 is a real-valued parameter, 1 < a < 3/2 and C(a) = a/{2T( 1/a)} with a Gamma function u- 1 -x T(u), i.e., T(u) =fo' x a dx (u > 0).

We denote by Po, the n-fold products of probability measure Pe with the above density f(x - 0). An estimator On of 0 based on Xi,..., Xn is called a 2a-th order asymptotically median unbiased (AMU) estimator if for any E R', there exists a positive number 6 such that n (2a-1)/2

lim sup n-°°

n(2a -1)/2

lim sup n-°°

0: I0-711
1

{

Pe,n en C 0} - 1

e; 10-ql <6

Pe,n{On

2

0}

-

1 2

=0, =0.

We denote by Ala the class of all best asymptotically normal and 2a-th order AMU estimators . For a On 2a-th order AMU, Go(t, 0) + n (2a-1)/2G1 ( t, 0) is defined to be the 2a-th order asymptotic distribution of (8n - 0) (or On for short ) if for any t E R' and each 0 E R'

lim

n(2a

-1)/2 1 Po,n{V ( On - 0) s t} - Go(t , 0) - n (2a-'1/2G1(t, 0) I = 0 .

In order to obtain the bound for the distribution of 2a-th order AMU estimators of 0, for arbitrary but fixed Bo, we consider the problem of testing hypothesis H: 0 = 00 + to '/2 (t > 0) against the alternative K: 0 = Bo. Then the log-likelihood ratio test statistic Zn is given by Zn =

{ f( X; - Oo)lf(X1 - 00 - to '12 ;I1 log n

_ - 1 1(IX

-Oo

la-

IX- Oo- tn 1/21").

344 ASYMPTOTICS FOR WEIBULL TYPE DISTRIBUTION 727

In order to obtain the asymptotic cumulants of Z, we need the following lemma. LEMMA

2.1.

If ha (x) = (x + A )' -

xa

for d > 0, then

fo- h2X°dx=aF( 2- a1 )d2+a(a - 1)T(2 _-2a )d3 1 +Y d 2a + 1

+0 2a + 1

)

2a+ 1 '

f

ha(x)eX0dx=a2T(3

- 2a )d3 +0(d4)

where

_ a(a - 1)T(a - 1)F(3 - 2a) Y 2(2a-1)F(2-a) PROOF .

First we have

(2.1) f o he (x)e Xadx = f o (x + d)2ae xdx - 2 f o (x + A)axae adx + f0 x2ae XQdx.

Since for f3 > 0 (2.2)

fo xfi 1 eX"dx= aT(a)

it follows that (2.3) f o (x + 4)2ae X'dx d2a+1

a

2a+1+2a+1

a+ (x + d)2 txa -1e Xadx

d2a+ 1

a 2a+1 + 2a+1 • f o x3a + (2a + 1)x3a -' d + a(2a +

1)x3a-2A2

+ 3 a(2a - 1)(2a + 1)x3,-3d3 } e X'dx + O(A4)

345 728 MASAFUMI AKAHIRA AND KEI TAKEUCHI

2a+1 T(3

+ a) + 2d +aT (3-

+ 3 a(2a - 1)T (

a

)d2 42a

3 - a) d3 2a + 1

+1 + 0(d4)

From (2.2), we obtain ^

f0

(x + d)"x"e-Xadx a _ - f (x + d)a+1( xa-1 - X-XQdx a+1 0

a a+1

fo

x"+' + (a + 1)x"d +

+ 6 a(a - 1)(a

a

a(a + 1)x"-'d2

+ 1)xa-2d3 }(x2a-1 - xa-1)e XQdx

a 1 fog R(d)xa-'e XQdx + 0(d4) , a+ where R(d) _ (x + d )a+l - x"+1 - (a + 1)dx " - 2 a(a + 1)d2xa-' - 6 a(a - 1)(a + 1)d3xa-2 . Then the remainder term R(A) of the Taylor expansion is represented by R(A) = Ka fo (A - t)3(x + t)"-3dt , where 0 : 5 t : 5 it follows that fo(x+ t)

and K. = a(a + 1)(a - 1)(a - 2)/6. Since 1 - xa < e X° < 1,

a-3xa-l(1

- e XQ)dx
1 - e X0)dx + fl. x2a- 4(1 - e XO)dx

fo' x3a-4dx

+ fl00 x2a-4dx

346 ASYMPTOTICS FOR WEIBULL TYPE DISTRIBUTION 729

a

3(a-1)(3-2a) Since f o (x + t)a-3x° -' dx = t2°i-3B(a, 3 - 2a), we have a

a fo R(d) x -'- x' dx = f0 R(A)xa-'dx - f0 R(d)x -' ( 1 - e- x')dx

= Kafo (d - t)3 j fa (x + t)'-3xa-'

dx I dt + 0(.42a+1)

= KaB(a, 3 - 2a) f0 t2a- 3(d - t)3dt + o(d2a+`) = KaB(a, 3 - 2a)B(2a - 2,4

) A2a+l

+ o(d2a+)

(a + 1)(a - 1)T(a - 1)T(3 - 2a) d2a+I + o(d2a+I) 4(2a + 1)(2a - 1)F(2 - a) Hence , we obtain

fo (x + d)axaex. A a+ {T(3+ a)-T(2+ a )} +d+ 2 (T(3- a)-T(2- a

)}d2

+ 6 a(a- 1) j T(3- a)-T(2- a )}A3 + a(a - 1)T(a - 1)T(3 - 2a) d2a+I + O(d2a+I) 4(2a + 1)(2a - 1)r(2 - a) = a T\ 2+ a)+d+(a2 1)T(2- a )d2

+ 6 (a- 1)(a-2)T(2- a )d3

+ a(a - 1)T(a - 1)T(3 - 2a) d2a+I + (d2a+I) 4(2a + 1)(2a - 1)r(2 - a) and by (2.2), f o x2aex0dx = T(2 + I/ a)/ a. From (2.1) to (2. 4), we have

347 730 MASAFUMI AKAHIRA AND KEI TAKEUCHI

he(x)e x°dx=aT(2- 1 )d2+a(a- 1)F(2_ 2 )d3 a a 1 + Y A2a+1 +0 (d2a+1)

2a+1

,

where y = a(a - 1)T(a - 1)T(3 - 2a)/2(2a - 1)T(2 - a). We also obtain

f 0 he(x)e-"dx = f 0 {(x + 4) a - xa}3e-"dx x 3a-3e

= a3d3 0

x" dx +

O(A4)

=a2T(3- 2 )d3+O(d4). a Thus we complete the proof. In the following lemma we obtain the asymptotic mean, variance and third-order cumulant of Z, under H and K. LEMMA 2.2. The asymptotic mean, variance and third-order cumulant of Zn are given as follows: Under K: 0 = Bo, Eeo(ZZ) = 1

t2 _ k t2a+1n ( 2a-1)/2

2 2

+ o(n

(2a-1)/2)

VBo(Zn ) = 1,2 - kt2a+ln(2a-1)/2 + o(n(2a-1)/2)

K3,Bo( Zn)=4(n

(2a- 1)/2

),

and under H: 0 = Bo + to 1/2 EBo +rn

2(Zn)=

I 2 k 2a+1

(2a-1)/2 -(2a-1)/2 +o(n ) - 2 t+ 2 t n -

VBo+ln',2(Zn) = It2 - kt2a+ln(2a-1)/2 +

o(n (2a-1)/2)

(2a-1)/2

where 12 ]

I=EB[[ aB logf(X-o) = - EB [ 2 log f (X - B) = a(a - 1)T( 1 - (1 / a))/T(1 / a)

J

348 ASYMPTOTICS FOR WEIBULL TYPE DISTRIBUTION

731

and

k=a{B(a + 1,a+1)+2a+1 }/r(1/a) with B(u, v) = fo' x"-'( 1 - x)°-'dx (u, v > O) . PROOF. Without loss of generality, we assume that No = 0. Putting n

We(x) _ l x l a - Ix -A I a with d > 0, we have Z n Yid (X;) where d = to "2. Since xa - (x - A)a (2.5)

Y% (x) = xa - (d - x)a (-

x)a

-(d

-x)a

for

x>_d,

for 0 < x < d , for x<0,

it follows that (2.6) Eo ['I%(X)] = C(a) [ - fo {(x + A)a - xa}e xadx

+ f o {x- - (A - x)0}e "^dx

f,

+ - Jx- - (x - d) a} e xadx ] = C(a)(I, + 12 + 13)

(say) .

Putting he (x) = (x + A)a - xa, we have from Lemma 2.1 (2.7) Ii+I3=

-f0

{(x

+A)a

a

-xa

}e xOdx +f - {xa- (x-d) }e xadx

= f o {(x + A) a - xa} {e (x+A)O - e "°} dx = f o he (x) {e ti°(x) - 1 } e x'dx _ - f o he (x) e xadx + 2 f o he j(x)e- X°dx + O(A)

1 )d2 d2a+1 +o(d2a +1). a 2a+1 +

349

732 MASAFUMI AKAHIRA AND KEI TAKEUCHI

We also obtain

(2.8) I2 =

f

{xa

- (A - x)a }e Xadx

= f o {xa - (d - x)a} { 1 - xa + 2 xza + O(x3a) } A 2a+1 ,42.

+l f° (e )a (1 - x )a dx + O(d3a+,)

2a + 1 + ={B(a+l,a+1)-

dza+l + O(d3a +1)

1

2a + 1

where B(u, v) denotes the Beta function. From (2.7) and (2.8), we have I,+Iz+I3=-aF(2- 1 )dz a + { B(a + 1, a + 1) +

y } 42a+1 + o(d2a+l) 2a+ 1

Since C(a) = a/{2T( 1/a)}, it follows from (2.6), (2.7) and (2 . 8) that (2.9) Eo ['YA (x)] = C( a)(Ii + 12 + 13) a(a-1)F(1-

1) a

J

2

te(a) + a{B(a + 1, a + 1) + (y/(2a + 1))) dza+, + o(dza+1) 2r(a) From (2.5), we obtain

(2.10)

Eo ['I%(X)] = f {(- x)a - (d - x)a }2f(x)dx +f' {xa - (d - x)a}z f (x) dx

+ f {xa - (x - d )a}z f(x) dx =f0 {xa - (x + d)a}zf(x)dx

350 ASYMPTOTICS FOR WEIBULL TYPE DISTRIBUTION 733

+fo

{xa

- (A - x)a }2f(x)dx

+ f °° {xa - (x - d)a }2f (x) dx

= C(a)(I, + 12 + 13 )

(say).

Since he (x) = (x + A)' - xa, it follows from Lemma 2.1 that (2.11)

Ii + 13' = f o {(x + d)° - xa12e-x.dx + f {x, - (x - d),12e x'dx x+e)°}dx = f p {(x +A ),_ xa}2 {e x' + e (

= f o h d (x){e °(x) + 1} e x°dx

= 2 f o hv(x)e x°dx - f o he(x)e xadx + O(d) =2aT(2- 1 )d2_ 2(l +Y)d2a +1+o (d2a+1). a 2a+ 1

Since 12, = f A

{xa

- ( d - x)a }2e x 'dx

=f d {xa - (d - x)a12(1 - xa) dx + O(d3a+1) =fo

{x2a

2a+1

2a + 1

- 2xa(d -

x)a

+ (d - x)2 a }dx + O(d3a+1)

- 2d2a+1 f ° O(d3a+1) 0 (d )a (1 d )a d dx +

2,42a +1 1B(a + 1, a+ 1) + O(d3a +1) 2a + 1 - 2d2«+ we obtain from (2.10) and (2.11)

(2.12) Eo[!e(X)] = C(a)(I1 + 12' + 13) a(a- 1)T(1 -1) a 2 T(a

351 734 MASAFUMI AKAHIRA AND KEI TAKEUCHI

a{B(a + 1, a + 1) + (y/(2a + 1))) d2a+l

+ o(d2a+') .

F(a ) From (2.5), we have

(2.13)

Eo [ `I%(X )] =

1 +

{(- x)a - (d - x)a}3 f(x)dx

fo

{xa

- (d - x)a}3f(x)dx

+f- {xa - (x - d)'}3 f (x) dx = f o {xa - (x + d)a}3 f (x) dx + f0 {xa - (d - x)a}3f(x)dx

+ f '{xa-(x-d)a}3f(x)dx = C(a)(Ii" + 13 + I3)

(say).

Since he (x) = (x + A)a - xa, it follows that

(2.14) 11" + 13 = - f o {(x + d)a - xa}3e xQdx + f {xa - (x - A )a}3e X'dx = f 0 {(x + A)a - xa } 3{e (x+e)' - e Xa}dx = f o he (x) {e "d(x) - 1 } e x'dx .

Since

fo he(x)e-xQdx = fo {(x + d)a - xa}°e xQdx = a4d4

f0 x4a-4e xadx + o(d4) = 0(d4)

it follows from (2.14) that (2.15) We also have

I +I3=0(A4).

352 ASYMPTOTICS FOR WEIBULL TYPE DISTRIBUTION 735

(2.16)

12" =

1

0

1) . {Xa - (A - x)a}3e XQdx = O(d3a+

From (2.13), (2.15) and (2.16), we obtain

(2.17) Eo[VIe(X)] = O(d4) . Putting d = to- 1/2 with t > 0, we obtain from (2.9), (2.12) and (2.17) Eo[Z„] = - nEo[!P1n'12(X)] a(a-1)TI1-1) a 2 1 t =

2T

(a)

a{B(a + 1, a + 1) + (y/(2a + 1))} 2a +1

+ o(

n

- (2a - 1 )/2

(2a-1)/2)

= 2 t2 - k t2a+In (2a-1)/2

+ O(n

(2a-1)/2

Vo(Z„) = nVo(Vf,n 2(X)) a(a - 1)r 1 - 1 ) a

T(

1

t

2

a)

a {B(a + 1, a + 1) + (y/(2a + 1))} tea +ln( 2a -1)/2 T(a ) (2a-1)/2) + o(n

= It2 - kt2a

K3,o(Z„)

+'

- ( 2a-1)/2 + o ( -(2a-1)/2)

nK3,o (Y%,,-=(X ))

nEo[{Yf... ... (X) - Eo[T,n '(X)]}3] = O(n

where

(2a

-1)/2)

353 736 MASAFUMI AKAHIRA AND KEI TAKEUCHI

0) }2 = - Eo I= Ee [} a0 log f(X -

]

[

2 log.f(X - e) J

= a(a - 1)T(1 - (1/a))/T(1/a) and

k=a{B(a+l,a+1)+2a+1 }/riia. In a similar way to the case under K: 0 = 0, we can obtain the asymptotic mean , variance and third-order cumulants under H: 0 = to 1/2 Thus we complete the proof. In order to get the bound for the 2a-th order asymptotic distribution of 2a-th order AMU estimators, we need the following. LEMMA 2.3. Assume that the asymptotic mean , variance and third order cumulant of Zn, under the distributions Po,n(O = 00, 00 + to 1/2), are given by the following form. E&(Zn) = µ(t, 0) + n (2a-1)/2C1 (t, 0) + o(n Ve(Zn)

=

u2(t, 0)

K3, e(Zn) =

o( n

+

(2a

n (2a-1)12C2

(t, 0) + o( n

(2a-1)/2) 12a-1)/2)

-1)/2)

Then 1 - (2a- 1)/2 Pe,n{Zn< ao} = 2 + o(n

if and only if ao

=,u(t, 0) +

Cl(t,0)

n (2a

-1)/2 + o(n(2a-1)/2)

The proof is essentially given in Akahira and Takeuchi (( 1981), pp. 132, 133). In the following theorem we obtain the 2a-th asymptotic bound for the distribution of 2a-th order AMU estimators of 0. THEOREM 2.1. The bound for the 2a-th order asymptotic distribution of 2a-th order AMU estimators of 0 is given by

,p(t) - CO ItI2acb (t)n(2a -1 )/2 sgn t + o(n (2a - 1)/2) ,

354 ASYMPTOTICS FOR WEIBULL TYPE DISTRIBUTION 737 that is, for any en E A2a

PQn{

In(

6n - 0):5: t

} :5 c (t) - Cot2a4

(t) n (2a- 1)/2

+ o(n

(2a-1) /2)

for all t>0, PB,n{ In (en - 0) C t} > 0(t) + CoI tI2ao(t)n(2a-1)/2 + o(n(2a-1)/2)

for all t<0,

where a{B(a + 1, a + 1) + (y/(2a + 1))} Co= 21a+(1/2)F(1/a) and 0(t) and 4(t) denote the standard normal distribution function and its density function, respectively. PROOF. Without loss of generality, we assume that 00 = 0. We consider the case when t > 0. In order to choose ao such that

(2.18)

Ptn '/ z,n{Zn

< ao} =

2

+ o(n

(2a-1)/2)

we have by Lemmas 2.2 and 2.3

ao = - 2 t2 +

2 t2a+ln( 2a-1)/2 + o

(n

(2a-1)/2)

Since (2.19) Po ,n{ZZ ? ao } = Po,n{ - (Zn - It2 - ao) <- It2} , putting Wn = - (Zn - It2 - ao), we have from Lemma 2.2 ) = ktza+ln( 2a-1)/2 + o n (2a 1)/z) (

(2.20)

E'o(Wn

(2.21)

Vo(Wn)

(2.22)

K3,o(Wn ) = o(n(2a -])/2)

= It2 - kt2a

+ln(2a-1)/2 + o(n(2a-1)/2)

We obtain by (2.19) to (2.22) and the Edgeworth expansion (2.23) Po,n{Zn?ao}=Po,n{Wn!5 It2} k 2a

-(2a - 1)/2 -(2a-1)/2

_ ^(^t) - 2^ t ^(f t)n + o(n ) .

355

738 MASAFUMI AKAHIRA AND KEI TAKEUCHI

Here , from (2.18), the definition of Z„ and the fundamental lemma of Neyman-Pearson it is noted that a test with the rejection region {Zn >_ ao} is 1/2 + o(n (2a-I)i2). the most powerful test of level Let On be any 2a-th order AMU estimator . Putting A& = {V 0n < t}, we have '1z

Ptn"', n (A&) = Ptn ' Z, n {an < to

} =

I

O (n

_ (2a- 1)/2

) .

2 +

Then it is seen that XA . of the indicator of A& is a test of level 1/ 2 + o(n (2a-')/2). From (2.23), we obtain for any en E A2a

Po,n{/ On


< Po,n{Wn< It2} k(V

t)

2

t2a

o( V I t)n )2a-1)l2 + o(n

(2a- 0/2 )

that is, (2.24)

Po,n{

In On < t} <- p (t) -

Cot2a

4, (

t)f

(2a -1)/2

+ o(o (2a

- 1)/2)

for all t > 0, where k a{B(a + 1, a + 1) + (y/(2a + 1))} Co = 21a+( 1/2) - 2 ja+('12)r(1 / a) Hence we see that the bound for the 2a-th order distribution of 2a-th order AMU estimators for all t > 0 is given by (2.24). In a similar way to the case t > 0, we can obtain the 2a-th order bound for all t < 0. Thus we complete the proof. Remark 2.1. The result of Theorem 2.1 holds for 2/ 3 < a < 1, where the information amount I must be expressed as a2T(2 - (1 / a))/ T(1 / a). The proof is omitted since it is essentially similar to the above.

3. The 2a-th order asymptotic distribution of the maximum likelihood estimator In this section we obtain the 2a-th order asymptotic distribution of the maximum likelihood estimator (MLE) and compare it with the 2a-th order asymptotic bound obtained in the previous section. We denote by 0o and OML the true parameter and the MLE, respective-

ly. It is seen that for real t , O ML < 00 +

to 112 if and only if

356 ASYMPTOTICS FOR WEIBULL TYPE DISTRIBUTION 739 n

(a

log f (Xi -

00 - In-'/2 ) < 0 .

Without loss of generality, we assume that 00 = 0. Hence we see that for each real t

(3.1)

if and only if I E (dl A) log f(X; - to-'/z) > 0 . s7n

OML < to '/z

Since a f-log f(x)= - al x i -' sgn x, we put () 32 Un = - I E (dl A) log f(X; - to '^z ) ,In =1 a

i/z I a i sgn (X; - to '/z) n II = I X; - to

_

In order to obtain the asymptotic cumulants of Un, we need the following lemma. LEMMA 3 .1.

f

If he(x) = (x + d)a - xa for d > 0, then

a

x -Ie X'he(x)dx=T(2- II )d+a

2

IT ( 2- 2 ) .4 2

Y dza + o(dza) - 2a

(3.3)

fo

xza -ze X"he(x)dx = T( 3 - a)A+

2

(a - I)T 3- a dz

+o(A2),

(3.4)

fo

fo (3.5)

x3a-3e

Xuhe (x)dx = (3- 3-)T ( 3 - 3) d + O(d a a

xa I X'ha (x)dx = aT ( 3

f xza-zeX'he(

?a

x)dx = aT ( 4 -

a

dz +0(43)

)A2 + O(43)

)

,

357

740 MASAFUMI AKAHIRA AND KEI TAKEUCHI

fo x3° - 3e x°he (x)dx = O(A2) ,

(3.6)

f oxa-'e X'hA(x)dx = O(d3 ) ,

fo

xz

° ze

x'he (x) A = O(d 3) .

PROOF. From (2.2), we have fo x a- Ie xvhe(x)dx a a-1 f

2a-1

x°

=fa (x+d) x e dx-fox e dx I (x + A)a+1{( a - 1)xa-2 - ax 2a z }ex°dx - 1 a+1 0 a

a + 1 fo {

xa

+1 + ( a+ 1)dxa

+

a(

1)

a2

A2xa -1

I

. {(a - 1)xa-2 - ax2a-2}e x°dx

a+1

f

R*(d) xa

2e

xadx -

a

+ O(d 3) ,

where R*(A) = (x + A)

a+t - xa+1 - (a

+ 1)dxa - 2 a(a + 1)d2xa-1 .

In a similar way to (2.4), we obtain

fo

R *(d)xa -

2e "dx

= 2 a(a + 1)(a - 1) f o (d - t)2 (f o (x + t)a-2xa-2e xadx) dt

= I a(a + 1)(a - 1) B(2a - 2, 3) B(a - 1, 3 - 2a) A2a + o(d2a) - (a + 1)T(a - 1)T(3 - 2a) d2a + o(d2a) . 4(2a - 1)T(2 - a) Hence , we have

358 ASYMPTOTICS FOR WEIBULL TYPE DISTRIBUTION 741

fo XI-1 e- x'hA (x) 3a-1 -x°

2a-1 -xQ

a+1 (a-1)fo x e dx-afo x e dx + (a - 1)(a + 1)d fo x2"-2e-XQdx - a(a + 1)d fo x3a-2e-XQdx + 2 a(a - 1)(a + 1)d2

-

a2(a +

f

0

1)d2 0

fo

x2"-

x3a-3e X

-

3e-XQdx

Qdx

1

2 1 A a - 1)T(a - 1)T( 3 - 2a) d2a + o(d2a a 4(2a - 1)T(2 - a)

a+ 1{ a a 1 -2+ a (a- 1)(a+ 1)T(2- I )d

-(a+l)T(3- 1 )d a

+ 2 (a- 1)(a+ 1)T(2- a) d2 - 2 a(a+1)T ( 3- 2 )d2} 1 - (a - 1)1'(a - 1)1'(3 - 2a) A2a + O(d3) a 4(2a - 1)F(2 - a) T(2- a ) d + a 2 I T

2- a )d2 2a Ala+o(d2a).

In a similar way to the above , we can obtain (3.3) and (3.4). We also have 2 fo xa -'e XQhe (x)dx = f o (x + d )2ax" 'e XQdx - 2 f o (x + d)"x2" 'e XQdx

+f0:

x3a-le

XQdx

359 742 MASAFUMI AKAHIRA AND KEI TAKEUCHI

= f 0^ {x2a + 2ax2a -'d + a(2a - 1)x2°`- 2d2 } xa -'e ".dx - 2 fo j xa + axa-'d + 2 a(a - 1)xa-2d2 l x2a-Ie- "dx

+ 2 + O(d 3)

a

aF(3_ 2 )d2+O(A) , a

and similarly get (3 . 5) and (3.6). From (2.2), we obtain fo

xa

-' e X"he (x)dx =

fo

xa

'e "Q(axa 'd )3dx + o(d3)

=a2T(4- 3 )d3+o(d3) a =

O(A)

,

and similarly have (3.7). Thus we complete the proof. In the following lemma , we obtain the asymptotic mean, variance and third order cumulant of U. LEMMA 3.2. The asymptotic mean, variance and third order cumulant of U„ are given for t > 0 as follows:

Eo (U„) = - I t + - t2an (2a- 1)/2 + o (n (2a-1)i2) 2 (2a -I)/2) Vo(U„) = I+ o(n -1)/2 ) K3,o(U„) = o(n(2a

1

where I is given in Lemma 2.2 and

k' = PROOF. (3.8)

a2{B(a, a + 1) + (y/ a)}

T(1/a)

Putting d = tn'/2, we obtain

Eon l X - d l a- ' sgn (X - d)] = C(a) { f,- (x - A)'-'e-"dx -

fo (A - x)'-'eX.dx

360 ASYMPTOTICS FOR WEIBULL TYPE DISTRIBUTION 743

-f 0 (d - x)«-'e I xl adx

= C(a)

I fo,

x« i e(x +d)° dx -

Jo (x + d)«

I

'e x' dx

[

a-1 - x « -1 } e x.dx - x" = C(a) f0 xa_ 1 (e_^4)° e - } dx - f o {(x + A) d

- f o (d - x)'- 'e- xadx = C(a)(J, + J2 + J3)

(say) .

Putting hd (x) = (x + d)« - x«, we have from Lemma 3.1

(3.9)

«

-'{e (x +d)' - e J, = f, 0 x = fo, x«-Iex 0

__

}dx

°{e ha(x) - 1}dx

x«-te

Jo

-x"

xahd(x)dx + 2 J0 x« 'e xQhv (x)dx + O(d3)

=-I'(2- a )d- a2 1 T 2- a )d2+ 2 T(3- a )d2 d2 « +

O(A2 a)

+ Za

(1

_ +)r(1 _1a'A+

a )d2 a2 1 2 - 2

+ 2 d2«+o(d2a).

From (2.2), we obtain

(3.10)

- J2 =

f0

-,e x"dx - f o x«- I e x0dx (x + d)«

« a

+ f o (x + d)«x« 'e x°dx - a .

Since by a similar way to (2.4)

361 744 MASAFUMI AKAHIRA AND KEI TAKEUCHI

f o (x + d)axa-'e X°dx = a +F 2- a )d+ a2 1 T(2-

a

)d2

Y

j2a

2a

+o(d2a),

it follows from (3.10) that

(3.11)

)dJ2= -T( 2a a

( 1- a

)d2

a

2 Ala

+

+ O(d2a)

Za

We also have

(3.12)

d a' -X°

-J3=,fo(d-x)-e dx =

fo (d

-

x)a-'( 1

- xa)dx +

O(A3a)

a a

-d2aro'(1- d)a'(^ )a^ dx+O(d3a)

= da - B(a, a + 1),42a + 0('43a) . a

From (3.8), (3.9), (3.11) and (3.12), we obtain (3.13)

Eo[IX-dla-'sgn(X-d)] =C(a){-2(a- 1)a- 1 )d a a + ( B(a, a + 1) + Y )deal+OWet) a JJ =-(a-1)F(1 (1/a))d

T(1/a)

+ a{B(a, a + 1) + (y/ a)} d2a + O(d2a) . 2T(1/a) Next, we have

(3.14)

Eo[IX - d 12a-2]

362 ASYMPTOTICS FOR WEIBULL TYPE DISTRIBUTION 745

= C(a) { f (x = C(a) f f o x2a

d)2a

-2e- x°dx +f

-2e(x+d)°

(d - x)2a-2e I xl °dx

I

dx + f 0 (x + d)2a72e-x°dx G 2a-2 x°

+ f 0 (d - x) e dx

[

(x +,J), x2a 2e- x °dx + f0 {e = C(a) 2 fo

+f0 {(x +

d)2a-2 -

ex° } x2a

2dx

x2a-2}e-x°dx

G 2a-2 x° +f0

(d - x) e dx

= C(a)[Ji + J2' + J3 + Jd]

(say) .

From (2.2), we obtain

(3.15) J(=2 f0 x2a-2e x°dx = a T ( 2 - a) = a ( 1 - a)F(l - a Since

f

0

Jz = 0 {e

(x +e)°

- ex. } x2a-2dx

= f°° x2a- 2e x°{e ha(x) - 1) dx 0 _ _f0

x2a-

_ 6 f0

ze x°he (x)dx + f 0 x2a ze xhv(x)dx

xz°-2e

x° he (x) dx + O(d3) ,

it follows from Lemma 3.1 that (3.16)

J2=-T( 3- a )d+(a -1)T(3- 3 )d2+0(d3).

From (2.2), we have

- 2 - x2a-2je x°dx (3.17) J3 = f0 {(x + d)2a

363 746 MASAFUMI AKAHIRA AND KEI TAKEUCHI

1 42 a - I

+

a f -(x

2a- 1 2a- 1 0

+ A)2a- Ix

a

-'e- x*dx

- a r (.^2aa> 1

1

2a

-1 _ 1 2a - 1

2a -14 aT +

2a

a

a

{f0 x32edx + (2a - 1)d f0 x3a-3e-x'dx

1

+ (a - 1)(2a - 1)42

fo

2a-i

-aT 2-

2a1 1 d

x3a-4e-

xadx

_

}

_

a+ 2a1 1 T 3

+T(3- a )4 +(a-1)T(3-

3

+0 (A 2)

-

a

)42 +O(43)

1 42a-I+T (3- a )d+(a -1)F(3- a )d2 2a - 1 +0(42). We also obtain

(3.18)

J4

=f0 (4 -x) = f' (A - x

za -z

)za

X. -2dx

za -i

2a -142a-

x

e dx

43a

(4

_ x)2a -2dx + O(44.- 1)

' fo (4 )a (1 - a

) 2a -

2 d

dx + O(A 1

i

2a- 1

- B(a + 1, 2a - 1)A ' + O(44a ^)

From (3.14) to (3.18), we have Eo[IX-4Iza-2] =

a r 1 '(1/a) al a 1 + I'(1/ )) T(3- a )42+0(42),

a

)

364 ASYMPTOTICS FOR WEIBULL TYPE DISTRIBUTION 747

hence , by (3.13) (3.19)

Vo ( IX - AI a-'sgn(X -d))

=aT(1 /a)r(1 +(a- 1)

-a)

[ F(1/a)T(3- a T1-la))_ 2

-(a- 1){ (F(1/ ) ]A2+o(42). Third, we have (3.20)

Eo [I X- A

I3a

-

3 sgn (X-d)]

= C(a) { f- (x -

d)3a-3e x°dx -fo (A_ x)3a - 3e X. dx 3a -f 0 (d - x) 3e iXi°dx }

I

- x3a- 3

= C(a) f0 -

f

{e (x+d)° - e x°}dx

m

0

fA

{(X +

(A

d)3a - 3 - x3a - 31

-

3

e-

x

e °dx

1

(say) .

= C(a)(Ji" + J2" + J3') Since JI,

x3a-3 -f0

_

{

e

-

(x

3e

+d)°

- e x° }dx = f0 x3a-3ex°

° h d( x)dx +

2

f0

x3a

{e ha(X) _ 1 }dx

3e X°he(x) dx + o(d2)

it follows by Lemma 3.1 that

(3.21) Jr = -3I 1-a)T(3- a )d+O(d2). From (2.2), we have

365 748 MASAFUMI AKAHIRA AND KEI TAKEUCHI (3.22) - J2" = f 0

+ d)3a-3 - x3a-3 le

{(X

x-°dx

= 3(a - 1)Afo x3a-4e ".dx + o(A) =3(1- a)r(3- a )A+o(A)

and also (3.23)

d - J3' = f 0 (d - x )3a-3e -X adx 3a-2 _ 3 - 2 -

A4a -2

f

d ( d

-3 )a ( 1 - d )la

d dx +

O(A5a-2)

A3a-2

3a - 2

+ O(A4a-2) .

From (3.20) to (3.23), we obtain

Eo[IX-

A13a -

3(a- 1). r

3sgn (X-A)]=

a

)A+o(A),

hence (3.24) K3, o (I X

- Al"

=Eo[{IX

sgn (X- A))

-Ala-1

sgn(X

-A)- Eo[IX- Al a-lsgn (X-A)]}3]

=O(A). Putting A = to 1/2 with t > 0, we have from (3.2), (3.13), ( 3.19) and (3.24) Eo(U„) =

a\/n Eo[I X -

_ - a(a - 1)

to 1 /21a

-1

r(1- (I/ a)) T(1/a)

sgn (X - to '/2)] I

+ a2{B(a, a+ 1) + (Y/ a)) 2a - (ea-1)/2 2r(1/a) + o(n

(2a-l)/2)

I t + kf

t2an( 2a -1)/2

+o(

n

(2a -1)/ 2

)

Vo(U„) = a2Vo(I X - to-1/21 a-1 sgn (X - to 1/2))

366 ASYMPTOTICS FOR WEIBULL TYPE DISTRIBUTION 749

T(1 - (I/a)) + o n (2a-1)/2) = a a- 1 ( T(1 ) /a) = I+ o(n (2a-1)/2) 3

K3,o(U

n) = K3,o(I X - tol121a-1 sgn (X - tn- '/2))

= o(

n -(2a- 1)/2)

where I= a(a - 1)T( 1 - (1 / a))/T(1 / a) and k'= a2{B(a, a + 1) + (y/ a)}/ T(1 /a). This completes the proof. In the following theorem , we obtain the 2a-th order asymptotic distribution of the MLE of 0. THEOREM 3 . 1. BML of 0 is given by

(3.25)

P8, n{

The 2a-th order asymptotic distribution of the MLE

In (BML - 0) <- t} C1 I t l2a o ( t)n (2a- 1)/2 sgn t = 2(t) -

+ (

n (2a-1)/2)

where C, = (2a + 1) Co with Co = a{B(a + 1, a + 1) + (y/ (2a + 1))}/ {2la+l l/2'T( 1 / a)}, and also the MLE is not 2a-th order asymptotically efficient in the sense that its 2a-th order asymptotic distribution does not uniformly attain the bound given in Theorem 2.1. IX)" is symmetric about the PROOF. Since the density f(x) = C(a)e origin , we see that the MLE of 0 is a 2a-th order AMU. We consider the case when t > 0. Using the Edgeworth expansion , we have by (3.1), (3.2) and Lemma 3.2 Po,n{v( 0ML =

PP,n{Un

0)

C

:5 t}

01 t2a`N ( V t)nl2a -,1/2 + o(n(

_ k(^t) -

2a-1)/2

2 ,[1-

that is,

(3.26)

PO,n{ In (OML - 0) < t} k' 2a (2a - 1)/2 (2a-,)/2 + o(n ) 2Ia +( 1/2) t s (t)n

367

750 MASAFUMI AKAHIRA AND KEI TAKEUCHI

4( t)n (2a - 1)/2 + o (n (2a -1)/2) = 0(t) - Clt2a ,

where C1 = k'/{2Ia

+( 1/2) }.

In a similar way to the case t > 0, we obtain for t < 0 In (OML - 0)

(3.27) PB,n{ =

0(t)

+

< t}

C1I t I2ao( t) n (2a-1)/2

+ o(

n

(2a-1)/2)

Hence (3.26) and (3.27) imply (3.25). Since k' = a2{B(a, a+ 1) + (y/ a)}/ ['(1/a) and Co = a{B(a + 1, a +. 1) + (y/(2a + 1))}/{2Ia+(1/2)T(l /a)], it is seen that C1 = (2a + 1) Co. Since Cl > Co for 1 < a < 3/ 2, it follows from Theorem 2.1 and (3.25) that the MLE is not 2a-th order asymptotically efficient in the sense that its 2a-th order asymptotic distribution does not uniformly attain the bound given in Theorem 2.1. This completes the proof. Remark 3 . 1. In the double exponential distribution case, that is, the case when a = 1, it is shown in Akahira and Takeuchi (1981) that the bound for the second order asymptotic distribution of the second order AMU estimators of 0 is given by t2 o(n 1/2) rh(t) - 6 0( t)n '/2 sgn t +

,

and the second order distribution of the MLE of 0, i.e., the median of X1,..., X, is given by

(3.28)

fi(t) - 2 0(t)n_n '/2 sgn t + o(n 'l2)

The results coincide with the case when a = 1 is substituted in the formulae of Theorems 2.1 and 3 . 1, but note that the proofs of these theorems do not include the case for a = 1 since it does not automatically hold that F(a)=(a- 1)T( a- 1)fora= 1.

4. The amount of the loss of asymptotic information of the maximum likelihood estimator In the section we obtain the amount of the loss of asymptotic information of the MLE BML using its second order asymptotic distribution (3.25). Differentiating the right-hand side of (3.25) w.r.t. t, we have the second order asymptotic density g(t) of In (BML - 0) as follows: (4.1) g(t) = 0 (t){1 - Cl(2altl2a-I

_ Itl2a+1)n (2a-1)/2} + o (n(2a-1)/2)

368 ASYMPTOTICS FOR WEIBULL TYPE DISTRIBUTION

751

for -oc 0

I

(4.2)

^tl

a^

(t) dt = 27c f(,'o tae 11 /2dt

=

2a/2

f (after transformation u = t2/2) a2 1 ).

Since, for sufficiently large n, d

d log g(t) t - C, {2a(2a -

1)l tl2a - 2 - (2a +

1)l tl2a}

it follows from (4.1) and (4.2) that the asymptotic information amount of the MLE is given by

IML = nI

IM L

f dt log g(t) 12g(t)dt

(p(t)[t2 + C,{4a(2a -

= nI f

- 2(3a + 1)ltl2 + o(n(3/2)-a) 2a+(3/2)

= nI { 1 - C, T(a +

I)l

t l2a-1

a +1 + ltl 2a +3 } n -(2 a - 1)/2 ]dt

1)n-(2a-1)/2

} + 0(n(3/2)-a) .

Hence, the amount of the loss L of asymptotic information of the MLE is given by 2a+(3/2 )

L = nI - IML = C,IT( a + 1)n(3/2)-a + o(n(3/2)-a) 2a+(1/2)a (2a + 1)T(a + 1){B(a + 1, a + 1 ) + (y/(2a + 1 _ Y,j a-(1/2)r(1/a) + o(n(3/2)-a) .

369 752 MASAFUMI AKAHIRA AND KEI TAKEUCHI

In a similar way to the above , it follows from (3.28) that, in the double-exponential distribution case , namely , when a = 1, the amount of the loss of asymptotic information of the MLE is given by 2 2n/7c + o(f), since I= 1. Acknowledgement The authors wish to thank the referee for pointing out two errors in the original version. REFERENCES

Akahira, M. (1975). Asymptotic theory for estimation of location in non-regular cases, II: Bounds of asymptotic distributions of consistent estimators , Rep. Statist . Appl. Res. JUSE, 22, 99-115. Akahira, M. (1986). The Structure of Asymptotic Deficiency of Estimators, Queen 's Papers in Pure and Appl. Math ., No. 75 , Queen 's University Press, Kingston, Ontario, Canada. Akahira , M. (1987). Second order asymptotic comparison of estimators of a common parameter in the double exponential case, Ann. Inst. Statist. Math., 39, 25-36. Akahira , M. (1988a). Second order asymptotic properties of the generalized Bayes estimators for a family of non-regular distributions , Statistical Theory and Data Analysis II, Proceedings of the Second Pacific Area Statistical Conference, (ed. K. Matusita), 87-100, North-Holland , Amsterdam-New York. Akahira, M. (1988b ). Second order asymptotic optimality of estimators for a density with finite cusps, Ann. Inst. Statist. Math., 40, 311-328. Akahira, M. and Takeuchi , K. (1981). Asymptotic Efficiency of Statistical Estimators: Concepts and Higher Order Asymptotic Efficiency, Lecture Notes in Statistics 7, Springer , New York. Akahira, M., Hirakawa, F. and Takeuchi , K. (1988 ). Second and third order asymptotic completeness of the class of estimators , Probability Theory and Mathematical Statistics, Proceedings of the Fifth Japan- USSR Symposium on Probability Theory, (eds. S. Watanabe and Yu . V. Prokhorov), Lecture Notes in Mathematics 1299, 11-27, Springer, Berlin. Ghosh, J. K., Sinha , B. K. and Wieand , H. S. (1980). Second order efficiency of the mle with respect to any bounded bowl -shaped loss function , Ann. Statist., 8, 506-521. Pfanzagl, J. and Wefelmeyer , W. (1978). A third -order optimum property of maximum likelihood estimator , J. Multivariate Anal., 8, 1-29. Pfanzagl, J. and Wefelmeyer, W. (1985). Asymptotic Expansions for General Statistical Models, Lecture Notes in Statistics 31, Springer , Berlin.

Sugiura, N. and Naing , M. T. (1987). Improved estimators for the location of double exponential distribution , Contributed Papers of 46 Session of ISI, Tokyo, 427-428.

370 SEQUENTIAL ANALYSIS, 8(4), 333-359 (1989)

THIRD ORDER ASYMPTOTIC EFFICIENCY OF THE SEQUENTIAL MAXIMUM LIKELIHOOD ESTIMATION PROCEDURE Masafumi Akahira Kei Takeuchi Institute of Mathematics Research Center for Advanced University of Tsukuba Science and Technology Ibaraki 305 University of Tokyo Japan 4 -6-1 Kowaba, Meguro-ku Tokyo 156, Japan Keywords and Phrases :

Sequential estimation procedure ; stopping rule ; third

order asymptotic efficiency ; maximum likelihood estimation procedure ; Wald identity ; asymptotically median unbiased estimator ; Edgeworth expansion.

ABSTRACT Under suitable regularity conditions , the third order asymptotic bounds for distributions of regular estimators are obtained . It is shown that the modified maximum likelihood estimation procedure combined with appropriate stopping rule is uniformly third order asymptotically efficient in the sense that its asymptotic distribution attains the bound uniformly in stopping rules up to the third order. 1JNTRODUCTION We consider a class of sequences of sequential estimation procedures {IIa : a=1,2, •••}, where for each a , IIQ denotes a sequential estimation procedure, that is, an estimating method combined with a stopping rule. We consider such a sequence that the expected sample size tends to infinity as a--- , also that the sample size •n is almost constant in the sense that Va(n)/{Ea(n )}2- 0 as a--00 or more precisely Va(n)/Ea( n)=O(1) as a-- -. In the previous paper by the same authors ( 1987), the Bhattacharyya type bound for asymptotic variances of estimation procedures is obtained , up to the second order , and it is also shown that the modified maximum likelihood estimation procedure attains the boun', if the stopping rule is properly determined. 333

371 334

AKAHIRA

AND

TAKEUCHI

In this paper it is shown that the third order asymptotic bounds for distributions of regular estimators which are completely similar to the fixed sample case, can be obtained, and it is also shown that for a choice of appropriate stopping rule, we may attain the third order bound uniformly which is generally impossible for the fixed sample case. 2. Wald's identity and moments Suppose that XI, ••• , Xn are independently and identically distributed (i. i. d.) random variables according to a distribution with a density f(x, 8) with respect to a a-finite measure p , where 0 is a real -valued parameter. We assume an usual set of regularity conditions on f(x , 0). And now we assume that a sequential sampling rule is given , with which we continue to observe X1, X2, ••• , Xn until n=No, and make an inference (estimation , test, etc.) based on XI, , XNO. Whether or not we stop sampling at n is decided based on XI, , Xn. In the subsequent discussions we consider asymptotic case where we actually consider a sequence of sequential inference rules {IIQ }(a=1, 2, ••• ) in which expected sample size va (8)=Eg0(n) approaches to infinity as a-a, and consider limiting properties of inference taken. In what follows we actually consider asymptotically almost fixed sample size rule, that is, we assume that n/va(8 ) converges in probability to 1 uniformly in 8 as a--. More precisely we assume the following (A.1) For any fixed point 80, va(8)/va(8o ) = 0(1), Ee,Q(n) =% (O) + O(1/v,,(8)), Vo,Q(n)/vQ(8)=O ( 1),Eo,Q(nk)/{va ( 8)}k=0(1) (k = 2,3,4), and {(ak/aek)va(e)}/vQ(8) = O(1) (k =1,2 ), uniformly in a neighborhood of 80, where E8,0(•) and V0, 0(•) designate expectation and variance , respectively. As the basic tool of the analysis we shall make frequent use of the following fundamental lemma of A.Wald (1959 ). Suppose that XI, , Xn are i.i.d. random variables. Proposition 2.1. Let Zn = g(X1)+ ••• +g(Xn) and 4(t)=E[exp {itg(XI)}]. Then we have E[eitZ„ / {4(t)}n] = 1.

This proposition can be easily generalized to the following : Proposition 2 . 2. LetZ„i = gi(X1)+ ••• +gi(Xn )(j=1, ••• , k), and 4 ( t1, , tk) =E(exp {itlgl ( X1)+ ••• +itkgk(X1)}]. Then we have Psp(it1Z +--.+itkZR)

E

)=1. l

From this we have the following lemma.

(2.1)

372 THIRD

ORDER

ASYMPTOTIC

EFFICIENCY

335

Lemma 2 .1. Let Zni U= 1,••,k) be defined as above , and denote pj =E[gj(Xi)] Q=1,---,k) and p J1,...jk.)= E[{gjl (X1)-pjl}...{ gjk.(X1 )- Pjk .}], where (j1 ,... jk,}c{1,...,k}.

Then we have

E[Zni] =E(n)pj (j =1, ••• ,k) ; E[(Znu -pjl )( Zn12 - p12)] =E(n)p(11J2); E[(Znii -pjl)(Zn12 - Pj2 )(Zn13 - pJ3 )] = E(n)p(jl j2j3) + E(nZnii ) p(2J3)

+ E(nZni2 )p(j 1 j3 ) + E(nZ j3 )p(j i j2) E[(Zn11-pji)(Z i2-pJ2 )(Zrj3-pJ3XZn14-pj4)] =E(n)p(11 J2J3J4 ) - E(n2){p 0 1 J2)p(J3J4) + 003)02M +p(11J4)P(12J3)}+E(nZn11Zn12 )p(j3 j4) + E(nZai2Zr,i3 ) pUlj4) +E(nZn13Zn14 )p(jij2)+ E(nZn11Zn13 )p(12 j4) + E(nZni2Zni4 )p(ji j3)+ E(nZ,j1Znj4 )p(12j3) +E(nZnji )p(j2J3J4) + E(nZni2 )p(1 j3J4) +E(nZni3 )p(l j2J4) + E(nZni4 )p(11 j2j3),

where {it, •••jk'}c{l,•• ,k}, provided that the differentiation under the integral sign of the left-hand side of (2.1) is allowed. Proof.

We shall give the proof of the last equality since the previous ones

are far easier to prove . For simplicity we shall write 1,2,3,4 instead of j1, j2 , j3, j4, and also assume pja=0 (a=1,2,3,4) without loss of generality. Putting tj=itj (j =1,2,3,4) in Proposition 2.2 we have ezp(t1Za 1-E L

+t2Z.t n

+t^.n)

JI

(01,t2, 3,Qr

= E[exp{tiZnl + t2Zn2 + t3Zn3 + t4Zn4 - n4r(ti,t2,t3,t4)}]

=E(e$) (say ), ( 2.2) where ip( tl,t2,t3,t4)= 1og4(ti,t2 , t3,t4). Differentiating both sides of (2.2) we obtain

a4

E(e'1')=0

m1m2at3at4

Since differentiation and expectation can be interchanged, we have ( O_EI at

a4

t

1"?34

_E at1at2at3\at4e^^^

373 336

AKAHIRA AND TAKEUCHI

-E

1Iat 1 & 1&

I(

&"

+

4 2 33

-E[a((

at 4 a!

)e*1

a3.t + -t. 2k +

t

+ +±'2k'

&1 &2&3 4 813814 &2 812814 813 at4 812813 &4 &3 &2

)

e^

_ a -E ael ((+ ++34+2++u+3+$4$23++4+34)2)e-O} 1

E 1 (1234 + +134+2+$34$12++124+3 + +24+13++14$23+'P4 '123

++14+3+2++4+13+2+ t4+3+12+4)1+234 + +14)24)34

+$1$3 24 + +1$4 23 + +14AA4)e+J'

where +a = alt/ata, +ap = a2+/ataatp, +apr = a31t/ataatpatr and +apr6 = a4Vataatpatrat8 (a,P,r,S = 1,••• ,4). Then we obtain 0 = E[ - np ( 1,2,3,4 ) - np(1,3 ,4)Zn2 + n2p(3 ,4)p(1,2) - np(1,2,4)Zn3 + n2p(2,4)p(1,3 ) + n2p(1,4)p(2,3 ) - np(1,2 , 3)Zn4 -np(1,4)Zn2Zn3 -np(1,3 ) Zn2Zn4-np ( 1,2)Zn3Zn4 - np(2,3 ,4)Zn1-np(3,4)Zn1Zn2 - np(2,4)Zn 1Zn3 - np(2,3 )Zn1Zn4 + Zn1Zn2Zn3Zn4],

hence E[Zn 1Zn2Zn3Zn4] = E(n)p(1 ,2,3,4) -E(n2){p(1 , 2)p(3,4) + p(1,3)p(2,4) + p(1,4)p( 2,3)} + E(nZn 1Zn2)p(3,4) + E(nZn2Zn3)p(1,4) +E(nZn3Zn4) p(1,2) +E(nZn1Zn3 )p(2,4) +E( nZn2Zn4)p(1,3) +E(nZn1Zn4) p(2,3) +E(nZn1 )p(2,3,4) +E(nZn2) p(1,3,4) + E(nZn3 ) p(1,2,4) +E(nZn4 ) p(1,2,3). This completes the proof. Remark 2 .1. From Lemma 2.1 it follows that the fourth order cumulant of Zni1, Zi2, Zi3 and Zni4 is given by K(Zni1, Zni2, Zni3, Znl4) = E(n)p(1, j2, j3,j4 ) - V(n){p(11,j2)p(j3,j4)+p(1J3)p(2J4) + A 1,J4)p(2,13)} + Cov(n, Z& I Zni2 ) p(j3 j4) + Cov(n,Zl2Zi3)p(j 1 X14) + Cov(n, Zni3ZJ4 )p( 1 J2) + Cov(n,Zn11Z13 )p(12, J4 ) + Cov(n ,Zni2Zni4)p(j 1„13) + Cov(n,ZnilZni4)p(12j3 ) +E(nZnii) p(12j3,j4)

374 THIRD ORDER ASYMPTOTIC

EFFICIENCY

337

+ E(nZnj2 )11(j 1 j3J4 ) + E(nZl3 ) p(j 1 j2J4) +E(nZr,i4)11(j1i2i3)•

3. Asymptotic cumulants of sequential estimation procedures In the same situation as in the previous section, we shall obtain asymptotic cumulants of sequential estimation procedures . For simplicity we denote va(6) by v. Let ® be a parameter space which is assumed to be an open subset of Euclidean 1space RI. In order to obtain the asymptotic cumulants of sequential estimation procedures, we assume the following conditions. (A.2) {x:f(x,6)>0} does not depend on 0, and f( x,61)/f(x,02) is not equal to constant for all disjoint points 81, 62. (A.3) For almost all x[pJ, f( x,8) is four times continuously differentiable in 6. In the Taylor expansion 4 hi

log

ftzre) _ X e( i)(e,x)+h4R(x,h) i=1

R(x,h) is uniformly bounded by a function G(x) which has moments up to the fourth, where 60(6,x) = (ai/aei)e(6,x)(i = 1,2,3,4 ) with e( 6,x) = log f(x,6) (A.4) For each 0, E8[e(I)(6,X)] = 0, 0<1(0) =Ee[{e(I)(6,X)}2] _ -Ee[e(2)(6,X)]
'\/vi=1

and z = -^ (e, 3,(9,X.) 3,v

v L vi=l

t

-3J(O)-K(e)}.

The following lemma is very useful for calculations of asymptotic cumulants.

375 338

AKAHIRA AND TAKEUCHI

Suppose that Ye is a function of X1, ••• , Xr, and 0 and is Lemma M . differentiable in 0. Then

dy

I

0 1 ^, Yd _ vv d e Ee( Yd-

I/ J VV Ee\ ae

and d EB(Zl"Yd -Vv dOE0(zly -,/-,EO (Y0 a01r)

J.E0(zlV ae )'

provided that the differentiations under the integral signs ofEe(Ye) and Ee(Z1,,,Ye) are allowed , respectively. The proof is omitted since the lemma is similar to Lemmas 5.1.1. and 5.1.2 in Akahira and Takeuchi (1981) ( see also Lemmas 2.1.1 and 2.1.2 in Akahira, 1986). An estimator 9a=ea (X1, , X,) of 0 is called to be -*/v-consistent if for any e>0 and any q of ®, there exist a sufficiently small positive number 8 and a sufficiently large positive number L satisfying the following : A

lim

Pe {\/ v e le I <s

I0a-0I?t}<e.

For each k=1,2,•••, a '/v-consistent estimator 0a is called k-th order asymptotically median unbiased (or k-th order AMU) estimator if for any q E there exists a positive number 8 such that lim

lim CF-

e... •I -q I< 8

SUP

qi <

v(k-

i)1z1 p {0

1 5e}- I =e, a 2

n 1 1Y`k-lua1Pe{ea?e}- I = 0.

2

A Let C be the class of the all bias -adjusted BAN estimators 0a which are third order AMU and asymptotically expanded as _ A zIV (0) 1 1 1 ^/v(9a-0 )= 1(0) +V -Q+vR+ oP(v),

where Q=Op(1), aQ/a0=Op(1), R=Op(1), aRIa0=Op(1), and the distribution of v/v(0a-0) admits the Edgeworth expansion up to the order v-1. If an estimator 0a belongs to the class C, then we call it C-estimator. However, in the subsequent discussion, the term R is not explicitely needed, hence we simply write

z (0) 1 ^v (e.-8)= «0) + ^-Q+O (v)' V v

(3.1)

although the above stochastic expansion is necessary in order to validate the proofs, since they actually depends on two facts that ( i) the asymptotic distribution of dv (9a - 0) is equivalent to that of

376 THIRD ORDER ASYMPTOTIC

EFFICIENCY

339

Z,,Y(e) 1 1 Q+x r(e) +^, v up to the order v-1, and that ( ii) the asymptotic moments of R and their derivatives are of 0(1).

Theorem 3 . 1. Assume that the conditions (A.1) to (A.5) hold. Let 6a be an Cestimator with (3.1). Then its asymptotic cumulants xi (i =1,2,3,4 ) up to the fourth and the order o(v-1) are given as follows : For Ta = Vv (6,-0)

µ1(e) µ2(e) 1 K1=Ea(Ta)=-+-+o 1/v v v

KZ-Ve( °)

K3=K3e( a

1 2µl(9) 2 ✓ µ1(e) c(9) + v +oY I(s)+ v1(9) v I(9)

p3(9) r3(e) I )= Ee[{Ta-Ee (a)}3)= =+-+o

r4,,(T

v

v

v

(i4(9) r4(O) )}2= - + - +.(-1 ) V V V

where 1 3J(9)+2K(O) 3J µt(9)= 61(9) X3(9) with P3( O)= - - 2

t3(e> yr (e)

1

µ2(e) = 61(9) r 3(9) with

3 r3(e)= 21(0) Ee[Z

( 1 ,V Q- µt(9))2)

I2(2J(9)+K(9)XJ(9)+K(9)) - 3H (9)+4L(9)+12N(9 ) X4(0)= J4 IS(e) (9)

3 JZ(O) 14(9) (M(O)- 1(0)

5v' 27v'2 4v" + (30J(O)+ 19K(O)) + - , 314(O)v J3(O)v2 13(9)v

3 J v' (n-v)1 Ee(W^ with W= ZZ -( - )Z - r4(9)= -v--,Y J4(9) 1 v vv z(6) = V9(Q), v'=(a/aO)va(0), and v" = (a2/a02)va(0).

Remark 3.1. If v=va(6) is independent of 0, then it is seen that the cumulants in Theorem 3.1 coincide with those in non-sequential case given in Theorem 2.1.1 in Akahira (1986 ), since v'=v"= 0 and E9(W2) = M(0) - {J2(0)/I(0)}.

377

340

AKAHIRA

AND

Remark 3 .2. Note that pl(9),

TAKEUCHI

{33(0) and p4(0) in Theorem 3.1 are

n independent of the specific estimator 0a.

A Theorem 3 .2. Assume that the conditions (A.1) to (A.5) hold. Let 9Q be an Cestimator with (3.1). Then the Edgeworth expansion of the distribution of 9Q up to the order v-1 is given by Pe{V I(9)v (9a-e);5 t)

I(e) ./I(0) p3(e) 2 12(e){p4(e )+ r4(e)}

e $(t)-

=fi(t)-

(t3- 3t)$(t)

24v

6w

13(0)p2(e) I(e){de )+ µi(0)} - 10t3 + 15t)$(t)- 2v 140) 72v (16

12(0 ) P 3(o)pl(e) 2 1

I(0)vi(0) r3(0) 2

6v t $(t)-

sv

t(t -3)$(t)+o(Y).

(3.2)

where 111 (9), 112(9), p3(9), p4(9), T3(9), r4(()) and i(9) are given in Theorem 3.1, and (t)- 40)dx with $(s)=(1/^/2rt)e 12 f 2

Theorem 3.3 Assume that the conditions (A.1) to (A.5) hold. Let 6, be any Cestimator with (3.1). Let Ee(WQ*)=rEg(W2)/1+o(1) with W=Z2,v-{(J/1)-(v%v)}ZI,v -{(n-v)I/-/v }, Then the asymptotic bound for the distributions of 0, and a fixed stopping rule up to the order v- t is given as follows :

P8{V I(e)r (ea-e)5 t) (())V

(Z) $(t)- 6Vv

P3(0) 2 I2(9

t $ (t)-

) p4(e) 3

24v

(t -3t)$(t)

13(9)2(0) I(0)p2(0) (t6-10t3 + 15t (t)- 2v t$(t) 72v IZ(e)S3(e)pl(e) 2 - t(t -3 6v

(t)-

2v

1 Ap(t){J(0)+R(9)+

4vI3(e) v

EE(W2) t Ap(t){(-+rV'l(e))2+ 2vI2(0) 2 4

I(0)}2+o V

(i)

(3.3)

378 THIRD ORDER ASYMPTOTIC

EFFICIENCY

341

for all t > O (t<0), where p1(0), p3( 0) and p4(0) are given in Theorem 3.1, and Q*=Q-Pt(e).

Corollary 3.1. Assume that the conditions (A.1) to (A.5) hold. Let 0a be any C-estimator with ( 3.1). Then the asymptotic bound for the distributions of 0. and any stopping rule satisfying (A.1) up to the order v- l is given by pe('/I(e)v(9a-9) S t} 5 I(e)V'1( 9)P3(9) (9)^ (9)

(^) 00) 6d v

t,+ (t) - 24v (t'-3e(t)

I'(e)pa(O) s I(9)p2(0) (t -lot + 15t(t)- v *(9) 72v 2v (e)p1(9) 6v

t(t2-3)t(e)- I e4(t )(J(9)+K(9)+ 2v' -I(e)}2+o (1) (3.4) 4vI (9) v v

for all t > O(t
Denote by snit, the maximum likelihood(ML) estimator of 0 based on the sample (XI, • • • , X,). Let 9ML* be the estimator modified from the ML estimator to be third order AMU. Then it follows from a similar way to Akahira and Takeuchi (1981) that under the condition (A.1) zto -v Ztr 3J+K

(eML 9)= I -( y ) I +Ij:/r(Z1v2;,- 21 Z1.)+oo(7y

(3.5)

Theorem 3.4. Assume that the conditions (A.1) to (A.5) hold. If the stopping rule is so determined that the observation is stopped at n satisfying R - e(2) --

A.

(3.6)

with C(9)= Jt01(9)

L(O)

,-(e) -- 1 {2L(9)+M(9)+N(9)}

1(8ve) + 1(0) -2v(O)

(3.7)

21(0)

and some random variable a with Eo(e)=o(1), and the stopping rule of (3.6) satisfies (A.1), then the modified maximum likelihood estimation procedure combined with the stopping rule is uniformly third order asymptotically efficient in the sense that its asymptotic distribution attains the bound (3.4), up to the order v- I, uniformly in t and stopping rule.

379

342

AKAHIRA

AND

TAKEUCHI

It has been well known that for fixed sample estimations Ee(W2) =Eo[{Z2,v-(J/I)Z1,v}2) represents the minimum loss of information for any best asymptotically normal estimator (Fisher, 1925; Rao, 1961) or the curvature of the model (Efron , 1975 ; Amari, 1985 ). The above result establishes that by the choice of appropriate stopping rule the loss of information can be reduced to zero (!), or the model can be made flat (!), and it is also to be noted that the right-hand side of (3.6) can be interpreted as the "realized" or ex post amount of information , and the stopping rule concerned is that we continue to sample until we get the "realized" amount of information is (nearly) equal to the expected amount of information corresponding to the predetermined expected sample size . In practical cases it may be troublesome but not really quite difficult to determine c precisely, and also to check the condition (A.1), but we may rather stop sampling as soon as the righthand side of (3.6 ) exceeds v ( 9*ML)I(8*ML)+c (K*ML). Then the expected sample size will not be exactly equal to 1, but the difference Ee(n)-v (6) will be of small magnitude of o(1).

Proof of Theorem 3.1. Putting U=Ta-E0(Ta), we have U = dv(A, -Ee(9Q))

(3.8) where Z1 ,,, I and pl denote Z1,49), 1(0) and p1 (O), respectively. Since UY=(U- Z[.r )z+ I Z1 I. I Zi, .

it follows that 1 2 1 Ee(U2= y ye(@*)+ I Ee(Z1.vU)- +0(Y ). where Q*=Q-p1.

Since = a (V;(e*- Ee(9n)))

=2 U-v-Ee(9A ")

= 2 U- -V v { EE(eA - 9)+ 1 J

V

Zl,r

=2vU ^v dO

1 11 1 - +YQ+Ph +1 ^vl

(3.9)

380 343

THIRD ORDER ASYMPTOTIC EFFICIENCY

2v

U - ,/ ;

l ae (vEe(Q))+1+o( Y))

+lµ' ) -^ v +o( 1_) _-• U-Vv(- v'µ v 2 S1 v 2v -y'U+ v' ,1 - 1 -Pi- -µ -V r +o(7=), 2v v4/ v S/ v v

(3.10)

it follows by Lemma 3.1 that E0 1,v Lq= -L± ROM- -L 199(

au)

2vvVvµl

Vvµl 1+o(1) 1111 v

(3.11)

=1+yµ1 zµ1+o(y), v

where v and pl' denote v(A)=(a/a6)v0(0) and pl' =(a/aO)pl(0),respectively. From (3.8), (3.9) and (3.11) we have

K2= V9(T0) = Ee(U2) -1+2µ -2vµ I

vl

+ 1Ve (Q*)+O(1 v v2I v

=1+2 µ' 2y'

1

(3.12)

I vl I v2l I v v

Since ZI." 9 3 ZI" 2 3 2 3 U9=(U- I )+_(U__ )ZIv+ZZI."U -ZIP". 2 219

it follows that Ee(U2= 3 Ee(ZI"Q*2) + 2IEe(ZjYU2)-

2 Ee(Zl.)+ o(1).

(3.13)

Since Ee(U^ =

I +0 (y) and r(e)=

) = 2J(O)+ K(O),

it follows from (3.10 ) and Lemma 3.1 that 2J+K v' 1 1 a aU +o(-) E 4Z U^ =-- E (U2)E ( U-)=- 2 v321 e 1," ^/ a8 a S/ v e a8

where J and K denote J( O) and K(e), respectively. Since 1 3v' Ee(Z91) _ K + -I+o( Y), S/V

(3.14)

381 344

AKAHIRA

AND

TAKEUCHI

we have from (3.13) and (3.14) K3 = K3 06(Ta) = Ee(U3) 3 3 2J+K Y' 2YI ^• 21 7v 13 v"I

-L (- K = 3v' )+0( 1 ) 213 V v rag

27B•(Z1.rQ*')+ o(!). v;13 3R1i+ 1 1 1 ^3+ v T3(A)+o(Y-).

where P3= D3(O) - ,r{(3J+2K)/(VcI3))-{3Y%(,3/212)} and r3 ( ()) = 3Ee(Z1 ,YQ*2)/(2 I.

Since Z 4 U4=(U- I r)^+ 3Z1.r(U- j )3- Z4,,U_+ a Z1.rU3+ 1 Z4 12 31 314

it follows that (3.16)

B•(U')=- B•(Z4 U3)+ 31Be(Z1.,U3)+ !jE•(Z; r) 12 From (3.10), (3 . 15) and Lemma 3.1 we obtain •,^, l a 3 eat/ 1 a 3J+2K 3v' \1 B•(ZlrU3)= N/va9B•(U3)- YB•(U )_ ^Yd9 - ^v13 vial=/

-

3

a Iva 8

1 v

U2

1 2vU--/

v(l+Yp" -

! p1

)l1

+o(1).

3J+ 2 K 3Y' g v; Y;12 )

- ^tB•(U3,+ 3(1 + v pi - plus•(U)+o (Y ).

(3.17)

Since azlr -Z - v1 Z - 111 aB ;r 2v lr

it follows from (3.10), (3 .14) and Lemma 3.1 that _ az B•(Zi.U!)= 1- d B•C+1^,U •)- 1-B4 -1'rU )- 2B•(Z1.vUl) ^YdO N/ Y

I d (2J+ K + r' / - i 6 f (Z - s Z3, - 7 )U=, ^/r t ;r13 v^ll ^r • ;r r IV;

(3.18)

382 THIRD ORDER ASYMPTOTIC

EFFICIENCY

Z_B.1z U+ r, µ - I-µt-,V; +o(^). V= 7r LL `2v 1 1 A

W-Z;•-(1

_;)Z,-.

345

(3.19)

(3.20)

V

Vr

we have by Lemma 3.1 B.(WU.*)=6e1 W(U- _^ •)_, + t Be(Zl.•U1n- !B0(Z^•w) 8 1 a 2

1 au 2 1 aw SWU

(3.21)

_ Beal w)+N/Y

Since by Lemma 3.1 (3.22)

B(nZ = -L±8 W = we obtain from (3.11) Be(UW)=Be(z;•U)- j Be(Zl •U)-Be( IU )+ Y• Be(Zl."4

-Be(I Zl•Z

;,

I -Be(n rZ1.) + Be(ZD+o(1)= 0(1). vI r IV;

(3.23)

From (3.8) we have

-U- VYABe(O

- 9)-%/;

=

=2vU-Vv (YBE(Q)}v+o(Y) v' - 1 =-U-^v+0(-). 2v Vv

(3.24)

Then by (3.20) and (3.22) Be( W)= Bel(ZvU-N/ vxZR•-(^-v )Zl.y-(^v^} +0(1)

-Be[ sv (ZR•U-(I - r• ) ZI •U-

2YBl I Zi. Zr•' I (I

(n -

vu U}J +0(l)

dY

v,)Z1.•- (

J

n

=0(1).

v)Z^v^+o(1) Y N/;

(3.25)

383 346

AKAHIRA

AND

TAKEUCHI

Since I'(9) =2J(9)+K(9) and J'(9) =L(8)+M(8)+N(A), it follows that ;r v' Z +Z +o (1) M d8 2v 2,1 Ar P a J v' 1 J(2J+K) vs v' J V. v' nl a^v) (L+M +M+ .V `/y)+ P(1). Y v )Z 1,r +( I y XZ2.r 2,, I 2 Is

((I

=(I

and a (n-v)I v7 + (n-vX2J+K) - (n-v)v7+o (1) a8 vv )_-`/ v 2v312 P

Hence aw azs r a J- v' a (n-v)I) v)Zly)-a k as _ (( ^v

-Z +Z ,v 3,v- v

2v 2

j(L+M+MZI,r -(I-Y)(Zxr -2v Z1 v" v.2

,v --v)+J( 12 ^Zlv

(n-vX2J +K) (n-vWI

VI +(--Y2)Z,r+-+P(1).

v which implies by (3.22) E

2v32

1

rZ2v+Z1.vZ3,v -

- (L+M +MZiv-

J(2J+K) 2

vZl,r

(I- L)(Z1,,Z^v-2-Z1v

v" v'2 2

(n-vX2J+K)

+ I2 Z1.v+( v y2 )7.1,r V

r

nZ

^vrn (n-vyv'I

v 7lv+ 2v32 21'vJ

.

= 1 v J+L - J_• - -(L+M+N) I 2v Vv ^v J V' v' I v ' J(2J+K) v" vi2 -(I-vXJ-2vI v V v)+ I +(v-v2X

Iv'2l (2J+K)v v 2v2

v' 1 J(J+K) i{ I -(M+M-W+K) v

Iv' 21v'2 - s }. V2 +v

(3.26)

384 THIRD

ORDER

ASYMPTOTIC

EFFICIENCY

We also have from (3.20) and (3.24) r r W)= Ee1(2U- dv) {Z$„ -(I- Y)z IV 199( r r =EeI - I z U-( I

_ y' El( Zlv2$v

2v I

347

N/- 1 +o(1)

r

_ (n-vk u IT v )2.U V-v ✓

+0(l)

y y^sl,v n-v

1221.,+

vI ^v ZI^'1+o(1)

(3.27)

=0(1).

By Lemma 3.1, (3.23) and (3.26) we obtain a'

E0(Zi W)

A Ee(21.YW)- y Ee(W

Y )- v E9(ZI.V

LW- )

1 d v' J '1y 1 1 1 e[ w( w+_ ! N/vdO v n-TVE 2 z - ^/vl

I I J(J+K) v' !v" 21,121 1 t I -(M+N)-W+K)v + Y - y2 +O(T)

^v

1 v'! , 1 1 _J

=-^/v(v) Ee(W) vv 1 :/

v'

2v)Ee(Z1•W)

1 v' Iv-_ 21,14 1 1 _ IJ(J+K) ;v 1 -(M+M-W+I^v + v - +o(-) v2 -V v 1 v" v' v'2 v{v1+vW+K)v2Il-^/vEe(W2) 1

(J(J+K)

i ^/v -(M

v' Iv" 2Ivi2 j +M-(J+K)-+-- v s + 0 (

1_lj(J+K)_(M+N)+Jvg_31v'2 v"1 1 +2l- - E(W2)+o(-). vv I v v2 v -Vv a Vv From (3 .21), (3.23), (3.25), (3 .26), (3.27) and (3 .28) we have

(3.28)

2 v ' Iv" 2121 E (WU2)=- i1 J(,+K) -(M+M-(J+K)v + v J e 12;/ v V2

+

Jv' 3Ivi2 v" 1 +21 1) -(M+N)+ --+ 1 E W2)+o( ^/v I v 2,2 v ;2: v a{

1 r J(J+K) !^v

_ - V4

1 I J(J+K)

Iv2^

v

v' 1 1 -(M+N)-(3J+K)- + Ee(W)+o(-

I2^/

v

I

v

I^/

v

"

v

(3.29)

385

348

AKAHIRA AND TAKEUCHI

From (3.20) we obtain (((

r' I ) 2vN/y U21

=Ee1 (w+(I - 2y )Z1 .,-vvll Usl

(3.30)

.,U)-VrIE(UZ). =Ee(WU2 )+( I - 2Y)Ee(Z1

Hence we have from (3.19) and (3.30)

Ee(Z1

1 d (2J+K v' 1 +, X/ 1 2 7v^ I

--

--E (WU^- 1_(J- 3v )E (Z U2)+IE(U2) v e N/ v• I 2v 2 j,,"ff.(Z"U)+(_^_pj1-pi -vlr 2v y312

I d

,(n}+o(1) r

2J+K v' 1

^/vd9 v;I2 r-"11 - 1_Ee(WUZ)- 1(J- 3v)Ee(Z1,U^+IEe(U^ 2v ^/v Vv-2(Y µl-1P; - 1)E0(Z1 ,U)+o(y

(3.31)

Since by Lemma 2.1 and (3.22) 2 31 E. K (.2 + 61Ee(nZi.,)+ v Ee(nZ1.)+o(v ) H yz Ee(Zi .)= V 312 = H + 3KY '

2 31 - vz Ee(n2)+

(nZ,,)+o(v ), vl Ee

it follows from (3.16), (3.17), (3.31) and (3.32) that E (O_-2 1 d 2J+%+

v 1-E (WUZ) O , / v ^ ^/ v12 v`^ZI ^/ y e

TV (I 2v

-2(y2p1 -

)Ee(Z1.,U)+IEe(U)

v Ni -1 )Ee(Z1,,U))

(3.32)

386 349

THIRD ORDER ASYMPTOTIC EFFICIENCY

8 1

d 3J+ 2K 3v' 3v'

+ 31 -- - - - E (U^ 2v" N/ v rl3 v"12

+3(1+y'p'- sp1)Ee(U2) v

s

+-L 314

-3Be(ns1+6//Ee(nZ",)I+o(- ). V2 v v 1 Hv+3 qvv

(3.33)

Since by Lemma 3.1, (3.18 ) and (3.22) az E,(nZ, ,= 1_ d Ee(nZl^- 1 EE(n 1Y) 1,dO N/ v ag

) I V d (Y,--)- f E (nZ - v2 - I _E (no)r +0(1) d9 ^/v ` a 2" 2v^ ^ e 1 1 d v' v2 I

= -(-)+-+-E (nZ )+o(1), ^vdO Vv 2v2 V'v 6 2"

it follows from (3 .23) that

klI

E9( nZ21)-y E0(n)J

.

s

( 2 d( v-)+ v2 + 2 9.(n2)- 2-Ee nZ2,)- I Ee(n^ 12, I-vvdO 7v IV 2 2 V4 1 I v vEe(ns)-I^vE(fZ^y)j { ( 13v{ `^/vde^/v)+ v2 j+1 1

2

2

1

9v1

vde(^v)+v+IvtEal(Zv v/

lsEa(Z^`^+v)

= 1 ' J " ')Z, )21 M + 1 +o(1) . - -_ j ? _ d (y__ )+ - } +vL9 [( W+(- - p :v 12 v I v ••') J3, %/v dR v v ,9 14 1

``

2 ± ( v'

13v`-V vde^v)

+r2} v It

1 2 J V. I_ v' 2 2 l +14V Ee( W +2(I-v)Z1rW+( I v)Zi.v/

2v,2 2Jv' J2 M i 1 1 1 1( 2 d v ' 12 7, A ^/ v vs Iv 12 1 7, • v

-M 14

1 1

+I

+o( v

(3.34)

387 350

AKAHIRA

AND

TAKEUCHI

Since J'(9)=L(9)+ M(8)+N(9 ) and 1V(9)=H(e)+3N (O), we have from (3.33) and (3.34) Ee(U4)=sEe(U2)+ zS2(- ,+µ1-lµ^ )Ee(ZlY0 v I I V2 i 3v I (J + EO(WU^+ 1 1 1 d r 2J + K )v' +E(WU^+-( ---)E(Z 1 U'J +--` + a v I 2v a ,. N / X/vde w12 ,32j

8 1 d ( 3J + 2K 3v 3v' + 31 N/ v d9 i3 vi12) 2v Ee(U^ 1 v' 1 1 H 3Kv' +3(Y Pj - vzµl )Ee(U2) + 3^(Y + Yz )

1 1( 2_ d v' 2,'2 2J,' J2 M 1 2 1 ++(--(-)+---+--- +E(W)+o(-) 14V e v 12 13v N ( vde v/ v ,2 Iv 12 I

= jEe(U2)+ z(-1+ V I

2 Nl-

vµ0(1+vµ;- Zµ1)

de /+ 12^vlIv2.v I2^/v 1 12+ v3121J

I2^/v1

v' J 3J 1 2 -(3 J + K )- + - Ee( W2) + ^ v 72^/v I 2v 72^v 8

1

d( + 31 l 7=VA

3J+2 K 3v ' vv13

v3^12 I

2J+ K v' z 312, /v l V

3v' (

3J+2K

3v'

2,3a \

v vI3

,31212

1 H 3K-1 1-v' 11 +3(µ 1 µ))+-(-+-) , v2 l l 3I4 v v2 2v'2 2Jv' J2 M l 1 1 1( 2 d v' E(W2)+1 ++ --(-)+---+---J+4 -o(-) I2 13, ^/ vde X/v v2 I, I2 I 1 4, e

V

2v'2 2 d 2J+K v' 8 6 2 3 1 ; --P )+ -( +- ) += -E e (U)--+- (v-µ ,V3121 13,3 2,/ v de 12 I2 2 1 v ; I2

v

- 2 (J(J+K) v' 2 J 3v' 2J+K - v' -M-N-(3J + 2K)- + (---X- -) v32l v I v - X72^/ v l 2v 38 ( 1 d ( ,

^/

I

3J+2K

` ^vr3

3v'

3v '

J

,31212 I -

2v^2 `\ -

3J+2K - 3v' 1

^/vl' vlZ

388 351

THIRD ORDER ASYMPTOTIC EFFICIENCY

1 ' 1 1 H 3Kv' +3(-p' --µ)- +-(-+-) 314 v v2 v 1 2 I

2v'2 2J-/ J2 M 1 + 3 2 1 1 1 2 d v' +- (-)+---+----E0(w )+o(-) v 1/ v 1^

v2 Iv 12 14v V

2(2J'+K') 6 2)_ 3 = 1Ee(U )-12+ 14 v M 2N H l4v

4(2J+K)2

J(5J+4K)

16v

16v

8(3J'+2)V)

14v 314v

8(3J+2K)(2J+K)

314v 16v

v' 27v'2 4v" 3 1 + -(150J+95K)+ - - - + -E (W)+o( 314v2

-I

Ee(U2)-

13v2

13v3

2 + 12(2J+

14v

9

v

+12N J +K) - 3H+4L+12N v

v

3 J2 5v' 27v'2 4v" (M-V+ -(30J+ 19K)+--14v

13v3

3I4v2

I3v2

+IEB(W2)+o(1 ) Y v

= 6E (U2)- 3 + -4 + 3 E ( W2)+o (1) (say). I e 12 v 14v 9 v

(3.35)

Since by (3.12) {V9(Ta)}2= {Eg(U2)}2= 1 Ee(U2)- 1 +o (v ).

12

it follows from (3.35) that K4=Ee[(TQ -Ed(T))4)-3 {V0(Ta)}2 6 3 =Ee(U)- -Ee(U)+ z I 04 =- +- EeW2 v

_ 04 + 1 r4(0) v v

(say).

(3.36)

389 352

AKAHIRA

AND

TAKEUCHI

We put pl(e) P2(9) Vy

v

v

It follows from (3.12), (3 . 15) and (3.36) that the Edgeworth expansion of the distribution of 0Q up to the order v- I is given by

,/i pl

'Vip=

=1'()- ^v fi(t)-

y=*t)

13 s P3

12

24v(04+r4xt3-3t$(fIL+IpI

IVi^

v $(O- 3(t=-1)$1 - I (y 6Vv

72v (ts-10t3+15t)4(f

1%/1r

1'0

3 t '9(t2 z. eau)- 6v ( 1)$(n- 6'p' at'-

t

(3.37)

2

where o( n= J- $(t)dzwith$(t)=(1/d2n)e-: rl A

(See also Kendall and Stuart, 1969). By the third order AMU condition of 0Q we have from (3.37) A 1 VIpl Q1µ2 IVIP3 I'1173 pe{e 5 e} = Z - -- $(o)- Y $(o)+ 6^Y 40( 0)+ 6v t(O)+o(v )

1 1 =2+a(y),

hence IP3 173 µ1= 6 and µz= 6

Therefore we obtain 1(0)03(0) 1(0)73(0) 1 Ee(Ta)= 6V'v + 6v +o(v). Thus we complete the proof.

Proof of Theorem 3.2. It follows from (3 .37) with (3.38). Proof of Theorem 3.3. From (3.2) we have pe(VI(e)v (9a-e)5 t}

(3.38)

390 353

THIRD ORDER ASYMPTOTIC EFFICIENCY

13 s

=tee)- - t

12 P4

=$(t)

24v

(t' -344 (d-

- -*(t)- Isg3N1 t(12-3* (fl-

P3 v

72

(`s- 1 ot3 +15 )$t

Ee(W2Xt'-3t*(t) L-

+0(j). 4 I Ee(Z1.,Q•2)t=$(A ).

- 2 Ve(Q*) $(t)

(3.39)

where Q* =Q-µl, and I, P1, P2, p3 and p4 denote 1 (0), 111(e), p2(0),03(0) and p.(e). By Lemma 3.1 we obtain E (aQ

2 1aE (Q *'^ - Q*) Q*')=Jvae e 1,. a Jv a ag

E(Z

2Ee l(d9 Vv

(Jv(Ta-Zj')1^Q*I+o(1)

_ Z r := - 2-EeE y (Ta )Q*+J v ±(T0- ZjY))Q*I+o(1)

l

Vv 2Jv

-2EeIQ*{2v u Jv

v +-((2J+JM

-I aB^i)JJ+0(1)

= EE(Q* ^^r)+o(1)

= 2 Ee[Q*(

Z;,-2-Z,, _ JY

)J+0(1)

JY I Q *jW - )Zi ., - 11 i,g.l+(I 2v

= 2

= Ee(WQ*)+ o(1),

where J

1i=ZS<^-

Y•

(I-

(n-v)l

)Z1,,-

Jy

+0(1)

(3.40)

391 354

AKAHIRA AND TAKEUCHI

We also have from Lemma 3.1 Ee(Z1,YWQ* )= T ^Ee(WR')- YEe(

R')- JEe(W )

• =- 1-E (a Q')- 1-E (W

^/ v e ag •S/ v e ag

)+o(1).

(3.41)

Since aW v'

nJ i

- =- - Z 2v +z3,v- Y - I (L+ ht+N)z1,v ae - v' Z - nl )+ J(2J+I^ Z +( YM-)Z + v'I -(J- v'XZ I v 2,Y 2v 1 .Y Vv 1 •r 12 v vZ 1 r ^ v

- (n-vX2J+ K) + (n-v)WI +o (1) Vv 2v542 Y ' it follows Lemma 3.1 that

I-Ee(

(3.42)

Q')=0(1).

Since 1

aQ

_1

a (

_

Z

V v d8 v ag' a I Z = 2v (Ta- I v )+

v

+ I2 {(2J

aT

aZ +K)ZI.v

-I

^v

v' 1 v' - P1 2J+K 2 Zlv -Q'+- Ta-^/v--+ 2v 2v T/v ,/v I 1 v' n1 1 v' { - + N2 +0(1) I ^v (Z -Z- -)+ ^/v 2v v v - I v' 2J+K v' --1J J 3V' I W+(I- 2v1, )Z1 vl +2v T+(a I2 +27v12l ,v v +P(1) 2 12 + Iv / 1••+ P(1) II W+ 2vTa+(J+K

itfollows from (3.23), (3.41) and (3.42) that Ee(Z1vWQ•)=--LEe(W ag)+0(1)

392 THIRD

ORDER

ASYMPTOTIC

= jEe(W)--E9(W7a

EFFICIENCY

)-(JI2K+ Ivr

355

)Ee(WZI.,)+o(1)

_ ^Ee(W) + o(1). By Lemma 3.1 we have

1 d 1 aZIv 1 )- -'vEe( as Q*)- J-EB(ZIY a ) Ee(ZiYQ*)= Vvd6Ee(ZIvQ*

- L EOQ*jW +(J3 V

v

l

I

✓

2v

( f

1111

Z (

---Ee(ZI"( aa-g(V; To- IY)/^1+0(1) ` -Ee(WQ*)- 1- Eel -(T.-Z")ZI,, -vv Vv 2-V v I Z +-VVIae (Ta- jv)

1, 1+o(1)

r Z =-2 E9 [(To- [v)Ziv

-Ee

ll -T \/ vC(2J+K)Z iv-I az 'v/jZIVJ+o(1) vi + I Z\ 112Y a777

az ltv =-2 Ee(Ta Iv)-12KEe(Ziv)+IEe( 0 Ziv)+o(1) 0 V 2J+K 1 f ( I +IEe Zi W

2v

J v' 11

+(I- Y)Ziv - vvI

+0(1)

✓

2J+K _J 3v' 2v I + I 2v +0(1)

If we take QO* as Q* such that Vg(Q*) is minimized under the conditions

E8(WQ*)= jEe(W2)+o(1).

Ee(ZIvWQ *)= -Ee(W2)+ o(1),

(3.43)

(3.44)

393 356

AKAHIRA

Ee(Zi,,Q *)=-

AND

TAKEUCHI

J+K 2v' 1 - v +0(1)

(3.45)

for real r, then Qo* must have the form Qo* =Ao+A1W+A2WZ1,v+A3Z12,v ,

where Ai (i=0,1,2,3) are constants . Since Cove(W,Zi,vW)=o(1), Covg(W,Z12,v) =o(1), and Covg(Z1,vW,Z12,v)=o(1), it follows that 12 1 2 1 2-/ V9(Q0*)= (r + -)E9(W )+-(J+ K+ I)2+o(1).

12

(3.46)

v

214

Hence we have from (3.39), (3 .40), (3.43 ), (3.44), (3 .45) and (3.46) Pe(V I(9)v (60-9)6-t}

2

2

124v (t3-3t*(t)- 72v (ts-1Ot3 +15t$(t)

;5 13'203

( ) •(t)- 6^v ta¢(t)-

lµ12

12p3µ1

- 2v 90(t)- 6v

2

do(t)

dt - 3)^(t)

2v

1

(W2(t2-3 ) 4l2Ee

+IV9(Q0 )+ 2Ee(Z1.,Qo 1+o(Y)

_ ... - *()I I Ee( W2)(t2 - 3)+ (r2+ )Ee(W^+ 1 W+K+ W l 4I 213 v

e

1

IVI--

L Ee(W^t +o (v ) + Ee(W2) (( t2 - 3 l 1 2V 1 d^(t) 1 -+(Ir2+1 )+'JIrtf- i*(tXJ +K+-I)2+o(-) 2v

l

4

11

4v

V

v

Ee(wll it 2 + '1 1 2v' 1 _ ti(t) (Z+VIr) +4 --t$(tXJ+ K+ v I)2+o(-) V 2v12 4v1' for all t>0 (t<0), where "•••" denotes first six terms of the above right-hand side. This completes the proof.

Proof of Corollary 3.1. It is straightforward from Theorem 3.3. Proof of Theorem 3.4. From (3.5) we have n -v

Zl,r Zl,r

3J+K 2 1

V;(e -e)=-+ (z --n- _z +.P( ML I^/ v

2r

^/v 21v 4r

394 THIRD ORDER ASYMPTOTIC EFFICIENCY

357

_3J+ =Z1'^+ 71,v {W +(J--)Z I V I' I2V1 V

Z =- + --Z t'vw I'Y

K Z2 +o( 1 P ^/ v 21^/ v t'v

213. . 1,v-12V

Zt ,Y+o,(: v) .

(3.47)

Vv

In a similar way to Theorem 2.2 in Takeuchi and Akahira (1988), we obtain by Taylor expansion of both sides of (3.6) about 0 (n-v)I=Vv (Z2,v-

J 3J+K n-v v' I JvZ 1,v ;v Zl'v+

IZI)

1 4L+3M+6N+H 2 ZI,vZ3,v 212v nZI'V

+ 212-- (v"I+2v (2J+K)+v(2L+2M+5N+H))Zi v+c( 6)+o(1),

(3.48)

where v =ve(0), v'=vo (0)=(a^ae)va(9), v"=v' ,(0)=(a2/a92)va (9), Z1,v=Z1,v(e), Z2,v=Z2,v(9), Z3,v=Z3,0), I=I(8), J=J(9), K=K(9), L=L(9), M=M(9), N=N(9) and H=H(9). In order to determine n satisfying Ee(n)=v+o (1), we have from (3.48) C(0) = Jv' L v" I Iv + I - 2v + 21

(2L + M+M .

From (3.48) we also obtain J v' n-v Z2,v-(I - v )Zt v- -I= p(1), -'/v

that is,

W

=op(1),

(3.49)

hence by (3.47)

'Vvl0 -9)=Zjv-

J+ K Zl,v- v - Z2,v+0 (- ). ✓v 213V v 12V Vv

Since , by Takeuchi and Akahira ( 1988), - A 1 1 2v 2 1 J+K Ve(^v(eML-9))=I+ 2( 1 + v) +o(y),

2I

the estimator O*ML modified from the ML estimator OML to be third order AMU has the same asymptotic variance as that of eML. Then it follows from Akahira (1986) and Akahira and Takeuchi ( 1981) that the modified ML estimator ;*ML has the

395 358

AKAHIRA AND TAKEUCHI

following asymptotic distribution up to the order v-1: Pot-,/ I(eh49) (e*L_ 9)s t }

I(OW I( 9)

=q'(t)-

(3(9)

6^ Y

2 t $(t)-

I2(9)p4(0)

24v

I3(9)02(9)

(t3-31)4(t)

I(9)µ12(9)

3 (15-1013 +15t)4(t)-

72v

t4(t)

2v

I2(9)(33(9)µ1(9) 1 2 ✓ 1 t(t2-3)¢(t)- ti(t){J(9)+K(9)+ -I( 9))2+o (-) , 6v 4v1 3 (9) V V

where µl( 9) 03(9) and {i4(9) are given in Theorem 3.1. Since, by (3.49), Ee(W2) =o( 1), it is seen from (3.4) that the asymptotic distribution of a*Mt . coincides with the bound up to the order v-1 uniformly in t and stopping rules . This completes the proof.

References Akahira, M. (1986 ). The Structure of Asymptotic Deficiency of Estimators. Queen's Papers in Pure and Applied Mathematics No.75 , Queen's University Press, Kingston , Ontario, Canada. Akahira, M. and Takeuchi, K. (1981). Asymptotic Efficiency of Statistical Estimators : Concepts and Higher Order Asymptotic Efficiency. Lecture Notes in Statistics 7, Springer-Verlag, NewYork. Amari, S. (1985). Differential-Geometrical Methods in Statistics. Lecture Notes in Statistics 28, Springer-Verlag, Berlin. Efron, B. (1975). Defining the curvature of a statistical problem (with application to second order efficiency). Ann. Statist., 3, 1189-1242.

Fisher, R. A. (1925). Theory of statistical estimation . Proc. Comb. Phil . Soc., 22, 700-725. Kendall, M. G. and Stuart, A. (1969). The Advanced Theory of Statistics, Vol.1, 3rd ed., Griffin, London. Rao, C. R. (1961). Asymptotic efficiency and limiting information. Proc. Fourth Berkeley Symp. Math. Statist. Prob., 1, 531-545.

396 THIRD

ORDER

ASYMPTOTIC

EFFICIENCY

359

Takeuchi , K. and Akahira, M. ( 1988). Second order asymptotic efficiency in terms of asymptotic variances of the sequential maximum likelihood estimation procedures . In : 2nd Pacific Area Statistical Conference , Statistical Theory and Data Analysis 11, North -Holland , Amsterdam, 191-196. Wald, A . ( 1959 ).

Sequential Analysis. John Wiley, New York.

Received December 1987 ; Revised June 1989 Recommended by J.K. Ghosh

397 Pub. Inst. Stat. Univ. Paris XXXV, fasc. 1, 1990, 3 a 9

First order asymptotic efficiency in semiparametric models implies infinite asymptotic deficiency * Masafumi Akahira(*) and Kei Takeuchi(**)

Abstract For semiparametric models, it is shown that, under fairly regularity conditions, the asymptotic deficiency of the maximum likelihood estimator or any regular best asymptotically normal estimator is infinity.

1. Introduction For various types of semiparametric models, it has been shown that uniformly asymptotically efficient estimators may be obtained under some regularity conditions ( e.g. see Hajek, 1962 and Takeuchi, 1971). More precisely let us assume that X1, . . . , X,, are independent

(') Institute of Mathematics, University of Tsukuba, Ibaraki 305, Japan (") Research Center for Advanced Science and Technology, University of Tokyo, Komaba, Meguro-ku, Tokyo 156, Japan Key words and phrases : Semiparametric models, asymptotic deficiency, maximum likelihood estimator. AMS Subject Classification (1980): 62F12, 62F10

*This paper is retyped with the correction of typographical errors.

398 and identically distributed random variables according to a density function f (x, 0) with respect to a a-finite measure µ, where 0 is a real-valued parameter, and, given 9, f belongs to some class F which can not be identified by a finite number of parameters. It is naturally required that fl(x,01) # f2(x,02) for any disjoint 01 and 02 and fl, f2 E F. Then we can construct estimators B. such that 9,, is asymptotically efficient for the model with known f, that is, in regular cases , v/n- (B,t - 0) is asymptotically normal N (0, if 1), where f (x, 9) }2 f (x, 0)dµ(x) jr = j { 80 log whatever true f E F be. The pertinent question here is that when we can also achieve uniform second order asymptotic efficiency ? or does the distribution of Vfn- (o - 0) attain the second order asymptotic bound ? In the subsequent discussion we will show that, in a sense , loss of third order asymptotic efficiency, or asymptotic deficiency , is generally infinity under fairly general conditions for semiparametric models, which although does not directly answer the above question, may shed some light on the possible higher order asymptotic efficiency of estimators under semiparametric models.

2. Infinite deficiency in semiparametric models Suppose that X1,. .. , X,. are independently and identically distributed (i.i.d.) real random variables with a density function f (x, 0, ^) with respect to a or-finite measure p, where 0 is a real-valued parameter to be estimated and ^ is a real-valued nuisance parameter. We assume the following conditions (A.1) to (A.5).

(A.1) The set {x : f (x, 0, ^) > 01 does not depend on 0.

399 (A.2) For almost all x[µ], f ( x, 0, ^) is three times continuously differentiable in 0 and ^. (A.3) For each 0 and each 6

0 < Ioo(9, 6) = E [{Po(9, 6, X)}2] = -E [Poo(9, e, X )] < 00,

o < Iii(0,6) = E [ {P,(e,6,X) }2] = - E [Pig(0 ,6,X)] < oo, where Po (0, 6, x) =

(a /a9)P(0, ^, x), too (0 , 6, x) _ (a2 /092)P(9, 6, x),

Pl (0, 6, x ) _ with

6, x ) and Pl, (e, 6, x) _ (a2/a62)P(9, ^, x)

1 (9, e, x) = log f (x, 9, 6).

(A.4) The parameters are defined to be "orthogonal" in the sense that E [Poi (9, e, X)] = 0, where Po (0,

^, x) = (a2/aea )P(e, e, x).

Note that the condition (A.4) is not necessarily restricted , because otherwise we can redefine the parameter g(9, ^) so that we have the above orthogonality.

(A.5) There exist Jooo

= E [Poo (9, , X)Po(0, ^, X)] ,

J001

= E [Poo (9, ^, X )Pl (B, ^, X )] ,

Jo,o = E [Poi (9, , X)Po(9, e, X )] , Joie =

E [Poi (9, ^, X )Pi (B, ^, X )] ,

Juo = E [Pil(9 , , X)Po(8, e, X)] , Kooo = E [{Po(0 , ^, X)13], 2 Koo, = E [{Po(o, e, X)} P1(0, e, X)] , Moooo = E [{Poo (0, ^, X)}2] - Ioo, Moroi = E [{Po1(0, ^, X)}2] and the following holds.

E [Pooo(O, X )j = -3Jooo - Kooo, E [tooi(0,

X)] = -Joao,

E[tou(0,X)] = -Joie, where Pooo(0,

e, x) _ (03/003) P(9, C, X), tool (0, C x) = (93/ae2oC)

1(0, e, x) and Poll (0, C, x) = (a3 /aeaC2) P(9, C, x).

400

From the condition (A.5) it is noted that Kooi = Joio - J001. We put Z0 = to(0'"'), Zi = tl(B,"),

Zoo = E { 'oo(0"' X) + loo} , Zoi = f E'0l(0", X:)• Let B* and ^* be the maximum likelihood estimators (MLEs) of 0 and respectively. Then we have the following. Theorem 2. 1. Assume that the conditions (A.1) to (A.5) hold. Then the MLE 0* of 0 has the following stochastic expansion. (B*-B)

1 1 3Jooo + Kooo 2 =j0oZo+ V n6IOOZooZo

-

2fI00

1 oInZoiZi Z0 + ^lo

- Joio Z Z - Joie Z2 + 0 ( 1

- i o 1 2^loolii 1 ^ Vfn The proof is given in the paper by Akahira and Takeuchi (1982) and also in section 4 . 2 of the monograph by Akahira ( 1986 ). We put 1 3Jooo + Kooo 2 Qo = I2 ZooZo 2I3 Zo, 00 00 2 Qi = 1 Zi (Zoi - Joio Z0 - Joie Z1 + Joii Z1 100111 \ Ioo Iii 2loolii Then we have the following:

Theorem 2.2. Assume that the conditions (A.1) to (A.5) hold. Then the asymptotic deficiency of the MLE 9* is given by d = loo {V (Qo ) +V (Q,)} _ 1 (Jooo + K000)2 1 ( J021 _ Joie Ioo (Mooooloo -j020 o) + 2I 0 + 100,11 I Moroi - 100 Iii

+ 2loolii'

401

where V(.) designates the asymptotic variance. The proof is given in the paper by Akahira and Takeuchi (1982) and also in section 4 . 2 of the monograph by Akahira ( 1986). Now, in order to consider a semiparametric model, we assume the condition (A.6). (A.6) For each 0, each ^ and almost all x[µ], f (x, 8, ^) = f (x - B, 01 and f(x,^) = f(-x,^)• Then we have the following corollary. Corollary 2.1. Assume that the conditions (A.1) to (A.6) hold. Then the asymptoic deficiency of the MLE B" is given by (2.1) d = oo (M0 0000 - Jooo) + (Jooo loo 000 )2 + 1020 111 (Mo101100 - j0210) Since, by the condition (A.6), Joll = 0, the proof is easily derived from Theorem 2.2. It is also noted that the inequalities M0000I00 Jooo ? 0 and Mo1oll00 - JOlo > 0 hold from the Schwarz inequality. Next we consider a semiparametric model of the type:

(2.2) f(x - 0,^) = CC { fo(x - 6)}1-e{g(x - 0)}£ for 0 < ^ < 1, where Cc is some constant, depending on ^, with Co = C1 = 1, and fo(x) and g(x) are certain density functions. Then we assume the condition (A.7). (A.7) {fo( x2du(x) = 1. .fo(x) f

It is easily seen that the condition (A.7) implies loo = 1. From (2.2) and (A.7) it follows that, by the Schwarz inequality, Mo101

j0210 =

J-^ f g(x) I2 fo

(x)dµ - {f- g(

= p (.fo, g) (say) is nonnegative. Then we have the following.

x)fo(x) fo(x)dµ

402

Theorem 2.3. Assume that the conditions (A.1) to (A.7) in the semiparametric model (2.2) hold. Let g(x) be of the form Kago (x) with some constant Ka for a > 0 and some function go(x). If p (fo, go) > 0, then the asymptotic deficiency of the MLE 9• tends to infinity as a -* oo. Proof. From the assumptions we have lim (Molol - Joao) = lim alp (fo, go) = 00, Q-400

a_+00

hence, by (2.1), it follows that the asymptotic deficiency of the MLE B` tends to infinity as a -^ oo. This completes the proof. Denote by F a semiparametric model consisting of density functions f (x) with f (x) = f (-x). Then we have the following. Corollary 2.2. Assume that for any fo E F there exists go E F such that p (fo, go) > 0, and that for any 6 with 0 < 6 < 1 and any a > 0, C(^, a) fo-fgo^ E F, which is denoted by f0(x, 6), where C(^, a) is some constant. If the conditions (A.1) to (A.7) on fa(x - 0, 6) for every a > 0 hold in the case when 0 is a location parameter, then the asymptotic deficiency of the MLE ( of 0 tends to infinity as a oo.

The proof is straightforward from Theorem 2.3. Example. Let fo(x) be a non-normal density and go(x) be a normal density with mean 0 and variance 1. Then we consider a mixture fo of densities fo and go as fo(x, e) = C(e, a) {fo(x)}'

{go ( x)}"oz for 0 < ^ < 1,

where C(^, v) is some constant with C(0, v) = 1. If the assumptions of Corollary 2.2 are satisfied for 1/a2 instead of a in the case when 0 is a location parameter, i.e., f, (x - 0,^) and go(x - 0), then the asymptotic deficiency of the MLE of 0 tends to infinity as o -4 0.

403 References Akahira, M. (1986).

The Structure of Asymptotic Deficiency of

Estimators. Queen's Papers in Pure and Applied Mathematics No.75, Queen's University Press, Kingston, Ontario, Canada Akahira, M. and Takeuchi, K. (1982). On asymptotic deficiency of estimators in pooled samples in the presence of nuisance parameters. Statistics & Decisions 1, 17-38. Hajek, J. (1962). Asymptotically most powerful rank order tests. Ann. Math. Statist., 33, 1124-1147. Takeuchi, K. (1971). A uniformly asymptotically efficient estimator of a location parameter. Jour. Amer. Statist. Assoc., 66, 292301. Recu en Juin 1988

404 Austral. J. Statist., 32(3), 1990 , 281-291

LOSS OF INFORMATION ASSOCIATED WITH THE ORDER STATISTICS AND RELATED ESTIMATORS IN THE DOUBLE EXPONENTIAL DISTRIBUTION CASE MASAFUMI AKAHIRA1 AND KEI TAKEUCHI2

University of Tsukuba and University of Tokyo

Summary Fisher (1934) derived the loss of information of the maximum likelihood estimator (MLE) of the location parameter in the case of the double exponential distribution. Takeuchi & Akahira (1976) showed that the MLE is not second order asymptotically efficient. This paper extends these results by obtaining the (asymptotic) losses of information of order statistics and related estimators, and by comparing them via their asymptotic distributions up to the second order. Key words: Loss of information ; double exponential distribution ; second order asymptotic efficiency; maximum likelihood estimator ; median.

1. Introduction Fisher (1934), starting from his fundamental paper (1922), discussed estimators of the location parameter of a double exponential (two-sided exponential) distribution as a typical example of non-regular estimation. He showed that the maximum likelihood estimator (MLE), which is equal to the sample median in this case, has asymptotic loss of information of order Vn, as compared to constant order in regular cases. In Takeuchi & Akahira (1976) we showed that the MLE does not have second order asymptotic efficiency, unlike regular cases. This paper extends these results by showing that it is possible to construct an estimator which is asymptotically Received May 1989; revised September 1989.

'Institute of Mathematics, University of Tsukuba, Ibaraki 305, Japan. 2Research Center for Advanced Science and Technology, University of Tokyo, 4-6-1 Komaba, Meguro-ku , Tokyo 156, Japan. Acknowledgements. The authors thank the associate editor and the referee for valuable comments , in particular for pointing out the form of proof used after Lemma 2.2.

405 282 MASAFUMI AKAHIRA AND KEI TAKEUCHI

better than the MLE both in terms of asymptotic variance and asymptotic loss of information in the second order, but that it is impossible to have an estimator which is uniformly better than the MLE in the second order expansion of the distribution. We thus conclude that there is no second order asymptotically efficient estimator.

2. The Loss of Information Supose that Xi,. .. , Xn are independently and identically distributed (i.i.d.) random variables with the double exponential density function. f(x,6) = fo(x - 0) = i exp(-Ix - 01). First we compute the amount of information contained in the order statistics around the median, or equivalently the loss of information by discarding all other order statistics. We use the following lemma. Lemma 2 .1. (Fisher, 1925; Rao 1961) The loss of information associated with any statistic T, which is a function of a sample of size n obtained from the population with the density function f (x, 0), is given by

EB[Ve( E a9logf(Xi,0) I T)] , where E9 (.) denotes unconditional expectation and Ve(• I T) the conditional variance given T. Now let Tk be the set of the central 2k + 1 order statistics X(,_k+1) < • • • < X(,+k+l) obtained from a sample of size n ='2s + 1 from the double exponential distribution. Then we have the following lemma. Lemma 2 .2. For each k = 0,1, ... , s - 1 the loss of information Lk associated with Tk is given by

/ 2s+1 \ Lk = Ee [v9 I sgn(X= - 0) I Tk Il (2.1) 2(2s + 1)! (2u3_k_1 - us_k-2 )( 1- u)a+k du. (2.2) (s - k - 1)!(s + k)! j Proof. Equation ( 2.1) follows directly from Lemma 2.1 and the assumed form of the density function . Given Tk, the set of the first s - k order statistics X(1 ), ..., X(,_k) is a random ordered sample from the distribution with

406 LOSS OF INFORMATION IN THE DOUBLE EXPONENTIAL DISTRIBUTION 283

a density fo(x)/Fo(x(,-k+1)) on x < x(,_k+1), where Fo(x) is the double exponential distribution function of the Xi. Similarly, the last s - k are from fo(x)/[1- FO(x(•+k+1)] on x > x(,+k+1). Define u = Fo(X(,_k+l)), v = 1-Fo(X(,+k+1))+ wi = Fo(X(i)), zi = 1-Fo(X(i)). Then, ignoring the ordering of the first and last (s-k) X s, w1i ..., w,_k and z ,+k+2,..., x2s+i are random samples from uniform distributions on [0, u] and [0, v] respectively. Given Tk, each sgn(Xi - 0) is determined for s - k + 1 < i < s + k + 1, so we have

1

2s+1

V sgn(Xi - 9) I Tk

k)(V1 + V2),

i =1

where V1 = 1 - { fo sgn( x - 0) dw /u}2, which corresponds to the conditional variance of X for X < X(,_ k+1), V2 corresponds to that of X for X > X(,+k+1), and

w = Fo(x - 0) _

f

2e-Ix-91

l 1_e_I8I

for x < 0, for .T > 0.

Then we get V1 =

0 for X(,_ k+l) < 9, { (2u - 1)/u2 for X(,_ k+l) > 0, 0 for X(,+ k+1) > 0,

V2 = { (2v - 1)/v2 for X(s+k+1) < 9, and hence Lk = (s - k){E(V1) + E(V2)}. Symmetry now implies that

E(V1) E(V2)

(s (k) ^s +1 k)^ J 31(2 u2 1)us-k(1

- u)s+k du,

and from this result equation (2.2) follows as required. This result can also be expressed in the following form. Theorem 2.1. For each k = 0,1'. ..' s - 2, the loss of information Lk associated with Tk is given by

_ 2s 1 2, k+1 +^k`2(k-j+1)( 2s 12 2s Lk s-j) 2(2s+1) - (s)(2) s-k-1 0 s-k-1 (2.3)

407 284 MASAFUMI AKAHIRA AND KEI TAKEUCHI

Proof. The essence of the proof lies in recognizing the integrals in (2.2) as incomplete beta functions, which in turn are tail probabilities of binomial random variables. Specifically, using Y(n) to denote a symmetric binomial random variable with parameter n, i.e., Pr{Y(n) = p} _ (p) (2 )", we have 1

n!

(p - 1)!(n - p)!xp-1(1 - x)"_p dx = Pr{Y(n) < p - 1} fil11 for integers p = 1, ..., n. Then (2.2) gives

Lk = 2s Pr{Y(2s - 1) < s - k - 2} 2(2s .}, l) - 2 Pr{Y(2s) < s - k - 1} - s - k - 1 Using symmetry twice, this equals

-Pr{IY(2s)-sl
Since s Pr{Y(2s - 1) = i - 1} = i Pr{Y(2s) = i}, the sum in (2.4) equals k 2E(1- ) Pr{Y(2s) = s - j} . sk-1

j-o

Substitution in (2.4) completes the proof of (2.3). Remark 2 .1. For the case k = 0 , Theorem 2.1 yields Lo s+l 1 2d1 1[( )(s)(2) 2(2s+1) s-1 This expression is consistent with a result in Fisher (1934, p.300).

3. The Asymptotic Loss of Information Evaluating the three terms in Theorem 2.1 for fixed k and large s shows that asymptotically the loss of information is

^^ [1+0(1)] -4(k+1)+OI s). (3.1)

408 LOSS OF INFORMATION IN THE DOUBLE EXPONENTIAL DISTRIBUTION 285

But in order to improve this quantity we must increase k with s, and (3.1) suggests that we take k = r^/s + o(-4/s) with a non-negative constant r.

From Theorem 2.1 and the Cramer-Rao inequality it is follows that for any unbiased estimator 9 which is a function of Tk only, we have for fixed k Lk = n { 1 + -Lk + ° VB(B) I - IT ( n) }, A = n. ll

(3.2)

where II and IT,, denote the Fisher information at B and Tk respectively. ✓n + o( ^/n) for p = r1/2 > 0 Theorem 3 . 1. For k = rVs + o(Vs) = a passociated with Tk is given by and n = 2s + 1, the loss of information , Lk, Lk = 4 [O( p) - p1 l - 4, (p)}] ✓n + o(,/n),

(3.3)

where t (x) = f x. ¢(u ) du with 0( u) = e-ss'/Z/ 27. Proof. Using the normal approximation to the binomial distribution we have

Pr{Y(2s) = s} =

1 (1 ✓

8s + 0(s-2)) . (3.4)

The summation term in (2.3) equals 2 k - + 1 Pr{Y( 2s) = s + j} sk-1

k = 2(k + 1) > s

-

Pr{Y(2s ) = s + j} -

k - 1 i=o

11

k > j Pr{Y(2s) = s + j}

s -k-1 j=o

f P0(u)du -s r,Is 1

s-rls

2

J PuO( u)du+°\ ✓s

after approximation by Riemann integrals,

2(r✓s + 1 ) ^ P 0(u) du s - r%/s - 1 Jo

1 - e-r2

✓(2s ) + 0 1 (3.5) ,/(2r) s - r^/s - 1 (V/s )

By (2.3), ( 3.4) and (3.5) we obtain /' P Lk = 2(2s + 1){ -

✓s + Ts J

0(u) du +

✓( rs)e-r2 + °(

✓s)

= 4[4(p) - p{1 - gy(p)}] ✓n + of,/n), which completes the proof. We now consider the explicit form of estimators depending on Tk. As the simplest one, we put 9k = (X(s-k+1) + X(,+k+1))/2. The proof of the next result is in Section 4.

409

286 MASAFUMI AKAHIRA AND KEI TAKEUCHI

p.%/n+o( ✓n) where p = ro > 0 and n = 2s + 1, the estimator 9k has the stochastic expansion

Theorem 3.2. Fork = rVs+o(.v/s) _

✓n(Bk- 9)

=Z+ (Z-

PA Z -p1+(Z + PAZ + P 1

+ op (n

7' )

and asymptotic variance Ve(%/n9k) = 1 + cPn 1 (1 + 0(1)), (3.6) where Z = ^/n{Fo(X(.-k+l) - 9) + Fo(X(s+k+l) - 9) - 1} with Fo(.) the distribution function of X;, equivalently, Z = Jn(U(s-k+l) + U(s+k+l) - 1) with U(1), ..., U(n) the order statistics from the uniform distribution on the interval (0, 1), and cP=2[2-0(p)+p{21P (P)- i}].

(3.7)

Remark 3.1. Since ¢(x) > x{1- fi (x)} for x > 0, it follows for any p > 0 that cP > p. Define po by -11(po) = 3/4, so that po 0.67. Then c, has a minimum value at p = po. Hence 9k has minimum variance at p = po. Remark 3.2. In particular, for k = 0 i.e., p = 0, we obtain from Theorem 3.2, Ve(%/nio)=1+

( /. (3.8) 2)+0 n ✓ rn ✓

Note that 8o is the median, i.e., the MLE in this case. Remark 3 .3. Since

Ve(ek) = n-1

{1 + cvn

3[ 1 + 0(1)]},

it follows from (3.2) that c, > Lk/`/n. In this case we have from (3.3) and Theorem 3.2,

cP -

1 n Lk = p.

Corollary 3.1. For k = r%s+o(,/s) = 2 p✓n+o(%/n) where p = r%/2 > 0 and n = 2s + 1, the bound of the loss of information Lk associated with the estimator 6k is given by Lk < cP0%n + o(,/n), where po is defined in Remark 3.1.

Proof. From (3.2) we have V9(Bk) > (n - Lkwhich implies that Lk
+ o( ✓n)=cP✓n+ o( ✓n)•

410 LOSS OF INFORMATION IN THE DOUBLE EXPONENTIAL DISTRIBUTION 287

Therefore, by putting p = po we have Lk - c,. ✓n + o( ✓n) Remark 3.4. Since po 0.67, it is seen that

Lk < cPa ✓n + o( ✓n) = 1.27 ✓n + o( ✓n) .

(3.9)

On the other hand, it follows from (3.1) that the loss Lo of the MLE Bo, i.e., the median, is given by Lo = Lo = ^^

✓n+o( ✓n) = 1.60 ✓n+o( ✓n). (3.10)

From (3.9) and (3.10) we see that the estimator 9k is asymptotically better than the MLE Bo in the sense of Lk < Lo + o(/n). In Akahira & Takeuchi (1981, p.97) it is shown that the asymptotic distribution of 80 up to the order n-1/2 is given by Pr { ✓n(9o - 0) < t} = .(t ) - 2 n- Jt20( t)sgn t + o(n- )

= Fn(t) (say). (3.11) Then the asymptotic density fn(t) of eo is given by fn(t) = Fn(t) = ¢(t) - 1n-1(2t - t3)q5(t)sgnt+o(n-l). Hence the asymptotic variance of 60 is given by

Ve( ✓neo) = J_oot2mThtt__ 1 + ^() (1+0(1)) which is consistent with (3.8), and identically so when n = 2s + 1. Next we obtain the asymptotic distribution of 0k up to the order n-1/2. The proof is given in Section 4. Theorem 3 .3. For k = r✓s+o( ✓s) = 2 p ✓n+o( ✓n) where p = r'/2 > 0 and n = 2s + 1, the asymptotic distribution of Bk up to the order n-,j is given by

Pr { ✓n(Bk - 0) < t}

=4(t)-2^)[-pt+ {(t-p)It-pl+(t+p)It+pl}]+o(n-4). z

Remark 3 .5. Letting r = 0, we see that, for k = 0, the asymptotic distribution of the estimator 60 is consistent with that given by (3.11). From (3.11) and Theorem 3.3 we have the following (the proof is straightforward).

411

288 MASAFUMI AKAHIRA AND KEI TAKEUCHI

Corollary 3 .2. For estimators 9k and Bo, Pr{Vn(9o -9)
{Vn( 6k-9) p,

Pr{ /n(Oo-9)Pr{\/n(Bk-0) Pr { Jn(Bk -

0) < t} + o(1 /%/n) for 0 < t < p,

Pr {Vn( Bo -9)
for-p
Remark 3.6. From Corollary 3.2 we see that, for 0 < t < p, eo is asymptotically better than 8k in the sense of the concentration probability. That is, Pr {\/njBo

- 01

< t} > Pr

{'/nl9k

- 01

< t} + 0(1/\/n)

for 0 < t < p,

and, for t > p, Bk is asymptotically better than Bo in the same sense as the above, i.e.,

Pr {VnIBo -

91

< t} < Pr {\/nOk -

01

< t} + 0(1 /V/n)

for t > p.

Recently, Sugiura & Naing (1989) extended the above results to the other estimators using a weighted linear combination of the sample median and pairs of order statistics.

4. Proofs In this section we give the proofs of Theorems 3.2 and 3.3, based on the following lemmas.

Lemma 4 .1. Let 1'1,. .. , Y„ be i. i.d. random variables with an exponential density function e-11/B /0 for y > 0, (4.1) f(y,e) 0 fory<0, and let g(Y1 i ... , Yn) be homogeneous in Y1, ... , Yn of degree 0 so that g(KYi,••.,KYn) = g(Y1i...,Yn) for all non -zero K . Then g (Y1,...,Y,,) and E' , Yi are independent. Proof. It is clear that F,'1 Y; is a complete sufficient statistic for 0. Since the distribution of g(Y1 i ... , Yn) is independent of 0, then, by the theorem of Basu ( 1955), it is independent of E 1 Y.

412 LOSS OF INFORMATION IN THE DOUBLE EXPONENTIAL DISTRIBUTION 289

Lemma 4.2. Under the same conditions as Lemma 4.1, E[9(Yi ,...,Yn)] =

E [g(Yi , ..., Yn)(Ei_ 1 Yi)a) E[(F,ni= 1 Ys')a ]

for all real numbers a. The proof is straightforward from Lemma 4.1. Proof of Theorem 3.2. Without loss of generality , we assume that 0 = 0. Let Fo(x) be a distribution function of Xi (i = 1, . . ., n). Then we have X(i) = Fo 1(U(i)) (i = 1, ..., n), where the U(i) s are order statistics from the uniform distribution on the interval (0,1). We also obtain

Fo'(u) =-{ sgn(u-Z )}log(1-I2u-1I). We Put U = U(,_k+1) and V = U(,+k+l). Then we have n Bk = 1ni {F-1(U)+F-1(V)}

= n(U+V - 1) + n- 1 {ni(U - i)Ini(U - 2)I} +n_ i {n4 (V - 2)Ini(V I 2) }+op(n-i)• Putting X = n3 (U - Z ), Y = n (V - 2 ), we obtain n30k=X 1-Y+ n( X If 1'1,. • • , Yea+1 are i.i.d. random variables with the exponential density function as at (4.1) with 0 = 1, then it is known that j

2s+1

U(j)Yi/Yi i=1

i=1

for each j = 1, ... , 2s + 1. Using this fact we have, by Lemma 4.2, a-k+l 2a+1

2

1)2) = Ef L( \ Flt=1 Yi - ^.i=s+k+2 Yi l E[(U + V ^'`` a+ li ^ / J /^ i=1 s-k+l 2s+1 2 E[(Fi=1 Y. - Fi=s+k+1 Yi) E[( Ei=11

s-k+1 (s + 1)(2s + 1)

Yi)2]

413 290 MASAFUMI AKAHIRA AND KEI TAKEUCHI Hence we obtain

V (X + Y) = E[(X + Y)2] = E [{ ,/n (U + V -1)} 2] = s s + 1 1 = 1 - - + o( n-3). (4.3) Since E(X - Y) = E[ni(U - V)] = -p, it follows similarly that V(X - Y) = E[{\/n(U - V)}2] - p2 = n'4 (p + 0(1)) . We put

Z=X+Y;

W = s1"4 (X - Y + P).

Since X=

2(X +Y+X -Y) = 2(Z-P+s-1/4W);

Y = 2{(X + Y) - (X - Y)} = 12(Z + p we have from (4.2) niik

= Z+;n-4[(Z-P)IZ-PI +(Z+P)IZ+PI] +op(n'4). (4.4)

Hence

V(Vnek ) = E(Z2) + 2n-i{E[Z (Z - p)IZ-pI] +E[Z(Z+P)IZ+PI]} +o(n -3). (4.5) For any constant c we have

E[Z(Z - c)IZ - cI] = E[IZ - cI3] + cE[(Z - c)IZ - CI] . (4.6) We obtain

f00 00 00

I

00

I

X

- CI 3O( x) dx = 2( c2 + c)¢(c) + (c3 + 3c) {2-t(c) - 1}, (4.7) cIo(x) dx

= - 2c0(c) + (c2 + 1){1 - 20(c)} . (4.8)

414 LOSS OF INFORMATION IN THE DOUBLE EXPONENTIAL DISTRIBUTION 291

Since Z is asymptotically normally distributed with mean 0 and variance 1 - r//s = 1 - p/ Jn, we have from (4.6)-(4.8) that

E[Z(Z - c)IZ - cI] = 40( c) + 2c{2-P(c) -1} . Hence we obtain

E[Z(Z - P)IZ -

pI] = 40( p) + 2p{24^(p) - 1} . (4.9)

From (4.3), (4.5) and (4 .9) we have

V(n39k) = 1 + 2n-i [20 (p) + P{2t ( P) - Z }] + o(n- ) . The coefficient of n-3 here yields c,, as at (3.7 ), proving (3.6). Proof of Theorem 3.3. Assume without loss of generality that 9 = 0. From (4.4) we have that Pr{i/nOk < t} equals

Pr{Z+4n -3[(Z-P)IZ-pI+(Z+P)IZ+pl] < t+op(n A)} =Pr {Z
n-'[(t-P ) lt-PI +(t+P)lt+p1] +op (

n-')}.

We see that Z is asymptotically normally distributed with mean 0 and variance 1 + o(1/,1s) = 1 + o(1 /Vn). Hence we obtain Pr{/n9k
+2[(t- P)lt -PI

+ (t+P)It+PI] } +op(n-1)} _ fi(t) - Zn 0(t) (pt + sgn(t)[(ItI - P)+]Z)

+ o(n-7).

References AKAHIRA , M. & TAKEUCHI , K. (1981 ). Asymptotic Efficiency of Statistical Estimators: Concepts and Higher Order Asymptotic Efficiency. Lecture Notes in Statistics 7. New York: Springer-Verlag. BASU, D. (1955). On statistics independent of a complete sufficient statistic. Sankhya 15, 377-380. FISHER, R.A. (1922 ). On the mathematical foundations of theoretical statistics . Philos. Trans. Roy. Soc. London Ser. A 222, 309-368. FISHER, R.A. (1925). Theory of statistical estimation . Proc . Cambridge Philos. Soc. 22, 700-725. FISHER, R.A. (1934 ). Two new properties of mathematical likelihood . Proc . Roy. Soc. London Ser. A 144, 285-307. RAO, C . R. (1961 ). Asymptotic efficiency and limiting information . Proc . Fourth Berkeley Symp. Math . Statist. Probab. 1, 531-545. SUGIURA, N. & NAING, M.T. (1989 ). Improved estimators for the location of double exponential distribution . Comm . Statist. A-Theory Methods 18, 541-554. TAKEUCHI , K. & AKAHIRA , M. (1976 ). On the second order asymptotic efficiencies of estimators . In Proc. Third Japan-USSR Symp. Probab. Theory, eds. G. Maruyama and J .V. Prokhorov, 604-638. Lecture Notes in Mathematics 550. Berlin: SpringerVerlag.

415

Ann. Inst. Statist. Math. Vol. 43, No. 2, 297-310 (1991)

BOOTSTRAP METHOD AND EMPIRICAL PROCESS MASAFUMI AKAHIRA ' AND KEI TAKEUCHI2 'Institute of Mathematics, University of Tsukuba, Tsukuba, Ibaraki 305, Japan 2Research Center for Advanced Science and Technology, University of Tokyo, 4-6-1 Komaba, Meguro-ku, Tokyo 156, Japan

(Received August 24, 1989; revised February 22, 1990)

Abstract. In this paper we consider the sampling properties of the bootstrap process, that is, the empirical process obtained from a random sample of size n (with replacement) of a fixed sample of size n of a continuous distribution. The cumulants of the bootstrap process are given up to the order n-1 and their unbiased estimation is discussed. Furthermore, it is shown that the bootstrap process has an asymptotic minimax property for some class of distributions up to the order n-1/2_

Key words and phrases: Bootstrap process, cumulants, unbiased estimators, asymptotic minimax property.

1. Introduction The bootstrap method may be reviewed from different viewpoints . In this paper , we intend to consider the sampling properties of the bootstrap process, that is, the empirical process derived from the bootstrap sampling, i.e. that obtained from a random sample of size n (with replacement ) of a fixed sample of size n of a certain distribution . Let X1,..., Xn be a sample of size n from a population with the distribution function F(t) and let Xi , ... , Xn be a bootstrap sample of size n, that is, a random sample of size n f r o m X1, ... , Xn. Let the empirical distribution functions obtained from (X1,..., Xn) and (Xi , ... , X*) be denoted by Fn(t) and Fn* (t), respectively. It is well known that /h(Fn(t) - F(t)) approaches a Gaussian process as n --+ oo, and, given Fn(t), f (FF(t) - Fn( t)) conditionally approaches a Gaussian process with the same variance and covariance of / (Fn(t) - F(t)) with F(t) replaced by Fn(t). Hence V/'n-(F,(t) - Fn( t)) can be considered to be the consistent estimator of V/n-(Fn(t) - F(t)). Note that /(Fn(t) - F(t)) can not be usually observed since F(t) is unknown , whereas the distribution of V/n-(Fn* (t) - Fn (t)) can be completely computed from the sample. We further investigate how V/'n-(F,F(t) - Fn(t)) will differ from /(Fn(t) - F (t)) in higher order terms and we discuss possible improvements on F,, (t). In many problems of statistical inference, the procedures will depend on the distribution of a statistic Tn under an unknown distribution F(t), which in many 297

416 298 MASAFUMI AKAHIRA AND KEI TAKEUCHI

cases can be discussed in terms of V^n-(Fn (t) -F(t)), at least asymptotically. Hence v,'n-(Fn (t) - FF( t)) can be used instead of V/'n-(Fn(t) - F(t)) in the derivative of the asymptotic distribution of the statistic . It will be shown that the method is in a sense asymptotically efficient in a nonparametric (or semiparametric) framework. A first order approximation was considered in the work of Efron ( 1979 , 1982) and Beran (1982 ) considered a second order approximation from a different viewpoint. The purpose of this paper is to compute the cumulants up to the order n - 1 and to show that the bootstrap process is in a sense, asymptotically, the best estimator of the empirical process up to the order n-1/2, whereas in terms of the order n-1 there are many complications and although a slight improvement is possible over the usual bootstrap process, no uniformly optimal results seem to be obtainable. The bootstrap method is used to estimate the distribution of some statistic Tn under a general unknown population distribution and it is shown that it is asymptotically best up to the second order in the sense that the estimator of the asymptotic variance as well as that of the asymptotic distribution of Tn can not be uniformly improved if the class of possible population distributions is sufficiently wide.

2. Unbiased estimation of cumulants of the empirical process ,W(t) _

In the framework of Section 1, we put Wn (t) = /i(Fn(t)-F(t)) and J(F,F(t) - Fn (t)). Consequently we have the following. LEMMA 2 . 1. follows:

The cumulants of Wn(t) are given, up to the fourth order, as

E[ Wn(t)] = 0, Cov(Wn(t l ), Wn(t2)) =F(t1)(1 - F(t2)) for t1

t2,

,c3(Wn(tl), Wn(t2), Wn ( t3))

=(1/vrn-)

(t1)(1 - 2F(t2))(1 - F(t3))

for t1 < t2 <

t3,

/( F k4(Wn(tl), Wn(t2), Wn(t3), Wn(t4))

_ (1/n)F(ti)(1 - F(t4))(1 - 4F(t2) - 2F(t3) + 6F(t2)F(t3)) for

t

4

-

The T h e proof is given in Section 4, but Lemma 2.1 may be also derived from Lemma 3.1 of Withers (1983). From Lemma 2.1 we have the following. LEMMA 2 .2. Given Fn( t), the conditional cumulants of WW(t) are given, up to the fourth order, as follows: E[W,(t) I F'n(t)] /_ 0, COV(Wn(tl), Wn

(t2)

I Fn( tl), F'n(t2)) = Fn (t l) (1 - Fn(t2)) for

tl < t2,

ic3(Wn(tl), Wn(t2), Wn(t3) I Fn(t1), Fn(t2), Fn(t3))

= (1/N/n)Fn(t1)(1 - 2Fn(t2))(1- Fn(t3)) for t1 < t2 < t3,

417

BOOTSTRAP METHOD AND EMPIRICAL PROCESS 299 #c4(Wn(tl), Wn(t2), Wn(t3), Wn( t4) I Fn ( tl), Fn (t2 ), F.(6), Fn(t4))

_ (1/n)Fn(t1)(1 - Fn(t4))(1 - 4Fn(t2) - 2Fn(t3) + 6Fn( t2)F.(t3)) for t1 < t2
E[Fn(t1)(1 - Fn(t2))] = {1 - (1/n)}F(tl)(1 - F(t2)) for t1 < t2, E[F.(t1)(1 - 2Fn(t2))(1 - Fn(t3))] _ {1 - (1/n)}{1 - (2/n)}F(tl)(1 - 2F(t2))(1 - F(t3)) for t1 < t2 < t3i E[Fn(t1)(1 - Fn(t4))(1 - 4Fn(t2) - 2Fn(t3) + 6Fn(t2)Fn(t3))] _ {1 - (1/n)}{1 - (2/n)}{1 - (3 /n)}F(tl)(1 - F(t4)) (1 - 4F (t2) - 2F (t3) + 6F(t2)F(t3)) - (1/n){1 - (1 /n)}F(tl)(1 - F(t4)) for t1 < t2 < t3 < t4. The proof is given in Section 4. From Lemmas 2.2 and 2.3 it is seen that, given Fn (t), the conditional cumulants of Wn (t) are not unbiased estimators of the corresponding cumulants of Wn(t). LEMMA 2.4. The ( unconditional ) cumulants of ,W(t) are given, up to the fourth order, as follows:

E[WW (t)] = 0, Cov(W, (tl), W, (t2)) = {1- (1 /n) }F(tl )( 1 - F(t2 ))

for

ti

<

t2,

i3(Wn ( t1), Wn ( t2), Wn(t3))

_ (1/i){1 - (1/n)}{1 - (2/n)}F(tl)(1 - 2F(t2 ))( 1 - F(t3)) for t1 < t2 < t3i k4(W, (tl), W, (t2), W,(t3), W,(t4))

_ (1/n){1 - (1/n)}{1 - (2/n)}{1 - (3/n)}F (tl)(1 - F(t4)) (1 - 4F(t2) - 2F(t3 ) + 6F(t2 ) F(t3)) - (1/n2){1 - ( 1/n)}F(tl )( 1 - F(t4))

+ (1/n)F(tl )( 1 - F(t4))(3 - 8F(t2) - 4F(t3 ) + 12F(t2 )F(t3)) - (2/n2)F (tl)(1 - F(t4))(3 - 1OF (t2) - 5F(t3) + 15F (t2)F(t3)) + ( 3/n3)F (t1)(1 - F (t4))(1 - 4F(t2) - 2F (t3) + 6F (t2)F(t3)) + o(1/n3) for

tl
The proof is given in Section 4. From Lemmas 2.1, 2.2 and 2.3 we also have the following.

418 300 MASAFUMI AKAHIRA AND KEI TAKEUCHI

THEOREM 2 .1. The unbiased estimators of the covariance Cov(Wn(tl), Wn(t2)) and the third order cumulant Ic3 (Wn(tl), Wn(t2), Wn(t3)) are given by {n/(n = 1) } Cov(W,(tl), W.* (t2) I F. (t1), Fn (t2)) = {1/(n - 1)}Fn(tl)(1- Fn(t2)) for t1 < {n2/(n - 1 ) (n - 2) }k3 ( Wn(tl), W„ (t2),

W.* (t3) I

Fn (4),

Fn (t 2

t2,

), Fn (6))

_ {n./(n - 1)(n - 2)}Fn(t1)(1 - 2Fn(t2))(1 - Fn(t3)) for t1 < t2 < t3i respectively. PROOF . From Lemmas 2.1, 2.2 and 2 . 3 we have for tl < t2 E[Cov (Wn(tl ), Wn(t2) I Fn (tl), Fn(t2))] = E[Fn ( t1)(1 - Fn(t2))]

= {1 - (1/n)}F(tl)(1 - F(t2)) = {(n - 1)/n} Cov(Wn(tl), Wn(t2)), hence {n/(n-1)}Cov(WW(tl), Wn(t2) I Fn(tl), Fn(t2)) = {n/(n-1)}Fn(t1)(1-Fn(t2)) is an unbiased estimator of Cov(Wn(ti), Wn(t2)). In a similar way we obtain for tl < t2 < t3

E[ic3(Wn (t l),

Wn(t2 ), Wn( t3 ) I F,-(t1 ), Fn(t2), Fn (

t3))]

= E[(1/ /) Fn(tl )( 1- 2Fn (t2))(1- Fn(t3))] = (1/'){(n - 1)(n - 2)/n2}F (tl)(1 - 2F (t2))(1 - F(t3)) = {(n - 1)(n - 2)/n2}k3(Wn(tl ), Wn(t2 ), Wn(t3)),

hence {n2/(n - 1 )(n - 2)}ic3 ( Wn(tl ), Wn(t2 ), Wn(t3 ) I Fn(t1 ), Fn(t2), Fn(t3)) = {n^/-n-/(n - 1)(n - 2)}Fn (t1)(1 - 2Fn (t2))(1 - Fn(t3)) is an unbiased estimator of ic3 (Wn(tl), Wn(t2), Wn(t3)). Thus we complete the proof. Remark 2 .1. Let Xi, ... , Xn_1 be a bootstrap sample of size n -1, that is, a random sample of size n - 1 from X 1 ,..., Xn. We put Wn_1(t) = /i(Fn-1(t) -

Fn(t)) with the empirical distribution Fn* -1(t) of X,..., Xn_1. Then it follows from Lemmas 2.1, 2.2 , 2.3 and Theorem 2.1 that Cov(Wn_1(tl),

Wn -1

(t2) I Fn(tl), Fn( t2)) = {n/(n -1)}Fn(t1)(1- Fn(t2))

is also an unbiased estimator of Cov(Wn(tl), Wn(t2)), but Cov(WW(tl), Wn(t2) I Fn(t1), Fn(t2)) is not unbiased for it. Hence it is desirable to use the bootstrap sample of size n-1 in place of size n . And also the biases of higher order cumulants become smaller.

419

BOOTSTRAP METHOD AND EMPIRICAL PROCESS 301

3. Minimax property of the bootstrap estimator In this section we consider the estimation problem based on the i.i.d. sample X1,..., X,,, on some real parameter 9 which can be defined as a functional 9 = 'Y(F) of a continuous distribution F. Then the natural estimator is 9n = XF(Fn), where F,,, is the empirical distribution function. We shall show that the bootstrap estimator of the distribution of 9n has a minimax property for some parametric family of distributions. We assume the following condition.

(A.1) The functional ' is Frechet differentiable up to the third order, that is, there are functions 8W/oF, 82W/OFBF and 83W/8FOFOF such that (3.1)

W (G) - T (F) =

- F) J 00 (aW /8F)d(G 00 W + (1/2) (G - F)d(G - F) 1-0000 1- 00 + (1/6) J J : 1_oo (83I /8F8F'0F) • dF(G - F ) d(G - F)d(G - F)

+ o(IIG - F'II3), where JIG - FII = sup., IG(x) - F(x)I. Putting WW,,(x) = ,fn-(Fn(x) - F( x)), we have from (3.1) (3.2)

/(9n - 0) 00 O1( x)dWn(x) 00 00 + (1/2vlrn-)-00 r f-00 02(x, y)dWn(x)dWn(y) + (1/6n) J 00 rM roc 03(X, y, z)dWn(x)dWn(y)dWn(z) + op(1/n),

where q51(x) = (8W/8F)(x), 02(x, y) = (82W /OFBF )(x, y) and 03(x, y, z) (OW3/OFOFOF)(x, y, z). We also assume that the following holds.

(A.2) fcbi(x)dF(x) = 0, fcb2(x, t)dF(t) = 0, f=,y)dF(s) 2(s f 03(x, y, u)dF(u) = f 03(x, t, z)dF(t) = f 03(s, y, z)dF( s) = 0, and the functions (62(x, y) and q3(x, y, z) are symmetric in (x, y) and (x, y, z), respectively. Furthermore, using Tn = V/-n-(9n - 0), we assume the following condition. (A.3)

E(TT) < oc.

LEMMA 3.1. Assume that the conditions (A.1), (A.2) and (A.3) hold. Then the asymptotic cumulants of Tn are given as follows.

420 302 MASAFUMI AKAHIRA AND KEI TAKEUCHI

. E(Tn) = (1/2^){fw + o(1/n)

02(x, x)dF(x) - : F 02(x, y) dF(x)dF(y)} oo

= (1/v/'n-)bl + o( 1/n) (say), oo

V(Tn ) =

f

+

oooo

q51

2

i(x)dF(x) }

02(x)dF(x ) - {f f 00 1 ^ 00.

( x)c2(x, x)dF(x)

- 3 f FOO 01(x)02(x, y)dF(x)dF(y) +2I f "Oc51( x)dF(x )) f°° f00 q2( x, y)dF (x)dF(y)} 00 . 00 }z m + (1/2n){ f 02(x, x)dF(x) - f f 00 02(x, y)dF(x) dF(y) l ocoo

00 )

00

y , y)dF(x)dF(y) + (1/n){

0 00

- 2 f. La 00 00

00

o L ao

O1(x)(a3(X,

y, z)dF (x)dF(y)dF(z)

+ f f f f 01(x)03( y, z, u) • dF(x)dF(y)dF(z)dF(u) } + o(1/n) vo + (1/n)vl + o(1/ n) (say), J

ic3(Tn) = E[{Tn - E(Tn)}3]

{

(1/vfn-) f

0i (x)dF(x ) - 3

(

f - 02 (x)dF(x )) (f ^ 01(x) dF(x))

+2 (f lo 01(x)dF(x)) 3

(

+ 3 f l 01(x)dF(x)) j - 3( j 02(x)dF(x

2(X , x)dF(x)

)) f f 00 2

cb2 (x, y)dF (x)dF(y)

+3 1 f &1(x)dF(x )) f. f 0 02 (x, y)dF(x)dF(y) + (3/2) (

c .

01 (x)dF(x ))2 f2( x , x)dF(x)

+ o(1/n), = (1//)/33 + 0 (1 /n) (say),

rc4(Tn) = E[{Tn - E(Tn)}4] - 3{V(Tn)}2 = 0(1/n).

421

BOOTSTRAP METHOD AND EMPIRICAL PROCESS 303

The proof is omitted since Lemma 3. 1 is similar to Theorem 3.1 of Withers (1983). Remark 3 .1. With the condition (A.3) and from the fact that there exists a finite positive constant c such that P(sup ^I Fn(x) - F(x)I > r I < ce-2r2

(Dvoretzky et al. (1956)),

lX

holds for all r > 0 and all positive integers n, it follows that the above expansion of the remainder term is valid. For an estimator 6* based on the bootstrap sample X*,..., Xn* of size n, we put T,n = Ji(6* - 6). LEMMA 3 .2. Assume that the conditions (A.1), (A.2) and (A.3) hold. Then the conditional asymptotic cumulants of Tn, given the empirical distribution function Fn, have the following form. E[Tn I Fn] _ (1/vrn-)bl + (1/n)ei + op(1/n),

V(T, I Fn) = vo + (1/01;2 + (1/n)vl + op(1/n), ic3(Tn

IF

n) = (

l£4(Tn I Fn) =

1 //)/33

/c4(Tn)

+ (1/n )S3 + op(1 /n),

+ op(1/n),

where ^1 = Op(1), 6 = Op(1), 6 = Op( 1), and b1, vo, v1 and 03 are constants given in Lemma 3.1. The proof is given in Section 4. Remark 3 . 2. In order to evaluate the bootstrap estimator 9*, it is seen from Lemmas 3 . 1 and 3 . 2 that the variance of e2 = /i(V(TT I Fn)-V(Tn ))+op(1/J) plays an important part. LEMMA 3.3. is given by

Under the conditions (A.1), (A.2) and (A.3), the variance of ^2

^

V(2) = (x){ f

2

(x) - 2m}2dF(x) - { f0(x ) dF(x) - 2m2 } ,

where m = f 01(x )dF(x). The proof is given in Section 4. Now we consider a parametric family F = {Fe:0 E O} of distribution functions, where O is an open set of R1 involving the origin. Take Feo as the previous distribution function F. We assume that, for each 0 E 6, the distribution function Fe is absolutely continuous with respect to a o-finite measure p, and denote dFe(x)/dp(x) by fo(x). For each 0 E ®, we put

00 ve = f 00

12 0i(x)fe(x)4 - S F O1(x)fe(x)dµ } .

l

JJJ

422 304 MASAFUMI AKAHIRA AND KEI TAKEUCHI

Since 8n = IQ (F,,) is an asymptotically unbiased estimator of 8, we have by Taylor's expansion of vo around 0 = Bo vB,. = voo + [ave/a8]0=00(8n - 8o ) + o(1/'), hence the variance of ven is given by Veo (vb.) = ([avo/88 ] o=e0 )2 Voo ( 8n) + o(1/n). Assume that the Fisher information amount 1(0) exists, i.e.

0 < I(8) =

f

{81og fo x)/86}2 fex)d p < oo, 00

then we have by Cramer-Rao's inequality that (3.3)

nVoo(ve,.)

>- ([avo/06]e= 00)2/I(0) +o(1),

provided that the differentiation under the integral sign is allowed. We further restrict our attention to a family of subclasses .F, = {Fo: dFo(x)/dµ = fo (x) with the form log (fo(x)/foo (x)) = c(8) + 0&i (x) a.e. [p] with c(0) = 0} of T, where b(x) is a function with finite variance at fo0. Then we have the following. THEOREM 3.1. Assume that the conditions (A.1), (A.2) and (A.3) hold. Then the bootstrap estimator 8* has a minimax property in the above family, i.e. max min nVoo (ven) = nVoo (vb.) + o(1), YO

bn

provided that the differentiation under the integral sign is allowed. The proof is given in Section 4. Remark 3.3. From Theorem 3.1 we see that the maximum of relative efficiency of the bootstrap estimator 8n is equal to 1 + o(1), i.e.

max 1( ein nVoo (ven ) Y,P

1 /nV0o(v.) ]

= 1+0(1)-

It also follows from Theorem 3.1 that in a semiparametric situation where the class of distributions is sufficiently wide to include FP, it is impossible to get an estimator with a smaller asymptotic variance than ve,.

423 BOOTSTRAP METHOD AND EMPIRICAL PROCESS 305

4. Proofs

In this section the proofs of lemmas and theorems are given. In order to prove Lemma 2.1 we have the following. LEMMA 4. 1. Let Z be a real random variable. Assume that, for each i = 1, 2, 3, 4 , Yi = 1 for Z < ci,, Yi = 0 for Z > ci, where cl < c2 < c3 < c4. Then i3 (Y1 ,

Y2, Y3) = E[(Y1 - p1)( Y2

- P2 )(Y3 -P3 )]

= Pi(1 - 2p2 )( 1 -M,

#c 4 (Yi, Y2, Y3, Y4) = E[(Yi -P1)( Y2 -p2 )(Y3 -p3 )(Y4 -p4 )] - Cov(YI, Y2) Cov(Y3i Y4) - Cov(YI, Y3) Cov(Y2, Y4) - Cov(YI, Y4) Cov(Y2, Y3) = pi(1 - p4)(1 - 4P2 - 2P3 + 6p2p3), where for each i = 1, 2, 3 , 4, pi = P{Z < ci} and Cov (•, •) denotes the covariance. PROOF . It is seen that pi < P2 < P3 < p4 . Since E (Y2) = pi (i = 1, 2, 3), E(Y1Y2) = E(YI) = pi, E(Y2Y3) = E(Y2) = P2, E(YIY2Y3) = E(YI) = PI, it follows that

r-3 (Y1, Y2, Y3) = E[(Y1 -p1)(Y2 -P2)(Y3 -P3)] = E(Y1Y2Y3) - p1E(Y2Y3) - p2E(Y1Y3) - p3E(Y1Y2) + 2p1P2P3 = Pi (1 - 2p2)(1 - P3)• In a similar way, we have

E[(Y1 -PI)(Y2 - p2 )(Y3

-p3)(Y4 -p4 )]

= {pl(1 - 2p2)(1 - P3) + plp2p3}(1 - p4), Cov(Yi, Yj) = pi (1 - pj) (1 < i < j < 4).

Hence we obtain r-4 (Y1, Y2, Y3, Y4) = E[(Y1 -pl)(Y2 - p2)(Y3 -p3 )(Y4 -P4)]

- Cov(YI, Y2) Cov(Y3i Y4) - Cov(Y1i Y3) Cov(Y2, Y4) - Cov(Y1i Y4) Cov(Y2i Y3) = Pi(1 - P4)(1 - 4P2 - 2P3 + 6P2P3)• Thus we complete the proof. PROOF OF LEMMA 2.1. Since WW,,(t) _ / (Fn(t) - F(t)), it is easily seen that E [Wn(t)] = 0 and Cov ( Wn(t1 ), Wn(t2)) = F (tl)(1-F (t2)) for t1 < t2. From Lemma 4.1 we have ic3(Wn ( tl), Wn ( t2), Wn (t3)) = (1/f) F(t1)(1 - 2F(t2 ))( 1 - F(t3)) for t1 < t2 < t3i k4( Wn( tl

), Wn(t2), Wn(t3), Wn(t4))

_ (1/n)F(ti)(1 - F(t4))(1 - 4F(t2) - 2F(t3) + 6F(t2)F(t3)) for

t1 < t2 < t3 < t4.

424 306 MASAFUMI AKAHIRA AND KEI TAKEUCHI This completes the proof.

In order to prove Lemma 2.3 we have the following. LEMMA 4 . 2. Suppose that , for each i = 1, 2, 3 , 4, Yi is a real random variable with mean E (YY) = mi. Then E[YI(1 - Y2)] = MI (1 - m2) - a12,

E[YI(1 - 2Y2)(1 - Y3)] = m1 - m1m3 - 2m1m2 + 2m1m2m3 - a13 - 2a12 + 2m10'23 + 2m2913 + 2m3o'12 + 2K123, E[YI(1 - Y4) (1 - 4Y2 - 2Y3 +6Y2Y3)] = m1 - 4(o*12 + m1m2) - 2(0'13 + m1m3 ) - ( a14 + m1m4) + 6(k123 + m10`23 + m20`13 + m3(712 + m1m2m3) + 4(IC124 + m1024 + m20`14 + m4a12 + m1m2m4) + 2(/134 + m10`34 + m3cr14 + m4a13 + m1m3m4) - 6(1234 + m41123 + m36124 + m2tc134 + 7121 /234 + m1m40'23 + m2m40`13 + m3m40`12 + m1m3a24 + m2m30`14 + m1m20r34 + m1m2m3m4 + 0` 120'34 + 0`13o24 + a14923),

where, for 1 < i < j < k < r < 4, Uij = Cov(Y, Yj), Icijk = Ic3(Yi, Yj, Yk) and kijkr = K4(Yi, Yj, Yk, Yr).

PROOF. The first one is easily derived. Since E(Y1Y2Y3) = K123 + m10-23 + m20`13 + m3cr12 + m1m2m3, it follows that E[Y1(1 - 2Y2)(1 - Y3)] = E(Y1) - E(Y1Y3) - 2E(YIY2) + 2E(Y1Y2Y3) = m1 - m1m3 - 2m1m2 + 2m1m2m3 - a13 - 2a12 + 2m10'23 + 2m20-13 + 2m3912 + 2K123•

Since E[Y1Y2Y3Y4] _ K1234 + 0` 120'34 + a13o24 + 0*14o23 + m4K123 + m3K124 + m2rv134 + m1ic234 + m1m4a23 + m2m4a13 + m3m41712 + m1m3a24 + m2m30*14 + m1m20`34 + m1m2m3m4,

it follows that E[YI(1 - Y4)(1 - 4Y2 - 2Y3 + 6Y2Y3)] = ml - 4E(Y1Y2) - 2E(Y1Y3) - E(Y1Y4) + 6E(YIY2Y3) + 4E(Y1Y2Y4) + 2E(YIY3Y4) - 6E(Y1Y2Y3Y4),

425 BOOTSTRAP METHOD AND EMPIRICAL PROCESS 307

hence the desired result follows. PROOF OF LEMMA 2.3. For i = 1, 2, 3, 4, we put Y = Fn,(ti ) and mi = F(ti) = E[FF,(ti )]. Then we have Qij = (1/n)mi(1 - mj ), rcijk = (1/n / )mi(1 2mj)(1 - mk ), lcijkr = ( 1/n2)mi ( 1 - mr)(1 - 4mj - 2mk + 6mjmk) for 1 < i < j < k < r < 4 . From Lemmas 4.1 and 4.2 we have the conclusion of Lemma 2.3 . PROOF OF LEMMA 2.4. From Lemmas 2.2 and 2.3 it follows that

E[W,(t)] = E[E[W,(t) I Fn(t)]] = 0, Cov(Wn*(t1), Wn(t2)) = E[Cov(Wn*(tl), Wn*,(t2) I Fn(t1), Fn(t2))] = {1 - (1/n) }F(tl)(1 - F(t2)) for t1 < n k3(Wn (tl), W (t 2), Wn (t3))

t2,

= E[i3(Wn(t1), Wn(t2), Wn(t3) I Fn(t1), Fn(t2), Fn(t3))] (1/n)}{1 - (2/n )}F(t1)(1 - 2F(t2))(1 - F(t3))

for

tl < t2 < t3.

Fn( t

3 ),

In a similar way, we have i4(Wn(tl), Wn(t2), Wn(t3), Wn(t4)) =

E[k4(W,(tl), Wn( t2), Wn (t3 ), Wn(t4) I Fn(tl), Fn (t2), +COV(COV(Wn*,(t1), Wn* (t2) I Fn (6), Fn (t2)) , COV(W,n*,(t3), Wn(t4 ) I Fn(t3 ) ,

+Cov(Cov(WW*,(tl), Wn*,(t3) I

Fn (t4))]

Fn (t4)))

Fn(tl), Fn (t3)),

C0V(Wn*(t2), W, (t4) I Fln(t2),

Fn( t4)))

+ Cov(Cov(Wn*(t1), Wn*(t4) I Fn(tl), Fn(t4)), COV(Wn*,(t2), Wn(t3) Fn( t2), Fn(t3))) = E[k4(Wn(t1), Wn(t2), Wn*,(t3), Wn(t4) I Fn(tl), Fn(t2), Fn(t3) , Fn(t4))] + 74 (say).

Since, by Lemma 2.2,

Cov(Wn*(ti), Wn*

I Fn(ti), Fn(tj)) = Fn(ti)(1 - Fn (t i))

for

it follows that

CoV(CoV(Wn*(t1), Wn*,(t2)

I

Fn (t1),

Fn(t2)),

COV(Wn*, (t3), Wn (t4) I Fn(t3), Fn(t4)))

= Cov(Fn(tl)(1 - Fn(t2)), F'n(t3)(1 - Fn(t4))) = E[Fn(tl)(1 - Fn(t2))Fn(t3)(1 - Fn(t4))]

- E[Fn(tl)(1 - Fn(t2))]E[Fn(t3)(1 - Fn(t4))]•

ti :5 tj,

426

308 M