Censored partial regression

Biostatistics (2003), 4, 1, pp. 109–121 Printed in Great Britain

Censored partial regression ´ ∗ ´ NEZ-ANT ˜ JESUS ORBE, EVA FERREIRA, VICENTE NU ON Departamento de Econometr´ıa y Estad´ıstica, Facultad de Ciencias Econ´omicas y Empresariales, Universidad del Pa´ıs Vasco-Euskal Herriko Unibertsitatea, Avda. Lehendakari Agirre 83, E-48015 Bilbao, Spain [email protected] S UMMARY In this work we study the effect of several covariates on a censored response variable with unknown probability distribution. A semiparametric model is proposed to consider situations where the functional form of the effect of one or more covariates is unknown, as is the case in the application presented in this work. We provide its estimation procedure and, in addition, a bootstrap technique to make inference on the parameters. A simulation study has been carried out to show the good performance of the proposed estimation process and to analyse the effect of the censorship. Finally, we present the results when the methodology is applied to AIDS diagnosed patients. Keywords: Bootstrap; Censorship; Duration models; Kaplan–Meier; Nonparametric estimation.

1. I NTRODUCTION In this paper we propose a new methodology to estimate and make inference in the context of a censored partial regression model. We have a dataset that contains information about the survival time for AIDS-diagnosed patients and some covariates. Our aim is to analyse the effect of these covariates on the patients’ survival time starting from the diagnosis time of the illness up to death or up to the end of the study; that is, there is censorship in the sample. The covariates are: sex, age at diagnosis, transmission category, disease at AIDS diagnosis and the period of diagnosis; the latter is used to capture the effect of the introduction of the AZT treatment (started in the middle of 1987). In a previous study, Orbe et al. (1996) used a dummy variable to capture this effect, a quite restrictive specification, since it seems more realistic to assume a more gradual effect. This motivates the proposal of a censored partial regression model, introducing all of the covariates in a parametric and linear form, except for the period of diagnosis, which is introduced through a nonparametric term. As is well known, the traditional methods applied in standard problems in Statistics cannot be used for censored data. There are two main classes of regression models that analyse the dependence between the duration and the covariates: the proportional hazards models (Cox, 1972) and the accelerated failure time models (see e.g. Lawless (1982)). The effect of the covariates in the Cox model is multiplicative on the baseline hazard. The advantage of this model, and the main reason for its extensive use, lies in the possibility to estimate the parameters of interest without any assumption on the distribution of the duration variable. However, the assumption of a proportional hazard function for different individuals is very restrictive, and, in some cases, this proportionality is not verified by the data. ∗ To whom correspondence should be addressed

c Oxford University Press (2003) 

110

J. O RBE ET AL.

On the other hand, in the accelerated failure time models, the survival time (or a monotonic transformation) is modelled linearly in the covariates. Usually, the estimation of these models is carried out assuming a distribution for the duration. However, some authors have developed methods to avoid the need for this assumption. Miller (1976); Koul et al. (1981); Buckley (1979), among others, proposed the first estimation procedures in this direction. In the last decade, Ritov (1990); Tsiatis (1990); Wei et al. (1990); Jones (1997); Lin et al. (1998); Gray (2000) and Jin et al. (2001) used rank tests to estimate the parameters within this class of models. Stute (1993) proposed an estimation procedure based on weighted least squares, using the Kaplan–Meier weights. All of these proposals are designed for models where the effect of the covariates is specified in a parametric manner. To allow for more flexible situations, Gray (1992) and Hastie and Tibshirani (1993) extended the Cox model to allow for nonparametric effects of some covariates on the hazard functions. Kooperberg et al. (1995) proposed the HARE (HAzard REgression) models, where linear splines and selected tensor products are used to estimate the logarithm of the conditional hazard function, which has also been studied by Intrator and Kooperberg (1995). Leblanc and Crowley (1999) did a similar work but they modelled the relative hazard function instead. Since we are interested in modelling the direct effect of the covariates on the duration rather than on the hazard function, our work fits in the accelerated failure time models framework, which is quite appealing to medical investigators due to its ease of interpretation. Moreover and to our knowledge, there is no previous work dealing with semiparametric estimators for these models. The rest of the paper is outlined as follows. In Section 2, we present the censored partial regression model and provide the details for the estimation procedure. Section 3 describes a method that uses bootstrap resample techniques to make inference. Section 4 provides some simulation results that support the adequacy of the methodology to different situations. Section 5 presents the results of the application to AIDS-diagnosed patients and, finally, Section 6 presents the conclusions.

2. A CENSORED PARTIAL REGRESSION MODEL The proposal we put forward in this section extends the linear censored regression model, allowing for a possible nonparametric specification for one or more covariates. Stute (1993) presents a new methodology for a regression model with censored data that allows for either fixed or random covariables and requires weak assumptions for the probability distributions involved. Moreover, the estimators can be easily obtained using weighted least squares. In order to explain the method, let us assume that T1 , . . . , Tn are independent observations from some unknown distribution function F. Due to the censored data, we observe Yi = min(Ti , Ci ), where C1 , . . . , Cn denote the values for the censoring variable C. Let δi = I (Ti
Ci ) be the failure indicator. In addition, X i represents the k-order vector of covariates for the ith individual. This model requires the following identifiability conditions: (H.1) T and C are independent, and (H.2) P(Ti
Ci |Ti , X i ) = P(Ti
Ci |Ti ), for i = 1, . . . , n. Hypothesis (H.1) is a standard assumption in random censorship models and (H.2) indicates that, given the duration, the covariables do not provide any further information as to whether the observation is censored or not, a weaker assumption than the independence between C and X . The relation between the covariates and the logarithm of the duration is modelled as ln Ti = X i β + i , with E[i |X i ] = 0. The logarithm can be replaced, without loss of generality, by any other monotone transformation.
n The estimator of β can be obtained by weighted least squares, minimizing i=1 Win [ln Y(i) − X [i] β]2 , where ln Y(i) is the ith ordered value of the observed response variable ln Y , X [i] is the concomitant associated with ln Y(i) , and Win are the Kaplan–Meier weights. In a compact notation, the solution can be written as βˆ = (X T W X )−1 X T W ln Y, where ln Y = (ln Y(1) , . . . , ln Y(n) )T , W is a diagonal matrix

Censored partial regression

111

formed with the Kaplan–Meier weights and X = [X 1T , X 2T , . . . , X nT ]T is the matrix of covariates. Stute (1993) studied the consistency of this estimator and Stute (1996) its asymptotic normality. We now propose a generalization of the censored linear model to a censored partial linear model. That is, we will introduce a nonparametric component, where we do not specify any functional form for the effect of one or more covariates. In our application, the period of diagnosis (R) will be the variable introduced in a nonparametric form. Hence, the model can now be written as ln Ti = X i β + h(Ri ) + i .

(1)

The partial linear model (1) has been studied when the dependent variable is not censored. Heckman (1986) and Rice (1986) studied the case where h(·) is estimated using splines, and Speckman (1988) estimated h(·) using kernels. In this work we propose a method to fit (1) when the dependent variable is at risk of being censored. In order to fit model (1), we have to consider two main issues. On the one hand, the goodness of fit and, on the other hand, the smoothness of the proposed function to model the effect of the covariate included in the nonparametric component. As for the goodness of fit, this is controlled through the sum of the weighted squared residuals using the Kaplan–Meier weights, that
account for the existence of censored n observations in the sample. This term corresponds to the equation i=1 Win [ln Y(i) − X [i] β − h(R[i] )]2 , T that can be written in matrix form as (ln Y − Xβ − N h) W (ln Y − Xβ − N h), where X , W , and ln Y are as above. The d-order vector h contains the values h j = h(S j ) for the d distinct ordered values S1 < · · · < Sd , of the covariate R. The connection between S1 , . . . , Sd and R1 , . . . , Rn is given through the n × d incidence matrix N which assigns the respective value of the covariate R to the ith individual; that is, Ni j = 1 if Ri = S j , and zero otherwise. As for the smoothness, we measure it in the usual way using the integral of the square of second derivatives. Hence, the combination of both terms can be handled by minimizing the penalized weighted least squares expression  (ln Y − Xβ − N h)T W (ln Y − Xβ − N h) + α

[h ” (r )]2 dr,

(2)

where the solution for the function h(·) is searched in the space of functions that are twice differentiable over the domain of R and have absolutely continuous first derivative. The degree of smoothness is determined by α, the smoothing parameter. Large values of α produce smoother curves, while smaller values produce more wiggly curves. Given α > 0, the solution that minimizes (2) is a smoothing spline function with knots at S1 , . . . , Sd , which is a mathematical consequence of the choice of cubic [h ” (r )]2 dr as the penalty term. As α approaches to zero, the penalty term loses importance and the solution tends to the cubic spline that interpolates the data; that is, such that h(R[i] ) = ln Y(i) − X [i] β or the average if we have tieds. However, when α is large enough, the penalty term dominates and, thus, we obtain the weighted least squares solution to a linear model; that is, the solution will provide a linear function for h(·). In practice, the value of α can be selected using a data-driven criterion. The usual one is cross-validation and this will be the criterion we will use in the empirical analysis. Using the properties of cubic splines the integral term can be computed as αh T K h, where K is a square matrix of order d, that only depends on the location of the knots. Further details on splines and how to calculate the K matrix can be found in Green and Silverman (1994, chapter 2). Thus, the minimization problem can be rewritten as (ln Y − Xβ − N h)T W (ln Y − Xβ − N h) + αh T K h.

(3)

112

J. O RBE ET AL.

Taking derivatives in (3) with respect to β and h, and reordering the terms leads us to the following pair of simultaneous matrix equations: (N T W N

X T W Xβ + α K )h

= =

X T W (ln Y − N h) N T W (ln Y − Xβ)

(a) (b).

(4)

The uniqueness of the solution to (4) is given by the fact that (i) W is diagonal and with non-negative elements and, therefore, it is a symmetric non-negative definite matrix; (ii) the columns of X are linearly independent and (iii) there is no linear combination Xβ equal to a linear form δ1 + δ2 R. In practice, the estimations of β and h can be obtained iterating between equations 4(a) and 4(b), solving repeatedly for β and h, respectively, until convergence is achieved (i.e. using the backfitting algorithm). Conditions (i)–(iii) also guarantee the convergence of the backfitting algorithm and, in most situations, this convergence is quite fast. For example, for our application (see Section 5), this convergence has been achieved after only four iterations. The proof of the uniqueness of the solution to (4) and the convergence of the backfitting algorithm follows directly from Theorem 4.1 in Green and Silverman (1994). The complete estimation process can be described by the following steps: step 1: separate the repeated values of the response variable; step 2: calculate the Kaplan–Meier estimator Fˆn (Kaplan and Meier, 1958) for the distribution function F and compute the Kaplan–Meier weights Win ; step 3: put the observed response variable ln Y in increasing order and reorder the covariates in agreement with this given order; step 4: build the incidence matrix N . Now the steps for the backfitting algorithm are: step 5: obtain the initial estimated value of h (hˆ 0 ) by applying ordinary least squares between ln Y and N ; step 6: substitute h by hˆ 0 in 4(a) and obtain βˆ0 by weighted least squares using the Kaplan–Meier weights; step 7: substitute β by βˆ0 in 4(b) and obtain the new estimation of h (hˆ 1 ) applying a natural cubic spline smoother to the difference (ln Y − Xβ); step 8: go to step 6 and iterate until convergence is achieved. In order to analyse the statistical properties of the resulting estimators we believe it is important to point out some relevant considerations related to this issue. On the one hand, consistency and asymptotic properties for the partial linear model estimator without censored observations have been studied by Heckman (1986), and the resulting spline estimator is consistent for very general hypotheses where penalized weighted least squares are applied. Basically, it is required that the smoothing parameter α go to zero together with α 1/4 n tending to infinity. The idea behind this has to do with the local character of the nonparametric estimator. The smoothing parameter has to be small enough to fit the data but not too small such that there is no sufficient amount of data involved in the estimation procedure. On the other hand, in a pure parametric context, under (H.1) and (H.2) the estimator proposed by Stute is consistent and has an asymptotic normal distribution (Stute, 1993, 1996). For the partial linear model, the system of equations in (4) indicates that, if the parameter estimators are consistent, the nonparametric part appears as the solution of a penalized weighted least squares problem. Thus, one should expect a global consistent solution when considering the whole set of conditions. However, the formal proof of the asymptotic properties is not a trivial issue and needs further research, since the partial linear model has censored observations. Alternatively, to check the accuracy of the proposed estimator, in Section 4 we will carry out a simulation study, where the effect of the sample size and the censorship will be studied. To make inference on the estimators, we will propose a bootstrap technique, adapted to the particular dataset that will be analysed. 3. I NFERENCE USING BOOTSTRAP There are two main bootstrap resample methods for censored data in homogeneous models: Reid (1981) and Efron (1981), which have been compared in Akritas (1986). Since, in our case, we have covariates, we propose a new procedure to generate the bootstrap samples for the case of random

Censored partial regression

113

censorship and a heterogeneous model, that is more related with Efron’s proposal. This procedure does not assume any particular relation between the censoring variable and the covariates, being therefore very flexible. We summarize it in the following steps: step 1: fit model (1) as described in Section 2 and compute its residuals; step 2: resample the centered residuals to obtain the bootstrap sample 1∗ , . . . , n∗ ; ∗ ˆ step 3: generate ln Ti∗ = X i βˆ + (N h(R)) for i = 1, . . . , n; step 4: generate a vector of i + i ; ∗ ∗ ∗ Bernoulli variables δ , where P(δi = 1| ln Ti = ln ti∗ , X i = xi ) = 1 − Gˆ n (ln ti∗− ), for i = 1, . . . , n, and obtain the bootstrap indicator of censoring. Here, Gˆ n is the Kaplan–Meier estimator of the distribution function for the censoring variable; step 5: generate the censoring variable. If ln T ∗ = ln t ∗ and δ ∗ = 1, C ∗ is taken from Gˆ n restricted to [ln t ∗ , ∞), while if ln T ∗ = ln t ∗ and δ ∗ = 0, C ∗ is taken from Gˆ n restricted to [0, ln t ∗ ); step 6: Estimate model (1), for the bootstrap sample, using the same estimation   ” 2
n ∗ [ln Y ∗ − X β − h(R )]2 + α procedure as in step 1. That is: minβ,h i=1 Win h (r ) dr ; step 7: go [i] [i] (i) back to step 2 and repeat the process M times (i.e. M bootstrap samples are obtained). Notice that, in step 2, the residuals bootstrap sample has been obtained using the Kaplan–Meier estimator for the c.d.f. of the residuals. In step 5, C ∗ has distribution Gˆ n . Thus, no obvious restrictions have been imposed on the joint distribution of the covariates and C and, in particular, independence is not required. In practice, the number M depends on the objective of the study. To estimate the distribution of the estimators or confidence intervals, a large value as M = 1000 should be used. To obtain standard deviations, far lower values are sufficient. For more details about bootstrap procedures see, for example, Davison and Hinkley (1997) or Efron and Tibshirani (1993). 4. S IMULATION STUDIES In this section we proceed to empirically study the properties of the proposed model under different sample sizes and censoring levels. Furthermore, we will be particularly interested in the effect of the censoring mechanism on the estimation of the parametric and nonparametric components of our model. We will specifically study the performance of the estimator under three different structures. In the first two cases, the main hypotheses of the model hold, and in the third one, they do not apply. The latter can represent some particular situations that may arise in practice, as in the case of the application considered in Section 5. However, this really represents the worst scenario for the purpose of illustration. Case 1. In cases 1 and 2, we have generated a duration variable with log-normal probability distribution, following the model ln T = 2 + 1X 1 + 3X 2 + esinX 3 + ,

with  ∼ N (0, σ = 0.5),

(5)

where X 1 ∼ U [0, 3] and X 2 ∼ U [0, 1] are the explanatory variables. For the nonparametric component, we use the function esinX 3 , a function with two peaks and one valley, where X 3 has been obtained randomly generating equally likely integer values, between 0 and 10. The censoring variable is considered to be uniform, independent from the duration and from the explanatory variables, a usual assumption considered in other simulation studies (see e.g. Jones (1997) and Stute (1999)). Case 2. We now allow for possible dependence between the censoring variable and the covariates, but so that (H.2) still holds. The duration variable is generated as in case 1. Model (5) enables us to generate the censoring and the indicator variables using an algorithm based on steps 4 and 5 of the bootstrap procedure (Section 3), that puts no restriction on the joint distribution of the covariates and the censoring variable. We consider three levels of censoring, 10%, 20% and 33%, two different sample sizes, n = 100 and n = 200 and, for each combination, we generate 1000 samples. We now estimate the censored partial linear model, where the parameters estimated are the coefficients of X 1 and X 2 , and the nonparametric

114

J. O RBE ET AL. Table 1. β1 and β2 coefficients estimation: case 1

n = 100

n = 200

Censoring level % 10 20 33

Mean 0.9988 0.9977 0.9936

β1 Variance 0.0039 0.0045 0.0061

Mean 2.9939 2.9935 2.9843

10 20 33

0.9998 0.9998 0.9982

0.0018 0.0022 0.0030

3.0003 2.9989 2.9968

β2 Variance 0.0393 0.0475 0.0625 0.0171 0.0195 0.0262

MSE (total) 0.0432 0.0521 0.0689 0.0188 0.0217 0.0292

Table 2. β1 and β2 coefficients estimation: case 2

n = 100

n = 200

Censoring level % 10 20 33

Mean 0.9980 0.9987 0.9965

β1 Variance 0.0042 0.0052 0.0068

Mean 2.9913 2.9868 2.9751

10 20 33

1.0005 0.9996 0.9976

0.0018 0.0020 0.0026

3.0076 3.0088 2.9962

β2 Variance 0.0321 0.0392 0.0480 0.0166 0.0191 0.0254

MSE (total) 0.0364 0.0446 0.0555 0.0185 0.0212 0.0281

part will account for the term f (X 3 ) = 2 + esin X 3 . Table 1 summarizes the results for case 1, providing the estimated means and variances for the parameter estimators in all cases. The results for case 2 are presented in Table 2. The effect of the censoring, as expected, tends to increase the variance of the estimators, losing precision as the censoring level increases. In addition, and also as expected, the precision is improved as the sample size increases. For the nonparametric part, we use the measurement error ME = 2
11  (1/11) i=1 f (x3i ) −  f (x3i ) at the fixed knots x3i = {0, 1, . . . , 10}, where the function is actually estimated. The resulting values are presented using box plots (one for each censoring level) for n = 100 (Figure 1), and for n = 200 (Figure 2). The left panels in each figure present the results for case 1 and the right panels present the results for case 2. In each panel, we have the results for the different censoring levels considered: in the left box we have a 10% censoring level, in the middle box, 20%, and in the right box, 33%. Moreover, since the nonparametric function can estimate the function at any point between 0 and 10, we illustrate the estimation for the whole interval (see Figure 3) by plotting the true function together with the confidence intervals computed for case 2, for samples sizes n = 100 and n = 200, and a censoring level of 20%. As expected, the confidence interval becomes narrower when the sample size increases. We have also simulated cases for n = 500 and 1000, and the results behave as expected (figures are not shown). Similar results appear when we estimated the model in case 1. Case 3. Finally, we want to check the effect of removing some assumptions in our methodology. For an illustration, we have simulated data from a linear model where the duration variable follows a log-normal probability distribution: that is, ln T = θ0 + θ1 X + θ2 Z + ,

(6)

where X ∼ U [0, 1] and Z ∼ U [0, 1] are the explanatory variables and  ∼ N (0, σ = 0.5). The true values for the parameters are θ0 = −2, θ1 = θ2 = 0. The censoring variable is calculated as C =

115

0.2 0.0

0.1

ME

0.3

Censored partial regression

Case 1

Case 2 Censorship 10%, 20% and 33%

0.2 0.0

0.1

ME

0.3

Fig. 1. Measurement Error (ME) for the nonparametric component (n = 100).

Case 1

Case 2 Censorship 10%, 20% and 33%

Fig. 2. Measurement Error (ME) for the nonparametric component (n = 200).

1 − X , and this relation will imply that the censorship will be concentrated at the observations where X is close to one. In this case, the independence between the censoring and the duration variable does not hold and, therefore, the parametric estimators proposed under this assumption might lead to inconsistent estimators. There are different estimators that account for a weaker hypothesis where the assumption is the independence between the distributions of the censoring and the duration, but when conditioned to the covariates (Ritov, 1990). Unfortunately, the consistency of the estimators in this type of models is only proved in a framework where the observations of the dependent variable lie in a interval truncated from above. This fact can lead to inconsistent estimators when they are related with the whole distribution (e.g. duration mean). In this particular case, since the censoring is concentrated at the end of the study, we expect that the local character of our nonparametric estimation should avoid global errors in the parameter estimators. In order to provide an intuitive argument for this, note that, in the backfitting procedure, if the

116

J. O RBE ET AL. 5.5

5

4.5

4

3.5

3

2.5 95% C.I. (n=200 ) 95% C.I. (n=100 )

2

f(X3 )

1.5 0

1

2

3

4

5

6

7

8

9

10

X3

Fig. 3. Nonparametric component (case 2 with 20% of censorship).

Table 3. Estimations in the partial linear model θ2 Sample size 100 500 1000 2000

Mean 0.0072 −0.0026 −0.0019 0.0012

SD 0.2044 0.0817 0.0590 0.0432

nonparametric term is removed, the parameter estimators are consistent. For the nonparametric estimator at each point, we just use the ‘closer’ observations and this avoids the use of the information close to the end of study for this covariate. That is, the parametric part uses the whole set of information and the nonparametric part the local, which will lead to unreliable results only at the end of the study and for this specific covariate. However, it should lead to reliable global estimators for the rest of the covariates. To test for this, four different sample sizes are considered, n = 100, 500, 1000, 2000 and, for each sample size, we generate 1000 samples. The censorship level is kept fixed at 15% for all cases. We now estimate the partial regression model ln T = f (X ) + θ2 Z + . The results for the parametric component of the model are presented in Table 3, and Figure 4 shows the 95% empirical confidence bands for the estimation of the nonparametric part of the model, f (X ), for n = 1000. From Table 3, we can see that the estimation for the parameter θ2 in the model has a bias and a variance that decrease as the sample size increases, which is a good sign for the estimation process. As we can see from Figure 4, the resulting confidence bands for the nonparametric estimate covers the true value f (X ) = −2. As expected, the nonparametric estimator should not be interpreted at the end of the study. We also have computed the pure parametric model (6) using Stute’s method. Table 4 presents the results where we can observe a clear bias in the estimators. 5. A PPLICATION : STUDY OF THE SURVIVAL TIME FOR AIDS PATIENTS The dataset contains information about the survival time for AIDS-diagnosed patients who lived in the Basque Autonomous Community and the Autonomous Community of Navarra in Spain. We have a

Censored partial regression

117

-1

-1.5

-2

-2.5

-3

-3.5 95% C.I. (n=1000 )

f (X)

-4 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

X

Fig. 4. Nonparametric component (case 3 with 15% of censorship).

Table 4. Estimations in the fully parametric model Sample size 100 500 1000 2000

θ0 Mean −1.8593 −1.8900 −1.8650 −1.8687

SD 0.1537 0.0594 0.0445 0.0311

θ1 Mean −0.2906 −0.3086 −0.3102 −0.3205

SD 0.2058 0.0893 0.0618 0.0452

θ2 Mean −0.0429 0.0186 −0.0022 0.0063

SD 0.1980 0.0816 0.0593 0.0433

sample of 461 patients diagnosed with AIDS from 1984 until December 31, 1990. The duration variable under study measures the survival time for the patient from the illness diagnosis time up to death or up to the end of the study (censored observations). The evolution of the HIV virus has three stages. The first one is known as the ‘pre-antibody’ stage and it is the shortest one with a duration of only several months and goes from the infection date to the development of antibodies or seroconversion point. The second stage, the ‘incubation’ stage, is the largest one, starting with the seroconversion (the patient is classified as seropositive) and goes until the diagnosis of AIDS. Finally, the third stage gives the survival time from the AIDS diagnosis time, when the individual develops some of the illnesses related to AIDS, up to death or up to the end of the study. We study the duration in the third stage. As of December 31, 1992, there were 447 patients that had died and 14 patients that had survived, the censored observations. The covariates are: sex (Sex), age at diagnosis (Age), transmission category, disease at AIDS diagnosis and the period of diagnosis (Period). The transmission category is coded using five dummy variables for T-Sex, T-Drug, T-Blood, T-Moth-child and T-Others. The disease at AIDS diagnosis is coded using three dummy variables Disease1, diagnosis through an opportunistic infection; Disease2, diagnosis produced by a Kaposi’s sarcoma or some lymphoma; and Disease3, diagnosis through an HIV encephalopathy or a HIV wasting syndrome. For the effect of the AZT treatment (starting in the middle of 1987), we introduce the variable R which indicates the period of diagnosis (in quarters). Thus, this variable takes values from one (diagnosis in the second quarter of 1984) to 27 (diagnosis in the last quarter of 1990). Since the variable period of diagnosis has a clear relation with the censoring variable (i.e. C = t0 − R, where t0 = 35, 27 quarters under study and eight under follow-up), we are in a situation similar to case 3

118

J. O RBE ET AL. Table 5. Estimates of β and 95% confidence intervals (Censored Partial Regression Model) Variable Constant Sex Disease1 Disease2 T-Sex T-Drug T-Blood T-Moth-child Age

Coef 0.5266 0.0348 −0.0154 −0.0508 −0.1201 −0.0430 0.0702 0.3877 −0.0177

SD 0.4188 0.1357 0.2582 0.3218 0.2458 0.2090 0.2868 0.4727 0.0065

LL −0.3778 −0.2218 −0.5159 −0.6872 −0.5714 −0.4264 −0.5232 −0.7676 −0.0289

UL 1.2963 0.3025 0.4874 0.5627 0.3663 0.3810 0.6329 1.1138 −0.0028

presented in Section 4 and this will affect to parametric specifications of this variable. This fact, together with the issue of considering a possible gradual effect for the AZT motivates to propose the estimation of a partial regression model in Section 2, where all covariates are introduced parametrically, except the period of diagnosis, which is introduced through a nonparametric term. As indicated in the previous section, the partial estimator should only be affected at the end of the range of the variable R. However, since one of the main interests in the application was the study of the effect of AZT, introduced in 1987, on the survival of the patients (i.e. r = 13–14 in the partial linear model), corresponding to the mid-part of the study, the interpretation of the results can be done in a reliable manner. As for the inference on the estimators, since we know the relation between the censoring variable and one of the explanatory variables (i.e. period of diagnosis) we have changed steps 4 and 5 in the bootstrap procedure described in Section 3, to reproduce this relation in the resamples. The new steps 4 and 5 will calculate the censoring indicator using δi∗ = I (ti∗
ci ), where ci = t0 − ri , t0 = 35 quarters and ri is the period of diagnosis. With this procedure, we calculate standard deviations as  σˆ βˆk = (M − 1)−1

M i=1

i∗

βˆk

i∗

− M −1

M

βˆk

j∗

2 1/2  ,

j=1

where βˆk indicates the estimated βk in the ith bootstrap sample. The confidence intervals are computed using the percentile method (Efron, 1982), denoting by LL and UL the lower and upper limit respectively. Table 5 reports the results for the parametric component, and Figure 5 those for the nonparametric component. With regard to the covariates introduced in the parametric component, only the age of the patient has a significant, negative, effect on his/her survival time. As for the estimation of the nonparametric component, the period of diagnosis does not seem to have a significant effect at the beginning of the study. As time goes by, a significant and positive effect appears with a clear acceleration several quarters before the beginning of the administration of AZT (started in the middle of 1987, i.e. r = 13). This makes sense because patients whose diagnosis time was several quarters before starting the administration of AZT also received this treatment. Therefore, we can say that the introduction of AZT has a positive effect on the survival, increasing the survival time of patients. In order to better illustrate this, we can look at Figure 5 and consider the rest of the covariates to remain constant. We can observe that the difference, in terms of mean of ln T , for a patient diagnosed of AIDS at the beginning of the illness (i.e. r = 1, last quarter of 1984) and a patient, for example, diagnosed with AIDS just after starting the introduction of

Censored partial regression

119

2

1.5

1

0.5

0

-0.5 95% C.I. h (R)

-1

-1.5 1

4

7

10

13

R

16

19

22

25

Fig. 5. Estimation of the nonparametric component (application). This figure shows the function h(R), that represents the nonparametric estimation of the effect of the period covariate on the mean log-duration, together with the 95% confidence bands. From the figure we can observe a positive and gradual effect of the AZT treatment (that started in the middle of 1987, i.e. R = 13). The increase of the mean log-duration of patients is clear and it can be directly quantified by comparing the values in ordinates. This effect can not be easily captured using parametric specifications.

AZT (i.e. r = 13) is larger than 1. That is, the difference in mean survival time between those two periods increases approximately in almost four quarters. 6. D ISCUSSION As a summary, this paper proposes a censored partial regression model to analyse a response variable and the effect of some covariates under censored observations. The proposal is based on a distributionfree model, where the proportionality of the hazard functions is not required, and it allows us to model situations where the effect of one or more covariates is not parametrically specified. In addition, we model directly the effect of the covariates on the duration, which leads to an easier interpretation of the results. The simulation study indicates that this procedure produces good estimates for the parametric component and for the nonparametric one, even in situations where the hypotheses of the model do not hold. A new bootstrap technique is derived to make inference on the estimators. Finally, the application of the methodology to the dataset allows us to interpret the gradual effect of the AZT treatment, that cannot be easily capture using parametric specifications. This analysis shows the evolution of the survival time for the illness during the period under study. In the first quarters, the effect of the period of diagnosis on the survival time is small and, as time goes by, this effect increases and presents a strong acceleration. This can be due to the fact that AIDS was quite unknown at the beginning and, then, as it became widely known, the disease was diagnosed earlier. ACKNOWLEDGEMENTS The authors would like to thank Winfried Stute for valuable comments and discussion. This work was partially supported by Universidad del Pa´ıs Vasco/Euskal Herriko Unibertsitatea (UPV/EHU), Direcci´on General de Ense˜nanza Superior e Investigaci´on Cient´ıfica del Ministerio Espa˜nol de Educaci´on y Cultura

120

J. O RBE ET AL.

and Gobierno Vasco under research grants UPV 038.321-HA129/99, UPV 038.321-13631/2001, PB980149, PI-1999-70 and PI-1999-46.

R EFERENCES A KRITAS , M. G. (1986). Bootstrapping the Kaplan–Meier estimator. Journal of the American Statistical Association 81, 1032–1038. B UCKLEY , J. J. (1979). Linear regression with censored data. Biometrika 66, 429–436. C OX , D. R. (1972). Regression models and life-tables. Journal of the Royal Statistical Society, Series B 34, 187–220. DAVISON , A. C. AND H INKLEY , D. V. (1997). Bootstrap Methods and Their Application. Cambridge: Cambridge University Press. E FRON , B. (1981). Censored data and bootstrap. Journal of the American Statistical Association 76, 312–319. E FRON , B. (1982). The Jackknife, the Bootstrap and Other Resampling Plans, CBMS-NSF 38. Philadelphia: SIAM. E FRON , B. AND T IBSHIRANI , R. J. (1993). An Introduction to the Bootstrap. New York: Chapman and Hall. G RAY , R. J. (1992). Flexible methods for analyzing survival data using splines, with applications to breast cancer prognosis. Journal of the American Statistical Association 87, 942–951. G RAY , R. J. (2000). Estimation of the regression parameters and the hazard function in transformed linear survival models. Biometrics 56, 571–576. G REEN , P. J. AND S ILVERMAN , B. W. (1994). Nonparametric Regression and Generalized Linear Models. London: Chapman and Hall. H ASTIE , T. AND T IBSHIRANI , R. (1993). Varying-coefficient models. Journal of The Royal Statistical Society, Series B 55, 757–796. H ECKMAN , N. E. (1986). Spline smoothing in a partly linear model. Journal of The Royal Statistical Society, Series B 48, 244–248. I NTRATOR , O. AND KOOPERBERG , C. (1995). Trees and splines in survival analysis. Technical Report. University of Washington. J IN , D. Y., Y ING , Z. AND W EI , L. J. (2001). A simple resampling method by perturbing the minimand. Biometrika 88, 381–390. J ONES , M. P. (1997). A class of semiparametric regressions for the accelerated failure time model. Biometrika 84, 73–84. K APLAN , E. L. AND M EIER , P. (1958). Nonparametric estimation from incomplete observations. Journal of the American Statistical Association 53, 457–481. KOOPERBERG , C., S TONE , C. J. AND T RUONG , Y. K. (1995). Hazard regression. Journal of the American Statistical Association 90, 78–94. KOUL , H., S USARLA , V. AND VAN -RYZIN , J. (1981). Regression analysis with randomly right-censored data. The Annals of Statistics 9, 1279–1288. L AWLESS , J. F. (1982). Statistical Models and Methods for Lifetime Data. New York: Wiley. L EBLANC , M.

AND

C ROWLEY , J. (1999). Adaptive regression splines in the Cox model. Biometrics 55, 204–213.

L IN , D. Y., W EI , L. J. 85, 605–618.

AND

Y ING , Z. (1998). Accelerated failure time models for counting processes. Biometrika

M ILLER , R. G. (1976). Least squares regression with censored data. Biometrika 63, 449–464.

Censored partial regression

121

´ ˜ -A NT ON ´ , V. (1996). An´alisis de la supervivencia en enfermos de O RBE , J., F ERN ANDEZ , A. AND N U´ NEZ SIDA residentes en la Comunidad Aut´onoma del Pa´ıs Vasco y Navarra. Osasunkaria 12, 1–8. R EID , N. (1981). Estimating the median survival time. Biometrika 68, 601–608. R ICE , J. (1986). Convergence rates for partially splined models. Statistics and Probability Letters 4, 203–208. R ITOV , Y. (1990). Estimation in a linear regression model with censored data. The Annals of Statistics 18, 303–328. S PECKMAN , P. (1988). Kernel smoothing in partial linear models. Journal of the Royal Statistical Society, Series B 50, 413–436. S TUTE , W. (1993). Consistent estimation under random censorship when covariables are present. Journal of Multivariate Analysis 45, 89–103. S TUTE , W. (1996). Distributional convergence under random censorship when covariables are present. Scandinavian Journal of Statistics 23, 461–471. S TUTE , W. (1999). Nonlinear censored regression. Statistica Sinica 9, 1089–1102. T SIATIS , A. A. (1990). Estimating regression parameters using linear rank tests for censored data. The Annals of Statistics 18, 354–372. W EI , L. J., Y ING , Z. AND L IN , D. Y. (1990). Linear regression analysis of censored survival data based on rank tests. Biometrika 77, 845–851. [Received May 8, 2001; first revision August 8, 2001; second revision January 2, 2002; accepted for publication February 1, 2002]